Monday, March 11th, 2013

What if frameworks were protocol independent?

Why does a programming framework usually need to be for specific protocol? Wouldn't it be really cool to use Django to develop something other than an HTTP application?

I believe the future of these so-called frameworks needs to be protocol independent. Where you as the programmer can plug in a different backend and literally use the same logic code you created in a completely new way with as little modification as possible. This is what a true Model-View-Controller framework should be. I am rather curious on seeing how easy it would be to decouple the WSGI element from Django and make it pluggable, so that I can tell it to display the data in a GTK+ application, a dynamic FTP server, or even create a Gopher server using Django. This will allow me to easily utilize my existing Django API knowledge to create non-HTTP applications. I know I could just use the API itself and manually insert it into another server's code, but this seems kind of hackish. It might be possible to do it with a reusable app, and override the runserver command to use a different protocol. The only issues that stand, will be supporting sessions and other stateful features. Also the returning of content from an HTTPResponse will need to be retooled to use the other protocol.

What are your thoughts? Do you think frameworks should have an easy API for using alternate protocols with the framework?

Comment #1: Posted 2 years ago by James Mills

You can possibly already do this with the circuits (1) framework. This is because circuits' architecture is centered around loosely coupled components. In circuits.web for example (in theory) you could rip out the Server and HTTP components and replace them with something else whilst still keeping your Controller components in tack. The only problem I see with this is you'd have to send around "fake" Request/Response objects; but this could be decoupled as well.

Nice article :)

--JamesMills / prologic

Comment #2: Posted 2 years ago by James Mills


Comment #3: Posted 2 years ago by Israel Fruchter

Sound like a cool idea,

But, it won't happen Django is built for web application, which goes hand by hand with HTTP.
that won't be changing in the next couple of years.

when you talk about showing same application on GTK
you might want to use kivy for that (

BTW, you can still use the django API (i.e. jsut the ORM) without the template and views

Comment #4: Posted 2 years ago by encolpe

Did you ever take a look at Zope ?

Comment #5: Posted 2 years ago by Jokey

You can re-use the model part everywhere, just set the env variable for settings and import django. For the controller and view part, this is rather unlikely at least at the moment because a GUI application actually uses a controller for more than just redirecting an url to a function with some salt. Then the view part is another story. I mean, you could do it to some extend with Qt, as it can render windows solely on xml files but if that's a feasible idea from user perspective is another question to be answered.

Comment #6: Posted 2 years ago by joeyjojo

I got frustrated by this too and decided to mangle django a bit to work in a qwebview (PyQt)

Results here:

Its done in what you would probably describe as the hackish way in your post, converting the QnetworkRequest/Reply objects from QtWebkit into standard HttpRequest/Response objects

Comment #7: Posted 2 years ago by Lennart Regebro

God, stupid comment systems that doesn't work. I'll try again, but it'll be shorter this time.

There are two things involved here which are differet and which you seem to confuse:

1. Protocols, ie switching out http or something else. This is at least theoretically possible, but generally pointless. Why would you want to have FTP instead of HTTP?

2. Using GTK or any otehr UI instead of a browser. This has nothing to do with protocols. There are GUI frameworks that run on top of other frameworks, so that you from the same app can build a GTK, QT or Windows app. They have never become popular, as they tend to become big and limiting, as their feature set tend to become a smallest common denomitator of what they support. Since browsers are very different from the others, having browser support makes the thing even worse. So very few people does this, becuase it sucks.

What do do instead? Easy: build your app in HTML5 and JS, and build a REST server API. Because browsers are everywhere, so if you do that, you can run your app anywhere anyway.

Comment #8: Posted 2 years ago by joeyjojo

I got frustrated by this too and decided to mangle django a bit to work in a qwebview (PyQt)
Results here:
Its done in what you would probably describe as the hackish way in your post, converting the QnetworkRequest/Reply objects from QtWebkit into standard HttpRequest/Response objects

Comment #9: Posted 2 years ago by Tim Hoffman

django is pretty opinionated about a lot of things, I think other frameworks (or bit's of frameworks) would be a but more flexible, whole chunks of pyramid could easily be used outside of a WSGI/web stack.

Comment #10: Posted 2 years ago by Paulo Almeida

I've been playing with that idea, inspired by Alistair Cockburn's hexagonal architecture:

The idea is to decouple business logic from user interface and data backends (or, more generally, from everything that drives or is driven by the application), so you can easily swap components. It's fairly straightforward for the interface; you expose an API in the business logic and call it from the command line, wsgi application, etc. For the backend you can use dependency injection, which is simpler in Python than in static languages.

I love Django, but for these experiments I've been using Werkzeug, because of the flexibility it allows.

Comment #11: Posted 2 years ago by Ben Rousch

You should be looking at the Twisted Python framework. It has an agnostic backend and you plug in whatever protocols you need. It's been around a long time and is still actively developed.

Comment #12: Posted 2 years ago by Mike Fletcher

I'd have to second the "look at Twisted" thing, their primary tutorial is even "create a 'finger' application that runs on IRC, web, etc.". The downside is that the rigour of the callback-oriented programming style required is far less friendly than the synchronous ORM-mediated style of a Django application. Similarly, getting new programmers to rigourously separate concerns is not easy. You might even look at building a bridge using Twisted (it can host django via wsgi) to host your django-hydra if you really want to experiment with this kind of thing.

Comment #13: Posted 2 years ago by Laurence Rowe

So long as you ignore the edge case support for streaming files, it's easy enough to map WSGI onto another request/response protocol. I've done this in the past with the XMPP info/query stanza, though <iq> allows for multi-legged message exchanges which you can't map easily onto WSGI. I ended up serializing messages as JSON, though there are standards like HAL which define alternative serialization formats if you must support XML and JSON clients.

I expect this to be something we will all have to deal with much more in the future as WebSockets allow for richer protocols to be used more widely on the web.

Comment #14: Posted 1 year, 12 months ago by adwokat pszczyna

Nice blog here! Also your web site loads up very fast! What web host are you using? Can I get your affiliate link to your host? I wish my web site loaded up as quickly as yours lol

About Me

My Photo
Names Kevin, hugely into UNIX technologies, not just Linux. I've dabbled with the demons, played with the Sun, and now with the Penguins.

Kevin Veroneau Consulting Services
Do you require the services of a Django contractor? Do you need both a website and hosting services? Perhaps I can help.

This Month

If you like what you read, please consider donating to help with hosting costs, and to fund future books to review.

Python Powered | © 2012-2014 Kevin Veroneau