Wednesday, June 6th, 2012

A real plugable apps framework for Django

Truth be told, Django's current implementation of plugable apps is absolutely it's worst selling point. Apps in Django are not that easy to install, and sometimes can really interfere with existing functionality. It's sometimes very difficult to make multiple apps play together nicely. This article is not about an existing solution, but rather something I would come to expect in Django 2.0 release, as a major change to the framework.

Currently the method of using an existing app in Django, such as a new Authentication backend, or a debugging app like django-debug-toolbar requires heavily modifying your project's settings.py. In the end, the so-called plugable app is mingled with your project, and is hardly plugable. If you need to remove the app in the future, you don't just remove a single directory, you need to also remove many settings from your project's settings.py, and sometimes the urls.py file. This can amount for a lot of work, and remembering how each app is installed, and which settings each app uses. This can cause your project to break, and tables left in your database from an app which is no longer installed. This is the only part of Django which I frown upon the most. When working with Web2py, I admired the dedication done in app separation. However, I found that the apps in web2py were a little too separate, where it was hard to share data with different apps.

This is what I would like to see done in a future Django release to enable real plugable app support. Each app will contain their own settings.py, which when the app is added to the INSTALLED_APPS tuple, will load these settings into the main settings namespace. It will not append it as-is, however. For example, if the app-specific settings.py has set a MIDDLEWARE, it will apply the new middleware to the existing set of middleware using a dependency system. Anyone who uses Django should know that the placement of each middleware is special, and this is why some sort of dependency system will need to be implemented to cater to this. The other settings will simply be appended as needed to the existing settings. Thus, the placement of each app in the INSTALLED_APPS will matter.

This will enable true app separation and plugable app support in Django. It would also be rather nice if settings for apps could be changed via the admin interface. The settings which do not require some sort of Python code to generate that is. This could enable a better end-user experience, as some apps could definitely benefit from this type of configuration.

Let me know what you think of this idea in the comments below, and what you think of Django's current plugable app implementation.

Comment #1: Posted 2 years, 6 months ago by Matthias

A thousand times, yes! The pluggable app system in Django could definitely use an improvement, if not an entire overhaul. You hit the nail on the head with some of the frustrations I run into.

One other area of consideration is standalone, pluggable apps versus project dependent apps. As nice as "every app is independent of the project" sounds in theory, in practice this just isn't the case. It would be great if it were easier to distinguish between the two. In fact, the entire project->app hierarchy in Django can make some cases a little awkward.

Comment #2: Posted 2 years, 6 months ago by Luke Plant

I really don't see how a dependency system for middleware could possibly work.

Take a look at a few real middleware and you'll see that there requirements cannot be worked out automatically. For example, "fiber.middleware.AdminPageMiddleware" from django-fiber must handle the response before any middleware that does compression or it isn't going to work. 'doing compression' is impossibly to work out automatically, unless you have a specific 'does_compression' flag, and every middleware uses it correctly. Clearly this approach will not scale - you have to think about and plan for every possible 'type' of middleware in advance, and have flags for each one of them, which you can't do. Middleware can do almost anything.

And there are much more complex issues with different combinations of middleware.

I think this is a nice aim, but it is impractical in reality. Apps are simply not pluggable - you can't just grab two apps and necessarily think they will work together. Many will,

With regards to settings in the database, you can already do that using something like django-values. It shouldn't be done for most settings however - many settings like MIDDLEWARE_CLASSES must be considered 'code', because they have very big consequences on the running of your app. These belong in settings.py, and should be under revision control / code review etc.

I think the basic problem is that people are using settings.py for stuff that should have been in the database from the start.

Comment #3: Posted 2 years, 6 months ago by Corey Oordt

You might want to look at the comments of a blog post I wrote a few years ago: http://opensource.washingtontimes.com/blog/2009/sep/15/request-comments-auto-installation-apps-django/

In the years that have passed, I see a big issue with this approach: you really do need to "wire in" the app. For example:

* the order in which apps or middleware load can be important.
* You will also probably need to allow overriding the apps settings.
* how you want to implement the urls could vary from project to project.

While some sort of automatic way of implementing an app, using defaults, would be nice. It needs to be done in a way that allows customization.

Corey

Python Powered | © 2012-2014 Kevin Veroneau