dynamically loading django apps at runtime

Posted on

Question :

dynamically loading django apps at runtime

Is it possible to dynamically load Django apps at runtime? Usually, apps are loaded at initialization, using the INSTALLED_APPS tuple in settings.py. However, is it possible to load additional apps at runtime? I am encountering this issue in different situations. One situation, for example, arises during testing, when I would like to dynamically load or unload apps.

In order to make the problem more concrete, imagine I have a directory called apps where I put my apps and I would like to automatically install any new app that goes in there without manually editing the settings.py.

This is easy enough. Following the example code in

Django: Dynamically add apps as plugin, building urls and other settings automatically

we put the following code in settings.py to could loop over the names of all sub-directories in the app directory and increment the INSTALLED_APPS tuple in settings.py like this:

APPS_DIR = '/path_to/apps/'

for item in os.listdir(APPS_DIR):
    if os.path.isdir(os.path.join(APPS_DIR, item)):
        app_name = 'apps.%s' % item
    if app_name not in INSTALLED_APPS:
        INSTALLED_APPS += (app_name, )

After that, if I was in a Django shell, I could something like

from django.conf import settings

and the apps would be listed in settings.INSTALLED_APPS. And if I did

from django.core import management
management.call_command('syncdb', interactive=False)

that would create the necessary DB tables for the apps.

However, if I were to now add some more apps to the apps/ directory, without re-starting, these would not be listed in settings.INSTALLED_APPS, and so a subsequent call to the syncdb would have no effect.

What I would like to know is if there is something I could do — without restarting — to reload the settings and load/install new apps.

I have tried to directly import my settings.py, i.e.
from myproject import settings

and then reload that settings using the python builtin after any app directory changes. Although settings.INSTALLED_APPS is now changed to include the newly added apps, this ultimately makes no difference. For example,

from django.db import models
models.get_apps()

shows only the original apps in apps and not the newly added ones and likewise

management.call_command('syncdb', interactive=False)

will not see the newly added apps.

As I stated above, I am thinking about this situation particularly in the context of testings where I dynamically would add or remove apps.

Ps. I working with Django 1.6, but on the advice of @RickyA, I see that there are some substantial changes to Django’s treatment of applications in 1.7

https://docs.djangoproject.com/en/1.7/ref/applications/

I’m still not sure what this might mean for the problem I am facing.

Answer #1:

Update for Django 1.8 on how to load an app that is not loaded yet

from collections import OrderedDict
from django.apps import apps
from django.conf import settings
from django.core import management

new_app_name = "my_new_app"

settings.INSTALLED_APPS += (new_app_name, )
# To load the new app let's reset app_configs, the dictionary
# with the configuration of loaded apps
apps.app_configs = OrderedDict()
# set ready to false so that populate will work 
apps.ready = False
# re-initialize them all; is there a way to add just one without reloading them all?
apps.populate(settings.INSTALLED_APPS)

# now I can generate the migrations for the new app
management.call_command('makemigrations', new_app_name, interactive=False)
# and migrate it
management.call_command('migrate', new_app_name, interactive=False)
Answered By: Davide

Answer #2:

To answer my own question…

While I do not have a completely general solution to this problem, I do have one that is sufficient for the purposes of dynamically loading apps during testing.

The basic solution is simple, and I found it at a wee little bixly blog.

Continuing with my example above, if I was in a django shell and wanted to add and load some new apps that were added to my apps directory, I could do

import os
from django.conf import settings
from django.db.models import loading
from django.core import management

APPS_DIR = '/path_to/apps/'

for item in os.listdir(APPS_DIR):
    if os.path.isdir(os.path.join(APPS_DIR, item)):
        app_name = 'apps.%s' % item
    if app_name not in settings.INSTALLED_APPS:
        settings.INSTALLED_APPS += (app_name, )

and then

loading.cache.loaded = False
management.call_command('syncdb', interactive=False)
Answered By: mjandrews

Answer #3:

With Django 2.2 this work for me

from collections import OrderedDict
from django.apps import apps
from django.conf import settings
from django.core import management

new_app_name = "my_new_app"

settings.INSTALLED_APPS += (new_app_name, )
apps.app_configs = OrderedDict()
apps.apps_ready = apps.models_ready = apps.loading = apps.ready = False
apps.clear_cache()
apps.populate(settings.INSTALLED_APPS)

management.call_command('makemigrations', new_app_name, interactive=False)

management.call_command('migrate', new_app_name, interactive=False)
Answered By: Gersón Vizquel

Answer #4:

Yes! Anything (or almost everything) in Python is possible. You should use os.walk() in order to get all folders, subfolders and files within your apps path in order to get all of your apps including the nested ones.

def get_installed_apps():
    from os import walk, chdir, getcwd
    previous_path = getcwd()
    master = []
    APPS_ROOT_PATH = '/my/project/apps/folder'
    chdir(APPS_ROOT_PATH)
    for root, directories, files in walk(top=getcwd(), topdown=False):
        for file in files:
            if 'apps.py' in file:
                app_path = f"{root.replace(BASE_DIR + '/', '').replace('/', '.')}.apps"
                print(app_path)
                master.append(app_path)
    chdir(previous_path)
    return master


print(get_installed_apps())
Answered By: Jcc.Sanabria

Answer #5:

It is possible to dynamically load and unload applications in tests in Django >= 1.7 (and also in the current 3.0) by a override_settings() decorator:

@override_settings(INSTALLED_APPS=[...])  # added or removed some apps
class MyTest(TestCase):
    # some tests with these apps
    def test_foo(self):
        pass

It has been possible since September 2014, not before the question.

Many other topics from the question are solved by django.apps also in Django >= 1.7. Generally: Dynamic configuration at startup is easy in the current Django. Dynamic loading and unloading after startup can’t be recommended in production, even that it could work eventually.

Answered By: hynekcer

Leave a Reply

Your email address will not be published.