How to Have Different Django Settings for Development and Production, and environment isolation

Overview

Django projects should have several environments each with its own peculiarities. Each environment is optimized for a specific task, e.g.: develop your app, having packages that helps you test it, another with all the security measures for production, and so on.

In this guide we will split the standard settings file into specific settings optimized for each environment.

A common deployment architecture consists of a 4-tier architecture, having different settings for one of these tiers:

1. development (DEV)
2. testing (TEST)
3. staging (STAGE)
4. production (PROD)

A default Django app just starts with a single configuration file located in DJANGO_PROJECT/settings.py. This approach is fine for small projects but to fit in the above 4-tier architecture, the project needs to be changed to address two main problems:

• each environment should have a specific settings file
• each environment should have its own packages

However there are special config keys that should be left out of versioning, like the SECRET_KEY setting, used for cryptographic signing functionalities.

Settings

There are two main approaches to work with settings in different environments:

1. Use environment variables
2. split settings.py file into multiple files and use each one in each different environment

It is better to have sensitive keys as environment variables whenever possible so it leaves secret keys outside config files versioning using environment variables as recommended by http://12factor.net/config.

Splitting settings description

The default Config file that comes shipped with Django should be pulled apart into several settings for each environment: local, staging, test, production.

This can be done easily inheriting from a base config file, changing what the specific environment needs

Using different settings

To set SECRET_KEY and DJANGO_DEBUG variables in each environment add this to settings.py.

# Read SECRET_KEY from an environment variable
import os
SECRET_KEY = os.environ.get('DJANGO_SECRET_KEY', 'p@1a;sljfk=$adsk;lhPUO*R*P(@HBBGG)g')
DEBUG = bool( os.environ.get('DJANGO_DEBUG', True) )

This will set a default value for the SECRET_KEY variable at low security environments like development and testing, then you will have to set DJANGO_SECRET_KEY and DJANGO_DEBUG variables in production.

Variables can be set with environment variables e.g.: export A_SECRET_KEY=foobar1234 $ export A_SECRET_KEY=shhh1234, and placed in:

• .bashrc or .profile
• virtualenvwrapper’s bin/activate hook file

When using manage.py many commands accepts the parameter to specify a specific settings file: python manage.py runserver --settings=myproject.settings.local

Checking settings environment variables

To check that the secret environment key is being loaded, it is possible to check it from a python shell:

$ ./manage.py shell
Python 3.6.4 (default, Feb  5 2018, 16:52:44) 
[GCC 7.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
(InteractiveConsole)
>>> import os
>>> os.environ["A_SECRET_KEY"]
"shhh1234"

Then to get the value for a specific environment, the production config file in version control only needs to get this environment variable value: A_SECRET_KEY = os.environ["A_SECRET_KEY"]

Creating different settings files

When building a project, Django automatically creates a configuration file in $PROJECT_NAME/settings.py.

To split this file into multiple settings files (one for each environment local, testing, staging and production), the best way is to create a base.py config with common settings across all of them and create specific config files for each environment in specific files having a structure like:

.
└── REPO-ROOT (git repo)
    ├── ...
    └── PROJECT-ROOT
        ├── settings
        |   ├── __init__.py
        |   ├── base.py
        |   └── local.py
        ├── manage.py
        └── ...

To build this structure:

1. Create directory

Create the settings directory

$ mkdir $PROJECT_NAME/settings

2. Add init.py

Add __init__.py file to make this directory a Python package.

$ touch $PROJECT_NAME/settings/__init__.py

3. Create base.py from settings.py

Move the default settings file into the new settings directory and change its name.

$ mv $PROJECT_NAME/settings.py $PROJECT_NAME/settings/base.py

4. Create setting files

Create all the configuration files (local.py, testing.py, staging.py, production.py ) and specify to inherit base.py configurations adding from .base import * to each file:

$ for name in local.py testing.py staging.py production.py; do echo "from .base import *" >> $PROJECT_NAME/settings/${name} ; done

So the directory tree structure is:

myapp/settings/
├── base.py
├── __init__.py
├── local.py
├── production.py
├── staging.py
└── testing.py

Also, in app/settings/base.py you can have the following helpful variable to work with your project main path:

# Build paths inside the project like this: os.path.join(BASE_DIR, ...)
BASE_DIR = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__))))
# or
BASE_DIR = Path(__file__).resolve(strict=True).parent.parent.parent

5. Use new settings file

Use the new settings file in one of these ways:

• call scripts specifying a settings file as mentioned earlier
• set an environment variable and call scripts normally

5.1 Set environment variables

Configure the current environment to use the appropriate settings file, using PYTHONPATH and DJANGO_SETTINGS_MODULE:

 

$ export DJANGO_SETTINGS_MODULE=mysite.settings.local
$ export PYTHONPATH=~/path/to/my/project
#django-admin will use the above settings by default
$ django-admin runserver

Use the --settings parameter with manage.py or django-admin:

 

$ django-admin runserver --settings=mysite.settings.local --pythonpath=/path/to/my/project
Performing system checks...

System check identified no issues (0 silenced).

July 23, 2016 - 22:43:48
Django version 1.9.6, using settings 'mysite.settings.local'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Differences between manage.py and django-admin

Django has two administrative scripts: django_admin.py and manage.py (in the root of each Django project).

We use the new path in manage.py, changing this os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings.local') to os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'mysite.settings.local')

Generally, when working on a single Django project, it’s easier to use manage.py than django-admin. If you need to switch between multiple Django settings files, use django-admin with **DJANGO_SETTINGS_MODULE** or the **--settings** command line option.

Packages for each environment

Now a similar approach for handling package dependencies.

Each environment has to have a set of packages that fits its purpose and operating system requirements. We have to configure a python virtual environment so it is possible to install packages in development, that are not needed in production and viceversa, or that can be installed in different Operating Systems.

Each environment needs a specific file, having a base.txt requirement file with common packages across environments and then adding the needed packages for each environment.

.
└── REPO-ROOT git repo
    ├── .gitignore
    ├── requirements
    |   ├── base.txt
    |   ├── local.txt
    |   ├── production.txt
    |   └── test.txt
    ├── ...
    └── PROJECT-ROOT
        ├── manage.py
        └── ...

Using pip it is possible to specify which file has the list of packages you want:

pip install [options] -r <requirements file> [package-index-options] ...
  -r, --requirement <file>    Install from the given requirements file. This option can be used multiple times.

So to make it possible for each environment to inherit the packages from the base.txt requirement file using pip, each new file should begin with: -r base.txt:

/requirements/base.txt:

# list of packages present in all environments

/requirements/local.txt:

-r base.txt

/requirements/test.txt:

-r base.txt

/requirements/production.txt:

-r base.txt

• To generate a requirements file: $ pip freeze $ pip freeze --local > requirements/base.txt
• To install the requirements file
  • in a local/development environment: $ pip install -r requirements/local.txt
  • in a testing environment: $ pip install -r requirements/testing.txt

Conclusion

We started having a settings.py single file and break it up into a new directory with specific environment settings.

As with any major change to the default installation, after generating these directories, it is a good practice to describe them in /docs/architecture.rst and what are the commands used to get them running in /docs/installation.rst for other developers or just for oneself when reviewing the project in the future.

Github

There is a Github with all of these steps applied to show how the project skeleton would look like at https://github.com/marcanuy/django-architecture-example

References

• Wikipedia 4-tier architecture
• https://docs.djangoproject.com/en/1.9/topics/settings/
• https://docs.djangoproject.com/en/1.9/ref/settings/
• https://developer.mozilla.org/en-US/docs/Learn/Server-side/Django/Deployment