Configure a Django project to handle multiple environments, creating different settings and handling packages for each one.

Process

Current
graph TB subgraph Django Project conf["settings.py"] pack[packages] end conf==>dev[Development Environment] pack==>dev conf==>prod[Production Server] pack==>prod
Goal
graph LR subgraph Production Server prod_conf["production_settings.py"] prod_pack["Production Server Packages"] end subgraph Development Environment dev_conf["development_settings.py"] dev_pack["Developer Environment Packages"] end

Overview

The development process of Django projects can have several environments, a common deployment architecture consists of a 4-tier architecture, consisting of software being deployed to each tier in the following order:

  1. development (DEV)
  2. testing (TEST)
  3. staging (STAGE)
  4. production (PROD)

A default Django app just starts with a single config file located in DJANGO_PROJECT/settings.py. This approach is fine for small projects but to fit in the above 4-tier architecture, the project needs to be changed to address two main problems:

  • each environment should have a specific settings file
  • each environment should have its own packages

The configuration file should be version controlled, even the developers local configuration file, all the developers of a project should use the same development configuration.

However there are special config keys that should be left out of versioning, like the SECRET_KEY setting (used for cryptographic signing functionalities)

The default Config file that comes shipped with Django should be pulled apart into several settings for each environment: local, staging, test, production. This can be done easily inheriting from a base config file, changing what the specific environment needs and leaving secret keys outside config files versioning using environment variables.

If virtualenvwrapper is being used, the default development settings parameter to work with manage.py can be specified in the postactivate hook: echo "export DJANGO_SETTINGS_MODULE=settings.local" >> $VIRTUAL_ENV/bin/postactivate

When using manage.py many commands accepts the parameter to specify a specific settings file: python manage.py runserver --settings=myproject.settings.local

To compare the current settings file with the one that installs Django by default: $ manage.py diffsettings

Setting environment variables

In a development environment, variables can be set with $ export A_SECRET_KEY=shhh1234, and placed in:

  • .bashrc or .profile
  • virtualenvwrapper’s bin/activate hook file

To check that the secret environment key is being loaded, it is possible to check it from a python shell:

import os
os.environ["A_SECRET_KEY"]
"shhh1234"

Then to get the value for a specific environment, the production config file in version control only needs to get this environment variable value: A_SECRET_KEY = os.environ["A_SECRET_KEY"]

In a production environment like Heroku, this can be done with: $ heroku config:set A_SECRET_KEY=shhh1234

Splitting settings

Splitting the default Django’s settings file into several files for different environments.

Django automatically creates a configuration file in <project_name>/settings.py, to break it up into local, testing, staging and production config files, the best way is to create a base.py config with common configurations accross all of them and create specific config files for each environment:

.
└── REPO-ROOT `git repo`
    β”œβ”€β”€ .gitignore
    β”œβ”€β”€ ...
    └── PROJECT-ROOT
        β”œβ”€β”€ settings
        |   β”œβ”€β”€ __init__.py
        |   β”œβ”€β”€ base.py
        |   └── local.py
        β”œβ”€β”€ manage.py
        └── ...
  1. Create the settings directory $ mkdir <project_name>/settings
  2. Add __init__.py file to make Python treat the settings directory as containing packages $ touch <project_name>/settings/__init__.py
  3. Move the default settings file into the settings directory and change its name $ mv <project_name>/settings.py <project_name>/settings/base.py
  4. Create all the configuration files (local.py, testing.py, staging.py, production.py ) and specify to inherit base.py configurations, for example, for the development file: echo "from .base import *" >> <project_name>/settings/local.py
  5. Use the new settings file in one of two ways:
    • set an environment variable and call scripts normally
    • call scripts specifying a settings file

Set environment variables

Configure the current environment to use the appropriate settings file, using PYTHONPATH and DJANGO_SETTINGS_MODULE:

 
  
  $ export DJANGO_SETTINGS_MODULE=mysite.settings.local
  $ export PYTHONPATH=~/path/to/my/project
  #django-admin will use the above settings by default
  $ django-admin runserver
  
  

Specify settings in a script parameter

Use the --settings parameter with manage.py or django-admin:

 

$ django-admin runserver --settings=mysite.settings.local --pythonpath=/path/to/my/project
Performing system checks...

System check identified no issues (0 silenced).

July 23, 2016 - 22:43:48
Django version 1.9.6, using settings 'mysite.settings.local'
Starting development server at http://127.0.0.1:8000/
Quit the server with CONTROL-C.

Differences between manage.py and django-admin

Django has two administrative scripts: django_admin.py and manage.py (in the root of each Django project).

manage.py is a wrapper of django-admin, the only difference between them is that manage.py sets the DJANGO_SETTINGS_MODULE environment variable to settings.py by default, if not changed.

So with the about settings scheme, it is better to use django-admin and choose the proper settings file.

Generally, when working on a single Django project, it’s easier to use manage.py than django-admin. If you need to switch between multiple Django settings files, use django-admin with DJANGO_SETTINGS_MODULE or the –settings command line option. – Django Docs

Review

We start having a settings.py single file and break it up into a new directory with specific environment settings:

graph LR A["settings.py"] B["settings/

base.py
local.py
testing.py
staging.py
production.py"] A-->B

Packages for each environment

Each environment has to have a set of packages that fits its purpose and operating system requirements. We have to configure a python virtual environment so it is possible to install packages in development, that are not needed in production and viceversa, or that can be installed in different Operating Systems.

Each environment needs a specific file, having a base.txt requirement file with common packages across environments and then adding the needed packages for each environment.

.
└── REPO-ROOT `git repo`
    β”œβ”€β”€ .gitignore
    β”œβ”€β”€ requirements
    |   β”œβ”€β”€ base.txt
    |   β”œβ”€β”€ local.txt
    |   β”œβ”€β”€ production.txt
    |   └── test.txt
    β”œβ”€β”€ ...
    └── PROJECT-ROOT
        β”œβ”€β”€ manage.py
        └── ...

Using pip it is possible to specify which file has the list of packages you want:

pip install [options] -r <requirements file> [package-index-options] ...
  -r, --requirement <file>    Install from the given requirements file. This option can be used multiple times.

So to make it possible for each environment to inherit the packages from the base.txt requirement file using pip, each new file should begin with: -r base.txt

# /requirements/base.txt
# list of packages present in all environments
# /requirements/local.txt
-r base.txt
# /requirements/test.txt
-r base.txt
# /requirements/production.txt
-r base.txt
  • To generate a requirements file: $ pip freeze $ pip freeze --local > requirements/base.txt
  • To install the requirements file
    • in a local/development environment: $ pip install -r requirements/local.txt
    • in a testing environment: $ pip install -r requirements/testing.txt

After

As with any major change to the default installation, after generating these directories, it is a good practice to describe them in /docs/architectrure.rst and what are the commands used to get them running in /docs/installation.rst for other developers or just for oneself when reviewing the project in the future.

References

Summary

graph TB env["Set environment variables"] env==>split["Split settings.py into separated files
with custom settings for each environment"] split==>pack["Configure different Packages for each environment"]
Uruguay
Marcelo Canina
I'm Marcelo Canina, a developer from Uruguay. I build websites and web-based applications from the ground up and share what I learn here.