How to set up Django tests to use a free PostgreSQL database in Heroku

Sqlite databases are great for testing purposes, they are fast and don’t need much setup, but when your Django app grows, you will need to test in an environment similar to production to spot as much errors as possible.

This guide assumes you have a Django app hosted at Heroku working with a PostgreSQL database add-on in production.

We are going to use PostgreSQL Hobby dev plan to get a free database just for testing against it.

This can be thought as a previous step to use a Continuous Integration service.

For example, for an app I am currently developing, tests summary passed from this (using sqlite):

Ran 251 tests in 294.559s

FAILED (failures=2, skipped=18)
Destroying test database for alias 'default'...

to this (using postgresql):

----------------------------------------------------------------------
Ran 249 tests in 1882.815s

FAILED (failures=1, errors=70, skipped=18)
Preserving test database for alias 'default'...

1. Create testing DB in Heroku

To create the database: heroku addons:create heroku-postgresql:hobby-dev.

2. Get credentials

Get the credentials using the above database color e.g.: heroku pg:credentials:url HEROKU_POSTGRESQL_BRONZE

$ heroku pg:credentials:url HEROKU_POSTGRESQL_BRONZE
Connection info string:
   "dbname=d1jjjjjjj23b host=ec2-54-225-222-121.compute-1.amazonaws.com port=5432 user=myuser password=hash sslmode=require"
Connection URL:
   postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b

Pay attention to the above database user and connection url as we will use it in the following steps.

3. Configure Django db settings

In the DATABASE section of yourapp/settings.py, set the DATABASES['TEST']['NAME'] key with the database name of the new db, e.g. d1jjjjjjj23b:

DATABASES = {
    'default': {
        'ENGINE': 'django.db.backends.sqlite3',
        'NAME': os.path.join(BASE_DIR, 'db.sqlite3'),
        'TEST': {
            'NAME': 'd1jjjjjjj23b',
        },
    }
}

# Heroku: Update database configuration from $DATABASE_URL.
import dj_database_url
db_from_env = dj_database_url.config(conn_max_age=500)
DATABASES['default'].update(db_from_env)

Note the usage of the package: https://github.com/kennethreitz/dj-database-url to get the database connection settings from an environment variable DATABASE_URL

4. Run tests with DATABASE_URL env variable

Set the env variable when running the test, and specify to not create the database (Heroku already created it):

DATABASE_URL="postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b" python manage.py test --keepdb

Now the test command will use the above database when running tests.

Possible Errors

If you don’t set the NAME key for the testing database, then the test command will complain from being unable to create the database with the message Got an error creating the test database: permission denied to create database, that is why we also add the --keepdb parameter to avoid the testing command to try to create the database again:

$ DATABASE_URL="postgres://myuser:hash@ec2-54-225-222-121.compute-1.amazonaws.com:5432/d1jjjjjjj23b" python manage.py test
Creating test database for alias 'default'...
/home/marcanuy/.virtualenvs/myapp/lib/python3.6/site-packages/django/db/backends/postgresql/base.py:269: RuntimeWarning: Normally Django will use a connection to the 'postgres' database to avoid running initialization queries against the production database when it's not needed (for example, when running tests). Django was unable to create a connection to the 'postgres' database and will use the first PostgreSQL database instead.
  RuntimeWarning
Got an error creating the test database: permission denied to create database

References

• https://docs.djangoproject.com/en/2.1/ref/databases/
• https://docs.djangoproject.com/en/2.1/topics/testing/
• https://docs.djangoproject.com/en/2.1/ref/settings/#std:setting-DATABASE-TEST
• https://docs.djangoproject.com/en/2.1/ref/settings/#std:setting-NAME
• https://github.com/kennethreitz/dj-database-url