Lessons Learned From Deploying Django on Heroku

This article describes my experience with the Heroku/Django tutorial at https://devcenter.heroku.com/articles/django. It largely follows the original tutorial, but points out the areas where the current tutorial seemed to be unclear on what was required.

Heroku logo.

This blog is neither endorsed nor sponsored by Heroku.


SYSTEM DETAILS
Ubuntu 12.04.1 LTS (VirtualBox)

Prerequisites:

I recommend that anyone following the tutorial do so in Linux if possible. Versions of many of these packages exist for Windows, but the setup process will be more complicated.

*NOTE: This is not mentioned in the original tutorial, but an error occurs without it. See the rest of this post.

Heroku is a cloud application platform: the Heroku servers manage web applications which are isolated into Unix-like environments called dynos, which execute a single command on a packaged version of the application called a slug. The slug is compiled from a Git repository of the files that make up the web applications.

Step 1: Configure PostGreSQL

The tutorial says that you need to have PostgreSQL installed “to test locally.” They do not explain how to set it up properly. To set up a database in PostgreSQL, you need to first make sure that it accepts local connections. Open the file pg_hba.conf in the text editor (vi, nano, Emacs, etc.) of your choice.

mainuser@webdevbox:~/$ sudo nano /etc/postgresql/9.1/main/pg_hba.conf

Look for a line similar to the below image:

pg_hba.conf

Above: The pg_hba.conf file open in nano.


The last parameter in the line will say something like “peer” or “md5.” Change it to “trust”: we want it to allow all local connections. Now, restart postgresql:

mainuser@webdevbox:~/$ sudo service postgresql restart
 * Restarting PostgreSQL 9.1 database server                             [ OK ] 

You should now be able to switch to the postgres user and access a postgresql prompt (postgres=#):

mainuser@webdevbox:~/$ sudo su postgres
postgres@webdevbox:~/$ psql
postgres=#

If you like, you can set a service password here. By default, the postgres account has no password on UNIX / Linux. For a brief explanation, see the section on “Service Password” in this post by Dave Page.

postgres=# \password postgres
Enter new password:

Note that this is not the superuser password; without further configuration, you will not be able to use su / sudo / other switching commands as the postgres user. Changing/setting the superuser password for postgres is beyond the scope of this post.

Now create a user and create a table with that user as an administrator, then exit postgres and the postgres user:

postgres=# CREATE USER testadmin WITH CREATEDB PASSWORD 'testing';
CREATE ROLE
postgres=# CREATE DATABASE django_testdb OWNER testadmin;
CREATE DATABASE
postgres=# \q
postgres@webdevbox:~/$ exit
mainuser@webdevbox:~/$

Remember the new user, password, and database name; you will need them later.

Step 2: Set Up a Local Django App

As the tutorial says, run these commands to install Django and some other dependencies:

mainuser@webdevbox:~/$ mkdir hellodjango && cd hellodjango
mainuser@webdevbox:~/hellodjango$ virtualenv venv --distribute
New python executable in venv/bin/python
Installing distribute...............done.
Installing pip...............done.
mainuser@webdevboxL~/hellodjango$ source venv/bin/activate
(venv)mainuser@webdevbox:~/hellodjango$ pip install Django psycopg2 dj-database-url

I encountered a bug while psycopg2 was being compiled. (If you installed python-dev, you probably will not see this.)

// Some output omitted
gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -DPSYCOPG_DEFAULT_PYDATETIME=1 -DPSYCOPG_VERSION="2.4.6 (dt dec pq3 ext)" -DPG_VERSION_HEX=0x090108 -DPSYCOPG_EXTENSIONS=1 -DPSYCOPG_NEW_BOOLEAN=1 -DHAVE_PQFREEMEM=1 -I/usr/include/python2.7 -I. -I/usr/include/postgresql -I/usr/include/postgresql/9.1/server -c psycopg/psycopgmodule.c -o build/temp.linux-i686-2.7/psycopg/psycopgmodule.o -Wdeclaration-after-statement

In file included from psycopg/psycopgmodule.c:27:0:

./psycopg/psycopg.h:30:20: fatal error: Python.h: No such file or directory

compilation terminated.

error: command 'gcc' failed with exit status 1

According to a psycopg2 FAQ, this error is caused by not having the python-dev package installed, so I installed it:

(venv)mainuser@webdevbox:~/hellodjango$ sudo apt-get install python-dev

The command should now work correctly. Continuing to follow the tutorial, you should run these commands:

(venv)mainuser@webdevbox:~/hellodjango$ pip install Django psycopg2 dj-database-url
// Output omitted
(venv)mainuser@webdevbox:~/hellodjango$ django-admin.py startproject hellodjango .
(venv)mainuser@webdevbox:~/hellodjango$ python manage.py runserver
0 errors found
Django version 1.4, using settings 'hellodjango.settings'
Development server is running at http://127.0.0.1:8000/
Quit the server with CONTROL-C.
^C
(venv)mainuser@webdevbox:~/hellodjango$ pip freeze > requirements.txt
(venv)mainuser@webdevbox:~/hellodjango$ cat requirements.txt
Django==1.4
psycopg2==2.4.5
dj-database-url==0.2.0

(CORRECTION): Contrary to both previous versions of this post, you can run pip freeze at any time. This will change requirements.txt by adding the new dependencies that have been installed.

If you edit requirements.txt manually, run pip install -r requirements.txt to install any new dependencies:

(venv)mainuser@webdevbox:~/hellodjango$ pip install -r requirements.txt

The contents of your requirements.txt file may look different from the example in the tutorial. As long as the lines in the tutorial are part of the requirements.txt file, everything should be fine.

Next, the tutorial states “Next, configure the application for the Heroku environment, including Heroku’s Postgres database. The dj-database-url module will parse the values of the DATABASE_URL environment variable and convert them to something Django can understand.” Add these lines to the end of helloheroku/settings.py as specified in the tutorial:

# Parse database configuration from $DATABASE_URL
import dj_database_url
DATABASES['default'] =  dj_database_url.config()

# Honor the 'X-Forwarded-Proto' header for request.is_secure()
SECURE_PROXY_SSL_HEADER = ('HTTP_X_FORWARDED_PROTO', 'https')

(CORRECTION) An earlier version of this post stated that values in settings.py’s DATABASES section needed to be changed. This was unnecessary. The DATABASE_URL environment variable must be set with Linux’s export command. Thanks to Zach Tomaszewski for pointing this out.

(venv)mainuser@webdevbox:~/hellodjango$ export DATABASE_URL=postgres://testadmin:testing@localhost/django_db

Here, testadmin is a postgresql user, testing is the password for that user, and django_db is the name of the local database. On the local machine, dj_database_url.config() reads the value of DATABASE_URL specified by export and returns its value, which becomes the value of DATABASES. On Heroku, dj_database_url.config() will read the value of DATABASES from Heroku’s environment variables instead.

Next, the Procfile needs to be created. As far as I know, it goes in the top-level directory of the project (hellodjango, where manage.py is).

(venv)mainuser@webdevbox:~/hellodjango$ nano Procfile
web: python manage.py runserver 0.0.0.0:$PORT --noreload

As suggested by the tutorial, I used Github’s Python .gitignore file as the project .gitignore file. Though the line *.py[cod] probably covers *.pyc, I added lines for venv and *.pyc anyway, to be safe. Then I initialized the Git repository:

(venv)mainuser@webdevbox:~/hellodjango$ git init
Initialized empty Git repository in /home/mainuser/hellodjango/.git/
(venv)mainuser@webdevbox:~/hellodjango$ git add .
(venv)mainuser@webdevbox:~/hellodjango$ git commit -m "My Django app."
[master (root-commit) 6c2765c] My Django app.
 9 files changed, 262 insertions(+)
 create mode 100644 .gitignore
 create mode 100644 Procfile
 create mode 100644 hellodjango/settings.py
 create mode 100644 hellodjango/urls.py
 create mode 100644 hellodjango/wsgi.py
 create mode 100644 manage.py
 create mode 100644 requirements.txt
 create mode 120000 venv/include/python2.7
 create mode 120000 venv/local/include

Step 3: Deploying The App to Heroku

When you first installed the Heroku toolbelt, you should have been asked to generate SSH keys:

(venv)mainuser@webdevbox:~/$ heroku login
Enter your Heroku credentials.
Email: admin@example.com
Password: same-as-heroku-account
Could not find an existing public key.
Would you like to generate one? [Yn] Y
Generating new SSH public key.
Uploading ssh public key /Users/mainuser/.ssh/id_rsa.pub.

Otherwise, follow the developers’ example to generate new keys with ssh-keygen -t rsa. Once you have a public SSH key (check in /Users/<username>/.ssh), you should be able to create a Heroku app.

(venv)mainuser@webdevbox:~/$ heroku create
Creating still-badlands-2085... done, stack is cedar
http://still-badlands-2085.herokuapp.com/ | git@heroku.com:still-badlands-2085.git
Git remote heroku added

Now push your code to the Heroku repository:

(venv)mainuser@webdevbox:~/$ git push heroku master
Counting objects: 16, done.
Compressing objects: 100% (11/11), done.
Writing objects: 100% (16/16), 4.34 KiB, done.
Total 16 (delta 0), reused 0 (delta 0)
-----> Python app detected
-----> No runtime.txt provided; assuming python-2.7.3.
-----> Preparing Python runtime (python-2.7.3)
-----> Installing Distribute (0.6.34)
-----> Installing Pip (1.2.1)
-----> Installing dependencies using Pip (1.2.1)
       Downloading/unpacking Django==1.4.4 (from -r requirements.txt (line 1))
         Running setup.py egg_info for package Django
           
       Downloading/unpacking argparse==1.2.1 (from -r requirements.txt (line 2))
         Running setup.py egg_info for package argparse
           
           no previously-included directories found matching 'doc/_build'
           no previously-included directories found matching 'env24'
           no previously-included directories found matching 'env25'
           no previously-included directories found matching 'env26'
           no previously-included directories found matching 'env27'
       Downloading/unpacking distribute==0.6.24 (from -r requirements.txt (line 3))
         Running setup.py egg_info for package distribute
           
           warning: no files found matching 'Makefile' under directory 'docs'
           warning: no files found matching 'indexsidebar.html' under directory 'docs'
       Downloading/unpacking dj-database-url==0.2.1 (from -r requirements.txt (line 4))
         Downloading dj-database-url-0.2.1.tar.gz
         Running setup.py egg_info for package dj-database-url
           
       Downloading/unpacking psycopg2==2.4.6 (from -r requirements.txt (line 5))
         Running setup.py egg_info for package psycopg2
           
           no previously-included directories found matching 'doc/src/_build'
       Installing collected packages: Django, argparse, distribute, dj-database-url, psycopg2
         Running setup.py install for Django
           changing mode of build/scripts-2.7/django-admin.py from 600 to 755
           
           changing mode of /app/.heroku/python/bin/django-admin.py to 755
         Running setup.py install for argparse
           
           no previously-included directories found matching 'doc/_build'
           no previously-included directories found matching 'env24'
           no previously-included directories found matching 'env25'
           no previously-included directories found matching 'env26'
           no previously-included directories found matching 'env27'
         Found existing installation: distribute 0.6.34
           Uninstalling distribute:
             Successfully uninstalled distribute
         Running setup.py install for distribute
           Before install bootstrap.
           Scanning installed packages
           Setuptools installation detected at /app/.heroku/python/lib/python2.7/site-packages
           Non-egg installation
           Removing elements out of the way...
           Already patched.
           /app/.heroku/python/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info already patched.
           
           warning: no files found matching 'Makefile' under directory 'docs'
           warning: no files found matching 'indexsidebar.html' under directory 'docs'
           Installing easy_install script to /app/.heroku/python/bin
           Installing easy_install-2.7 script to /app/.heroku/python/bin
           After install bootstrap.
           /app/.heroku/python/lib/python2.7/site-packages/setuptools-0.6c11-py2.7.egg-info already exists
         Running setup.py install for dj-database-url
           
         Running setup.py install for psycopg2
           building 'psycopg2._psycopg' extension

// gcc output omitted           

           no previously-included directories found matching 'doc/src/_build'
       Successfully installed Django argparse distribute dj-database-url psycopg2
       Cleaning up...
-----> Collecting static files
       0 static files copied.

-----> Discovering process types
       Procfile declares types -> web
-----> Compiled slug size: 29.2MB
-----> Launching... done, v6
       http://still-badlands-2085.herokuapp.com deployed to Heroku

To git@heroku.com:still-badlands-2085.git
 * [new branch]      master -> master
(venv)mainuser@webdevbox:~/hellodjango$ 

If you sign in to your Heroku account in a browser and navigate to My Apps, then to your-app-name-####, your app should be visible:

Heroku dashboard.

Above: The Heroku dashboard for app still-badlands-2085. If the “web” box is checked and the dyno count is 1, your app may be running already.


Now, you should be able to start the app with heroku ps:scale web=#, where # is the number of web dynos. Free accounts can have more than one app, but if more than one dyno is running for an extended period, you will exceed your 750 free hours (750 hours = 31.25 days) per month. Use heroku ps to view the app state, heroku open to view the app in your default browser, and heroku logs to view recent log information.

You can also start the app from your browser by checking the “web” checkbox, then clicking “Apply Changes.”

(venv)mainuser@webdevbox:~/hellodjango$ heroku ps:scale web=1
Scaling web processes... done, now running 1
(venv)mainuser@webdevbox:~/hellodjango$ heroku ps
=== web: `python manage.py runserver 0.0.0.0:$PORT --noreload`
web.1: up 2013/02/19 18:07:46 (~ 6m ago)

(venv)mainuser@webdevbox:~/hellodjango$ heroku open
Opening still-badlands-2085... done

The Django example app.

Above: The still-badlands-2085 app in Google Chrome.


Now you should be able to sync your Heroku database.

(venv)mainuser@webdevbox:~/hellodjango$ heroku run python manage.py syncdb
Running `python manage.py syncdb` attached to terminal... up, run.6600
Creating tables ...
Creating table auth_permission
Creating table auth_group_permissions
Creating table auth_group
Creating table auth_user_groups
Creating table auth_user_user_permissions
Creating table auth_user
Creating table django_content_type
Creating table django_session
Creating table django_site

You just installed Django's auth system, which means you don't have any superusers defined.
Would you like to create one now? (yes/no): yes
Username (leave blank to use 'u55828'): someadmin
E-mail address: admin@example.com
Password: somethingbetter
Password (again): somethingbetter
Superuser created successfully.
Installing custom SQL ...
Installing indexes ...
Installed 0 object(s) from 0 fixture(s)
(venv)mainuser@webdevbox:~/hellodjango$

If, like me, you had at first forgotten to configure PostgreSQL and do an export, you will see errors like this if you try to run the server locally:

(venv)mainuser@webdevbox:~/hellodjango$ python manage.py syncdb
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 443, in execute_from_command_line
    utility.execute()
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 382, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 196, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 232, in execute
    output = self.handle(*args, **options)
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/base.py", line 371, in handle
    return self.handle_noargs(**options)
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/core/management/commands/syncdb.py", line 57, in handle_noargs
    cursor = connection.cursor()
  File "/home/mainuser/hellodjango/venv/local/lib/python2.7/site-packages/django/db/backends/dummy/base.py", line 15, in complain
    raise ImproperlyConfigured("settings.DATABASES is improperly configured. "
django.core.exceptions.ImproperlyConfigured: settings.DATABASES is improperly configured. Please supply the ENGINE value. Check settings documentation for more details.

If this happens, set up the database as described in Step 1 and edit settings.py as described in Step 2, then try again.

Step 4: Change The Web Server To Gunicorn

If you’ve been following the Heroku tutorial so far, there’s one last step: switching the webserver to gunicorn.

Gunicorn logo.

Gunicorn is a Python port of Ruby’s WSGI HTTP server, Unicorn.

  1. In settings.py, change DEBUG to False. You should always do this before deploying a Django application with gunicorn, because gunicorn does not respect the DEBUG flag. In general, always change DEBUG to False before deploying any production Django application.
    DEBUG = False
    
  2. Add the line gunicorn==0.16.1 to your requirements.txt
  3. Change your Procfile’s web: line to web: gunicorn hellodjango.wsgi
  4. Run pip to update your dependencies:
    (venv)mainuser@webdevbox:~/hellodjango$ pip install -r requirements.txt
    
  5. Run foreman start to run the application locally. Use Ctrl-C to quit the server when you’re done.
    (venv)mainuser@webdevbox:~/hellodjango$ foreman start
    18:36:03 web.1  | started with pid 3420
    18:36:04 web.1  | 2013-02-19 18:36:04 [3423] [INFO] Starting gunicorn 0.16.1
    18:36:04 web.1  | 2013-02-19 18:36:04 [3423] [INFO] Listening at: http://0.0.0.0:5000 (3423)
    18:36:04 web.1  | 2013-02-19 18:36:04 [3423] [INFO] Using worker: sync
    18:36:04 web.1  | 2013-02-19 18:36:04 [3426] [INFO] Booting worker with pid: 3426
    ^CSIGINT received
    18:41:06 web.1  | 2013-02-19 18:41:06 [3426] [INFO] Worker exiting (pid: 3426)
    18:41:06 system | sending SIGTERM to all processes
    18:41:06 web.1  | 2013-02-19 18:41:06 [3423] [INFO] Handling signal: int
    SIGTERM received
    18:41:06 web.1  | terminated by SIGTERM
    
  6. Push your changes to Heroku:
    (venv)mainuser@webdevbox:~/hellodjango$ git add Procfile requirements.txt hellodjango/settings.py
    (venv)mainuser@webdevbox:~/hellodjango$ git commit -m "Changed server to gunicorn."
    (venv)mainuser@webdevbox:~/hellodjango$ git push heroku master
    Counting objects: 7, done.
    Compressing objects: 100% (3/3), done.
    Writing objects: 100% (4/4), 364 bytes, done.
    Total 4 (delta 2), reused 0 (delta 0)
    -----> Python app detected
    -----> No runtime.txt provided; assuming python-2.7.3.
    -----> Using Python runtime (python-2.7.3)
    -----> Installing dependencies using Pip (1.2.1)
           Downloading/unpacking gunicorn==0.16.1 (from -r requirements.txt (line 7))
             Running setup.py egg_info for package gunicorn
               
           Installing collected packages: gunicorn
             Running setup.py install for gunicorn
               
               Installing gunicorn_paster script to /app/.heroku/python/bin
               Installing gunicorn script to /app/.heroku/python/bin
               Installing gunicorn_django script to /app/.heroku/python/bin
           Successfully installed gunicorn
           Cleaning up...
    -----> Collecting static files
           0 static files copied.
    
    -----> Discovering process types
           Procfile declares types -> web
    -----> Compiled slug size: 29.5MB
    -----> Launching... done, v7
           http://still-badlands-2085.herokuapp.com deployed to Heroku
    
    To git@heroku.com:still-badlands-2085.git
       6c2765c..697eb0f  master -> master
    mainuser@webdevbox:~/hellodjango$ 
    
  7. Run heroku logs. You should see some reference to gunicorn in the recent log entries.

Step 5: Stopping Your App

If you’re doing this for a course project like I did, or doing some testing later, you’ll want to leave the app running. When you’re done, shut off the process by scaling back the number of dynos to 0:

(venv)mainuser@webdevbox:~/hellodjango$ heroku ps:scale web=0
Scaling web processes... done, now running 0
(venv)mainuser@webdevbox:~/hellodjango$ 

Though I may have missed something, I could not find any advice on the official site on how to stop an app completely outside of the web interface (where you can uncheck the web) checkbox to stop the dyno). This tip comes from Stack Overflow.

Overall Thoughts

In my opinion, the biggest weakness of the tutorial is that it does not explain how to set up the Postgresql database or link to a tutorial which does. Maybe the developers assumed that anyone who is experienced enough in web development to be using Heroku would have used Postgresql before. Maybe they just expected people to Read The Manual. Otherwise, I found Heroku to be fairly easy to deploy and use.

This post was last revised on February 27, 2013. See Corrections to Heroku Posts for more details.

Advertisements