Although Django is a great all-in-one tool for creating websites in a very short time, its deployment can be a pain since there are a lot of steps you need to take and a lot of bits and pieces you need to fiddle with. If you don't take the easy path of using a service such as Heroku, especially looking at their limitations; you need to prepare your server by looking at a lot of guides and best practices etc. Of course you may not care about the best practices or the security of your server and want to put together a testing environment with Django's development server, but for a production environment you will need to go through some of the phases such as:
If you do some research on the internet to find a good deployment guide for all these steps, you will be disappointed. Most of the Django deployment guides are outdated or include various bad practices, or just are not explanatory. In the end, you will need to look at some individual guides for each tool you decide to use. This makes the deployment process of Django to be tiresome.
Since I had the problems above, I decided to write a guide about how I carry out the deployment steps. I actually use a scribble of notes that I prepared for myself beforehand to write this post. Therefore, I don't guarantee the readability and clarity of this guide. If you get stuck at some step, please leave a comment below so that I can fix or update some of the parts of this post.
Throughout the guide, I will give the commands for each steps which include some parts you need to edit. Please change these according to your project:
This guide considers the usage of following tools and environments:
We need to separate our "settings.py" file into three files. Create a settings folder instead of the "settings.py" file, add a "__init__.py" file that containsfrom .dev import * to this folder. Add these setting files to this folder:
from .base import *
DEBUG=True
try:
from .local import *
except ImportError:
pass
You can add "dev.py", "base.py" and "__init__.py" to the version control.
You should be using Vagrant in your development environment to run the project. I assume that you already have a running Vagrant setup and know how to login to your Vagrant instance using SSH and run commands on it.
Also, before starting to take steps below, please ensure that you secured your deployment server. You must have a user with sudo access.
Our project will be accessed by the user named with our project's name. In other words, automated deployment will be carried out by a user named "<<project name>>". Thus the project files will reside on this user's home directory. We need to copy the project into the folder "/home/<<project name>>". To do that, first install Git to the server:
sudo mkdir /home/<<project name>>
cd /home/<<project name>>
sudo apt-get update
sudo apt-get install git
git config --global user.name "<<your name>>"
git config --global user.email "<<your email>>"
Generate and install your SSH key into the server and Github: https://help.github.com/articles/generating-a-new-ssh-key-and-adding-it-to-the-ssh-agent/ OR you can choose to clone the repository over the HTTPS address. Doing latter will cause the requirement of entering Github username and password.
Clone the git repository that contains the project into the target folder:
sudo git clone <<repo address>>
Install required libraries with following commands:
sudo apt-get install build-essential python3-dev python-setuptools
sudo apt-get install libjpeg-dev libtiff-dev zlib1g-dev libfreetype6-dev liblcms2-dev
sudo apt-get install postgresql libpq-dev
In this step, we will create our project's OS user and also the user for Postgresql. We then create the database of the project and grant the user access to the database.
sudo adduser <<project name>>
sudo chown -R <<project name>>:<<project name>> /home/<<project name>>
sudo su - postgres -c "createuser -s <<project name>>"
sudo su - <<project name>> -c "createdb <<project name>>"
sudo su - postgres
psql
ALTER USER <<project name>> WITH PASSWORD '<<db user password>>';
GRANT ALL PRIVILEGES ON DATABASE <<project name>> TO <<project name>>;
\q
logout
Virtualenv is very important for Python projects since it is required to isolate the project's environment and requirements from other projects running on the server. We install and configure virtualenv:
sudo easy_install -U pip
sudo pip install virtualenv virtualenvwrapper stevedore virtualenv-clone
sudo /usr/local/bin/virtualenv /home/<<project name>>/.virtualenvs/<<project name>> \
--python=/usr/bin/python3
sudo chown -R <<project name>>:<<project name>> /home/<<project name>>
sudo echo home/<<project name>> > /home/<<project name>>/.virtualenvs/<<project name>>/.project
Since we don't include our database settings and secret key in version control, we need to create settings files for these settings manually in our production environment. You should apply the following steps with our newly created "<<project name>>" user.
Inside your Django settings folder, edit "__init__.py" as follows:
from .prod import *
Create "prod.py" with following content:
from .base import *
DEBUG = False
DATABASES = {
'default': {
'ENGINE': 'django.db.backends.postgresql',
'NAME': '<<project name>>',
'USER': '<<project name>>',
'PASSWORD': '<<db user password>>',
'HOST': '', # Set to empty string for localhost.
'PORT': '', # Set to empty string for default.
'CONN_MAX_AGE': 600, # number of seconds database connections should persist for
}
}
try:
from .local import *
except ImportError:
pass
Create "local.py" with following content:
SECRET_KEY = '<<secret key>>'
We will configure Gunicorn to bind to our Django project socket file. This step has some changes according to your Ubuntu version, refer to the warning to see changes for Ubuntu 16.04.
Make sure your "requirements.txt" file is similar to this:
Django==1.9.7
psycopg2>=2.6
Pillow==2.7.0
django-ckeditor>=5.0.3
gunicorn==19.6.0
Activate the virtual environment and install the requirements:
sudo su - <<project name>>
source /home/<<project name>>/.virtualenvs/<<project name>>/bin/activate
pip install -r /home/<<project name>>/requirements.txt
deactivate
exit
Create gunicorn service entry:
sudo nano /etc/init/gunicorn.conf
Write following contents and save:
description "Gunicorn application server handling <<project name>>"
start on runlevel [2345]
stop on runlevel [!2345]
respawn
setuid <<project name>>
setgid www-data
chdir /home/<<project name>>
exec .virtualenvs/<<project name>>/bin/gunicorn --workers 5 --bind unix:/home/<<project name>>/<<project name>>.sock <<project name>>.wsgi:application
Activate gunicorn service:
sudo service gunicorn start
Warning: In Ubuntu 16.04, init service is changed to "systemd" so you need to apply different steps to configure gunicorn. If so, apply following steps instead:
sudo nano /etc/systemd/system/gunicorn.service
[Unit]
Description=gunicorn daemon
After=network.target
[Service]
User=<<project name>>
Group=www-data
WorkingDirectory=/home/<<project name>>/<<project name>>
ExecStart=/home/<<project name>>/.virtualenvs/<<project name>>/bin/gunicorn --workers 3 --bind unix:/home/<<project name>>/<<project name>>/<<project name>>.sock <<project name>>.wsgi:application
[Install]
WantedBy=multi-user.target
sudo systemctl start gunicorn
sudo systemctl enable gunicorn
Edit /etc/nginx/sites-available/<<project name>> file to include following server settings:
server {
listen 80;
server_name <<site address>>;
location = /favicon.ico { access_log off; log_not_found off; }
location /static/ {
root /home/<<project name>>/<<project name>>;
}
location / {
include proxy_params;
proxy_pass http://unix:/home/<<project name>>/<<project name>>.sock;
}
}
Test nginx and start if there is no error:
sudo nginx -t
sudo service nginx restart
Warning: Make sure that the group "www-data" and user "www-data" has read/write permissions for the socket file.
Using project's user, activate the virtual environment and run following tasks to apply initial Django management tasks. You may need to run python3.5 instead of python3 command if you use Python 3.5:
python3 manage.py migrate
python3 manage.py collectstatic
python3 manage.py compilemessages
python3 manage.py createsuperuser
Restart gunicorn and check if the site is working.
Add the project user to the sudo group:
sudo adduser <<project name>> sudo
Make sure <<project user>> is owner of ".git/FETCH_HEAD".
To carry out the deployment with Fabric, we need to configure SSH login for our project's user. Create a SSH key for our "<<project name>>" user. Add the public key to the user's ".ssh" folder on the server. Move the private key to "vagrant_data" folder in your development environment and rename the keyfile to "ssh_key". The "vagrant_data" should be at the project root with the same level as the "fabfile.py" file. Do not add the key file to the repository.
Create "fabfile.py" file in your project directory and add it to the repository. The contents of the "fabfile.py" should be as following:
from fabric.api import run, env, cd, sudo, prefix, local
from contextlib import contextmanager as _contextmanager
env.activate = 'source /home/<<project name>>/.virtualenvs/<<project name>>/bin/activate'
env.directory = '/home/<<project name>>'
env.key_filename = 'vagrant_data/ssh_key'
@_contextmanager
def virtualenv():
with prefix(env.activate):
yield
def deploy():
with cd(env.directory):
run('git pull origin master')
with virtualenv():
sudo('pip install -r ' + env.directory + '/requirements.txt')
run('python3 manage.py migrate')
run('python3 manage.py collectstatic')
# run('python3 manage.py compilemessages')
sudo('service gunicorn restart')
According to this configuration, when we run the deploy command:
Warning: If you are using Ubunto 16.04 or later, you need to change the last line to sudo('systemctl restart gunicorn')
You can add new commands to the Fabric file according to your needs, please refer to the Fabric documentation to understand how Fabric works.
Add following to the requirements.txt and install requirements to the Vagrant box (login to your Vagrant box through SSH and run pip install -r requirements.txt from the project root folder):
Fabric3==1.11.1.post1
Also instead of adding fabric to your project "requirements.txt" file, you may choose to install it manually using pip install command since it is not required to be installed to the server.
You will need to apply the following steps when you update the project and want to deploy to the server.
SSH into the Vagrant box and run fab deploy. You will need to supply the host connection string to Fabric when you run the command. Write it as "<<project name>>@<<server ip>>:22":
source .virtualenvs/<<project name>>/bin/activate
fab deploy
No hosts found. Please specify (single) host string for connection: <<project name>>@<<server ip>>:22
[<<project name>>@xx.xxx.xxx.xx:22] run: git pull origin master
[<<project name>>@xx.xxx.xxx.xx:22] Login password for '<<project name>>':
As seen above, it will ask for the passphrase for the private key in your "vagrant_data" folder. Enter the passphrase. If asked for the Github credentials, enter them. An example run for "quenchless" application (one of the projects of mine) can be seen below:
(quenchless) vagrant@vagrant-ubuntu-trusty-32:~/quenchless$ fab deploy
No hosts found. Please specify (single) host string for connection: quenchless@46.101.214.89:22
[quenchless@46.101.214.89:22] run: git pull origin master
[quenchless@46.101.214.89:22] Login password for 'quenchless':
[quenchless@46.101.214.89:22] out: Username for 'https://github.com': berkersonmez
[quenchless@46.101.214.89:22] out: Password for 'https://berkersonmez@github.com':
[quenchless@46.101.214.89:22] out: From https://github.com/berkersonmez/quenchless
[quenchless@46.101.214.89:22] out: * branch master -> FETCH_HEAD
[quenchless@46.101.214.89:22] out: Already up-to-date.
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] run: pip install -r /home/quenchless/requirements.txt
[quenchless@46.101.214.89:22] out: Requirement already satisfied (use --upgrade to upgrade): Django==1.9.7 in ./.virtualenvs/quenchless/lib/python3.4/site-packages (from -r /home/quenchless/requirements.txt (line 1))
[quenchless@46.101.214.89:22] out: Requirement already satisfied (use --upgrade to upgrade): psycopg2>=2.6 in ./.virtualenvs/quenchless/lib/python3.4/site-packages (from -r /home/quenchless/requirements.txt (line 2))
[quenchless@46.101.214.89:22] out: Requirement already satisfied (use --upgrade to upgrade): Pillow==2.7.0 in ./.virtualenvs/quenchless/lib/python3.4/site-packages (from -r /home/quenchless/requirements.txt (line 3))
[quenchless@46.101.214.89:22] out: Requirement already satisfied (use --upgrade to upgrade): django-ckeditor>=5.0.3 in ./.virtualenvs/quenchless/lib/python3.4/site-packages (from -r /home/quenchless/requirements.txt (line 4))
[quenchless@46.101.214.89:22] out: Requirement already satisfied (use --upgrade to upgrade): gunicorn==19.6.0 in ./.virtualenvs/quenchless/lib/python3.4/site-packages (from -r /home/quenchless/requirements.txt (line 5))
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] run: python3 manage.py migrate
[quenchless@46.101.214.89:22] out: Operations to perform:
[quenchless@46.101.214.89:22] out: Apply all migrations: berkersonmez, sessions, auth, admin, contenttypes
[quenchless@46.101.214.89:22] out: Running migrations:
[quenchless@46.101.214.89:22] out: No migrations to apply.
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] run: python3 manage.py collectstatic
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] out: You have requested to collect static files at the destination
[quenchless@46.101.214.89:22] out: location as specified in your settings:
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] out: /home/quenchless/quenchless/static
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] out: This will overwrite existing files!
[quenchless@46.101.214.89:22] out: Are you sure you want to do this?
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] out: Type 'yes' to continue, or 'no' to cancel: yes
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] out: 0 static files copied to '/home/quenchless/quenchless/static', 1427 unmodified.
[quenchless@46.101.214.89:22] out:
[quenchless@46.101.214.89:22] sudo: service gunicorn restart
[quenchless@46.101.214.89:22] out: sudo password:
[quenchless@46.101.214.89:22] out: Sorry, try again.
[quenchless@46.101.214.89:22] out: sudo password:
[quenchless@46.101.214.89:22] out: gunicorn stop/waiting
[quenchless@46.101.214.89:22] out: gunicorn start/running, process 20367
[quenchless@46.101.214.89:22] out:
Done.
Disconnecting from 46.101.214.89... done.
Congratulations, the application has been deployed!
400 Bad Request when opening the pageMake sure that Django settings file has the ALLOWED_HOSTS setting with the domain you are trying to access the project from:
ALLOWED_HOSTS = ("yourdomain.com",)