Common Gateway Interface

This interface is the oldest one, supported by nearly every web server out of the box. Programs using CGI to communicate with their web server need to be started by the server for every request. So, every request starts a new Python interpreter – which takes some time to start up – thus making the whole interface only usable for low load situations.

The upside of CGI is that it is simple – writing a program which uses CGI is a matter of about three lines of code. But this simplicity comes at a price: it does very few things to help the developer.

Writing CGI programs, while still possible, is not recommended anymore. With WSGI (more on that later) it is possible to write programs that emulate CGI, so they can be run as CGI if no better option is available.

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

# enable debugging
import cgitb; cgitb.enable()

print "Content-Type: text/plain;charset=utf-8"
print

print "Hello World!"

mod_python

People coming from PHP often find it hard to grasp how to use Python in the web. Their first thought is mostly mod_python because they think that this is the equivalent to mod_php. Actually it is not really. It does embed the interpreter into the Apache process, thus speeding up requests by not having to start a Python interpreter every request. On the other hand, it is by far not “Python intermixed with HTML” as PHP often does. The Python equivalent of that is a template engine. mod_python itself is much more powerful and gives more access to Apache internals. It can emulate CGI, it can work an a “Python Server Pages” mode similar to JSP which is “HTML intermangled with Python” and it has a “Publisher” which destignates one file to accept all requests and decide on what to do then.

But mod_python has some problems. Unlike the PHP interpreter the Python interpreter uses caching when executing files, so when changing a file the whole web server needs to be re-started to update. Another problem ist the basic concept – Apache starts some child processes to handle the requests and unfortunately every child process needs to load the whole Python interpreter even if it does not use it. This makes the whole web server slower. Another problem is that as mod_python is linked against a specific version of libpython, it is not possible to switch from an older version to a newer (e.g. 2.4 to 2.5) without recompiling mod_python. mod_python is also bound to the Apache web server, so programs written for mod_python cannot easily run on other web servers.

These are the reasons why mod_python should be avoided when writing new programs. In some circumstances it might be still a good idea to use mod_python for deployment, but WSGI makes it possible to run WSGI programs under mod_python as well.

FastCGI and SCGI

FastCGI and SCGI try to solve the performance problem of CGI in another way. Instead of embedding the interpreter into the web server, they create long-running processes which run in the background. There still is some module in the web server which makes it possible for the web server to “speak” with the background process. As the background process is independent from the server, it can be written in any language of course also in Python. The language just needs to have a library which handles the communication with the web server.

The difference between FastCGI and SCGI is very small, as SCGI is essentially just a “simpler FastCGI”. But as the web server support for SCGI is limited most people use FastCGI instead, which works the same way. Almost everything that applies to SCGI also applies to FastCGI as well, so we’ll only write about the latter.

These days, FastCGI is never used directly. Just like mod_python it is only used for the deployment of WSGI applications.

#!/usr/bin/env python
# -*- coding: UTF-8 -*-

from cgi import escape
import sys, os
from flup.server.fcgi import WSGIServer

def app(environ, start_response):
    start_response('200 OK', [('Content-Type', 'text/html')])

    yield '<h1>FastCGI Environment</h1>'
    yield '<table>'
    for k, v in sorted(environ.items()):
         yield '<tr><th>%s</th><td>%s</td></tr>' % (escape(k), escape(v))
    yield '</table>'

WSGIServer(app).run()

mod_wsgi

mod_wsgi is an attempt to get rid of the low level gateways. As FastCGI, SCGI, mod_python are mostly used to deploy WSGI applications anyway, mod_wsgi was started to directly embed WSGI aplications into the Apache web server. The benefit from this approach is that WSGI applications can be deployed much easier as is is specially designed to host WSGI applications – unlike the other low level methods which have glue code to host WSGI applications (like flup which was mentioned before). The downside is that mod_wsgi is limited to the Apache web server, other servers would need their own implementations of mod_wsgi.

It supports two modes: the embedded mode in which it integrates with the Apache process and the daemon mode which is more FastCGI-like. Contrary to FastCGI, mod_wsgi handles the worker-processes by itself which makes administration easier.

WSGI Servers

The code that is used to connect to various low level gateways like CGI or mod_python is called WSGI server. One of these servers is flup which was already mentioned and supports FastCGI, SCGI as well as AJP. Some of these servers are written in Python as flup is, but there also exist others which are written in C and can be used as drop-in replacements.

There are quite a lot of servers already available, so a Python web application can be deployed nearly everywhere. This is one big advantage that Python has compared with other web techniques.

Case study: MoinMoin

What does WSGI give the web application developer? Let’s take a look on one long existing web application written in Python without using WSGI.

One of the most widely used wiki software is MoinMoin. It was created in 2000, so it predates WSGI by about three years. While it now includes support for WSGI, older versions needed separate code to run on CGI, mod_python, FastCGI and standalone. Now, this all is possible by using WSGI and the already-written gateways. For running with on FastCGI flup can be used, for running a standalone server wsgiref is the way to go.

Templates

Mixing of HTML and Python code is possible with some libraries. While convenient at first, it leads to horribly unmaintainable code. That’s why templates exist. Templates are, in the simplest case, just HTML files with placeholders. The HTML is sent to the user’s browser after filling out the placeholders.

Python already includes such simple templates:

# a simple template
template = "<html><body><h1>Hello %s!</h1></body></html>"
print template % "Reader"

The Python standard library also includes some more advanced templates usable through string.Template, but in HTML templates it is needed to use conditional and looping contructs like Python’s for and if. So, some template engine is needed.

Now, Python has a lot of template engines which can be used with or without a framework. Some of these are using a plain-text programming language which is very easy to learn as it is quite limited while others use XML so the template output is always guaranteed to be valid XML. Some frameworks ship their own template engine or recommend one particular. If one is not yet sure, using these is a good idea.

Data persistence

Data persistence, while sounding very complicated is just about storing data. This data might be the text of blog entries, the postings of a bulletin board or the text of a wiki page. As always, there are different ways to store informations on a web server.

Often relational database engines like MySQL or PostgreSQL are used due to their good performance handling very large databases consisting of up to millions of entries. These are queried using a language called SQL. Python programmers in general do not like SQL too much, they prefer to work with objects. It is possible to save Python objects into a database using a technology called ORM. ORM translates all object-oriented access into SQL code under the hood, the user does not need to think about it. Most frameworks use ORMs and it works quite well.

A second possibility is using files that are saved on the hard disk (sometimes called flatfiles). This is very easy, but is not too fast. There is even a small database engine called SQLite which is bundled with Python in the sqlite module and uses only one file. This database can be used to store objects via an ORM and has no other dependencies. For smaller sites SQLite is just enough. But it is not the only way in which data can be saved into the file systems. Sometimes normal, plain text files are enough.

The third and least used possibility are so-called object oriented databases. These databases store the actual objects instead of the relations that OR-mapping creates between rows in a database. This has the advantage that nearly all objects can be saven in a straightforward way, unlike in relational databases where some objects are very hard to represent with ORMs.

Frameworks often give the users hints on which method to choose, it is usually a good idea to stick to these unless there are some special requirements which require to use the one method and not the other.

Some notable frameworks

There is an incredible number of frameworks, so there is no way to describe them all. It is not even neccessary, as most of these frameworks are nothing special and everything that can be done with these can also be done with one of the popular ones.