Dynamically Configured Mass Virtual Hosting
This document describes how to efficiently serve an arbitrary number of virtual hosts with the Apache httpd webserver.
Motivation
The techniques described here are of interest if your
httpd.conf contains many
<VirtualHost> sections that are
substantially the same, for example:
NameVirtualHost 111.22.33.44
<VirtualHost 111.22.33.44>
ServerName www.customer-1.com
DocumentRoot /www/hosts/www.customer-1.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-1.com/cgi-bin
</VirtualHost>
<VirtualHost 111.22.33.44>
ServerName www.customer-2.com
DocumentRoot /www/hosts/www.customer-2.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-2.com/cgi-bin
</VirtualHost>
# blah blah blah
<VirtualHost 111.22.33.44>
ServerName www.customer-N.com
DocumentRoot /www/hosts/www.customer-N.com/docs
ScriptAlias /cgi-bin/ /www/hosts/www.customer-N.com/cgi-bin
</VirtualHost>
The basic idea is to replace all of the static
<VirtualHost> configurations with a mechanism
that works them out dynamically. This has a number of
advantages:
- Your configuration file is smaller, so Apache starts more quickly and uses less memory.
- Adding virtual hosts is simply a matter of creating the appropriate directories in the filesystem and entries in the DNS - you don't need to reconfigure or restart Apache.
The main disadvantage is that you cannot have a different log file for each virtual host; however, if you have many virtual hosts, doing this can be a bad idea anyway, because of the number of file descriptors needed. It is better to log to a pipe or a fifo, and arrange for the process at the other end to distribute the logs to the customers. (This can also be used to accumulate statistics, etc.).
Overview
A virtual host is defined by two pieces of information: its
IP address, and the contents of the Host: header
in the HTTP request. The dynamic mass virtual hosting technique
used here is based on automatically inserting this information into the
pathname of the file that is used to satisfy the request. This
can be most easily done by using mod_vhost_alias
with Apache 2.0. Alternatively, mod_rewrite can be used.
Both of these modules are disabled by default; you must enable
one of them when configuring and building Apache if you want to
use this technique.
A couple of things need to be `faked' to make the dynamic
virtual host look like a normal one. The most important is the
server name, which is used by Apache to generate
self-referential URLs etc. It is configured with the
ServerName directive, and it is available to CGIs
via the SERVER_NAME environment variable. The
actual value used at run time is controlled by the UseCanonicalName
setting. With UseCanonicalName Off, the server name
is taken from the contents of the Host: header in the
request. With UseCanonicalName DNS, it is taken from a
reverse DNS lookup of the virtual host's IP address. The former
setting is used for name-based dynamic virtual hosting, and the
latter is used for IP-based hosting. If Apache cannot work out
the server name because there is no Host: header,
or the DNS lookup fails, then the value configured with
ServerName is used instead.
The other thing to `fake' is the document root (configured
with DocumentRoot and available to CGIs via the
DOCUMENT_ROOT environment variable). In a normal
configuration, this is used by the core module when
mapping URIs to filenames, but when the server is configured to
do dynamic virtual hosting, that job must be taken over by another
module (either mod_vhost_alias or
mod_rewrite), which has a different way of doing
the mapping. Neither of these modules is responsible for
setting the DOCUMENT_ROOT environment variable so
if any CGIs or SSI documents make use of it, they will get a
misleading value.
Simple Dynamic Virtual Hosts
This extract from httpd.conf implements the
virtual host arrangement outlined in the Motivation section above, but in a
generic fashion using mod_vhost_alias.
# get the server name from the Host: header
UseCanonicalName Off
# this log format can be split per-virtual-host based on the first field
LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
# include the server name in the filenames used to satisfy requests
VirtualDocumentRoot /www/hosts/%0/docs
VirtualScriptAlias /www/hosts/%0/cgi-bin
This configuration can be changed into an IP-based virtual
hosting solution by just turning UseCanonicalName
Off into UseCanonicalName DNS. The server
name that is inserted into the filename is then derived from
the IP address of the virtual host.
A Virtually Hosted Homepages System
This is an adjustment of the above system, tailored for an
ISP's homepages server. Using a slightly more complicated
configuration, we can select substrings of the server name to
use in the filename so that, for example, the documents for
www.user.isp.com are found in
/home/user/. It uses a single cgi-bin
directory instead of one per virtual host.
# all the preliminary stuff is the same as above, then
# include part of the server name in the filenames
VirtualDocumentRoot /www/hosts/%2/docs
# single cgi-bin directory
ScriptAlias /cgi-bin/ /www/std-cgi/
There are examples of more complicated
VirtualDocumentRoot settings in the
mod_vhost_alias documentation.
Using Multiple Virtual Hosting Systems on the Same Server
With more complicated setups, you can use Apache's normal
<VirtualHost> directives to control the
scope of the various virtual hosting configurations. For
example, you could have one IP address for general customers' homepages,
and another for commercial customers, with the following setup.
This can, of course, be combined with conventional
<VirtualHost> configuration sections.
UseCanonicalName Off
LogFormat "%V %h %l %u %t \"%r\" %s %b" vcommon
<Directory /www/commercial>
Options FollowSymLinks
AllowOverride All
</Directory>
<Directory /www/homepages>
Options FollowSymLinks
AllowOverride None
</Directory>
<VirtualHost 111.22.33.44>
ServerName www.commercial.isp.com
CustomLog logs/access_log.commercial vcommon
VirtualDocumentRoot /www/commercial/%0/docs
VirtualScriptAlias /www/commercial/%0/cgi-bin
</VirtualHost>
<VirtualHost 111.22.33.45>
ServerName www.homepages.isp.com
CustomLog logs/access_log.homepages vcommon
VirtualDocumentRoot /www/homepages/%0/docs
ScriptAlias /cgi-bin/ /www/std-cgi/
</VirtualHost>
More Efficient IP-Based Virtual Hosting
The configuration changes suggested to turn the first example into an IP-based virtual hosting setup result in a rather inefficient setup. A new DNS lookup is required for every request. To avoid this overhead, the filesystem can be arranged to correspond to the IP addresses, instead of to the host names, thereby negating the need for a DNS lookup. Logging will also have to be adjusted to fit this system.
# get the server name from the reverse DNS of the IP address
UseCanonicalName DNS
# include the IP address in the logs so they may be split
LogFormat "%A %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
# include the IP address in the filenames
VirtualDocumentRootIP /www/hosts/%0/docs
VirtualScriptAliasIP /www/hosts/%0/cgi-bin
Simple Dynamic
Virtual Hosts Using
This extract from httpd.conf does the same
thing as the first example. The first
half is very similar to the corresponding part above, except for
some changes, required for backward compatibility and to make the
mod_rewrite part work properly; the second half
configures mod_rewrite to do the actual work.
There are a couple of especially tricky bits: by default,
mod_rewrite runs before other URI translation
modules (mod_alias etc.) - so if you wish to use these modules, mod_rewrite must be configured to accommodate
them. Also, some magic is required to do a
per-dynamic-virtual-host equivalent of
ScriptAlias.
# get the server name from the Host: header
UseCanonicalName Off
# splittable logs
LogFormat "%{Host}i %h %l %u %t \"%r\" %s %b" vcommon
CustomLog logs/access_log vcommon
<Directory /www/hosts>
# ExecCGI is needed here because we can't force
# CGI execution in the way that ScriptAlias does
Options FollowSymLinks ExecCGI
</Directory>
# now for the hard bit
RewriteEngine On
# a ServerName derived from a Host: header may be any case at all
RewriteMap lowercase int:tolower
## deal with normal documents first:
# allow Alias /icons/ to work - repeat for other aliases
RewriteCond %{REQUEST_URI} !^/icons/
# allow CGIs to work
RewriteCond %{REQUEST_URI} !^/cgi-bin/
# do the magic
RewriteRule ^/(.*)$ /www/hosts/${lowercase:%{SERVER_NAME}}/docs/$1
## and now deal with CGIs - we have to force a MIME type
RewriteCond %{REQUEST_URI} ^/cgi-bin/
RewriteRule ^/(.*)$ /www/hosts/${lowercase:%{SERVER_NAME}}/cgi-bin/$1 [T=application/x-httpd-cgi]
# that's it!
A
Homepages System Using mod_rewrite
This does the same thing as the second example.
RewriteEngine on
RewriteMap lowercase int:tolower
# allow CGIs to work
RewriteCond %{REQUEST_URI} !^/cgi-bin/
# check the hostname is right so that the RewriteRule works
RewriteCond ${lowercase:%{SERVER_NAME}} ^www\.[a-z-]+\.isp\.com$
# concatenate the virtual host name onto the start of the URI
# the [C] means do the next rewrite on the result of this one
RewriteRule ^(.+) ${lowercase:%{SERVER_NAME}}$1 [C]
# now create the real file name
RewriteRule ^www\.([a-z-]+)\.isp\.com/(.*) /home/$1/$2
# define the global CGI directory
ScriptAlias /cgi-bin/ /www/std-cgi/
Using a Separate Virtual Host Configuration File
This arrangement uses more advanced mod_rewrite
features to work out the translation from virtual host to document
root, from a separate configuration file. This provides more
flexibility, but requires more complicated configuration.
The vhost.map file should look something like
this:
www.customer-1.com /www/customers/1
www.customer-2.com /www/customers/2
# ...
www.customer-N.com /www/customers/N
The httpd.conf should contain the following:
RewriteEngine on
RewriteMap lowercase int:tolower
# define the map file
RewriteMap vhost txt:/www/conf/vhost.map
# deal with aliases as above
RewriteCond %{REQUEST_URI} !^/icons/
RewriteCond %{REQUEST_URI} !^/cgi-bin/
RewriteCond ${lowercase:%{SERVER_NAME}} ^(.+)$
# this does the file-based remap
RewriteCond ${vhost:%1} ^(/.*)$
RewriteRule ^/(.*)$ %1/docs/$1
RewriteCond %{REQUEST_URI} ^/cgi-bin/
RewriteCond ${lowercase:%{SERVER_NAME}} ^(.+)$
RewriteCond ${vhost:%1} ^(/.*)$
RewriteRule ^/(.*)$ %1/cgi-bin/$1 [T=application/x-httpd-cgi]