6 Building Extensions: Tips and Tricks

Whenever possible, the Distutils try to use the configuration information made available by the Python interpreter used to run the setup.py script. For example, the same compiler and linker flags used to compile Python will also be used for compiling extensions. Usually this will work well, but in complicated situations this might be inappropriate. This section discusses how to override the usual Distutils behaviour.

6.1 Tweaking compiler/linker flags

Compiling a Python extension written in C or C++ will sometimes require specifying custom flags for the compiler and linker in order to use a particular library or produce a special kind of object code. This is especially true if the extension hasn't been tested on your platform, or if you're trying to cross-compile Python.

In the most general case, the extension author might have foreseen that compiling the extensions would be complicated, and provided a Setup file for you to edit. This will likely only be done if the module distribution contains many separate extension modules, or if they often require elaborate sets of compiler flags in order to work.

A Setup file, if present, is parsed in order to get a list of extensions to build. Each line in a Setup describes a single module. Lines have the following structure:

 module ... [sourcefile ...] [cpparg ...] [library ...]
 

Let's examine each of the fields in turn.

• module is the name of the extension module to be built, and should be a valid Python identifier. You can't just change this in order to rename a module (edits to the source code would also be needed), so this should be left alone.

• sourcefile is anything that's likely to be a source code file, at least judging by the filename. Filenames ending in .c are assumed to be written in C, filenames ending in .C, .cc, and .c++ are assumed to be C++, and filenames ending in .m or .mm are assumed to be in Objective C.

• cpparg is an argument for the C preprocessor, and is anything starting with -I, -D, -U or -C.

• library is anything ending in .a or beginning with -l or -L.

If a particular platform requires a special library on your platform, you can add it by editing the Setup file and running python setup.py build. For example, if the module defined by the line

foo foomodule.c

must be linked with the math library libm.a on your platform, simply add -lm to the line:

foo foomodule.c -lm

Arbitrary switches intended for the compiler or the linker can be supplied with the -Xcompiler arg and -Xlinker arg options:

foo foomodule.c -Xcompiler -o32 -Xlinker -shared -lm

The next option after -Xcompiler and -Xlinker will be appended to the proper command line, so in the above example the compiler will be passed the -o32 option, and the linker will be passed -shared. If a compiler option requires an argument, you'll have to supply multiple -Xcompiler options; for example, to pass -x c++ the Setup file would have to contain -Xcompiler -x -Xcompiler c++.

Compiler flags can also be supplied through setting the CFLAGS environment variable. If set, the contents of CFLAGS will be added to the compiler flags specified in the Setup file.

6.2 Using non-Microsoft compilers on Windows

6.2.1 Borland C++

This subsection describes the necessary steps to use Distutils with the Borland C++ compiler version 5.5.

First you have to know that Borland's object file format (OMF) is different from the format used by the Python version you can download from the Python or ActiveState Web site. (Python is built with Microsoft Visual C++, which uses COFF as the object file format.) For this reason you have to convert Python's library python24.lib into the Borland format. You can do this as follows:

coff2omf python24.lib python24_bcpp.lib

The coff2omf program comes with the Borland compiler. The file python24.lib is in the Libs directory of your Python installation. If your extension uses other libraries (zlib,...) you have to convert them too.

The converted files have to reside in the same directories as the normal libraries.

How does Distutils manage to use these libraries with their changed names? If the extension needs a library (eg. foo) Distutils checks first if it finds a library with suffix _bcpp (eg. foo_bcpp.lib) and then uses this library. In the case it doesn't find such a special library it uses the default name (foo.lib.)¹

To let Distutils compile your extension with Borland C++ you now have to type:

python setup.py build --compiler=bcpp

If you want to use the Borland C++ compiler as the default, you could specify this in your personal or system-wide configuration file for Distutils (see section 5.)

6.2.2 GNU C / Cygwin / MinGW

This section describes the necessary steps to use Distutils with the GNU C/C++ compilers in their Cygwin and MinGW distributions.²For a Python interpreter that was built with Cygwin, everything should work without any of these following steps.

These compilers require some special libraries. This task is more complex than for Borland's C++, because there is no program to convert the library.

First you have to create a list of symbols which the Python DLL exports. (You can find a good program for this task at http://starship.python.net/crew/kernr/mingw32/Notes.html, see at PExports 0.42h there.)

pexports python24.dll >python24.def

Then you can create from these information an import library for gcc.

dlltool --dllname python24.dll --def python24.def --output-lib libpython24.a

The resulting library has to be placed in the same directory as python24.lib. (Should be the libs directory under your Python installation directory.)

If your extension uses other libraries (zlib,...) you might have to convert them too. The converted files have to reside in the same directories as the normal libraries do.

To let Distutils compile your extension with Cygwin you now have to type

python setup.py build --compiler=cygwin

and for Cygwin in no-cygwin mode³ or for MinGW type:

python setup.py build --compiler=mingw32

If you want to use any of these options/compilers as default, you should consider to write it in your personal or system-wide configuration file for Distutils (see section 5.)

Footnotes

...foo.lib.)¹

This also means you could replace all existing COFF-libraries with OMF-libraries of the same name.

... distributions.²

Check http://sources.redhat.com/cygwin/ and http://www.mingw.org/ for more information

... mode³

Then you have no POSIX emulation available, but you also don't need cygwin1.dll.

---

Installing Python Modules

Previous: 5 Distutils Configuration Files Up: Installing Python Modules Next: About this document ...

---

Release 2.4.1, documentation updated on 30 March 2005.