Boost.Python Build and Test HOWTO

1   Requirements

Boost.Python requires Python 2.21 or newer.

2   Background

There are two basic models for combining C++ and Python:

The key distinction between extending and embedding is the location of C++' main() function: in the Python interpreter executable, or in some other program, respectively. Note that even when embedding Python in another program, extension modules are often the best way to make C/C++ functionality accessible to Python code, so the use of extension modules is really at the heart of both models.

Except in rare cases, extension modules are built as dynamically-loaded libraries with a single entry point, which means you can change them without rebuilding either the other extension modules or the executable containing main().

3   Getting Boost.Python Binaries

Since Boost.Python is a separately-compiled (as opposed to header-only) library, its user relies on the services of a Boost.Python library binary. The Boost Getting Started Guide will walk you through the steps of installing one. If building binaries from source, you might want to supply the --with-python argument to bjam (or the --with-libraries=python argument to configure), so only the Boost.Python binary will be built, rather than all the Boost binaries.

Note

Of course it's possible to use other build systems to build Boost.Python and its extensions, but they are not officially supported by Boost and 99% of all “I can't build Boost.Python” problems come from trying to use another build system.

If you want to use another system anyway, we suggest that you follow these instructions, and then invoke bjam with the -ofilename option to dump the build commands it executes to a file, so you can see what your build system needs to do.

4   Choosing a Boost.Python Library Binary

The Boost.Python binary comes in both static and dynamic flavors. Take care to choose the right flavor for your application.2

4.1   The Dynamic Binary

The dynamic library is the safest and most-versatile choice:

  • A single copy of the library code is used by all extension modules built with a given toolset.3
  • The library contains a type conversion registry. Because one registry is shared among all extension modules, instances of a class exposed to Python in one dynamically-loaded extension module can be passed to functions exposed in another such module.

4.2   The Static Binary

It might be appropriate to use the static Boost.Python library in any of the following cases:

  • You are extending python and the types exposed in your dynamically-loaded extension module don't need to be used by any other Boost.Python extension modules, and you don't care if the core library code is duplicated among them.
  • You are embedding python in your application and either:
    • You are targeting a Unix variant OS other than MacOS or AIX, where the dynamically-loaded extension modules can “see” the Boost.Python library symbols that are part of the executable.
    • Or, you have statically linked some Boost.Python extension modules into your application and you don't care if any dynamically-loaded Boost.Python extension modules are able to use the types exposed by your statically-linked extension modules (and vice-versa).

5   Configuring Boost.Build

As described in the Boost.Build reference manual, a file called user-config.jam in your home directory (%HOMEDRIVE%%HOMEPATH% on Windows) is used to describe the build resources available to the build system. You'll need to tell it about your Python installation.

Users of Unix-Variant OSes

If you are using a unix-variant OS and you ran Boost's configure script, it may have generated a user-config.jam for you.4 If your configure/make sequence was successful and Boost.Python binaries were built, your user-config.jam file is probably already correct.

5.1   The Basics

If you have a fairly “standard” python installation for your platform, there's very little you need to do to describe it. Simply adding

import toolset : using ;
using python ;

to a user-config.jam file in your home directory6 should be enough.

5.2   Advanced Configuration

On the other hand, if you have several versions of Python installed, or Python is installed in an unusual way, you may want to supply any or all of the following optional parameters to using python:

version
the version of Python to use. Should be in Major.Minor format, for example, 2.3. Do not include the subminor version (i.e. not 2.5.1). If you have multiple Python versions installed, the version will usually be the only additional argument required.
cmd-or-prefix
preferably, a command that invokes a Python interpreter. Alternatively, the installation prefix for Python libraries and header files. Use the alternative formulation if there is no appropriate Python executable available.
includes
the #include path for Python headers.
libraries
the path to Python library binaries. On MacOS/Darwin, you can also pass the path of the Python framework.
condition
if specified, should be a set of Boost.Build properties that are matched against the build configuration when Boost.Build selects a Python configuration to use.
extension-suffix
A string to append to the name of extension modules before the true filename extension. You almost certainly don't need to use this. Usually this suffix is only used when targeting a Windows debug build of Python, and will be set automatically for you based on the value of the <python-debugging> feature. However, at least one Linux distribution (Ubuntu Feisty Fawn) has a specially configured python-dbg package that claims to use such a suffix.

5.3   Advanced Configuration Examples

Note that in the examples below, case and especially whitespace are significant.

  • If you have both python 2.5 and python 2.4 installed, user-config.jam might contain:

    using python : 2.5 ;  # Make both versions of Python available
    
    using python : 2.4 ;  # To build with python 2.4, add python=2.4
                          # to your command line.
    

    The first version configured (2.5) becomes the default. To build against python 2.4, add python=2.4 to the bjam command line.

  • If you have python installed in an unusual location, you might supply the path to the interpreter in the cmd-or-prefix parameter:

    using python : : /usr/local/python-2.6-beta/bin/python ;
    
  • If you have a separate build of Python for use with a particular toolset, you might supply that toolset in the condition parameter:

    using python ;  # use for most toolsets
    
    # Use with Intel C++ toolset
    using python
         : # version
         : c:\\Devel\\Python-2.5-IntelBuild\\PCBuild\\python # cmd-or-prefix
         : # includes
         : # libraries
         : <toolset>intel # condition
         ;
    
  • You can set up your user-config.jam so a bjam built under Windows can build/test both windows and cygwin python extensions. Just pass <target-os>cygwin in the condition parameter for the cygwin python installation:

    # windows installation
    using python ;
    
    # cygwin installation
    using python : : c:\\cygwin\\bin\\python2.5 : : : <target-os>cygwin ;
    

    when you put target-os=cygwin in your build request, it should build with the cygwin version of python:5

    bjam target-os=cygwin toolset=gcc

    This is supposed to work the other way, too (targeting windows python with a cygwin bjam) but it seems as though the support in Boost.Build's toolsets for building that way is broken at the time of this writing.

  • Note that because of the way Boost.Build currently selects target alternatives, you might have be very explicit in your build requests. For example, given:

    using python : 2.5 ; # a regular windows build
    using python : 2.4 : : : : <target-os>cygwin ;
    

    building with

    bjam target-os=cygwin
    

    will yield an error. Instead, you'll need to write:

    bjam target-os=cygwin/python=2.4
    

[1]Note that although we tested earlier versions of Boost.Python with Python 2.2, and we don't think we've done anything to break compatibility, this release of Boost.Python may not have been tested with versions of Python earlier than 2.4, so we're not 100% sure that python 2.2 and 2.3 are supported.
[2]

Information about how to identify the static and dynamic builds of Boost.Python:

Be sure to read this section even if your compiler supports auto-linking, as Boost.Python does not yet take advantage of that feature.

[3]Because of the way most *nix platforms share symbols among dynamically-loaded objects, I'm not certainextension modules built with different compiler toolsets will always use different copies of the Boost.Python library when loaded into the same Python instance. Not using different libraries could be a good thing if the compilers have compatible ABIs, because extension modules built with the two libraries would be interoperable. Otherwise, it could spell disaster, since an extension module and the Boost.Python library would have different ideas of such things as class layout. I would appreciate someone doing the experiment to find out what happens.
[4]configure overwrites the existing user-config.jam in your home directory (if any) after making a backup of the old version.
[5]Note that the <target-os>cygwin feature is different from the <flavor>cygwin subfeature of the gcc toolset, and you might need handle both explicitly if you also have a MinGW GCC installed.
[6]

Create the user-config.jam file if you don't already have one. For Windows users, the home directory can be found by typing:

ECHO %HOMEDRIVE%%HOMEPATH%

into a Windows command prompt