mirror of
https://github.com/boostorg/python.git
synced 2026-01-25 06:22:15 +00:00
This commit was manufactured by cvs2svn to create branch 'RC_1_30_0'.
[SVN r17693]
This commit is contained in:
@@ -1,864 +0,0 @@
|
||||
.. This is a comment. Note how any initial comments are moved by
|
||||
transforms to after the document title, subtitle, and docinfo.
|
||||
|
||||
.. Need intro and conclusion
|
||||
.. Exposing classes
|
||||
.. Constructors
|
||||
.. Overloading
|
||||
.. Properties and data members
|
||||
.. Inheritance
|
||||
.. Operators and Special Functions
|
||||
.. Virtual Functions
|
||||
.. Call Policies
|
||||
|
||||
++++++++++++++++++++++++++++++++++++++++++++++
|
||||
Introducing Boost.Python (Extended Abstract)
|
||||
++++++++++++++++++++++++++++++++++++++++++++++
|
||||
|
||||
|
||||
.. bibliographic fields (which also require a transform):
|
||||
|
||||
:Author: David Abrahams
|
||||
:Address: 45 Walnut Street
|
||||
Somerville, MA 02143
|
||||
:Contact: dave@boost-consulting.com
|
||||
:organization: `Boost Consulting`_
|
||||
:date: $Date$
|
||||
:status: This is a "work in progress"
|
||||
:version: 1
|
||||
:copyright: Copyright David Abrahams 2002. All rights reserved
|
||||
|
||||
:Dedication:
|
||||
|
||||
For my girlfriend, wife, and partner Luann
|
||||
|
||||
:abstract:
|
||||
|
||||
This paper describes the Boost.Python library, a system for
|
||||
C++/Python interoperability.
|
||||
|
||||
.. meta::
|
||||
:keywords: Boost,python,Boost.Python,C++
|
||||
:description lang=en: C++/Python interoperability with Boost.Python
|
||||
|
||||
.. contents:: Table of Contents
|
||||
.. section-numbering::
|
||||
|
||||
|
||||
.. _`Boost Consulting`: http://www.boost-consulting.com
|
||||
|
||||
==============
|
||||
Introduction
|
||||
==============
|
||||
|
||||
Python and C++ are in many ways as different as two languages could
|
||||
be: while C++ is usually compiled to machine-code, Python is
|
||||
interpreted. Python's dynamic type system is often cited as the
|
||||
foundation of its flexibility, while in C++ static typing is the
|
||||
cornerstone of its efficiency. C++ has an intricate and difficult
|
||||
compile-time meta-language, while in Python, practically everything
|
||||
happens at runtime.
|
||||
|
||||
Yet for many programmers, these very differences mean that Python and
|
||||
C++ complement one another perfectly. Performance bottlenecks in
|
||||
Python programs can be rewritten in C++ for maximal speed, and
|
||||
authors of powerful C++ libraries choose Python as a middleware
|
||||
language for its flexible system integration capabilities.
|
||||
Furthermore, the surface differences mask some strong similarities:
|
||||
|
||||
* 'C'-family control structures (if, while, for...)
|
||||
|
||||
* Support for object-orientation, functional programming, and generic
|
||||
programming (these are both *multi-paradigm* programming languages.)
|
||||
|
||||
* Comprehensive operator overloading facilities, recognizing the
|
||||
importance of syntactic variability for readability and
|
||||
expressivity.
|
||||
|
||||
* High-level concepts such as collections and iterators.
|
||||
|
||||
* Strong support for writers of re-usable libraries.
|
||||
|
||||
* C++ idioms in common use, such as handle/body classes and
|
||||
reference-counted smart pointers mirror Python reference semantics.
|
||||
|
||||
* Exception-handling for effective management of error conditions.
|
||||
|
||||
Given Python's rich 'C' interoperability API, it should in principle
|
||||
be possible to expose C++ type and function interfaces to Python with
|
||||
an analogous interface to their C++ counterparts. However, the
|
||||
facilities provided by Python alone for integration with C++ are
|
||||
relatively meager. Some of this, such as the need to manage
|
||||
reference-counting manually and lack of C++ exception-handling
|
||||
support, comes from the limitations of the 'C' language in which the
|
||||
API is implemented. Most of the remaining issues can be handled if
|
||||
the code understands the C++ type system. For example:
|
||||
|
||||
* Every argument of every wrapped function requires some kind of
|
||||
extraction code to convert it from Python to C++. Likewise, the
|
||||
function return value has to be converted from C++ to Python.
|
||||
Appropriate Python exceptions must be raised if the conversion
|
||||
fails. Argument and return types are part of the function's type,
|
||||
and much of this tedium can be relieved if the wrapping system can
|
||||
extract that information through introspection.
|
||||
|
||||
* Passing a wrapped C++ derived class instance to a C++ function
|
||||
accepting a pointer or reference to a base class requires knowledge
|
||||
of the inheritance relationship and how to translate the address of
|
||||
a base class into that of a derived class.
|
||||
|
||||
The Boost.Python Library (BPL) leverages the power of C++
|
||||
meta-programming techniques to introspect about the C++ type system,
|
||||
and presents a simple, IDL-like C++ interface for exposing C++ code in
|
||||
extension modules.
|
||||
|
||||
===========================
|
||||
Boost.Python Design Goals
|
||||
===========================
|
||||
|
||||
The primary goal of Boost.Python is to allow users to expose C++
|
||||
classes and functions to Python using nothing more than a C++
|
||||
compiler. In broad strokes, the user experience should be one of
|
||||
directly manipulating C++ objects from Python.
|
||||
|
||||
However, it's also important not to translate all interfaces *too*
|
||||
literally: the idioms of each language must be respected. For
|
||||
example, though C++ and Python both have an iterator concept, they are
|
||||
expressed very differently. Boost.Python has to be able to bridge the
|
||||
interface gap.
|
||||
|
||||
It must be possible to insulate Python users from crashes resulting
|
||||
from trivial misuses of C++ interfaces, such as accessing
|
||||
already-deleted objects. By the same token the library should
|
||||
insulate C++ users from low-level Python 'C' API, replacing
|
||||
error-prone 'C' interfaces like manual reference-count management and
|
||||
raw ``PyObject`` pointers with more-robust alternatives.
|
||||
|
||||
Support for component-based development is crucial, so that C++ types
|
||||
exposed in one extension module can be passed to functions exposed in
|
||||
another without loss of crucial information like C++ inheritance
|
||||
relationships.
|
||||
|
||||
Finally, all wrapping must be *non-intrusive*, without modifying or
|
||||
even seeing the original C++ source code. Existing C++ libraries have
|
||||
to be wrappable by third parties who only have access to header files
|
||||
and binaries.
|
||||
|
||||
==========================
|
||||
Hello Boost.Python World
|
||||
==========================
|
||||
|
||||
And now for a preview of Boost.Python, and how it improves on the raw
|
||||
facilities offered by Python. Here's a function we might want to
|
||||
expose::
|
||||
|
||||
char const* greet(unsigned x)
|
||||
{
|
||||
static char const* const msgs[] = { "hello", "Boost.Python", "world!" };
|
||||
|
||||
if (x > 2)
|
||||
throw std::range_error("greet: index out of range");
|
||||
|
||||
return msgs[x];
|
||||
}
|
||||
|
||||
To wrap this function in standard C++ using the Python 'C' API, we'd
|
||||
need something like this::
|
||||
|
||||
extern "C" // all Python interactions use 'C' linkage and calling convention
|
||||
{
|
||||
// Wrapper to handle argument/result conversion and checking
|
||||
PyObject* greet_wrap(PyObject* args, PyObject * keywords)
|
||||
{
|
||||
int x;
|
||||
if (PyArg_ParseTuple(args, "i", &x)) // extract/check arguments
|
||||
{
|
||||
char const* result = greet(x); // invoke wrapped function
|
||||
return PyString_FromString(result); // convert result to Python
|
||||
}
|
||||
return 0; // error occurred
|
||||
}
|
||||
|
||||
// Table of wrapped functions to be exposed by the module
|
||||
static PyMethodDef methods[] = {
|
||||
{ "greet", greet_wrap, METH_VARARGS, "return one of 3 parts of a greeting" }
|
||||
, { NULL, NULL, 0, NULL } // sentinel
|
||||
};
|
||||
|
||||
// module initialization function
|
||||
DL_EXPORT init_hello()
|
||||
{
|
||||
(void) Py_InitModule("hello", methods); // add the methods to the module
|
||||
}
|
||||
}
|
||||
|
||||
Now here's the wrapping code we'd use to expose it with Boost.Python::
|
||||
|
||||
#include <boost/python.hpp>
|
||||
using namespace boost::python;
|
||||
BOOST_PYTHON_MODULE(hello)
|
||||
{
|
||||
def("greet", greet, "return one of 3 parts of a greeting");
|
||||
}
|
||||
|
||||
and here it is in action::
|
||||
|
||||
>>> import hello
|
||||
>>> for x in range(3):
|
||||
... print hello.greet(x)
|
||||
...
|
||||
hello
|
||||
Boost.Python
|
||||
world!
|
||||
|
||||
Aside from the fact that the 'C' API version is much more verbose than
|
||||
the BPL one, it's worth noting that it doesn't handle a few things
|
||||
correctly:
|
||||
|
||||
* The original function accepts an unsigned integer, and the Python
|
||||
'C' API only gives us a way of extracting signed integers. The
|
||||
Boost.Python version will raise a Python exception if we try to pass
|
||||
a negative number to ``hello.greet``, but the other one will proceed
|
||||
to do whatever the C++ implementation does when converting an
|
||||
negative integer to unsigned (usually wrapping to some very large
|
||||
number), and pass the incorrect translation on to the wrapped
|
||||
function.
|
||||
|
||||
* That brings us to the second problem: if the C++ ``greet()``
|
||||
function is called with a number greater than 2, it will throw an
|
||||
exception. Typically, if a C++ exception propagates across the
|
||||
boundary with code generated by a 'C' compiler, it will cause a
|
||||
crash. As you can see in the first version, there's no C++
|
||||
scaffolding there to prevent this from happening. Functions wrapped
|
||||
by Boost.Python automatically include an exception-handling layer
|
||||
which protects Python users by translating unhandled C++ exceptions
|
||||
into a corresponding Python exception.
|
||||
|
||||
* A slightly more-subtle limitation is that the argument conversion
|
||||
used in the Python 'C' API case can only get that integer ``x`` in
|
||||
*one way*. PyArg_ParseTuple can't convert Python ``long`` objects
|
||||
(arbitrary-precision integers) which happen to fit in an ``unsigned
|
||||
int`` but not in a ``signed long``, nor will it ever handle a
|
||||
wrapped C++ class with a user-defined implicit ``operator unsigned
|
||||
int()`` conversion. The BPL's dynamic type conversion registry
|
||||
allows users to add arbitrary conversion methods.
|
||||
|
||||
==================
|
||||
Library Overview
|
||||
==================
|
||||
|
||||
This section outlines some of the library's major features. Except as
|
||||
neccessary to avoid confusion, details of library implementation are
|
||||
omitted.
|
||||
|
||||
------------------
|
||||
Exposing Classes
|
||||
------------------
|
||||
|
||||
C++ classes and structs are exposed with a similarly-terse interface.
|
||||
Given::
|
||||
|
||||
struct World
|
||||
{
|
||||
void set(std::string msg) { this->msg = msg; }
|
||||
std::string greet() { return msg; }
|
||||
std::string msg;
|
||||
};
|
||||
|
||||
The following code will expose it in our extension module::
|
||||
|
||||
#include <boost/python.hpp>
|
||||
BOOST_PYTHON_MODULE(hello)
|
||||
{
|
||||
class_<World>("World")
|
||||
.def("greet", &World::greet)
|
||||
.def("set", &World::set)
|
||||
;
|
||||
}
|
||||
|
||||
Although this code has a certain pythonic familiarity, people
|
||||
sometimes find the syntax bit confusing because it doesn't look like
|
||||
most of the C++ code they're used to. All the same, this is just
|
||||
standard C++. Because of their flexible syntax and operator
|
||||
overloading, C++ and Python are great for defining domain-specific
|
||||
(sub)languages
|
||||
(DSLs), and that's what we've done in BPL. To break it down::
|
||||
|
||||
class_<World>("World")
|
||||
|
||||
constructs an unnamed object of type ``class_<World>`` and passes
|
||||
``"World"`` to its constructor. This creates a new-style Python class
|
||||
called ``World`` in the extension module, and associates it with the
|
||||
C++ type ``World`` in the BPL type conversion registry. We might have
|
||||
also written::
|
||||
|
||||
class_<World> w("World");
|
||||
|
||||
but that would've been more verbose, since we'd have to name ``w``
|
||||
again to invoke its ``def()`` member function::
|
||||
|
||||
w.def("greet", &World::greet)
|
||||
|
||||
There's nothing special about the location of the dot for member
|
||||
access in the original example: C++ allows any amount of whitespace on
|
||||
either side of a token, and placing the dot at the beginning of each
|
||||
line allows us to chain as many successive calls to member functions
|
||||
as we like with a uniform syntax. The other key fact that allows
|
||||
chaining is that ``class_<>`` member functions all return a reference
|
||||
to ``*this``.
|
||||
|
||||
So the example is equivalent to::
|
||||
|
||||
class_<World> w("World");
|
||||
w.def("greet", &World::greet);
|
||||
w.def("set", &World::set);
|
||||
|
||||
It's occasionally useful to be able to break down the components of a
|
||||
Boost.Python class wrapper in this way, but the rest of this paper
|
||||
will tend to stick to the terse syntax.
|
||||
|
||||
For completeness, here's the wrapped class in use:
|
||||
|
||||
>>> import hello
|
||||
>>> planet = hello.World()
|
||||
>>> planet.set('howdy')
|
||||
>>> planet.greet()
|
||||
'howdy'
|
||||
|
||||
Constructors
|
||||
============
|
||||
|
||||
Since our ``World`` class is just a plain ``struct``, it has an
|
||||
implicit no-argument (nullary) constructor. Boost.Python exposes the
|
||||
nullary constructor by default, which is why we were able to write:
|
||||
|
||||
>>> planet = hello.World()
|
||||
|
||||
However, well-designed classes in any language may require constructor
|
||||
arguments in order to establish their invariants. Unlike Python,
|
||||
where ``__init__`` is just a specially-named method, In C++
|
||||
constructors cannot be handled like ordinary member functions. In
|
||||
particular, we can't take their address: ``&World::World`` is an
|
||||
error. The library provides a different interface for specifying
|
||||
constructors. Given::
|
||||
|
||||
struct World
|
||||
{
|
||||
World(std::string msg); // added constructor
|
||||
...
|
||||
|
||||
we can modify our wrapping code as follows::
|
||||
|
||||
class_<World>("World", init<std::string>())
|
||||
...
|
||||
|
||||
of course, a C++ class may have additional constructors, and we can
|
||||
expose those as well by passing more instances of ``init<...>`` to
|
||||
``def()``::
|
||||
|
||||
class_<World>("World", init<std::string>())
|
||||
.def(init<double, double>())
|
||||
...
|
||||
|
||||
Boost.Python allows wrapped functions, member functions, and
|
||||
constructors to be overloaded to mirror C++ overloading.
|
||||
|
||||
Data Members and Properties
|
||||
===========================
|
||||
|
||||
Any publicly-accessible data members in a C++ class can be easily
|
||||
exposed as either ``readonly`` or ``readwrite`` attributes::
|
||||
|
||||
class_<World>("World", init<std::string>())
|
||||
.def_readonly("msg", &World::msg)
|
||||
...
|
||||
|
||||
and can be used directly in Python:
|
||||
|
||||
>>> planet = hello.World('howdy')
|
||||
>>> planet.msg
|
||||
'howdy'
|
||||
|
||||
This does *not* result in adding attributes to the ``World`` instance
|
||||
``__dict__``, which can result in substantial memory savings when
|
||||
wrapping large data structures. In fact, no instance ``__dict__``
|
||||
will be created at all unless attributes are explicitly added from
|
||||
Python. BPL owes this capability to the new Python 2.2 type system,
|
||||
in particular the descriptor interface and ``property`` type.
|
||||
|
||||
In C++, publicly-accessible data members are considered a sign of poor
|
||||
design because they break encapsulation, and style guides usually
|
||||
dictate the use of "getter" and "setter" functions instead. In
|
||||
Python, however, ``__getattr__``, ``__setattr__``, and since 2.2,
|
||||
``property`` mean that attribute access is just one more
|
||||
well-encapsulated syntactic tool at the programmer's disposal. BPL
|
||||
bridges this idiomatic gap by making Python ``property`` creation
|
||||
directly available to users. So if ``msg`` were private, we could
|
||||
still expose it as attribute in Python as follows::
|
||||
|
||||
class_<World>("World", init<std::string>())
|
||||
.add_property("msg", &World::greet, &World::set)
|
||||
...
|
||||
|
||||
The example above mirrors the familiar usage of properties in Python
|
||||
2.2+:
|
||||
|
||||
>>> class World(object):
|
||||
... __init__(self, msg):
|
||||
... self.__msg = msg
|
||||
... def greet(self):
|
||||
... return self.__msg
|
||||
... def set(self, msg):
|
||||
... self.__msg = msg
|
||||
... msg = property(greet, set)
|
||||
|
||||
Operators and Special Functions
|
||||
===============================
|
||||
|
||||
The ability to write arithmetic operators for user-defined types that
|
||||
C++ and Python both allow the definition of has been a major factor in
|
||||
the popularity of both languages for scientific computing. The
|
||||
success of packages like NumPy attests to the power of exposing
|
||||
operators in extension modules. In this example we'll wrap a class
|
||||
representing a position in a large file::
|
||||
|
||||
class FilePos { /*...*/ };
|
||||
|
||||
// Linear offset
|
||||
FilePos operator+(FilePos, int);
|
||||
FilePos operator+(int, FilePos);
|
||||
FilePos operator-(FilePos, int);
|
||||
|
||||
// Distance between two FilePos objects
|
||||
int operator-(FilePos, FilePos);
|
||||
|
||||
// Offset with assignment
|
||||
FilePos& operator+=(FilePos&, int);
|
||||
FilePos& operator-=(FilePos&, int);
|
||||
|
||||
// Comparison
|
||||
bool operator<(FilePos, FilePos);
|
||||
|
||||
The wrapping code looks like this::
|
||||
|
||||
class_<FilePos>("FilePos")
|
||||
.def(self + int()) // __add__
|
||||
.def(int() + self) // __radd__
|
||||
.def(self - int()) // __sub__
|
||||
|
||||
.def(self - self) // __sub__
|
||||
|
||||
.def(self += int()) // __iadd__
|
||||
.def(self -= int()) // __isub__
|
||||
|
||||
.def(self < self); // __lt__
|
||||
;
|
||||
|
||||
The magic is performed using a simplified application of "expression
|
||||
templates" [VELD1995]_, a technique originally developed by for
|
||||
optimization of high-performance matrix algebra expressions. The
|
||||
essence is that instead of performing the computation immediately,
|
||||
operators are overloaded to construct a type *representing* the
|
||||
computation. In matrix algebra, dramatic optimizations are often
|
||||
available when the structure of an entire expression can be taken into
|
||||
account, rather than processing each operation "greedily".
|
||||
Boost.Python uses the same technique to build an appropriate Python
|
||||
callable object based on an expression involving ``self``, which is
|
||||
then added to the class.
|
||||
|
||||
Inheritance
|
||||
===========
|
||||
|
||||
C++ inheritance relationships can be represented to Boost.Python by adding
|
||||
an optional ``bases<...>`` argument to the ``class_<...>`` template
|
||||
parameter list as follows::
|
||||
|
||||
class_<Derived, bases<Base1,Base2> >("Derived")
|
||||
...
|
||||
|
||||
This has two effects:
|
||||
|
||||
1. When the ``class_<...>`` is created, Python type objects
|
||||
corresponding to ``Base1`` and ``Base2`` are looked up in the BPL
|
||||
registry, and are used as bases for the new Python ``Derived`` type
|
||||
object [#mi]_, so methods exposed for the Python ``Base1`` and
|
||||
``Base2`` types are automatically members of the ``Derived`` type.
|
||||
Because the registry is global, this works correctly even if
|
||||
``Derived`` is exposed in a different module from either of its
|
||||
bases.
|
||||
|
||||
2. C++ conversions from ``Derived`` to its bases are added to the
|
||||
Boost.Python registry. Thus wrapped C++ methods expecting (a
|
||||
pointer or reference to) an object of either base type can be
|
||||
called with an object wrapping a ``Derived`` instance. Wrapped
|
||||
member functions of class ``T`` are treated as though they have an
|
||||
implicit first argument of ``T&``, so these conversions are
|
||||
neccessary to allow the base class methods to be called for derived
|
||||
objects.
|
||||
|
||||
Of course it's possible to derive new Python classes from wrapped C++
|
||||
class instances. Because Boost.Python uses the new-style class
|
||||
system, that works very much as for the Python built-in types. There
|
||||
is one significant detail in which it differs: the built-in types
|
||||
generally establish their invariants in their ``__new__`` function, so
|
||||
that derived classes do not need to call ``__init__`` on the base
|
||||
class before invoking its methods :
|
||||
|
||||
>>> class L(list):
|
||||
... def __init__(self):
|
||||
... pass
|
||||
...
|
||||
>>> L().reverse()
|
||||
>>>
|
||||
|
||||
Because C++ object construction is a one-step operation, C++ instance
|
||||
data cannot be constructed until the arguments are available, in the
|
||||
``__init__`` function:
|
||||
|
||||
>>> class D(SomeBPLClass):
|
||||
... def __init__(self):
|
||||
... pass
|
||||
...
|
||||
>>> D().some_bpl_method()
|
||||
Traceback (most recent call last):
|
||||
File "<stdin>", line 1, in ?
|
||||
TypeError: bad argument type for built-in operation
|
||||
|
||||
This happened because Boost.Python couldn't find instance data of type
|
||||
``SomeBPLClass`` within the ``D`` instance; ``D``'s ``__init__``
|
||||
function masked construction of the base class. It could be corrected
|
||||
by either removing ``D``'s ``__init__`` function or having it call
|
||||
``SomeBPLClass.__init__(...)`` explicitly.
|
||||
|
||||
Virtual Functions
|
||||
=================
|
||||
|
||||
Deriving new types in Python from extension classes is not very
|
||||
interesting unless they can be used polymorphically from C++. In
|
||||
other words, Python method implementations should appear to override
|
||||
the implementation of C++ virtual functions when called *through base
|
||||
class pointers/references from C++*. Since the only way to alter the
|
||||
behavior of a virtual function is to override it in a derived class,
|
||||
the user must build a special derived class to dispatch a polymorphic
|
||||
class' virtual functions::
|
||||
|
||||
//
|
||||
// interface to wrap:
|
||||
//
|
||||
class Base
|
||||
{
|
||||
public:
|
||||
virtual int f(std::string x) { return 42; }
|
||||
virtual ~Base();
|
||||
};
|
||||
|
||||
int calls_f(Base const& b, std::string x) { return b.f(x); }
|
||||
|
||||
//
|
||||
// Wrapping Code
|
||||
//
|
||||
|
||||
// Dispatcher class
|
||||
struct BaseWrap : Base
|
||||
{
|
||||
// Store a pointer to the Python object
|
||||
BaseWrap(PyObject* self_) : self(self_) {}
|
||||
PyObject* self;
|
||||
|
||||
// Default implementation, for when f is not overridden
|
||||
int f_default(std::string x) { return this->Base::f(x); }
|
||||
// Dispatch implementation
|
||||
int f(std::string x) { return call_method<int>(self, "f", x); }
|
||||
};
|
||||
|
||||
...
|
||||
def("calls_f", calls_f);
|
||||
class_<Base, BaseWrap>("Base")
|
||||
.def("f", &Base::f, &BaseWrap::f_default)
|
||||
;
|
||||
|
||||
Now here's some Python code which demonstrates:
|
||||
|
||||
>>> class Derived(Base):
|
||||
... def f(self, s):
|
||||
... return len(s)
|
||||
...
|
||||
>>> calls_f(Base(), 'foo')
|
||||
42
|
||||
>>> calls_f(Derived(), 'forty-two')
|
||||
9
|
||||
|
||||
Things to notice about the dispatcher class:
|
||||
|
||||
* The key element which allows overriding in Python is the
|
||||
``call_method`` invocation, which uses the same global type
|
||||
conversion registry as the C++ function wrapping does to convert its
|
||||
arguments from C++ to Python and its return type from Python to C++.
|
||||
|
||||
* Any constructor signatures you wish to wrap must be replicated with
|
||||
an initial ``PyObject*`` argument
|
||||
|
||||
* The dispatcher must store this argument so that it can be used to
|
||||
invoke ``call_method``
|
||||
|
||||
* The ``f_default`` member function is needed when the function being
|
||||
exposed is not pure virtual; there's no other way ``Base::f`` can be
|
||||
called on an object of type ``BaseWrap``, since it overrides ``f``.
|
||||
|
||||
Admittedly, this formula is tedious to repeat, especially on a project
|
||||
with many polymorphic classes; that it is neccessary reflects
|
||||
limitations in C++'s compile-time reflection capabilities. Several
|
||||
efforts are underway to write front-ends for Boost.Python which can
|
||||
generate these dispatchers (and other wrapping code) automatically.
|
||||
If these are successful it will mark a move away from wrapping
|
||||
everything directly in pure C++ for many of our users.
|
||||
|
||||
---------------
|
||||
Serialization
|
||||
---------------
|
||||
|
||||
*Serialization* is the process of converting objects in memory to a
|
||||
form that can be stored on disk or sent over a network connection. The
|
||||
serialized object (most often a plain string) can be retrieved and
|
||||
converted back to the original object. A good serialization system will
|
||||
automatically convert entire object hierarchies. Python's standard
|
||||
``pickle`` module is such a system. It leverages the language's strong
|
||||
runtime introspection facilities for serializing practically arbitrary
|
||||
user-defined objects. With a few simple and unintrusive provisions this
|
||||
powerful machinery can be extended to also work for wrapped C++ objects.
|
||||
Here is an example::
|
||||
|
||||
#include <string>
|
||||
|
||||
struct World
|
||||
{
|
||||
World(std::string a_msg) : msg(a_msg) {}
|
||||
std::string greet() const { return msg; }
|
||||
std::string msg;
|
||||
};
|
||||
|
||||
#include <boost/python.hpp>
|
||||
using namespace boost::python;
|
||||
|
||||
struct World_picklers : pickle_suite
|
||||
{
|
||||
static tuple
|
||||
getinitargs(World const& w) { return make_tuple(w.greet()); }
|
||||
};
|
||||
|
||||
BOOST_PYTHON_MODULE(hello)
|
||||
{
|
||||
class_<World>("World", init<std::string>())
|
||||
.def("greet", &World::greet)
|
||||
.def_pickle(World_picklers())
|
||||
;
|
||||
}
|
||||
|
||||
Now let's create a ``World`` object and put it to rest on disk::
|
||||
|
||||
>>> import hello
|
||||
>>> import pickle
|
||||
>>> a_world = hello.World("howdy")
|
||||
>>> pickle.dump(a_world, open("my_world", "w"))
|
||||
|
||||
In a potentially *different script* on a potentially *different
|
||||
computer* with a potentially *different operating system*::
|
||||
|
||||
>>> import pickle
|
||||
>>> resurrected_world = pickle.load(open("my_world", "r"))
|
||||
>>> resurrected_world.greet()
|
||||
'howdy'
|
||||
|
||||
Of course the ``cPickle`` module can also be used for faster
|
||||
processing.
|
||||
|
||||
Boost.Python's ``pickle_suite`` fully supports the ``pickle`` protocol
|
||||
defined in the standard Python documentation. There is a one-to-one
|
||||
correspondence between the standard pickling methods (``__getinitargs__``,
|
||||
``__getstate__``, ``__setstate__``) and the functions defined by the
|
||||
user in the class derived from ``pickle_suite`` (``getinitargs``,
|
||||
``getstate``, ``setstate``). The ``class_::def_pickle()`` member function
|
||||
is used to establish the Python bindings for all user-defined functions
|
||||
simultaneously. Correct signatures for these functions are enforced at
|
||||
compile time. Non-sensical combinations of the three pickle functions
|
||||
are also rejected at compile time. These measures are designed to
|
||||
help the user in avoiding obvious errors.
|
||||
|
||||
Enabling serialization of more complex C++ objects requires a little
|
||||
more work than is shown in the example above. Fortunately the
|
||||
``object`` interface (see next section) greatly helps in keeping the
|
||||
code manageable.
|
||||
|
||||
------------------
|
||||
Object interface
|
||||
------------------
|
||||
|
||||
Experienced extension module authors will be familiar with the 'C' view
|
||||
of Python objects, the ubiquitous ``PyObject*``. Most if not all Python
|
||||
'C' API functions involve ``PyObject*`` as arguments or return type. A
|
||||
major complication is the raw reference counting interface presented to
|
||||
the 'C' programmer. E.g. some API functions return *new references* and
|
||||
others return *borrowed references*. It is up to the extension module
|
||||
writer to properly increment and decrement reference counts. This
|
||||
quickly becomes cumbersome and error prone, especially if there are
|
||||
multiple execution paths.
|
||||
|
||||
Boost.Python provides a type ``object`` which is essentially a high
|
||||
level wrapper around ``PyObject*``. ``object`` automates reference
|
||||
counting as much as possible. It also provides the facilities for
|
||||
converting arbitrary C++ types to Python objects and vice versa.
|
||||
This significantly reduces the learning effort for prospective
|
||||
extension module writers.
|
||||
|
||||
Creating an ``object`` from any other type is extremely simple::
|
||||
|
||||
object o(3);
|
||||
|
||||
``object`` has templated interactions with all other types, with
|
||||
automatic to-python conversions. It happens so naturally that it's
|
||||
easily overlooked.
|
||||
|
||||
The ``extract<T>`` class template can be used to convert Python objects
|
||||
to C++ types::
|
||||
|
||||
double x = extract<double>(o);
|
||||
|
||||
All registered user-defined conversions are automatically accessible
|
||||
through the ``object`` interface. With reference to the ``World`` class
|
||||
defined in previous examples::
|
||||
|
||||
object as_python_object(World("howdy"));
|
||||
World back_as_c_plus_plus_object = extract<World>(as_python_object);
|
||||
|
||||
If a C++ type cannot be converted to a Python object an appropriate
|
||||
exception is thrown at runtime. Similarly, an appropriate exception is
|
||||
thrown if a C++ type cannot be extracted from a Python object.
|
||||
``extract<T>`` provides facilities for avoiding exceptions if this is
|
||||
desired.
|
||||
|
||||
The ``object::attr()`` member function is available for accessing
|
||||
and manipulating attributes of Python objects. For example::
|
||||
|
||||
object planet(World());
|
||||
planet.attr("set")("howdy");
|
||||
|
||||
``planet.attr("set")`` returns a callable ``object``. ``"howdy"`` is
|
||||
converted to a Python string object which is then passed as an argument
|
||||
to the ``set`` method.
|
||||
|
||||
The ``object`` type is accompanied by a set of derived types
|
||||
that mirror the Python built-in types such as ``list``, ``dict``,
|
||||
``tuple``, etc. as much as possible. This enables convenient
|
||||
manipulation of these high-level types from C++::
|
||||
|
||||
dict d;
|
||||
d["some"] = "thing";
|
||||
d["lucky_number"] = 13;
|
||||
list l = d.keys();
|
||||
|
||||
This almost looks and works like regular Python code, but it is pure C++.
|
||||
|
||||
=================
|
||||
Thinking hybrid
|
||||
=================
|
||||
|
||||
For many applications runtime performance considerations are very
|
||||
important. This is particularly true for most scientific applications.
|
||||
Often the performance considerations dictate the use of a compiled
|
||||
language for the core algorithms. Traditionally the decision to use a
|
||||
particular programming language is an exclusive one. Because of the
|
||||
practical and mental difficulties of combining different languages many
|
||||
systems are written in just one language. This is quite unfortunate
|
||||
because the price payed for runtime performance is typically a
|
||||
significant overhead due to static typing. For example, our experience
|
||||
shows that developing maintainable C++ code is typically much more
|
||||
time-consuming and requires much more hard-earned working experience
|
||||
than developing useful Python code. A related observation is that many
|
||||
compiled packages are augmented by some type of rudimentary scripting
|
||||
layer. These ad hoc solutions clearly show that many times a compiled
|
||||
language alone does not get the job done. On the other hand it is also
|
||||
clear that a pure Python implementation is too slow for numerically
|
||||
intensive production code.
|
||||
|
||||
Boost.Python enables us to *think hybrid* when developing new
|
||||
applications. Python can be used for rapidly prototyping a
|
||||
new application. Python's ease of use and the large pool of standard
|
||||
libraries give us a head start on the way to a first working system. If
|
||||
necessary, the working procedure can be used to discover the
|
||||
rate-limiting algorithms. To maximize performance these can be
|
||||
reimplemented in C++, together with the Boost.Python bindings needed to
|
||||
tie them back into the existing higher-level procedure.
|
||||
|
||||
Of course, this *top-down* approach is less attractive if it is clear
|
||||
from the start that many algorithms will eventually have to be
|
||||
implemented in a compiled language. Fortunately Boost.Python also
|
||||
enables us to pursue a *bottom-up* approach. We have used this approach
|
||||
very successfully in the development of a toolbox for scientific
|
||||
applications (scitbx) that we will describe elsewhere. The toolbox
|
||||
started out mainly as a library of C++ classes with Boost.Python
|
||||
bindings, and for a while the growth was mainly concentrated on the C++
|
||||
parts. However, as the toolbox is becoming more complete, more and more
|
||||
newly added functionality can be implemented in Python. We expect this
|
||||
trend to continue, as illustrated qualitatively in this figure:
|
||||
|
||||
.. image:: python_cpp_mix.png
|
||||
|
||||
This figure shows the ratio of newly added C++ and Python code over
|
||||
time as new algorithms are implemented. We expect this ratio to level
|
||||
out near 70% Python. The increasing ability to solve new problems
|
||||
mostly with the easy-to-use Python language rather than a necessarily
|
||||
more arcane statically typed language is the return on the investment
|
||||
of learning how to use Boost.Python. The ability to solve some problems
|
||||
entirely using only Python will enable a larger group of people to
|
||||
participate in the rapid development of new applications.
|
||||
|
||||
=============
|
||||
Conclusions
|
||||
=============
|
||||
|
||||
The examples in this paper illustrate that Boost.Python enables
|
||||
seamless interoperability between C++ and Python. Importantly, this is
|
||||
achieved without introducing a third syntax: the Python/C++ interface
|
||||
definitions are written in pure C++. This avoids any problems with
|
||||
parsing the C++ code to be interfaced to Python, yet the interface
|
||||
definitions are concise and maintainable. Freed from most of the
|
||||
development-time penalties of crossing a language boundary, software
|
||||
designers can take full advantage of two rich and complimentary
|
||||
language environments. In practice it turns out that some things are
|
||||
very difficult to do with pure Python/C (e.g. an efficient array
|
||||
library with an intuitive interface in the compiled language) and
|
||||
others are very difficult to do with pure C++ (e.g. serialization).
|
||||
If one has the luxury of being able to design a software system as a
|
||||
hybrid system from the ground up there are many new ways of avoiding
|
||||
road blocks in one language or the other.
|
||||
|
||||
.. I'm not ready to give up on all of this quite yet
|
||||
|
||||
.. Perhaps one day we'll have a language with the simplicity and
|
||||
expressive power of Python and the compile-time muscle of C++. Being
|
||||
able to take advantage of all of these facilities without paying the
|
||||
mental and development-time penalties of crossing a language barrier
|
||||
would bring enormous benefits. Until then, interoperability tools
|
||||
like Boost.Python can help lower the barrier and make the benefits of
|
||||
both languages more accessible to both communities.
|
||||
|
||||
===========
|
||||
Footnotes
|
||||
===========
|
||||
|
||||
.. [#mi] For hard-core new-style class/extension module writers it is
|
||||
worth noting that the normal requirement that all extension classes
|
||||
with data form a layout-compatible single-inheritance chain is
|
||||
lifted for Boost.Python extension classes. Clearly, either
|
||||
``Base1`` or ``Base2`` has to occupy a different offset in the
|
||||
``Derived`` class instance. This is possible because the wrapped
|
||||
part of BPL extension class instances is never assumed to have a
|
||||
fixed offset within the wrapper.
|
||||
|
||||
===========
|
||||
Citations
|
||||
===========
|
||||
|
||||
.. [VELD1995] T. Veldhuizen, "Expression Templates," C++ Report,
|
||||
Vol. 7 No. 5 June 1995, pp. 26-31.
|
||||
http://osl.iu.edu/~tveldhui/papers/Expression-Templates/exprtmpl.html
|
||||
106
doc/index.html
106
doc/index.html
@@ -1,106 +0,0 @@
|
||||
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
||||
|
||||
<html>
|
||||
<head>
|
||||
<meta name="generator" content=
|
||||
"HTML Tidy for Windows (vers 1st August 2002), see www.w3.org">
|
||||
<meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
|
||||
<link rel="stylesheet" type="text/css" href="boost.css">
|
||||
|
||||
<title>Boost.Python</title>
|
||||
</head>
|
||||
|
||||
<body link="#0000ff" vlink="#800080">
|
||||
<table border="0" cellpadding="7" cellspacing="0" width="100%" summary=
|
||||
"header">
|
||||
<tr>
|
||||
<td valign="top" width="300">
|
||||
<h3><a href="../../../index.htm"><img height="86" width="277" alt=
|
||||
"C++ Boost" src="../../../c++boost.gif" border="0"></a></h3>
|
||||
</td>
|
||||
|
||||
<td valign="top">
|
||||
<h1 align="center">Boost.Python</h1>
|
||||
|
||||
<h2 align="center">Index</h2>
|
||||
</td>
|
||||
</tr>
|
||||
</table>
|
||||
<hr>
|
||||
|
||||
<h2>Synopsis</h2>
|
||||
Welcome to version 2 of <b>Boost.Python</b>, a C++ library which enables
|
||||
seamless interoperability between C++ and the <a href=
|
||||
"http://www.python.org">Python</a> programming language. The new version
|
||||
has been rewritten from the ground up, with a more convenient and
|
||||
flexible interface, and many new capabilities, including support for:
|
||||
|
||||
<ul>
|
||||
<li>References and Pointers</li>
|
||||
|
||||
<li>Globally Registered Type Coercions</li>
|
||||
|
||||
<li>Automatic Cross-Module Type Conversions</li>
|
||||
|
||||
<li>Efficient Function Overloading</li>
|
||||
|
||||
<li>C++ to Python Exception Translation</li>
|
||||
|
||||
<li>Default Arguments</li>
|
||||
|
||||
<li>Keyword Arguments</li>
|
||||
|
||||
<li>Manipulating Python objects in C++</li>
|
||||
|
||||
<li>Exporting C++ Iterators as Python Iterators</li>
|
||||
|
||||
<li>Documentation Strings</li>
|
||||
</ul>
|
||||
The development of these features was funded in part by grants to <a
|
||||
href="http://www.boost-consulting.com">Boost Consulting</a> from the <a
|
||||
href="http://www.llnl.gov/">Lawrence Livermore National Laboratories</a>
|
||||
and by the <a href="http://cci.lbl.gov/">Computational Crystallography
|
||||
Initiative</a> at Lawrence Berkeley National Laboratories.
|
||||
<hr>
|
||||
|
||||
<h2>Contents</h2>
|
||||
|
||||
<dl class="index">
|
||||
<dt><a href="tutorial/index.html">Tutorial Introduction</a></dt>
|
||||
|
||||
<dt><a href="building.html">Building and Testing</a></dt>
|
||||
|
||||
<dt><a href="v2/reference.html">Reference Manual</a></dt>
|
||||
|
||||
<dt><a href="v2/configuration.html">Configuration Information</a></dt>
|
||||
|
||||
<dt><a href="v2/platforms.html">Known Working Platforms and
|
||||
Compilers</a></dt>
|
||||
|
||||
<dt><a href="v2/definitions.html">Definitions</a></dt>
|
||||
|
||||
<dt><a href="projects.html">Projects using Boost.Python</a></dt>
|
||||
|
||||
<dt><a href="support.html">Support Resources</a></dt>
|
||||
|
||||
<dt><a href="v2/faq.html">Frequently Asked Questions (FAQs)</a></dt>
|
||||
|
||||
<dt><a href="news.html">News/Change Log</a></dt>
|
||||
|
||||
<dt><a href="v2/progress_reports.html">LLNL Progress Reports</a></dt>
|
||||
|
||||
<dt><a href="v2/acknowledgments.html">Acknowledgments</a></dt>
|
||||
</dl>
|
||||
<hr>
|
||||
|
||||
<p>Revised
|
||||
<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan -->
|
||||
17 November, 2002
|
||||
<!--webbot bot="Timestamp" endspan i-checksum="39359" -->
|
||||
</p>
|
||||
|
||||
<p><i>© Copyright <a href="../../people/dave_abrahams.htm">Dave
|
||||
Abrahams</a> 2002. All Rights Reserved.</i></p>
|
||||
</body>
|
||||
</html>
|
||||
|
||||
Reference in New Issue
Block a user