Boost.Python is an open source C++ library which provides a concise
-IDL-like interface for binding C++ classes and functions to
-Python. Leveraging the full power of C++ compile-time introspection
-and of recently developed metaprogramming techniques, this is achieved
-entirely in pure C++, without introducing a new syntax.
-Boost.Python's rich set of features and high-level interface make it
-possible to engineer packages from the ground up as hybrid systems,
-giving programmers easy and coherent access to both the efficient
-compile-time polymorphism of C++ and the extremely convenient run-time
-polymorphism of Python.
Python and C++ are in many ways as different as two languages could
-be: while C++ is usually compiled to machine-code, Python is
-interpreted. Python's dynamic type system is often cited as the
-foundation of its flexibility, while in C++ static typing is the
-cornerstone of its efficiency. C++ has an intricate and difficult
-compile-time meta-language, while in Python, practically everything
-happens at runtime.
-
Yet for many programmers, these very differences mean that Python and
-C++ complement one another perfectly. Performance bottlenecks in
-Python programs can be rewritten in C++ for maximal speed, and
-authors of powerful C++ libraries choose Python as a middleware
-language for its flexible system integration capabilities.
-Furthermore, the surface differences mask some strong similarities:
-
-
'C'-family control structures (if, while, for...)
-
Support for object-orientation, functional programming, and generic
-programming (these are both multi-paradigm programming languages.)
-
Comprehensive operator overloading facilities, recognizing the
-importance of syntactic variability for readability and
-expressivity.
-
High-level concepts such as collections and iterators.
-
High-level encapsulation facilities (C++: namespaces, Python: modules)
-to support the design of re-usable libraries.
-
Exception-handling for effective management of error conditions.
-
C++ idioms in common use, such as handle/body classes and
-reference-counted smart pointers mirror Python reference semantics.
-
-
Given Python's rich 'C' interoperability API, it should in principle
-be possible to expose C++ type and function interfaces to Python with
-an analogous interface to their C++ counterparts. However, the
-facilities provided by Python alone for integration with C++ are
-relatively meager. Compared to C++ and Python, 'C' has only very
-rudimentary abstraction facilities, and support for exception-handling
-is completely missing. 'C' extension module writers are required to
-manually manage Python reference counts, which is both annoyingly
-tedious and extremely error-prone. Traditional extension modules also
-tend to contain a great deal of boilerplate code repetition which
-makes them difficult to maintain, especially when wrapping an evolving
-API.
-
These limitations have lead to the development of a variety of wrapping
-systems. SWIG is probably the most popular package for the
-integration of C/C++ and Python. A more recent development is SIP,
-which was specifically designed for interfacing Python with the Qt
-graphical user interface library. Both SWIG and SIP introduce their
-own specialized languages for customizing inter-language bindings.
-This has certain advantages, but having to deal with three different
-languages (Python, C/C++ and the interface language) also introduces
-practical and mental difficulties. The CXX package demonstrates an
-interesting alternative. It shows that at least some parts of
-Python's 'C' API can be wrapped and presented through a much more
-user-friendly C++ interface. However, unlike SWIG and SIP, CXX does
-not include support for wrapping C++ classes as new Python types.
-
The features and goals of Boost.Python overlap significantly with
-many of these other systems. That said, Boost.Python attempts to
-maximize convenience and flexibility without introducing a separate
-wrapping language. Instead, it presents the user with a high-level
-C++ interface for wrapping C++ classes and functions, managing much of
-the complexity behind-the-scenes with static metaprogramming.
-Boost.Python also goes beyond the scope of earlier systems by
-providing:
-
-
Support for C++ virtual functions that can be overridden in Python.
-
Comprehensive lifetime management facilities for low-level C++
-pointers and references.
-
Support for organizing extensions as Python packages,
-with a central registry for inter-language type conversions.
-
A safe and convenient mechanism for tying into Python's powerful
-serialization engine (pickle).
-
Coherence with the rules for handling C++ lvalues and rvalues that
-can only come from a deep understanding of both the Python and C++
-type systems.
-
-
The key insight that sparked the development of Boost.Python is that
-much of the boilerplate code in traditional extension modules could be
-eliminated using C++ compile-time introspection. Each argument of a
-wrapped C++ function must be extracted from a Python object using a
-procedure that depends on the argument type. Similarly the function's
-return type determines how the return value will be converted from C++
-to Python. Of course argument and return types are part of each
-function's type, and this is exactly the source from which
-Boost.Python deduces most of the information required.
-
This approach leads to user guided wrapping: as much information is
-extracted directly from the source code to be wrapped as is possible
-within the framework of pure C++, and some additional information is
-supplied explicitly by the user. Mostly the guidance is mechanical
-and little real intervention is required. Because the interface
-specification is written in the same full-featured language as the
-code being exposed, the user has unprecedented power available when
-she does need to take control.
The primary goal of Boost.Python is to allow users to expose C++
-classes and functions to Python using nothing more than a C++
-compiler. In broad strokes, the user experience should be one of
-directly manipulating C++ objects from Python.
-
However, it's also important not to translate all interfaces too
-literally: the idioms of each language must be respected. For
-example, though C++ and Python both have an iterator concept, they are
-expressed very differently. Boost.Python has to be able to bridge the
-interface gap.
-
It must be possible to insulate Python users from crashes resulting
-from trivial misuses of C++ interfaces, such as accessing
-already-deleted objects. By the same token the library should
-insulate C++ users from low-level Python 'C' API, replacing
-error-prone 'C' interfaces like manual reference-count management and
-raw PyObject pointers with more-robust alternatives.
-
Support for component-based development is crucial, so that C++ types
-exposed in one extension module can be passed to functions exposed in
-another without loss of crucial information like C++ inheritance
-relationships.
-
Finally, all wrapping must be non-intrusive, without modifying or
-even seeing the original C++ source code. Existing C++ libraries have
-to be wrappable by third parties who only have access to header files
-and binaries.
And now for a preview of Boost.Python, and how it improves on the raw
-facilities offered by Python. Here's a function we might want to
-expose:
-
-char const* greet(unsigned x)
-{
- static char const* const msgs[] = { "hello", "Boost.Python", "world!" };
-
- if (x > 2)
- throw std::range_error("greet: index out of range");
-
- return msgs[x];
-}
-
-
To wrap this function in standard C++ using the Python 'C' API, we'd
-need something like this:
-
-extern "C" // all Python interactions use 'C' linkage and calling convention
-{
- // Wrapper to handle argument/result conversion and checking
- PyObject* greet_wrap(PyObject* args, PyObject * keywords)
- {
- int x;
- if (PyArg_ParseTuple(args, "i", &x)) // extract/check arguments
- {
- char const* result = greet(x); // invoke wrapped function
- return PyString_FromString(result); // convert result to Python
- }
- return 0; // error occurred
- }
-
- // Table of wrapped functions to be exposed by the module
- static PyMethodDef methods[] = {
- { "greet", greet_wrap, METH_VARARGS, "return one of 3 parts of a greeting" }
- , { NULL, NULL, 0, NULL } // sentinel
- };
-
- // module initialization function
- DL_EXPORT init_hello()
- {
- (void) Py_InitModule("hello", methods); // add the methods to the module
- }
-}
-
-
Now here's the wrapping code we'd use to expose it with Boost.Python:
-
-#include <boost/python.hpp>
-using namespace boost::python;
-BOOST_PYTHON_MODULE(hello)
-{
- def("greet", greet, "return one of 3 parts of a greeting");
-}
-
-
and here it is in action:
-
->>> import hello
->>> for x in range(3):
-... print hello.greet(x)
-...
-hello
-Boost.Python
-world!
-
-
Aside from the fact that the 'C' API version is much more verbose,
-it's worth noting a few things that it doesn't handle correctly:
-
-
The original function accepts an unsigned integer, and the Python
-'C' API only gives us a way of extracting signed integers. The
-Boost.Python version will raise a Python exception if we try to pass
-a negative number to hello.greet, but the other one will proceed
-to do whatever the C++ implementation does when converting an
-negative integer to unsigned (usually wrapping to some very large
-number), and pass the incorrect translation on to the wrapped
-function.
-
That brings us to the second problem: if the C++ greet()
-function is called with a number greater than 2, it will throw an
-exception. Typically, if a C++ exception propagates across the
-boundary with code generated by a 'C' compiler, it will cause a
-crash. As you can see in the first version, there's no C++
-scaffolding there to prevent this from happening. Functions wrapped
-by Boost.Python automatically include an exception-handling layer
-which protects Python users by translating unhandled C++ exceptions
-into a corresponding Python exception.
-
A slightly more-subtle limitation is that the argument conversion
-used in the Python 'C' API case can only get that integer x in
-one way. PyArg_ParseTuple can't convert Python long objects
-(arbitrary-precision integers) which happen to fit in an unsigned
-int but not in a signedlong, nor will it ever handle a
-wrapped C++ class with a user-defined implicit operatorunsigned
-int() conversion. Boost.Python's dynamic type conversion
-registry allows users to add arbitrary conversion methods.
Although this code has a certain pythonic familiarity, people
-sometimes find the syntax bit confusing because it doesn't look like
-most of the C++ code they're used to. All the same, this is just
-standard C++. Because of their flexible syntax and operator
-overloading, C++ and Python are great for defining domain-specific
-(sub)languages
-(DSLs), and that's what we've done in Boost.Python. To break it down:
-
-class_<World>("World")
-
-
constructs an unnamed object of type class_<World> and passes
-"World" to its constructor. This creates a new-style Python class
-called World in the extension module, and associates it with the
-C++ type World in the Boost.Python type conversion registry. We
-might have also written:
-
-class_<World> w("World");
-
-
but that would've been more verbose, since we'd have to name w
-again to invoke its def() member function:
-
-w.def("greet", &World::greet)
-
-
There's nothing special about the location of the dot for member
-access in the original example: C++ allows any amount of whitespace on
-either side of a token, and placing the dot at the beginning of each
-line allows us to chain as many successive calls to member functions
-as we like with a uniform syntax. The other key fact that allows
-chaining is that class_<> member functions all return a reference
-to *this.
It's occasionally useful to be able to break down the components of a
-Boost.Python class wrapper in this way, but the rest of this article
-will stick to the terse syntax.
-
For completeness, here's the wrapped class in use:
Since our World class is just a plain struct, it has an
-implicit no-argument (nullary) constructor. Boost.Python exposes the
-nullary constructor by default, which is why we were able to write:
-
->>> planet = hello.World()
-
-
However, well-designed classes in any language may require constructor
-arguments in order to establish their invariants. Unlike Python,
-where __init__ is just a specially-named method, In C++
-constructors cannot be handled like ordinary member functions. In
-particular, we can't take their address: &World::World is an
-error. The library provides a different interface for specifying
-constructors. Given:
This does not result in adding attributes to the World instance
-__dict__, which can result in substantial memory savings when
-wrapping large data structures. In fact, no instance __dict__
-will be created at all unless attributes are explicitly added from
-Python. Boost.Python owes this capability to the new Python 2.2 type
-system, in particular the descriptor interface and property type.
-
In C++, publicly-accessible data members are considered a sign of poor
-design because they break encapsulation, and style guides usually
-dictate the use of "getter" and "setter" functions instead. In
-Python, however, __getattr__, __setattr__, and since 2.2,
-property mean that attribute access is just one more
-well-encapsulated syntactic tool at the programmer's disposal.
-Boost.Python bridges this idiomatic gap by making Python property
-creation directly available to users. If msg were private, we
-could still expose it as attribute in Python as follows:
The ability to write arithmetic operators for user-defined types has
-been a major factor in the success of both languages for numerical
-computation, and the success of packages like NumPy attests to the
-power of exposing operators in extension modules. Boost.Python
-provides a concise mechanism for wrapping operator overloads. The
-example below shows a fragment from a wrapper for the Boost rational
-number library:
The magic is performed using a simplified application of "expression
-templates" [VELD1995], a technique originally developed for
-optimization of high-performance matrix algebra expressions. The
-essence is that instead of performing the computation immediately,
-operators are overloaded to construct a type representing the
-computation. In matrix algebra, dramatic optimizations are often
-available when the structure of an entire expression can be taken into
-account, rather than evaluating each operation "greedily".
-Boost.Python uses the same technique to build an appropriate Python
-method object based on expressions involving self.
C++ inheritance relationships can be represented to Boost.Python by adding
-an optional bases<...> argument to the class_<...> template
-parameter list as follows:
When the class_<...> is created, Python type objects
-corresponding to Base1 and Base2 are looked up in
-Boost.Python's registry, and are used as bases for the new Python
-Derived type object, so methods exposed for the Python Base1
-and Base2 types are automatically members of the Derived
-type. Because the registry is global, this works correctly even if
-Derived is exposed in a different module from either of its
-bases.
-
C++ conversions from Derived to its bases are added to the
-Boost.Python registry. Thus wrapped C++ methods expecting (a
-pointer or reference to) an object of either base type can be
-called with an object wrapping a Derived instance. Wrapped
-member functions of class T are treated as though they have an
-implicit first argument of T&, so these conversions are
-neccessary to allow the base class methods to be called for derived
-objects.
-
-
Of course it's possible to derive new Python classes from wrapped C++
-class instances. Because Boost.Python uses the new-style class
-system, that works very much as for the Python built-in types. There
-is one significant detail in which it differs: the built-in types
-generally establish their invariants in their __new__ function, so
-that derived classes do not need to call __init__ on the base
-class before invoking its methods :
Because C++ object construction is a one-step operation, C++ instance
-data cannot be constructed until the arguments are available, in the
-__init__ function:
-
->>> class D(SomeBoostPythonClass):
-... def __init__(self):
-... pass
-...
->>> D().some_boost_python_method()
-Traceback (most recent call last):
- File "<stdin>", line 1, in ?
-TypeError: bad argument type for built-in operation
-
-
This happened because Boost.Python couldn't find instance data of type
-SomeBoostPythonClass within the D instance; D's __init__
-function masked construction of the base class. It could be corrected
-by either removing D's __init__ function or having it call
-SomeBoostPythonClass.__init__(...) explicitly.
Deriving new types in Python from extension classes is not very
-interesting unless they can be used polymorphically from C++. In
-other words, Python method implementations should appear to override
-the implementation of C++ virtual functions when called through base
-class pointers/references from C++. Since the only way to alter the
-behavior of a virtual function is to override it in a derived class,
-the user must build a special derived class to dispatch a polymorphic
-class' virtual functions:
-
-//
-// interface to wrap:
-//
-class Base
-{
- public:
- virtual int f(std::string x) { return 42; }
- virtual ~Base();
-};
-
-int calls_f(Base const& b, std::string x) { return b.f(x); }
-
-//
-// Wrapping Code
-//
-
-// Dispatcher class
-struct BaseWrap : Base
-{
- // Store a pointer to the Python object
- BaseWrap(PyObject* self_) : self(self_) {}
- PyObject* self;
-
- // Default implementation, for when f is not overridden
- int f_default(std::string x) { return this->Base::f(x); }
- // Dispatch implementation
- int f(std::string x) { return call_method<int>(self, "f", x); }
-};
-
-...
- def("calls_f", calls_f);
- class_<Base, BaseWrap>("Base")
- .def("f", &Base::f, &BaseWrap::f_default)
- ;
-
The key element which allows overriding in Python is the
-call_method invocation, which uses the same global type
-conversion registry as the C++ function wrapping does to convert its
-arguments from C++ to Python and its return type from Python to C++.
-
Any constructor signatures you wish to wrap must be replicated with
-an initial PyObject* argument
-
The dispatcher must store this argument so that it can be used to
-invoke call_method
-
The f_default member function is needed when the function being
-exposed is not pure virtual; there's no other way Base::f can be
-called on an object of type BaseWrap, since it overrides f.
Admittedly, this formula is tedious to repeat, especially on a project
-with many polymorphic classes. That it is neccessary reflects some
-limitations in C++'s compile-time introspection capabilities: there's
-no way to enumerate the members of a class and find out which are
-virtual functions. At least one very promising project has been
-started to write a front-end which can generate these dispatchers (and
-other wrapping code) automatically from C++ headers.
-
Pyste is being developed by Bruno da Silva de Oliveira. It builds on
-GCC_XML, which generates an XML version of GCC's internal program
-representation. Since GCC is a highly-conformant C++ compiler, this
-ensures correct handling of the most-sophisticated template code and
-full access to the underlying type system. In keeping with the
-Boost.Python philosophy, a Pyste interface description is neither
-intrusive on the code being wrapped, nor expressed in some unfamiliar
-language: instead it is a 100% pure Python script. If Pyste is
-successful it will mark a move away from wrapping everything directly
-in C++ for many of our users. It will also allow us the choice to
-shift some of the metaprogram code from C++ to Python. We expect that
-soon, not only our users but the Boost.Python developers themselves
-will be "thinking hybrid" about their own code.
Serialization is the process of converting objects in memory to a
-form that can be stored on disk or sent over a network connection. The
-serialized object (most often a plain string) can be retrieved and
-converted back to the original object. A good serialization system will
-automatically convert entire object hierarchies. Python's standard
-pickle module is just such a system. It leverages the language's strong
-runtime introspection facilities for serializing practically arbitrary
-user-defined objects. With a few simple and unintrusive provisions this
-powerful machinery can be extended to also work for wrapped C++ objects.
-Here is an example:
Of course the cPickle module can also be used for faster
-processing.
-
Boost.Python's pickle_suite fully supports the pickle protocol
-defined in the standard Python documentation. Like a __getinitargs__
-function in Python, the pickle_suite's getinitargs() is responsible for
-creating the argument tuple that will be use to reconstruct the pickled
-object. The other elements of the Python pickling protocol,
-__getstate__ and __setstate__ can be optionally provided via C++
-getstate and setstate functions. C++'s static type system allows the
-library to ensure at compile-time that nonsensical combinations of
-functions (e.g. getstate without setstate) are not used.
-
Enabling serialization of more complex C++ objects requires a little
-more work than is shown in the example above. Fortunately the
-object interface (see next section) greatly helps in keeping the
-code manageable.
Experienced 'C' language extension module authors will be familiar
-with the ubiquitous PyObject*, manual reference-counting, and the
-need to remember which API calls return "new" (owned) references or
-"borrowed" (raw) references. These constraints are not just
-cumbersome but also a major source of errors, especially in the
-presence of exceptions.
-
Boost.Python provides a class object which automates reference
-counting and provides conversion to Python from C++ objects of
-arbitrary type. This significantly reduces the learning effort for
-prospective extension module writers.
-
Creating an object from any other type is extremely simple:
-
-object s("hello, world"); // s manages a Python string
-
-
object has templated interactions with all other types, with
-automatic to-python conversions. It happens so naturally that it's
-easily overlooked:
-
-object ten_Os = 10 * s[4]; // -> "oooooooooo"
-
-
In the example above, 4 and 10 are converted to Python objects
-before the indexing and multiplication operations are invoked.
-
The extract<T> class template can be used to convert Python objects
-to C++ types:
-
-double x = extract<double>(o);
-
-
If a conversion in either direction cannot be performed, an
-appropriate exception is thrown at runtime.
-
The object type is accompanied by a set of derived types
-that mirror the Python built-in types such as list, dict,
-tuple, etc. as much as possible. This enables convenient
-manipulation of these high-level types from C++:
This almost looks and works like regular Python code, but it is pure
-C++. Of course we can wrap C++ functions which accept or return
-object instances.
Because of the practical and mental difficulties of combining
-programming languages, it is common to settle a single language at the
-outset of any development effort. For many applications, performance
-considerations dictate the use of a compiled language for the core
-algorithms. Unfortunately, due to the complexity of the static type
-system, the price we pay for runtime performance is often a
-significant increase in development time. Experience shows that
-writing maintainable C++ code usually takes longer and requires far
-more hard-earned working experience than developing comparable Python
-code. Even when developers are comfortable working exclusively in
-compiled languages, they often augment their systems by some type of
-ad hoc scripting layer for the benefit of their users without ever
-availing themselves of the same advantages.
-
Boost.Python enables us to think hybrid. Python can be used for
-rapidly prototyping a new application; its ease of use and the large
-pool of standard libraries give us a head start on the way to a
-working system. If necessary, the working code can be used to
-discover rate-limiting hotspots. To maximize performance these can
-be reimplemented in C++, together with the Boost.Python bindings
-needed to tie them back into the existing higher-level procedure.
-
Of course, this top-down approach is less attractive if it is clear
-from the start that many algorithms will eventually have to be
-implemented in C++. Fortunately Boost.Python also enables us to
-pursue a bottom-up approach. We have used this approach very
-successfully in the development of a toolbox for scientific
-applications. The toolbox started out mainly as a library of C++
-classes with Boost.Python bindings, and for a while the growth was
-mainly concentrated on the C++ parts. However, as the toolbox is
-becoming more complete, more and more newly added functionality can be
-implemented in Python.
-
-
This figure shows the estimated ratio of newly added C++ and Python
-code over time as new algorithms are implemented. We expect this
-ratio to level out near 70% Python. Being able to solve new problems
-mostly in Python rather than a more difficult statically typed
-language is the return on our investment in Boost.Python. The ability
-to access all of our code from Python allows a broader group of
-developers to use it in the rapid development of new applications.
The first version of Boost.Python was developed in 2000 by Dave
-Abrahams at Dragon Systems, where he was privileged to have Tim Peters
-as a guide to "The Zen of Python". One of Dave's jobs was to develop
-a Python-based natural language processing system. Since it was
-eventually going to be targeting embedded hardware, it was always
-assumed that the compute-intensive core would be rewritten in C++ to
-optimize speed and memory footprint 1. The project also wanted to
-test all of its C++ code using Python test scripts 2. The only
-tool we knew of for binding C++ and Python was SWIG, and at the time
-its handling of C++ was weak. It would be false to claim any deep
-insight into the possible advantages of Boost.Python's approach at
-this point. Dave's interest and expertise in fancy C++ template
-tricks had just reached the point where he could do some real damage,
-and Boost.Python emerged as it did because it filled a need and
-because it seemed like a cool thing to try.
-
This early version was aimed at many of the same basic goals we've
-described in this paper, differing most-noticeably by having a
-slightly more cumbersome syntax and by lack of special support for
-operator overloading, pickling, and component-based development.
-These last three features were quickly added by Ullrich Koethe and
-Ralf Grosse-Kunstleve 3, and other enthusiastic contributors arrived
-on the scene to contribute enhancements like support for nested
-modules and static member functions.
-
By early 2001 development had stabilized and few new features were
-being added, however a disturbing new fact came to light: Ralf had
-begun testing Boost.Python on pre-release versions of a compiler using
-the EDG front-end, and the mechanism at the core of Boost.Python
-responsible for handling conversions between Python and C++ types was
-failing to compile. As it turned out, we had been exploiting a very
-common bug in the implementation of all the C++ compilers we had
-tested. We knew that as C++ compilers rapidly became more
-standards-compliant, the library would begin failing on more
-platforms. Unfortunately, because the mechanism was so central to the
-functioning of the library, fixing the problem looked very difficult.
-
Fortunately, later that year Lawrence Berkeley and later Lawrence
-Livermore National labs contracted with Boost Consulting for support
-and development of Boost.Python, and there was a new opportunity to
-address fundamental issues and ensure a future for the library. A
-redesign effort began with the low level type conversion architecture,
-building in standards-compliance and support for component-based
-development (in contrast to version 1 where conversions had to be
-explicitly imported and exported across module boundaries). A new
-analysis of the relationship between the Python and C++ objects was
-done, resulting in more intuitive handling for C++ lvalues and
-rvalues.
-
The emergence of a powerful new type system in Python 2.2 made the
-choice of whether to maintain compatibility with Python 1.5.2 easy:
-the opportunity to throw away a great deal of elaborate code for
-emulating classic Python classes alone was too good to pass up. In
-addition, Python iterators and descriptors provided crucial and
-elegant tools for representing similar C++ constructs. The
-development of the generalized object interface allowed us to
-further shield C++ programmers from the dangers and syntactic burdens
-of the Python 'C' API. A great number of other features including C++
-exception translation, improved support for overloaded functions, and
-most significantly, CallPolicies for handling pointers and
-references, were added during this period.
-
In October 2002, version 2 of Boost.Python was released. Development
-since then has concentrated on improved support for C++ runtime
-polymorphism and smart pointers. Peter Dimov's ingenious
-boost::shared_ptr design in particular has allowed us to give the
-hybrid developer a consistent interface for moving objects back and
-forth across the language barrier without loss of information. At
-first, we were concerned that the sophistication and complexity of the
-Boost.Python v2 implementation might discourage contributors, but the
-emergence of Pyste and several other significant feature
-contributions have laid those fears to rest. Daily questions on the
-Python C++-sig and a backlog of desired improvements show that the
-library is getting used. To us, the future looks bright.
Boost.Python achieves seamless interoperability between two rich and
-complimentary language environments. Because it leverages template
-metaprogramming to introspect about types and functions, the user
-never has to learn a third syntax: the interface definitions are
-written in concise and maintainable C++. Also, the wrapping system
-doesn't have to parse C++ headers or represent the type system: the
-compiler does that work for us.
-
Computationally intensive tasks play to the strengths of C++ and are
-often impossible to implement efficiently in pure Python, while jobs
-like serialization that are trivial in Python can be very difficult in
-pure C++. Given the luxury of building a hybrid software system from
-the ground up, we can approach design with new confidence and power.
In retrospect, it seems that "thinking hybrid" from the
-ground up might have been better for the NLP system: the
-natural component boundaries defined by the pure python
-prototype turned out to be inappropriate for getting the
-desired performance and memory footprint out of the C++ core,
-which eventually caused some redesign overhead on the Python
-side when the core was moved to C++.
We also have some reservations about driving all C++
-testing through a Python interface, unless that's the only way
-it will be ultimately used. Any transition across language
-boundaries with such different object models can inevitably
-mask bugs.
These features were expressed very differently in v1 of
-Boost.Python
-
-
-
-
-
-
-
-
diff --git a/doc/PyConDC_2003/bpl.pdf b/doc/PyConDC_2003/bpl.pdf
deleted file mode 100755
index 477ff4e4..00000000
Binary files a/doc/PyConDC_2003/bpl.pdf and /dev/null differ
diff --git a/doc/PyConDC_2003/bpl.txt b/doc/PyConDC_2003/bpl.txt
index c2ee19aa..8113f50e 100644
--- a/doc/PyConDC_2003/bpl.txt
+++ b/doc/PyConDC_2003/bpl.txt
@@ -9,10 +9,12 @@
:Author: Ralf W. Grosse-Kunstleve
+:status: Draft
:copyright: Copyright David Abrahams and Ralf W. Grosse-Kunstleve 2003. All rights reserved
.. contents:: Table of Contents
+
.. _`Boost Consulting`: http://www.boost-consulting.com
==========
@@ -629,22 +631,19 @@ virtual functions. At least one very promising project has been
started to write a front-end which can generate these dispatchers (and
other wrapping code) automatically from C++ headers.
-Pyste_ is being developed by Bruno da Silva de Oliveira. It builds on
-GCC_XML_, which generates an XML version of GCC's internal program
-representation. Since GCC is a highly-conformant C++ compiler, this
-ensures correct handling of the most-sophisticated template code and
-full access to the underlying type system. In keeping with the
-Boost.Python philosophy, a Pyste interface description is neither
-intrusive on the code being wrapped, nor expressed in some unfamiliar
-language: instead it is a 100% pure Python script. If Pyste is
-successful it will mark a move away from wrapping everything directly
-in C++ for many of our users. It will also allow us the choice to
-shift some of the metaprogram code from C++ to Python. We expect that
-soon, not only our users but the Boost.Python developers themselves
-will be "thinking hybrid" about their own code.
+Pyste builds on GCC_XML_, which generates an XML version of GCC's
+internal program representation. Since GCC is a highly-conformant C++
+compiler, this ensures correct handling of the most-sophisticated
+template code and full access to the underlying type system. In
+keeping with the Boost.Python philosophy, a Pyste interface
+description is neither intrusive on the code being wrapped, nor
+expressed in some unfamiliar language: instead it is a 100% pure
+Python script. If Pyste is successful it will mark a move away from
+wrapping everything directly in C++ for many of our users. We expect
+that soon, not only our users but the Boost.Python developers
+themselves will be "thinking hybrid" about their own code.
.. _`GCC_XML`: http://www.gccxml.org/HTML/Index.html
-.. _`Pyste`: http://www.boost.org/libs/python/pyste
---------------
Serialization
@@ -771,6 +770,15 @@ This almost looks and works like regular Python code, but it is pure
C++. Of course we can wrap C++ functions which accept or return
``object`` instances.
+.. =====================
+ Development history
+ =====================
+
+ XXX Outline of development history to illustrate that the
+ library is mature. XXX
+
+ This can be postponed for the PyConDC paper
+
=================
Thinking hybrid
=================
@@ -808,7 +816,7 @@ mainly concentrated on the C++ parts. However, as the toolbox is
becoming more complete, more and more newly added functionality can be
implemented in Python.
-.. image:: python_cpp_mix.jpg
+.. image:: python_cpp_mix.png
This figure shows the estimated ratio of newly added C++ and Python
code over time as new algorithms are implemented. We expect this
@@ -818,87 +826,6 @@ language is the return on our investment in Boost.Python. The ability
to access all of our code from Python allows a broader group of
developers to use it in the rapid development of new applications.
-=====================
- Development history
-=====================
-
-The first version of Boost.Python was developed in 2000 by Dave
-Abrahams at Dragon Systems, where he was privileged to have Tim Peters
-as a guide to "The Zen of Python". One of Dave's jobs was to develop
-a Python-based natural language processing system. Since it was
-eventually going to be targeting embedded hardware, it was always
-assumed that the compute-intensive core would be rewritten in C++ to
-optimize speed and memory footprint [#proto]_. The project also wanted to
-test all of its C++ code using Python test scripts [#test]_. The only
-tool we knew of for binding C++ and Python was SWIG_, and at the time
-its handling of C++ was weak. It would be false to claim any deep
-insight into the possible advantages of Boost.Python's approach at
-this point. Dave's interest and expertise in fancy C++ template
-tricks had just reached the point where he could do some real damage,
-and Boost.Python emerged as it did because it filled a need and
-because it seemed like a cool thing to try.
-
-This early version was aimed at many of the same basic goals we've
-described in this paper, differing most-noticeably by having a
-slightly more cumbersome syntax and by lack of special support for
-operator overloading, pickling, and component-based development.
-These last three features were quickly added by Ullrich Koethe and
-Ralf Grosse-Kunstleve [#feature]_, and other enthusiastic contributors arrived
-on the scene to contribute enhancements like support for nested
-modules and static member functions.
-
-By early 2001 development had stabilized and few new features were
-being added, however a disturbing new fact came to light: Ralf had
-begun testing Boost.Python on pre-release versions of a compiler using
-the EDG_ front-end, and the mechanism at the core of Boost.Python
-responsible for handling conversions between Python and C++ types was
-failing to compile. As it turned out, we had been exploiting a very
-common bug in the implementation of all the C++ compilers we had
-tested. We knew that as C++ compilers rapidly became more
-standards-compliant, the library would begin failing on more
-platforms. Unfortunately, because the mechanism was so central to the
-functioning of the library, fixing the problem looked very difficult.
-
-Fortunately, later that year Lawrence Berkeley and later Lawrence
-Livermore National labs contracted with `Boost Consulting`_ for support
-and development of Boost.Python, and there was a new opportunity to
-address fundamental issues and ensure a future for the library. A
-redesign effort began with the low level type conversion architecture,
-building in standards-compliance and support for component-based
-development (in contrast to version 1 where conversions had to be
-explicitly imported and exported across module boundaries). A new
-analysis of the relationship between the Python and C++ objects was
-done, resulting in more intuitive handling for C++ lvalues and
-rvalues.
-
-The emergence of a powerful new type system in Python 2.2 made the
-choice of whether to maintain compatibility with Python 1.5.2 easy:
-the opportunity to throw away a great deal of elaborate code for
-emulating classic Python classes alone was too good to pass up. In
-addition, Python iterators and descriptors provided crucial and
-elegant tools for representing similar C++ constructs. The
-development of the generalized ``object`` interface allowed us to
-further shield C++ programmers from the dangers and syntactic burdens
-of the Python 'C' API. A great number of other features including C++
-exception translation, improved support for overloaded functions, and
-most significantly, CallPolicies for handling pointers and
-references, were added during this period.
-
-In October 2002, version 2 of Boost.Python was released. Development
-since then has concentrated on improved support for C++ runtime
-polymorphism and smart pointers. Peter Dimov's ingenious
-``boost::shared_ptr`` design in particular has allowed us to give the
-hybrid developer a consistent interface for moving objects back and
-forth across the language barrier without loss of information. At
-first, we were concerned that the sophistication and complexity of the
-Boost.Python v2 implementation might discourage contributors, but the
-emergence of Pyste_ and several other significant feature
-contributions have laid those fears to rest. Daily questions on the
-Python C++-sig and a backlog of desired improvements show that the
-library is getting used. To us, the future looks bright.
-
-.. _`EDG`: http://www.edg.com
-
=============
Conclusions
=============
@@ -924,24 +851,3 @@ the ground up, we can approach design with new confidence and power.
.. [VELD1995] T. Veldhuizen, "Expression Templates," C++ Report,
Vol. 7 No. 5 June 1995, pp. 26-31.
http://osl.iu.edu/~tveldhui/papers/Expression-Templates/exprtmpl.html
-
-===========
- Footnotes
-===========
-
-.. [#proto] In retrospect, it seems that "thinking hybrid" from the
- ground up might have been better for the NLP system: the
- natural component boundaries defined by the pure python
- prototype turned out to be inappropriate for getting the
- desired performance and memory footprint out of the C++ core,
- which eventually caused some redesign overhead on the Python
- side when the core was moved to C++.
-
-.. [#test] We also have some reservations about driving all C++
- testing through a Python interface, unless that's the only way
- it will be ultimately used. Any transition across language
- boundaries with such different object models can inevitably
- mask bugs.
-
-.. [#feature] These features were expressed very differently in v1 of
- Boost.Python
diff --git a/doc/PyConDC_2003/python_cpp_mix.jpg b/doc/PyConDC_2003/python_cpp_mix.jpg
deleted file mode 100755
index 755a9605..00000000
Binary files a/doc/PyConDC_2003/python_cpp_mix.jpg and /dev/null differ
diff --git a/doc/tutorial/doc/default_arguments.html b/doc/tutorial/doc/default_arguments.html
index 2fbebd23..d0c1f426 100644
--- a/doc/tutorial/doc/default_arguments.html
+++ b/doc/tutorial/doc/default_arguments.html
@@ -83,7 +83,7 @@ respectively. In our foo function the minimum number of arguments is 1
and the maximum number of arguments is 4. The def(...) function will
automatically add all the foo variants for us:
Objects here, objects there, objects here there everywhere. More frequently
diff --git a/doc/tutorial/doc/quickstart.txt b/doc/tutorial/doc/quickstart.txt
index 844ffa1b..7efc63bf 100644
--- a/doc/tutorial/doc/quickstart.txt
+++ b/doc/tutorial/doc/quickstart.txt
@@ -994,7 +994,7 @@ respectively. In our [^foo] function the minimum number of arguments is 1
and the maximum number of arguments is 4. The [^def(...)] function will
automatically add all the foo variants for us:
- def("foo", foo, foo_overloads());
+ .def("foo", foo, foo_overloads());
[h2 BOOST_PYTHON_MEMBER_FUNCTION_OVERLOADS]
@@ -1330,258 +1330,6 @@ create a new scope around a class:
.value("blue", blue)
;
-[def Py_Initialize [@http://www.python.org/doc/current/api/initialization.html#l2h-652 Py_Initialize]]
-[def Py_Finalize [@http://www.python.org/doc/current/api/initialization.html#l2h-656 Py_Finalize]]
-[def PyRun_String [@http://www.python.org/doc/current/api/veryhigh.html#l2h-55 PyRun_String]]
-[def PyRun_File [@http://www.python.org/doc/current/api/veryhigh.html#l2h-56 PyRun_File]]
-[def Py_eval_input [@http://www.python.org/doc/current/api/veryhigh.html#l2h-58 Py_eval_input]]
-[def Py_file_input [@http://www.python.org/doc/current/api/veryhigh.html#l2h-59 Py_file_input]]
-[def Py_single_input [@http://www.python.org/doc/current/api/veryhigh.html#l2h-60 Py_single_input]]
-[def Py_XINCREF [@http://www.python.org/doc/current/api/countingRefs.html#l2h-65 Py_XINCREF]]
-[def Py_XDECREF [@http://www.python.org/doc/current/api/countingRefs.html#l2h-67 Py_XDECREF]]
-[def PyImport_AppendInittab [@http://www.python.org/doc/current/api/importing.html#l2h-137 PyImport_AppendInittab]]
-[def PyImport_AddModule [@http://www.python.org/doc/current/api/importing.html#l2h-125 PyImport_AddModule]]
-[def PyModule_New [@http://www.python.org/doc/current/api/moduleObjects.html#l2h-591 PyModule_New]]
-[def PyModule_GetDict [@http://www.python.org/doc/current/api/moduleObjects.html#l2h-594 PyModule_GetDict]]
-
-[page:0 Embedding]
-
-By now you should know how to use Boost.Python to call your C++ code from
-Python. However, sometimes you may need to do the reverse: call Python code
-from the C++-side. This requires you to ['embed] the Python interpreter
-into your C++ program.
-
-Currently, Boost.Python does not directly support everything you'll need
-when embedding. Therefore you'll need to use the
-[@http://www.python.org/doc/current/api/api.html Python/C API] to fill in
-the gaps. However, Boost.Python already makes embedding a lot easier and,
-in a future version, it may become unnecessary to touch the Python/C API at
-all. So stay tuned... :-)
-
-[h2 Building embedded programs]
-
-To be able to use embedding in your programs, they have to be linked to
-both Boost.Python's and Python's static link library.
-
-Boost.Python's static link library comes in two variants. Both are located
-in Boost's [^/libs/python/build/bin-stage] subdirectory. On Windows, the
-variants are called [^boost_python.lib] (for release builds) and
-[^boost_python_debug.lib] (for debugging). If you can't find the libraries,
-you probably haven't built Boost.Python yet. See [@../../building.html
-Building and Testing] on how to do this.
-
-Python's static link library can be found in the [^/libs] subdirectory of
-your Python directory. On Windows it is called pythonXY.lib where X.Y is
-your major Python version number.
-
-Additionally, Python's [^/include] subdirectory has to be added to your
-include path.
-
-In a Jamfile, all the above boils down to:
-
-[pre
- projectroot c:\projects\embedded_program ; # location of the program
-
- # bring in the rules for python
- SEARCH on python.jam = $(BOOST_BUILD_PATH) ;
- include python.jam ;
-
- exe embedded_program # name of the executable
- : #sources
- embedded_program.cpp
- : # requirements
- boost_python c:\boost\libs\python
- $(PYTHON_PROPERTIES)
- $(PYTHON_LIB_PATH)
- $(PYTHON_EMBEDDED_LIBRARY) ;
-]
-
-[h2 Getting started]
-
-Being able to build is nice, but there is nothing to build yet. Embedding
-the Python interpreter into one of your C++ programs requires these 4
-steps:
-
-# '''#include''' [^][br][br]
-
-# Call Py_Initialize() to start the interpreter and create the [^__main__] module.[br][br]
-
-# Call other Python C API routines to use the interpreter.[br][br]
-
-# Call Py_Finalize() to stop the interpreter and release its resources.
-
-(Of course, there can be other C++ code between all of these steps.)
-
-[:['[*Now that we can embed the interpreter in our programs, lets see how to put it to use...]]]
-
-[page:1 Using the interpreter]
-
-As you probably already know, objects in Python are reference-counted.
-Naturally, the [^PyObject]s of the Python/C API are also reference-counted.
-There is a difference however. While the reference-counting is fully
-automatic in Python, the Python/C API requires you to do it
-[@http://www.python.org/doc/current/api/refcounts.html by hand]. This is
-messy and especially hard to get right in the presence of C++ exceptions.
-Fortunately Boost.Python provides the [@../../v2/handle.html handle] class
-template to automate the process.
-
-[h2 Reference-counting handles]
-
-There are two ways in which a function in the Python/C API can return a
-[^PyObject*]: as a ['borrowed reference] or as a ['new reference]. Which of
-these a function uses, is listed in that function's documentation. The two
-require slightely different approaches to reference-counting but both can
-be 'handled' by Boost.Python.
-
-For a function returning a ['borrowed reference] we'll have to tell the
-[^handle] that the [^PyObject*] is borrowed with the aptly named
-[@../../v2/handle.html#borrowed-spec borrowed] function. Two functions
-returning borrowed references are PyImport_AddModule and PyModule_GetDict.
-The former returns a reference to an already imported module, the latter
-retrieves a module's namespace dictionary. Let's use them to retrieve the
-namespace of the [^__main__] module:
-
- handle<> main_module(borrowed( PyImport_AddModule("__main__") ));
- handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get()) ));
-
-Because the Python/C API doesn't know anything about [^handle]s, we used
-the [@../../v2/handle.html#handle-spec-observers get] member function to
-retrieve the [^PyObject*] from which the [^handle] was constructed.
-
-For a function returning a ['new reference] we can just create a [^handle]
-out of the raw [^PyObject*] without wrapping it in a call to borrowed. One
-such function that returns a new reference is PyRun_String which we'll
-discuss in the next section.
-
-[blurb __detail__ [*Handle is a class ['template], so why haven't we been using any template parameters?][br]
-[br]
-[^handle] has a single template parameter specifying the type of the managed object. This type is [^PyObject] 99% of the time, so the parameter was defaulted to [^PyObject] for convenience. Therefore we can use the shorthand [^handle<>] instead of the longer, but equivalent, [^handle].
-]
-
-[h2 Running Python code]
-
-To run Python code from C++ there is a family of functions in the API
-starting with the PyRun prefix. You can find the full list of these
-functions [@http://www.python.org/doc/current/api/veryhigh.html here]. They
-all work similarly so we will look at only one of them, namely:
-
- PyObject* PyRun_String(char *str, int start, PyObject *globals, PyObject *locals)
-
-PyRun_String takes the code to execute as a null-terminated (C-style)
-string in its [^str] parameter. The function returns a new reference to a
-Python object. Which object is returned depends on the [^start] paramater.
-
-The [^start] parameter is the start symbol from the Python grammar to use
-for interpreting the code. The possible values are:
-
-[table Start symbols
-
- [Py_eval_input] [for interpreting isolated expressions]
- [Py_file_input] [for interpreting sequences of statements]
- [Py_single_input] [for interpreting a single statement]
-]
-
-When using Py_eval_input, the input string must contain a single expression
-and its result is returned. When using Py_file_input, the string can
-contain an abitrary number of statements and None is returned.
-Py_single_input works in the same way as Py_file_input but only accepts a
-single statement.
-
-Lastly, the [^globals] and [^locals] parameters are Python dictionaries
-containing the globals and locals of the context in which to run the code.
-For most intents and purposes you can use the namespace dictionary of the
-[^__main__] module for both parameters.
-
-We have already seen how to get the [^__main__] module's namespace so let's
-run some Python code in it:
-
- handle<> main_module(borrowed( PyImport_AddModule("__main__") ));
- handle<> main_namespace(borrowed( PyModule_GetDict(main_module.get()) ));
- handle<>( PyRun_String("hello = file('hello.txt', 'w')\n"
- "hello.write('Hello world!')\n"
- "hello.close()", Py_file_input,
- main_namespace.get(), main_namespace.get()) );
-
-This should create a file called 'hello.txt' in the current directory
-containing a phrase that is well-known in programming circles.
-
-__note__ [*Note] that we wrap the return value of PyRun_String in a
-(nameless) [^handle] even though we are not interested in it. If we didn't
-do this, the the returned object would be kept alive unnecessarily. Unless
-you want to be a Dr. Frankenstein, always wrap [^PyObject*]s in [^handle]s.
-
-[h2 Beyond handles]
-
-It's nice that [^handle] manages the reference counting details for us, but
-other than that it doesn't do much. Often we'd like to have a more useful
-class to manipulate Python objects. But we have already seen such a class
-in the [@object_interface.html previous section]: the aptly named [^object]
-class and it's derivatives. What we haven't seen, is that they can be
-constructed from a [^handle]. The following examples should illustrate this
-fact:
-
- handle<> main_module(borrowed( PyImport_AddModule("__main__") ));
- dict main_namespace(handle<>(borrowed( PyModule_GetDict(main_module.get()) )));
- handle<>( PyRun_String("result = 5 ** 2", Py_file_input,
- main_namespace.ptr(), main_namespace.ptr()) );
- int five_squared = extract( main_namespace["result"] );
-
-Here we create a dictionary object for the [^__main__] module's namespace.
-Then we assign 5 squared to the result variable and read this variable from
-the dictionary. Another way to achieve the same result is to let
-PyRun_String return the result directly with Py_eval_input:
-
- object result(handle<>( PyRun_String("5 ** 2", Py_eval_input,
- main_namespace.ptr(), main_namespace.ptr()) ));
- int five_squared = extract(result);
-
-__note__ [*Note] that [^object]'s member function to return the wrapped
-[^PyObject*] is called [^ptr] instead of [^get]. This makes sense if you
-take into account the different functions that [^object] and [^handle]
-perform.
-
-[h2 Exception handling]
-
-If an exception occurs in the execution of some Python code, the PyRun_String function returns a null pointer. Constructing a [^handle] out of this null pointer throws [@../../v2/errors.html#error_already_set-spec error_already_set], so basically, the Python exception is automatically translated into a C++ exception when using [^handle]:
-
- try
- {
- object result(handle<>( PyRun_String("5/0", Py_eval_input,
- main_namespace.ptr(), main_namespace.ptr()) ));
- // execution will never get here:
- int five_divided_by_zero = extract(result);
- }
- catch(error_already_set)
- {
- // handle the exception in some way
- }
-
-The [^error_already_set] exception class doesn't carry any information in itself. To find out more about the Python exception that occurred, you need to use the [@http://www.python.org/doc/api/exceptionHandling.html exception handling functions] of the Python/C API in your catch-statement. This can be as simple as calling [@http://www.python.org/doc/api/exceptionHandling.html#l2h-70 PyErr_Print()] to print the exception's traceback to the console, or comparing the type of the exception with those of the [@http://www.python.org/doc/api/standardExceptions.html standard exceptions]:
-
- catch(error_already_set)
- {
- if (PyErr_ExceptionMatches(PyExc_ZeroDivisionError))
- {
- // handle ZeroDivisionError specially
- }
- else
- {
- // print all other errors to stderr
- PyErr_Print();
- }
- }
-
-(To retrieve even more information from the exception you can use some of the other exception handling functions listed [@http://www.python.org/doc/api/exceptionHandling.html here].)
-
-If you'd rather not have [^handle] throw a C++ exception when it is constructed, you can use the [@../../v2/handle.html#allow_null-spec allow_null] function in the same way you'd use borrowed:
-
- handle<> result(allow_null( PyRun_String("5/0", Py_eval_input,
- main_namespace.ptr(), main_namespace.ptr()) ));
- if (!result)
- // Python exception occurred
- else
- // everything went okay, it's safe to use the result
-
[page Iterators]
In C++, and STL in particular, we see iterators everywhere. Python also has
diff --git a/doc/tutorial/doc/using_the_interpreter.html b/doc/tutorial/doc/using_the_interpreter.html
index e63658e6..4c0e4de9 100644
--- a/doc/tutorial/doc/using_the_interpreter.html
+++ b/doc/tutorial/doc/using_the_interpreter.html
@@ -145,15 +145,13 @@ previous section: the aptly named object
class and it's derivatives. What we haven't seen, is that they can be
constructed from a handle. The following examples should illustrate this
fact:
Here we create a dictionary object for the __main__ module's namespace.
Then we assign 5 squared to the result variable and read this variable from
diff --git a/doc/v2/acknowledgments.html b/doc/v2/acknowledgments.html
index 3cc24edb..82cea6a2 100644
--- a/doc/v2/acknowledgments.html
+++ b/doc/v2/acknowledgments.html
@@ -31,15 +31,6 @@
Dave Abrahams is
the architect, designer, and implementor of Boost.Python.
-
Brett Calcott
- contributed and maintains the Visual Studio project files and
- documentation.
-
-
Gottfried
- Ganßauge supplied support for opaque pointer conversions,
- complete with documentation and a regression test (and I didn't
- even have to ask him for those)!
-
Martin Casado solved some sticky problems which allow us to build the
- Boost.Python shared library for AIX's crazy dynamic linking model.
-
Achim Domma contributed some
of the Object Wrappers and
HTML templates for this documentation. Dave Hawkes contributed
@@ -91,6 +71,16 @@
definition syntax. Pearu Pearson wrote some of the test cases that are in
the current test suite.
+
Brett Calcott
+ contributed and maintains the Visual Studio project files and
+ documentation.
-
- The short answer is: "you can't". This is not a
- Boost.Python limitation so much as a limitation of C++. The
- problem is that a Python function is actually data, and the only
- way of associating data with a C++ function pointer is to store it
- in a static variable of the function. The problem with that is
- that you can only associate one piece of data with every C++
- function, and we have no way of compiling a new C++ function
- on-the-fly for every Python function you decide to pass
- to foo. In other words, this could work if the C++
- function is always going to invoke the same Python
- function, but you probably don't want that.
-
-
If you have the luxury of changing the C++ code you're
- wrapping, pass it an object instead and call that;
- the overloaded function call operator will invoke the Python
- function you pass it behind the object.
-
-
For more perspective on the issue, see this
- posting.
-
-
opaque_pointer_converter<> is derived from
-
- to_python_converter
- and registers itself as an
-
- lvalue_from_pytype converter from Python objects
- into pointers to undefined types.
- Thus it may be used as a converter from opaque pointers into
- Python objects and vice versa.
-
-
Class template
- opaque_pointer_converter synopsis
Registers the instance as a
-
- lvalue_from_pytype converter from Python objects
- into opaque pointers.
-
The name is used for the type of the Python Objects created;
- it should be printable but needn't be an
- ntbs because the object type is
- not supposed to be user constructible within python scripts.
This macro must be used to define specializations of the
- type_id function
- which can't be instantiated for incomplete types.
-
Note
-
In order for this to work in a cross-module environment the macro must
- be invoked in every translation unit which uses the
- opaque_pointer_converter.
raw_function(...)
- is used to convert a function taking a tuple and a dict into a Python callable object
- which accepts a variable number of arguments and arbitrary keyword
- arguments.
-
-
Returns: a callable object which requires at least min_args arguments. When called, the actual non-keyword arguments will be passed in a tuple as the first argument to f, and the keyword arguments will be passed in a dict as the second argument to f.
-
-
-
return_opaque_pointer is a model of
-
- ResultConverterGenerator
- which can be used to wrap C++ functions returning pointers to
- undefined types such that the return value is copied into a
- new Python object.
-
In addition to specifying the return_opaque_pointer
- policy the
- BOOST_PYTHON_OPAQUE_SPECIALIZED_TYPE_ID macro must be
- used to define specializations for the
- type_id function
- on the type pointed to by returned pointer.
What if a function or method needs a policy and the user
-doesn't set one?
If a function/method needs a policy and one was not
-set, Pyste will issue a error. The user should then go in the interface file
-and set the policy for it, otherwise the generated cpp won't compile.
-
-
-
-
-
-
-
-
-Note that, for functions/methods that return const T&, the policy
-return_value_policy<copy_const_reference>() wil be used by default, because
-that's normally what you want. You can change it to something else if you need
-to, though.
+doesn't set one?
+If a function/method needs a policy and one was not set, Pyste will issue a error.
+The user should then go in the interface file and set the policy for it,
+otherwise the generated cpp won't compile.
diff --git a/pyste/doc/pyste.txt b/pyste/doc/pyste.txt
index 945b2364..b6fbb281 100644
--- a/pyste/doc/pyste.txt
+++ b/pyste/doc/pyste.txt
@@ -30,7 +30,7 @@ Here's the interface file for it, named [^world.pyste]:
and that's it!
-The next step is invoke Pyste in the command-line:
+The next step is invoke pyste in the command-line:
[pre python pyste.py --module=hello world.pyste]
@@ -47,21 +47,20 @@ Pyste supports the following features:
* Attributes
* Enums (both "free" enums and class enums)
* Nested Classes
-* Support for [^boost::shared_ptr] and [^std::auto_ptr]
[page Running Pyste]
-To run Pyste, you will need:
+To run pyste, you will need:
-* Python 2.2, available at [@http://www.python.org python's website].
+* Python 2.2, avaiable at [@http://www.python.org python's website].
* The great [@http://effbot.org elementtree] library, from Fredrik Lundh.
* The excellent GCCXML, from Brad King.
-Installation for the tools is available in their respective webpages.
+Installation for the tools is avaiable in their respective webpages.
[blurb
[$theme/note.gif] GCCXML must be accessible in the PATH environment variable, so
-that Pyste can call it. How to do this varies from platform to platform.
+that pyste can call it. How to do this varies from platform to platform.
]
[h2 Ok, now what?]
@@ -72,56 +71,31 @@ Well, now let's fire it up:
'''
>python pyste.py
-Pyste version 0.6.5
-
Usage:
pyste [options] --module= interface-files
where options are:
- -I add an include path
- -D define symbol
- --multiple create various cpps, instead of only one
- (useful during development)
- --out specify output filename (default: .cpp)
- in --multiple mode, this will be a directory
+ -I add an include path
+ -D define symbol
--no-using do not declare "using namespace boost";
use explicit declarations instead
--pyste-ns= set the namespace where new types will be declared;
- default is the empty namespace
- --debug writes the xml for each file parsed in the current
- directory
- -h, --help print this help and exit
- -v, --version print version information
+ default is "pyste"
'''
]
Options explained:
-The [^-I] and [^-D] are preprocessor flags, which are needed by GCCXML to parse
-the header files correctly and by Pyste to find the header files declared in the
+The [^-I] and [^-D] are preprocessor flags, which are needed by gccxml to parse the header files correctly and by pyste to find the header files declared in the
interface files.
-[^--multiple] tells Pyste to generate multiple cpps for this module (one for
-each header parsed) in the directory named by [^--out], instead of the usual
-single cpp file. This mode is useful during development of a binding, because
-you are constantly changing source files, re-generating the bindings and
-recompiling. This saves a lot of time in compiling.
-
-[^--out] names the output file (default: [^.cpp]), or in multiple mode,
-names a output directory for the files (default: [^]).
-
-[^--no-using] tells Pyste to don't declare "[^using namespace boost;]" in the
+[^--no-using] tells pyste to don't declare "[^using namespace boost;]" in the
generated cpp, using the namespace boost::python explicitly in all declarations.
Use only if you're having a name conflict in one of the files.
Use [^--pyste-ns] to change the namespace where new types are declared (for
-instance, the virtual wrappers). Use only if you are having any problems. By
-default, Pyste uses the empty namespace.
-
-[^--debug] will write in the current directory a xml file as outputted by GCCXML
-for each header parsed. Useful for bug reports.
-
-[^-h, --help, -v, --version] are self-explaining, I believe. ;)
+instance, the virtual wrappers). Use only if one of your header files declare a
+namespace named "pyste" and this is causing conflicts.
So, the usage is simple enough:
@@ -129,13 +103,7 @@ So, the usage is simple enough:
will generate a file [^mymodule.cpp] in the same dir where the command was
executed. Now you can compile the file using the same instructions of the
-[@../../doc/tutorial/doc/building_hello_world.html tutorial]. Or, if you prefer:
-
-[pre >python pyste.py --module=mymodule --multiple file.pyste file2.pyste ...]
-
-will create a directory named "mymodule" in the current directory, and will
-generate a bunch of cpp files, one for each header exported. You can then
-compile them all into a single shared library (or dll).
+[@../../doc/tutorial/doc/building_hello_world.html tutorial].
[h2 Wait... how do I set those I and D flags?]
@@ -152,15 +120,11 @@ which for Visual C++ 6 is normally located at:
with that, you should have little trouble setting up the flags.
-[blurb [$theme/note.gif][*A note about Psyco][br][br]
-Although you don't have to install [@http://psyco.sourceforge.net/ Psyco] to use Pyste, if you do, Pyste will make use of it to speed up the wrapper generation. Speed ups of 30% can be achieved, so it's highly recommended.
-]
-
[page The Interface Files]
The interface files are the heart of Pyste. The user creates one or more
interface files declaring the classes and functions he wants to export, and then
-invokes Pyste passing the interface files to it. Pyste then generates a single
+invokes pyste passing the interface files to it. Pyste then generates a single
cpp file with Boost.Python code, with all the classes and functions exported.
Besides declaring the classes and functions, the user has a number of other
@@ -221,14 +185,7 @@ with the function [^exclude]:
exclude(World.greet)
exclude(World.msg)
-To access the operators of a class, access the member [^operator] like this
-(supposing that [^C] is a class being exported):
-
- exclude(C.operator['+'])
- exclude(C.operator['*'])
- exclude(C.operator['<<'])
-
-The string inside the brackets is the same as the name of the operator in C++.[br]
+Easy, huh? [$theme/smiley.gif]
[page:1 Policies]
@@ -255,22 +212,15 @@ function:
[blurb
[$theme/note.gif] [*What if a function or method needs a policy and the user
-doesn't set one?][br][br] If a function/method needs a policy and one was not
-set, Pyste will issue a error. The user should then go in the interface file
-and set the policy for it, otherwise the generated cpp won't compile.
-]
-
-[blurb
-[$theme/note.gif]
-Note that, for functions/methods that return [^const T&], the policy
-[^return_value_policy()] wil be used by default, because
-that's normally what you want. You can change it to something else if you need
-to, though.
+doesn't set one?][br][br]
+If a function/method needs a policy and one was not set, Pyste will issue a error.
+The user should then go in the interface file and set the policy for it,
+otherwise the generated cpp won't compile.
]
[page:1 Templates]
-Template classes can easily be exported too, but you can't export the template
+Template Classes can easily exported too, but you can't export the "Template"
itself... you have to export instantiations of it! So, if you want to export a
[^std::vector], you will have to export vectors of int, doubles, etc.
@@ -297,7 +247,7 @@ rename the instantiations:
double_inst = Point("double") // another way to do the same
rename(double_inst, "DPoint")
-Note that you can rename, exclude, set policies, etc, in the [^Template] object
+Note that you can rename, exclude, set policies, etc, in the [^Template] class
like you would do with a [^Function] or a [^Class]. This changes affect all
[*future] instantiations:
@@ -318,7 +268,7 @@ If you want to change a option of a particular instantiation, you can do so:
[blurb [$theme/note.gif] [*What if my template accepts more than one type?]
[br][br]
-When you want to instantiate a template with more than one type, you can pass
+When you want to instantiate a Template with more than one type, you can pass
either a string with the types separated by whitespace, or a list of strings
'''("int double" or ["int", "double"]''' would both work).
]
@@ -329,20 +279,14 @@ Suppose you have this function:
std::vector names();
-But you don't want to export [^std::vector], you want this function
-to return a python list of strings. Boost.Python has excellent support for
-that:
+But you don't want to export a vector, you want this function to return
+a python list of strings. Boost.Python has an excellent support for that:
list names_wrapper()
{
list result;
- // call original function
vector v = names();
- // put all the strings inside the python list
- vector::iterator it;
- for (it = v.begin(); it != v.end(); ++it){
- result.append(*it);
- }
+ // put each string in the vector in the list
return result;
}
@@ -351,8 +295,9 @@ that:
def("names", &names_wrapper);
}
-Nice heh? Pyste supports this mechanism too. You declare the [^names_wrapper]
-function in a header named "[^test_wrappers.h]" and in the interface file:
+Nice heh?
+Pyste supports this mechanism too. You declare the [^names_wrapper] function in a
+header, like "[^test_wrappers.h]", and in the interface file:
Include("test_wrappers.h")
names = Function("names", "test.h")
@@ -364,13 +309,13 @@ You can optionally declare the function in the interface file itself:
"""
list names_wrapper()
{
- // code to call name() and convert the vector to a list...
+ // call name() and convert the vector to a list...
}
""")
names = Function("names", "test.h")
set_wrapper(names, names_wrapper)
-The same mechanism can be used with methods too. Just remember that the first
+The same mechanism can be done with methods too. Just remember that the first
parameter of wrappers for methods is a pointer to the class, like in
Boost.Python:
@@ -381,7 +326,7 @@ Boost.Python:
list names_wrapper(C* c)
{
- // same as before, calling c->names() and converting result to a list
+ // same as before, calling c->names() and converting result to a list
}
And then in the interface file:
@@ -389,13 +334,6 @@ And then in the interface file:
C = Class("C", "test.h")
set_wrapper(C.names, "names_wrapper")
-[blurb
-[$theme/note.gif]Even though Boost.Python accepts either a pointer or a
-reference to the class in wrappers for member functions as the first parameter,
-Pyste expects them to be a [*pointer]. Doing otherwise will prevent your
-code to compile when you set a wrapper for a virtual method.
-]
-
[page:1 Exporting All Declarations from a Header]
Pyste also supports a mechanism to export all declarations found in a header
@@ -430,38 +368,3 @@ the members of the header object like this:
rename(hello.World.greet, "Greet")
exclude(hello.World.set, "Set")
-
-[page:1 Smart Pointers]
-
-Pyste for now has manual support for smart pointers. Suppose:
-
- struct C
- {
- int value;
- };
-
- boost::shared_ptr newC(int value)
- {
- boost::shared_ptr c( new C() );
- c->value = value;
- return c;
- }
-
- void printC(boost::shared_ptr c)
- {
- std::cout << c->value << std::endl;
- }
-
-To make [^newC] and [^printC] work correctly, you have to tell Pyste that a
-convertor for [^boost::shared_ptr] is needed.
-
- C = Class('C', 'C.h')
- use_shared_ptr(C)
- Function('newC', 'C.h')
- Function('printC', 'C.h')
-
-For [^std::auto_ptr]'s, use the function [^use_auto_ptr].
-
-This system is temporary, and in the future the converters will automatically be
-exported if needed, without the need to tell Pyste about them explicitly.
-
diff --git a/pyste/doc/renaming_and_excluding.html b/pyste/doc/renaming_and_excluding.html
index 4f56bb92..29a8001b 100644
--- a/pyste/doc/renaming_and_excluding.html
+++ b/pyste/doc/renaming_and_excluding.html
@@ -50,15 +50,7 @@ with the function exclude:
exclude(World.msg)
-To access the operators of a class, access the member operator like this
-(supposing that C is a class being exported):
-Installation for the tools is available in their respective webpages.
+Installation for the tools is avaiable in their respective webpages.
GCCXML must be accessible in the PATH environment variable, so
-that Pyste can call it. How to do this varies from platform to platform.
+that pyste can call it. How to do this varies from platform to platform.
@@ -47,68 +47,37 @@ Well, now let's fire it up:
>python pyste.py
-Pyste version 0.6.5
-
Usage:
pyste [options] --module=<name> interface-files
where options are:
- -I <path> add an include path
- -D <symbol> define symbol
- --multiple create various cpps, instead of only one
- (useful during development)
- --out specify output filename (default: <module>.cpp)
- in --multiple mode, this will be a directory
+ -I <path> add an include path
+ -D <symbol> define symbol
--no-using do not declare "using namespace boost";
use explicit declarations instead
--pyste-ns=<name> set the namespace where new types will be declared;
- default is the empty namespace
- --debug writes the xml for each file parsed in the current
- directory
- -h, --help print this help and exit
- -v, --version print version information
+ default is "pyste"
Options explained:
-The -I and -D are preprocessor flags, which are needed by
-GCCXML to parse
-the header files correctly and by Pyste to find the header files declared in the
+The -I and -D are preprocessor flags, which are needed by gccxml to parse the header files correctly and by pyste to find the header files declared in the
interface files.
---multiple tells Pyste to generate multiple cpps for this module (one for
-each header parsed) in the directory named by --out, instead of the usual
-single cpp file. This mode is useful during development of a binding, because
-you are constantly changing source files, re-generating the bindings and
-recompiling. This saves a lot of time in compiling.
-
---out names the output file (default: <module>.cpp), or in multiple mode,
-names a output directory for the files (default: <module>).
-
---no-using tells Pyste to don't declare "using namespace boost;" in the
+--no-using tells pyste to don't declare "using namespace boost;" in the
generated cpp, using the namespace boost::python explicitly in all declarations.
Use only if you're having a name conflict in one of the files.
Use --pyste-ns to change the namespace where new types are declared (for
-instance, the virtual wrappers). Use only if you are having any problems. By
-default, Pyste uses the empty namespace.
-
---debug will write in the current directory a xml file as outputted by
-GCCXML
-for each header parsed. Useful for bug reports.
-
--h, --help, -v, --version are self-explaining, I believe. ;)
+instance, the virtual wrappers). Use only if one of your header files declare a
+namespace named "pyste" and this is causing conflicts.
will generate a file mymodule.cpp in the same dir where the command was
executed. Now you can compile the file using the same instructions of the
-tutorial. Or, if you prefer:
-will create a directory named "mymodule" in the current directory, and will
-generate a bunch of cpp files, one for each header exported. You can then
-compile them all into a single shared library (or dll).
+tutorial.
Wait... how do I set those I and D flags?
Don't worry: normally
GCCXML is already configured correctly for your plataform,
@@ -124,15 +93,6 @@ which for Visual C++ 6 is normally located at:
with that, you should have little trouble setting up the flags.
-
-
-
-A note about Psyco
-Although you don't have to install
-Psyco to use Pyste, if you do, Pyste will make use of it to speed up the wrapper generation. Speed ups of 30% can be achieved, so it's highly recommended.
-
-For std::auto_ptr's, use the function use_auto_ptr.
-
-This system is temporary, and in the future the converters will automatically be
-exported if needed, without the need to tell Pyste about them explicitly.
-Permission to copy, use, modify, sell and distribute this document
- is granted provided this copyright notice appears in all copies. This document
- is provided "as is" without express or implied warranty, and with
- no claim as to its suitability for any purpose.
-Template classes can easily be exported too, but you can't export the template
+Template Classes can easily exported too, but you can't export the "Template"
itself... you have to export instantiations of it! So, if you want to export a
std::vector, you will have to export vectors of int, doubles, etc.
@@ -55,7 +55,7 @@ rename the instantiations:
rename(double_inst, "DPoint")
-Note that you can rename, exclude, set policies, etc, in the Template object
+Note that you can rename, exclude, set policies, etc, in the Template class
like you would do with a Function or a Class. This changes affect all
future instantiations:
@@ -80,7 +80,7 @@ If you want to change a option of a particular instantiation, you can do so:
What if my template accepts more than one type?
-When you want to instantiate a template with more than one type, you can pass
+When you want to instantiate a Template with more than one type, you can pass
either a string with the types separated by whitespace, or a list of strings
("int double" or ["int", "double"] would both work).
The interface files are the heart of Pyste. The user creates one or more
interface files declaring the classes and functions he wants to export, and then
-invokes Pyste passing the interface files to it. Pyste then generates a single
+invokes pyste passing the interface files to it. Pyste then generates a single
cpp file with
Boost.Python code, with all the classes and functions exported.
diff --git a/pyste/doc/wrappers.html b/pyste/doc/wrappers.html
index 63c9a9ce..61bda6f8 100644
--- a/pyste/doc/wrappers.html
+++ b/pyste/doc/wrappers.html
@@ -30,21 +30,15 @@ Suppose you have this function:
std::vector<std::string> names();
-But you don't want to export std::vector<std::string>, you want this function
-to return a python list of strings.
-Boost.Python has excellent support for
-that:
+But you don't want to export a vector<string>, you want this function to return
+a python list of strings.
+Boost.Python has an excellent support for that:
list names_wrapper()
{
list result;
- // call original function
vector<string> v = names();
- // put all the strings inside the python list
- vector<string>::iterator it;
- for (it = v.begin(); it != v.end(); ++it){
- result.append(*it);
- }
+ // put each string in the vector in the list
return result;
}
@@ -54,8 +48,9 @@ that:
}
-Nice heh? Pyste supports this mechanism too. You declare the names_wrapper
-function in a header named "test_wrappers.h" and in the interface file:
+Nice heh?
+Pyste supports this mechanism too. You declare the names_wrapper function in a
+header, like "test_wrappers.h", and in the interface file:
Include("test_wrappers.h")
names = Function("names", "test.h")
@@ -68,14 +63,14 @@ You can optionally declare the function in the interface file itself:
"""
list names_wrapper()
{
- // code to call name() and convert the vector to a list...
+ // call name() and convert the vector to a list...
}
""")
names = Function("names", "test.h")
set_wrapper(names, names_wrapper)
-The same mechanism can be used with methods too. Just remember that the first
+The same mechanism can be done with methods too. Just remember that the first
parameter of wrappers for methods is a pointer to the class, like in
Boost.Python:
@@ -87,7 +82,7 @@ Boost.Python:
list names_wrapper(C* c)
{
- // same as before, calling c->names() and converting result to a list
+ // same as before, calling c->names() and converting result to a list
}
@@ -96,18 +91,6 @@ And then in the interface file:
C = Class("C", "test.h")
set_wrapper(C.names, "names_wrapper")
-
-
-
-
-Even though
-Boost.Python accepts either a pointer or a
-reference to the class in wrappers for member functions as the first parameter,
-Pyste expects them to be a pointer. Doing otherwise will prevent your
-code to compile when you set a wrapper for a virtual method.
-
-
-
diff --git a/pyste/example/.cvsignore b/pyste/example/.cvsignore
index 7ee0168b..ce1da4c5 100644
--- a/pyste/example/.cvsignore
+++ b/pyste/example/.cvsignore
@@ -1,2 +1 @@
-.sconsign
-*.obj
+*.cpp
diff --git a/pyste/example/basic.cpp b/pyste/example/basic.cpp
deleted file mode 100644
index 9e6ed996..00000000
--- a/pyste/example/basic.cpp
+++ /dev/null
@@ -1,8 +0,0 @@
-#include "basic.h"
-
-namespace basic {
-
- int C::static_value = 3;
- const int C::const_static_value = 100;
-
-}
diff --git a/pyste/example/basic.h b/pyste/example/basic.h
index 5ab6ca58..a618c192 100644
--- a/pyste/example/basic.h
+++ b/pyste/example/basic.h
@@ -1,15 +1,7 @@
-#ifndef BASIC_H
-#define BASIC_H
-
-
-#include
-
namespace basic {
struct C
{
- // test virtuallity
- C(): value(1), const_value(0) {}
virtual int f(int x = 10)
{
return x*2;
@@ -18,42 +10,16 @@ struct C
int foo(int x=1){
return x+1;
}
-
- const std::string& name() { return _name; }
- void set_name(const std::string& name) { _name = name; }
- std::string _name;
-
- // test data members
- static int static_value;
- static const int const_static_value;
-
- int value;
- const int const_value;
-
- // test static functions
- static int mul(int x=2, int y=3) { return x*y; }
};
-inline int call_f(C& c)
+int call_f(C& c)
{
return c.f();
}
-inline int call_f(C& c, int x)
+int call_f(C& c, int x)
{
return c.f(x);
}
-inline int get_static()
-{
- return C::static_value;
}
-
-inline int get_value(C& c)
-{
- return c.value;
-}
-
-}
-
-#endif
diff --git a/pyste/example/basic.pyste b/pyste/example/basic.pyste
index 4fe0b5b3..90978d19 100644
--- a/pyste/example/basic.pyste
+++ b/pyste/example/basic.pyste
@@ -1,5 +1,2 @@
Class('basic::C', 'basic.h')
Function('basic::call_f', 'basic.h')
-Function('basic::get_static', 'basic.h')
-Function('basic::get_value', 'basic.h')
-
diff --git a/pyste/example/enums.h b/pyste/example/enums.h
index 207001e0..398b66c0 100644
--- a/pyste/example/enums.h
+++ b/pyste/example/enums.h
@@ -1,6 +1,3 @@
-#ifndef ENUMS_H
-#define ENUMS_H
-
namespace enums {
enum color { red, blue };
@@ -20,5 +17,3 @@ struct X
};
}
-
-#endif
diff --git a/pyste/example/header_test.h b/pyste/example/header_test.h
index 286f5449..85cb5ee0 100644
--- a/pyste/example/header_test.h
+++ b/pyste/example/header_test.h
@@ -1,6 +1,3 @@
-#ifndef HEADER_TEST_H
-#define HEADER_TEST_H
-
#include
What if a function or method needs a policy and the user
-doesn't set one?
-