mirror of
https://github.com/boostorg/python.git
synced 2026-01-22 05:22:45 +00:00
480 lines
17 KiB
HTML
480 lines
17 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<html>
|
|
<head>
|
|
<meta name="generator" content=
|
|
"HTML Tidy for Cygwin (vers 1st February 2003), see www.w3.org">
|
|
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
|
|
<link rel="stylesheet" type="text/css" href="../boost.css">
|
|
<title>
|
|
Boost.Python - FAQ
|
|
</title>
|
|
</head>
|
|
<body link="#0000ff" vlink="#800080">
|
|
<table border="0" cellpadding="7" cellspacing="0" width="100%"
|
|
summary="header">
|
|
<tr>
|
|
<td valign="top" width="300">
|
|
<h3>
|
|
<a href="../../../../index.htm"><img height="86" width="277"
|
|
alt="C++ Boost" src="../../../../c++boost.gif" border=
|
|
"0"></a>
|
|
</h3>
|
|
</td>
|
|
<td valign="top">
|
|
<h1 align="center">
|
|
<a href="../index.html">Boost.Python</a>
|
|
</h1>
|
|
<h2 align="center">
|
|
Frequently Asked Questions (FAQs)
|
|
</h2>
|
|
</td>
|
|
</tr>
|
|
</table>
|
|
<hr>
|
|
<dl class="page-index">
|
|
<dt>
|
|
<a href="#question1">Is return_internal_reference efficient?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#question2">How can I wrap functions which take C++
|
|
containers as arguments?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#c1204">fatal error C1204:Compiler limit:internal structure
|
|
overflow</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#debugging">How do I debug my Python extensions?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#imul">Why doesn't my <code>*=</code> operator work?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#macosx">Does Boost.Python work with Mac OS X?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#xref">How can I find the existing PyObject that holds a C++
|
|
object?</a>
|
|
</dt>
|
|
<dt>
|
|
<a href="#ownership">How can I wrap a function which needs to take
|
|
ownership of a raw pointer?</a>
|
|
</dt>
|
|
</dl>
|
|
<hr>
|
|
<h2>
|
|
<a name="question1"></a>Is return_internal_reference efficient?
|
|
</h2>
|
|
<blockquote>
|
|
<b>Q:</b> <i>I have an object composed of 12 doubles. A const& to
|
|
this object is returned by a member function of another class. From the
|
|
viewpoint of using the returned object in Python I do not care if I get
|
|
a copy or a reference to the returned object. In Boost.Python Version 2
|
|
I have the choice of using copy_const_reference or
|
|
return_internal_reference. Are there considerations that would lead me
|
|
to prefer one over the other, such as size of generated code or memory
|
|
overhead?</i>
|
|
<p>
|
|
<b>A:</b> copy_const_reference will make an instance with storage for
|
|
one of your objects, size = base_size + 12 * sizeof(double).
|
|
return_internal_reference will make an instance with storage for a
|
|
pointer to one of your objects, size = base_size + sizeof(void*).
|
|
However, it will also create a weak reference object which goes in
|
|
the source object's weakreflist and a special callback object to
|
|
manage the lifetime of the internally-referenced object. My guess?
|
|
copy_const_reference is your friend here, resulting in less overall
|
|
memory use and less fragmentation, also probably fewer total cycles.
|
|
</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2>
|
|
<a name="question2"></a>How can I wrap functions which take C++
|
|
containers as arguments?
|
|
</h2>
|
|
<p>
|
|
Ralf W. Grosse-Kunstleve provides these notes:
|
|
</p>
|
|
<ol>
|
|
<li>
|
|
Using the regular <code>class_<></code> wrapper:
|
|
<pre>
|
|
class_<std::vector<double> >("std_vector_double")
|
|
.def(...)
|
|
...
|
|
;
|
|
</pre>This can be moved to a template so that several types (double, int,
|
|
long, etc.) can be wrapped with the same code. This technique is used in the
|
|
file
|
|
<blockquote>
|
|
scitbx/include/scitbx/array_family/boost_python/flex_wrapper.h
|
|
</blockquote>in the "scitbx" package. The file could easily be
|
|
modified for wrapping std::vector<> instantiations.
|
|
<p>
|
|
This type of C++/Python binding is most suitable for containers
|
|
that may contain a large number of elements (>10000).
|
|
</p>
|
|
</li>
|
|
<li>
|
|
Using custom rvalue converters. Boost.Python "rvalue converters"
|
|
match function signatures such as:
|
|
<pre>
|
|
void foo(std::vector<double> const& array); // pass by const-reference
|
|
void foo(std::vector<double> array); // pass by value
|
|
</pre>Some custom rvalue converters are implemented in the file
|
|
<blockquote>
|
|
scitbx/include/scitbx/boost_python/container_conversions.h
|
|
</blockquote>This code can be used to convert from C++ container
|
|
types such as std::vector<> or std::list<> to Python
|
|
tuples and vice versa. A few simple examples can be found in the file
|
|
|
|
<blockquote>
|
|
scitbx/array_family/boost_python/regression_test_module.cpp
|
|
</blockquote>Automatic C++ container <-> Python tuple
|
|
conversions are most suitable for containers of moderate size. These
|
|
converters generate significantly less object code compared to
|
|
alternative 1 above.
|
|
</li>
|
|
</ol>A disadvantage of using alternative 2 is that operators such as
|
|
arithmetic +,-,*,/,% are not available. It would be useful to have custom
|
|
rvalue converters that convert to a "math_array" type instead of tuples.
|
|
This is currently not implemented but is possible within the framework of
|
|
Boost.Python V2 as it will be released in the next couple of weeks. [ed.:
|
|
this was posted on 2002/03/10]
|
|
<p>
|
|
It would also be useful to also have "custom lvalue converters" such as
|
|
std::vector<> <-> Python list. These converters would
|
|
support the modification of the Python list from C++. For example:
|
|
</p>
|
|
<p>
|
|
C++:
|
|
</p>
|
|
<pre>
|
|
void foo(std::vector<double>& array)
|
|
{
|
|
for(std::size_t i=0;i<array.size();i++) {
|
|
array[i] *= 2;
|
|
}
|
|
}
|
|
</pre>Python:
|
|
<pre>
|
|
>>> l = [1, 2, 3]
|
|
>>> foo(l)
|
|
>>> print l
|
|
[2, 4, 6]
|
|
</pre>Custom lvalue converters require changes to the Boost.Python core
|
|
library and are currently not available.
|
|
<p>
|
|
P.S.:
|
|
</p>
|
|
<p>
|
|
The "scitbx" files referenced above are available via anonymous CVS:
|
|
</p>
|
|
<pre>
|
|
cvs -d:pserver:anonymous@cvs.cctbx.sourceforge.net:/cvsroot/cctbx login
|
|
cvs -d:pserver:anonymous@cvs.cctbx.sourceforge.net:/cvsroot/cctbx co scitbx
|
|
</pre>
|
|
<hr>
|
|
<h2>
|
|
<a name="c1204"></a>fatal error C1204:Compiler limit:internal structure
|
|
overflow
|
|
</h2>
|
|
<blockquote>
|
|
<b>Q:</b> <i>I get this error message when compiling a large source
|
|
file. What can I do?</i>
|
|
<p>
|
|
<b>A:</b> You have two choices:
|
|
</p>
|
|
<ol>
|
|
<li>
|
|
Upgrade your compiler (preferred)
|
|
</li>
|
|
<li>
|
|
Break your source file up into multiple translation units.
|
|
<p>
|
|
<code><b>my_module.cpp</b></code>:
|
|
</p>
|
|
<pre>
|
|
...
|
|
void more_of_my_module();
|
|
BOOST_PYTHON_MODULE(my_module)
|
|
{
|
|
def("foo", foo);
|
|
def("bar", bar);
|
|
...
|
|
more_of_my_module();
|
|
}
|
|
</pre><code><b>more_of_my_module.cpp</b></code>:
|
|
<pre>
|
|
void more_of_my_module()
|
|
{
|
|
def("baz", baz);
|
|
...
|
|
}
|
|
</pre>If you find that a <code><a href=
|
|
"class.html#class_-spec">class_</a><...></code> declaration can't fit
|
|
in a single source file without triggering the error, you can always pass a
|
|
reference to the <code>class_</code> object to a function in another source
|
|
file, and call some of its member functions (e.g. <code>.def(...)</code>) in
|
|
the auxilliary source file:
|
|
<p>
|
|
<code><b>more_of_my_class.cpp</b></code>:
|
|
</p>
|
|
<pre>
|
|
void more_of_my_class(class<my_class>& x)
|
|
{
|
|
x
|
|
.def("baz", baz)
|
|
.add_property("xx", &my_class::get_xx, &my_class::set_xx)
|
|
;
|
|
|
|
...
|
|
}
|
|
</pre>
|
|
</li>
|
|
</ol>
|
|
</blockquote>
|
|
<hr>
|
|
<h2>
|
|
<a name="debugging"></a>How do I debug my Python extensions?
|
|
</h2>
|
|
<p>
|
|
Greg Burley gives the following answer for Unix GCC users:
|
|
</p>
|
|
<blockquote>
|
|
Once you have created a boost python extension for your c++ library or
|
|
class, you may need to debug the code. Afterall this is one of the
|
|
reasons for wrapping the library in python. An expected side-effect or
|
|
benefit of using BPL is that debugging should be isolated to the c++
|
|
library that is under test, given that python code is minimal and
|
|
boost::python either works or it doesn't. (ie. While errors can occur
|
|
when the wrapping method is invalid, most errors are caught by the
|
|
compiler ;-).
|
|
<p>
|
|
The basic steps required to initiate a gdb session to debug a c++
|
|
library via python are shown here. Note, however that you should
|
|
start the gdb session in the directory that contains your BPL
|
|
my_ext.so module.
|
|
</p>
|
|
<pre>
|
|
(gdb) target exec python
|
|
(gdb) run
|
|
>>> from my_ext import *
|
|
>>> [C-c]
|
|
(gdb) break MyClass::MyBuggyFunction
|
|
(gdb) cont
|
|
>>> pyobj = MyClass()
|
|
>>> pyobj.MyBuggyFunction()
|
|
Breakpoint 1, MyClass::MyBuggyFunction ...
|
|
Current language: auto; currently c++
|
|
(gdb) do debugging stuff
|
|
</pre>
|
|
</blockquote>
|
|
<p>
|
|
Greg's approach works even better using Emacs' "<code>gdb</code>"
|
|
command, since it will show you each line of source as you step through
|
|
it.
|
|
</p>
|
|
<p>
|
|
On <b>Windows</b>, my favorite debugging solution is the debugger that
|
|
comes with Microsoft Visual C++ 7. This debugger seems to work with
|
|
code generated by all versions of Microsoft and Metrowerks toolsets;
|
|
it's rock solid and "just works" without requiring any special tricks
|
|
from the user.
|
|
</p>
|
|
<p>
|
|
Unfortunately for Cygwin and MinGW users, as of this writing gdb on
|
|
Windows has a very hard time dealing with shared libraries, which could
|
|
make Greg's approach next to useless for you. My best advice for you is
|
|
to use Metrowerks C++ for compiler conformance and Microsoft Visual
|
|
Studio as a debugger when you need one.
|
|
</p>
|
|
<h3>
|
|
Debugging extensions through Boost.Build
|
|
</h3>If you are launching your extension module tests with <a href=
|
|
"../../../tools/build">Boost.Build</a> using the
|
|
<code>boost-python-runtest</code> rule, you can ask it to launch your
|
|
debugger for you by adding "-sPYTHON_LAUNCH=<i>debugger</i>" to your bjam
|
|
command-line:
|
|
<pre>
|
|
bjam -sTOOLS=metrowerks "-sPYTHON_LAUNCH=devenv /debugexe" test
|
|
bjam -sTOOLS=gcc -sPYTHON_LAUNCH=gdb test
|
|
</pre>It can also be extremely useful to add the <code>-d+2</code> option
|
|
when you run your test, because Boost.Build will then show you the exact
|
|
commands it uses to invoke it. This will invariably involve setting up
|
|
PYTHONPATH and other important environment variables such as LD_LIBRARY_PATH
|
|
which may be needed by your debugger in order to get things to work right.
|
|
<hr>
|
|
<h2>
|
|
<a name="imul"></a>Why doesn't my <code>*=</code> operator work?
|
|
</h2>
|
|
<blockquote>
|
|
<b>Q:</b> <i>I have exported my class to python, with many overloaded
|
|
operators. it works fine for me except the</i> <code>*=</code>
|
|
<i>operator. It always tells me "can't multiply sequence with non int
|
|
type". If I use</i> <code>p1.__imul__(p2)</code> <i>instead of</i>
|
|
<code>p1 *= p2</code><i>, it successfully executes my code. What's
|
|
wrong with me?</i>
|
|
<p>
|
|
<b>A:</b> There's nothing wrong with you. This is a bug in Python
|
|
2.2. You can see the same effect in Pure Python (you can learn a lot
|
|
about what's happening in Boost.Python by playing with new-style
|
|
classes in Pure Python).
|
|
</p>
|
|
<pre>
|
|
>>> class X(object):
|
|
... def __imul__(self, x):
|
|
... print 'imul'
|
|
...
|
|
>>> x = X()
|
|
>>> x *= 1
|
|
</pre>To cure this problem, all you need to do is upgrade your Python to
|
|
version 2.2.1 or later.
|
|
</blockquote>
|
|
<hr>
|
|
<h2>
|
|
<a name="macosx"></a>Does Boost.Python work with Mac OS X?
|
|
</h2>
|
|
<blockquote>
|
|
<p>
|
|
The short answer: as of January 2003, unfortunately not.
|
|
</p>
|
|
<p>
|
|
The longer answer: using Mac OS 10.2.3 with the December Developer's
|
|
Kit, Python 2.3a1, and bjam's darwin-tools.jam, Boost.Python compiles
|
|
fine, including the examples. However, there are problems at runtime
|
|
(see <a href=
|
|
"http://mail.python.org/pipermail/c++-sig/2003-January/003267.html">http://mail.python.org/pipermail/c++-sig/2003-January/003267.html</a>).
|
|
Solutions are currently unknown.
|
|
</p>
|
|
<p>
|
|
It is known that under certain circumstances objects are
|
|
double-destructed. See <a href=
|
|
"http://mail.python.org/pipermail/c++-sig/2003-January/003278.html">http://mail.python.org/pipermail/c++-sig/2003-January/003278.html</a>
|
|
for details. It is not clear however if this problem is related to
|
|
the Boost.Python runtime issues.
|
|
</p>
|
|
</blockquote>
|
|
<hr>
|
|
<h2>
|
|
<a name="xref">How can I find the existing PyObject that holds a C++
|
|
object?</a>
|
|
</h2>
|
|
<blockquote>
|
|
"I am wrapping a function that always returns a pointer to an
|
|
already-held C++ object."
|
|
</blockquote>One way to do that is to hijack the mechanisms used for
|
|
wrapping a class with virtual functions. If you make a wrapper class with
|
|
an initial PyObject* constructor argument and store that PyObject* as
|
|
"self", you can get back to it by casting down to that wrapper type in a
|
|
thin wrapper function. For example:
|
|
<pre>
|
|
class X { X(int); virtual ~X(); ... };
|
|
X* f(); // known to return Xs that are managed by Python objects
|
|
|
|
|
|
// wrapping code
|
|
|
|
struct X_wrap : X
|
|
{
|
|
X_wrap(PyObject* self, int v) : self(self), X(v) {}
|
|
PyObject* self;
|
|
};
|
|
|
|
handle<> f_wrap()
|
|
{
|
|
X_wrap* xw = dynamic_cast<X_wrap*>(f());
|
|
assert(xw != 0);
|
|
return handle<>(borrowed(xw->self));
|
|
}
|
|
|
|
...
|
|
|
|
def("f", f_wrap());
|
|
class_<X,X_wrap>("X", init<int>())
|
|
...
|
|
;
|
|
</pre>Of course, if X has no virtual functions you'll have to use
|
|
<code>static_cast</code> instead of <code>dynamic_cast</code> with no runtime
|
|
check that it's valid. This approach also only works if the <code>X</code>
|
|
object was constructed from Python, because <code>X</code>s constructed from
|
|
C++ are of course never <code>X_wrap</code> objects.
|
|
<p>
|
|
Another approach to this requires some work on Boost.Python, but it's
|
|
work we've been meaning to get to anyway. Currently, when a
|
|
<code>shared_ptr<X></code> is converted from Python, the
|
|
shared_ptr actually manages a reference to the containing Python
|
|
object. I plan to make it so that when a shared_ptr<X> is
|
|
converted back to Python, the library checks to see if it's one of
|
|
those "Python object managers" and if so just returns the original
|
|
Python object. To exploit this you'd have to be able to change the C++
|
|
code you're wrapping so that it deals with shared_ptr instead of raw
|
|
pointers.
|
|
</p>
|
|
<p>
|
|
There are other approaches too. The functions that receive the Python
|
|
object that you eventually want to return could be wrapped with a thin
|
|
wrapper that records the correspondence between the object address and
|
|
its containing Python object, and you could have your f_wrap function
|
|
look in that mapping to get the Python object out.
|
|
</p>
|
|
<h2>
|
|
<a name="ownership">How can I wrap a function which needs to take
|
|
ownership of a raw pointer?</a>
|
|
</h2>
|
|
<blockquote>
|
|
<i>Part of an API that I'm wrapping goes something like this:</i>
|
|
<pre>
|
|
struct A {}; struct B { void add( A* ); }
|
|
where B::add() takes ownership of the pointer passed to it.
|
|
</pre>
|
|
<p>
|
|
<i>However:</i>
|
|
</p>
|
|
<pre>
|
|
a = mod.A()
|
|
b = mod.B()
|
|
b.add( a )
|
|
del a
|
|
del b
|
|
# python interpreter crashes
|
|
# later due to memory corruption.
|
|
</pre>
|
|
<p>
|
|
<i>Even binding the lifetime of a</i> to b via
|
|
with_custodian_and_ward doesn't prevent the python object a from
|
|
ultimately trying to delete the object it's pointing to. Is there a
|
|
way to accomplish a 'transfer-of-ownership' of a wrapped C++ object?
|
|
</p>
|
|
<p>
|
|
<i>--Bruce Lowery</i>
|
|
</p>
|
|
</blockquote>Yes: Make sure the C++ object is held by auto_ptr:
|
|
<pre>
|
|
class_<A, std::auto_ptr<A> >("A")
|
|
...
|
|
;
|
|
</pre>Then make a thin wrapper function which takes an auto_ptr parameter:
|
|
<pre>
|
|
void b_insert(B& b, std::auto_ptr<A> a)
|
|
{
|
|
b.insert(a.get());
|
|
a.release();
|
|
}
|
|
</pre>Wrap that as B.add. Note that pointers returned via
|
|
<code><a href="manage_new_object.html#manage_new_object-spec">manage_new_object</a></code>
|
|
will also be held by <code>auto_ptr</code>, so this
|
|
transfer-of-ownership will also work correctly.
|
|
<hr>
|
|
<p>
|
|
Revised
|
|
<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan -->
|
|
23 January, 2003
|
|
<!--webbot bot="Timestamp" endspan i-checksum="39359" -->
|
|
</p>
|
|
<p>
|
|
<i>© Copyright <a href="../../../../people/dave_abrahams.htm">Dave
|
|
Abrahams</a> 2002-2003. All Rights Reserved.</i>
|
|
</p>
|
|
</body>
|
|
</html>
|