boost/python
2
0
Fork 0
mirror of https://github.com/boostorg/python.git synced 2026-01-24 18:12:43 +00:00
Code Issues Projects Releases Wiki Activity

New export_converters() interface.

Browse Source 
[SVN r9604]
This commit is contained in:
Ralf W. Grosse-Kunstleve
2001-03-20 02:44:20 +00:00
parent e21d518511
commit feadcfe0a2
1 changed files with 145 additions and 102 deletions
Download Patch File Download Diff File Expand all files Collapse all files

247
doc/cross_module_dependencies.html
View File

@@ -1,15 +1,15 @@
<html>
<head>
<title>Cross-extension-module dependencies</title>
</head>
<body>
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.0//EN"
"http://www.w3.org/TR/REC-html40/strict.dtd">
<title>Cross-extension-module dependencies</title>
<div>
<img src="../../../c++boost.gif"
alt="c++boost.gif (8819 bytes)"
align="center"
width="277" height="86">
width="277" height="86">
</body>
<hr>
<h1>Cross-extension-module dependencies</h1>
@@ -27,11 +27,12 @@ provides for classes that are wrapped in the same extension module. To
a large extent this complexity can be hidden from the author of a
Boost.Python extension module, but not entirely.
<h1>The recipe</h1>
<hr>
<h2>The recipe</h2>
Suppose there is an extension module that exposes certain instances of
the C++ std::vector template library such that it can be used from
Python in the following manner:
the C++ <tt>std::vector</tt> template library such that it can be used
from Python in the following manner:
<pre>
import std_vector
@@ -40,79 +41,106 @@ v.push_back(5)
v.size()
</pre>
Suppose the std_vector module is done well and reflects all C++
functions that are useful at the Python level, for all C++ built-in
data types (std_vector.int, std_vector.long, etc.).
Suppose the <tt>std_vector</tt> module is done well and reflects all
C++ functions that are useful at the Python level, for all C++ built-in
data types (<tt>std_vector.int</tt>, <tt>std_vector.long</tt>, etc.).
<p>
Suppose further that there is statistic module with a C++ class that
has constructors or member functions that use or return a std::vector.
For example:
has constructors or member functions that use or return a
<tt>std::vector</tt>. For example:
<pre>
class xy {
private:
std::vector&lt;double&gt; m_x;
std::vector&lt;double&gt; m_y;
public:
xy(const std::vector&lt;double&gt;&amp; x, const std::vector&lt;double&gt;&amp; y) : m_x(x), m_y(y) {}
const std::vector&lt;double&gt;&amp; x() const { return m_x; }
const std::vector&lt;double&gt;&amp; y() const { return m_y; }
double correlation();
private:
std::vector&lt;double&gt; m_x;
std::vector&lt;double&gt; m_y;
}
</pre>
What is more natural then reusing the std_vector extension module to
expose these constructors or functions to Python?
What is more natural than reusing the <tt>std_vector</tt> extension
module to expose these constructors or functions to Python?
<p>
Unfortunately, what seems natural needs a little work in both the
std_vector and the statistics module.
<tt>std_vector</tt> and the <tt>statistics</tt> module.
<p>
In the std_vector extension module, std::vector&lt;double&gt; needs to be
exposed to Python with the x_class_builder&lt;&gt; template instead of the
regular class_builder&lt;&gt;. For example:
In the <tt>std_vector</tt> extension module,
<tt>std::vector&lt;double&gt;</tt> is exposed to Python in the usual
way with the <tt>class_builder&lt;&gt;</tt> template. To also enable the
automatic conversion of <tt>std::vector&lt;double&gt;</tt> function
arguments or return values in other Boost.Python C++ modules, the
converters that convert a <tt>std::vector&lt;double&gt;</tt> C++ object
to a Python object and vice versa (see last section for details) have
to be exported. For example:
<pre>
x_class_builder&lt;std::vector&lt;double&gt; &gt; v_double(std_vector_module, &quot;double&quot;);
class_builder&lt;std::vector&lt;double&gt; &gt; v_double(std_vector_module, &quot;double&quot;);
export_converters(v_double);
</pre>
In the extension module that wraps class xy we need to use
the import_class_builder&lt;&gt; template:
<pre>
import_class_builder&lt;std::vector&lt;double&gt; &gt; v_double(&quot;std_vector&quot;, &quot;double&quot;);
</pre>
That is all. All the properties that are defined for std_vector.double
in the std_vector Boost.Python module will be available for the
returned objects of xy.x() and xy.y(). Similarly, the constructor for
xy will accept objects that were created by the std_vector module.
<h1>Non-copyable types</h1>
The x_class_builder&lt;T&gt; instantiates template functions that invoke the
copy constructor of T. For a T that is non-copyable this will result in
compile-time error messages. In such a case, another variety of the
class_builder&lt;&gt;, the xptr_class_builder&lt;&gt; must be used.
In the extension module that wraps <tt>class xy</tt> we can now import
these converters with the <tt>import_converters&lt;&gt;</tt> template.
For example:
<pre>
xptr_class_builder&lt;store&gt; py_store(your_module, &quot;store&quot;);
import_converters&lt;std::vector&lt;double&gt; &gt; v_double_converters(&quot;std_vector&quot;, &quot;double&quot;);
</pre>
The corresponding import_class_builder&lt;&gt; does not need any special
attention:
That is all. All the attributes that are defined for
<tt>std_vector.double</tt> in the <tt>std_vector</tt> Boost.Python
module will be available for the returned objects of <tt>xy.x()</tt>
and <tt>xy.y()</tt>. Similarly, the constructor for <tt>xy</tt> will
accept objects that were created by the <tt>std_vector</tt>module.
<hr>
<h2>Placement of <tt>import_converters&lt;&gt;</tt> template instantiations</h2>
<tt>import_converts&lt;&gt;</tt> can be viewed as a drop-in replacement
for <tt>class_wrapper&lt;&gt;</tt>, and the recommendations for the
placement of <tt>class_wrapper&lt;&gt;</tt> template instantiations
also apply to to <tt>import_converts&lt;&gt;</tt>. In particular, it is
important that an instantiation of <tt>class_wrapper&lt;&gt;</tt> is
visible to any code which wraps a C++ function with a <tt>T</tt>,
<tt>T*</tt>, const <tt>T&amp;</tt>, etc. parameter or return value.
Therefore you may want to group all <tt>class_wrapper&lt;&gt;</tt> and
<tt>import_converts&lt;&gt;</tt> instantiations at the top of your
module's init function, then <tt>def()</tt> the member functions later
to avoid problems with inter-class dependencies.
<hr>
<h2>Non-copyable types</h2>
<tt>export_converters()</tt> instantiates C++ template functions that
invoke the copy constructor of the wrapped type. For a type that is
non-copyable this will result in compile-time error messages. In such a
case, <tt>export_converters_noncopyable()</tt> can be used to export
the converters that do not involve the copy constructor of the wrapped
type. For example:
<pre>
import_class_builder&lt;store&gt; py_store(&quot;noncopyable_export&quot;, &quot;store&quot;);
class_builder&lt;store&gt; py_store(your_module, &quot;store&quot;);
export_converters_noncopyable(py_store);
</pre>
<h1>Python module search path</h1>
The corresponding <tt>import_converters()</tt> statement does not need
any special attention:
The std_vector and statistics modules can now be used in the following
way:
<pre>
import_converters&lt;store&gt; py_store(&quot;noncopyable_export&quot;, &quot;store&quot;);
</pre>
<hr>
<h2>Python module search path</h2>
The <tt>std_vector</tt> and <tt>statistics</tt> modules can now be used
in the following way:
<pre>
import std_vector
@@ -124,14 +152,14 @@ xy.correlation()
</pre>
In this example it is clear that Python has to be able to find both the
std_vector and the statistics extension module. In other words, both
extension modules need to be in the Python module search path
(sys.path).
<tt>std_vector</tt> and the <tt>statistics</tt> extension module. In
other words, both extension modules need to be in the Python module
search path (<tt>sys.path</tt>).
<p>
The situation is not always that obvious. Suppose the statistics
module has a random function that returns a vector of random
numbers with a given length:
The situation is not always this obvious. Suppose the
<tt>statistics</tt> module has a random function that returns a vector
of random numbers with a given length:
<pre>
import statistics
@@ -141,15 +169,15 @@ xy = statistics.xy(x, y)
xy.correlation()
</pre>
A naive user will not easily anticipate that the std_vector module is
used to pass the x and y vectors around. If the std_vector module is in
the Python module search path, this form of ignorance is of no harm.
On the contrary, we are glad that we do not have to bother the user
with details like this.
A naive user will not easily anticipate that the <tt>std_vector</tt>
module is used to pass the <tt>x</tt> and <tt>y</tt> vectors around. If
the <tt>std_vector</tt> module is in the Python module search path,
this form of ignorance is of no harm. On the contrary, we are glad
that we do not have to bother the user with details like this.
<p>
If the std_vector module is not in the Python module search path, a
Python exception will be raised:
If the <tt>std_vector</tt> module is not in the Python module search
path, a Python exception will be raised:
<pre>
Traceback (innermost last):
@@ -161,16 +189,17 @@ ImportError: No module named std_vector
As is the case with any system of a non-trivial complexity, it is
important that the setup is consistent and complete.
<h1>Two-way module dependencies</h1>
<hr>
<h2>Two-way module dependencies</h2>
Boost.Python supports two-way module dependencies. This is best
illustrated by a simple example.
<p>
Suppose there is a module ivect that implements vectors of integers,
and a similar module dvect that implements vectors of doubles. We want
to be able do convert an integer vector to a double vector and vice
versa. For example:
Suppose there is a module <tt>ivect</tt> that implements vectors of
integers, and a similar module <tt>dvect</tt> that implements vectors
of doubles. We want to be able do convert an integer vector to a double
vector and vice versa. For example:
<pre>
import ivect
@@ -178,9 +207,10 @@ iv = ivect.ivect((1,2,3,4,5))
dv = iv.as_dvect()
</pre>
The last expression will implicitly import the dvect module in order to
enable the conversion of the C++ representation of dvect to a Python
object. The analogous is possible for a dvect:
The last expression will implicitly import the <tt>dvect</tt> module in
order to enable the conversion of the C++ representation of
<tt>dvect</tt> to a Python object. The analogous is possible for a
<tt>dvect</tt>:
<pre>
import dvect
@@ -188,44 +218,49 @@ dv = dvect.dvect((1,2,3,4,5))
iv = dv.as_ivect()
</pre>
Now the ivect module is imported implicitly.
Now the <tt>ivect</tt> module is imported implicitly.
<p>
Note that the two-way dependencies are possible because the
dependencies are resolved only when needed. This is, the initialization
of the ivect module does not rely on the dvect module, and vice versa.
Only if as_dvect() or as_ivect() is actually invoked will the
corresponding module be implicitly imported. This also means that, for
example, the dvect module does not have to be available at all if
as_dvect() is never used.
of the <tt>ivect</tt> module does not rely on the <tt>dvect</tt>
module, and vice versa. Only if <tt>as_dvect()</tt> or
<tt>as_ivect()</tt> is actually invoked will the corresponding module
be implicitly imported. This also means that, for example, the
<tt>dvect</tt> module does not have to be available at all if
<tt>as_dvect()</tt> is never used.
<h1>Clarification of compile-time and link-time dependencies</h1>
<hr>
<h2>Clarification of compile-time and link-time dependencies</h2>
Boost.Python's support for resolving cross-module dependencies at
runtime does not imply that compile-time dependencies are eliminated.
For example, the statistics extension module in the example above will
need to #include &lt;vector&gt;. This is immediately obvious from the
definition of class xy.
need to <tt>#include &lt;vector&gt;</tt>. This is immediately obvious
from the definition of <tt>class xy</tt>.
<p>
If a library is wrapped that consists of both header files and compiled
components (e.g. libdvect.a, dvect.lib, etc.), both the Boost.Python
extension module with the x_class_wrapper&lt;&gt; and the module with the
import_class_wrapper&lt;&gt; need to be linked against the object library.
Ideally one would build a shared library (e.g. libdvect.so, dvect.dll,
etc.). However, this introduces the issue of getting the search path
for the dynamic loading configured correctly. For small libraries it is
therefore often more convenient to ignore the fact that the object
files are loaded into memory more than once.
components (e.g. <tt>libdvect.a</tt>, <tt>dvect.lib</tt>, etc.), both
the Boost.Python extension module with the
<tt>export_converters()</tt> statement and the module with the
<tt>import_converters()</tt> statement need to be linked against
the object library. Ideally one would build a shared library (e.g.
<tt>libdvect.so</tt>, <tt>dvect.dll</tt>, etc.). However, this
introduces the issue of getting the search path for the dynamic loading
configured correctly. For small libraries it is therefore often more
convenient to ignore the fact that the object files are loaded into
memory more than once.
<p>
The main purpose of Boost.Python's support for resolving cross-module
dependencies at runtime is to allow for a modular system layout. With
this support it is straightforward to reflect C++ code organization at
the Python level. Without the cross-module support, a multi-purpose
module like std_vector would be impractical because the entire wrapper
code would somehow have to be duplicated in all extension modules that
use it, making them harder to maintain and harder to build.
module like <tt>std_vector</tt> would be impractical because the entire
wrapper code would somehow have to be duplicated in all extension
modules that use it, making them harder to maintain and harder to
build.
<p>
Finally, there is an important psychological component. If a group of
@@ -235,19 +270,27 @@ The situation is much more transparent if the work is represented by
a module with a recognizable name. This is not just a question of
strong egos, but also of getting credit and funding.
<h1>Why not use the x_class_builder universally?</h1>
<hr>
<h2>Why not use <tt>export_converters()</tt> universally?</h2>
There is some overhead associated with the Boost.Python cross-module
support. Depending on the platform, the code generated by
x_class_builder&lt;&gt; is roughly 10%-20% larger than that generated by
class_builder&lt;&gt;. For a large extension module with many wrapped
classes, this could mean a significant difference. Therefore the
general recommendation is to use x_class_wrapper&lt;&gt; only for classes
that are likely to be used as function arguments or return values in
other modules.
support. Depending on the platform, the size of the code generated by
<tt>export_converters()</tt> is roughly 10%-20% of that generated
by <tt>class_builder&lt;&gt;</tt>. For a large extension module with
many wrapped classes, this could mean a significant difference.
Therefore the general recommendation is to use
<tt>export_converters()</tt> only for classes that are likely to
be used as function arguments or return values in other modules.
<hr>
<address>
Author: Ralf W. Grosse-Kunstleve, March 2001
&copy; Copyright Ralf W. Grosse-Kunstleve 2001. Permission to copy,
use, modify, sell and distribute this document is granted provided this
copyright notice appears in all copies. This document is provided "as
is" without express or implied warranty, and with no claim as to its
suitability for any purpose.
<p>
Updated: March 2001
</address>
</html>
</div>