boost/python
2
0
Fork 0
mirror of https://github.com/boostorg/python.git synced 2026-01-24 06:02:14 +00:00
Code Issues Projects Releases Wiki Activity

temp file before branching

Browse Source 
[SVN r9515]
This commit is contained in:
Ralf W. Grosse-Kunstleve
2001-03-09 02:40:06 +00:00
parent a559480716
commit 33ea0dbdee
1 changed files with 253 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

253
doc/cross_module_dependencies.html Normal file
View File

@@ -0,0 +1,253 @@
<html>
<head>
<title>Cross-extension-module dependencies</title>
</head>
<body>
<img src="../../../c++boost.gif"
alt="c++boost.gif (8819 bytes)"
align="center"
width="277" height="86">
</body>
<hr>
<h1>Cross-extension-module dependencies</h1>
It is good programming practice to organize large projects as modules
that interact with each other via well defined interfaces. With
Boost.Python it is possible to reflect this organization at the C++
level at the Python level. This is, each logical C++ module can be
organized as a separate Python extension module.
<p>
At first sight this might seem natural and straightforward. However, it
is a fairly complex problem to establish cross-extension-module
dependencies while maintaining the same ease of use Boost.Python
provides for classes that are wrapped in the same extension module. To
a large extent this complexity can be hidden from the author of a
Boost.Python extension module, but not entirely.
<h1>The recipe</h1>
Suppose there is an extension module that exposes certain instances of
the C++ std::vector template library such that it can be used from
Python in the following manner:
<pre>
import std_vector
v = std_vector.double([1, 2, 3, 4])
v.push_back(5)
v.size()
</pre>
Suppose the std_vector module is done well and reflects all C++
functions that are useful at the Python level, for all C++ built-in
data types (std_vector.int, std_vector.long, etc.).
<p>
Suppose further that there is statistic module with a C++ class that
has constructors or member functions that use or return a std::vector.
For example:
<pre>
class xy {
private:
std::vector&lt;double&gt; m_x;
std::vector&lt;double&gt; m_y;
public:
xy(const std::vector&lt;double&gt;&amp; x, const std::vector&lt;double&gt;&amp; y) : m_x(x), m_y(y) {}
const std::vector&lt;double&gt;&amp; x() const { return m_x; }
const std::vector&lt;double&gt;&amp; y() const { return m_y; }
double correlation();
}
</pre>
What is more natural then reusing the std_vector extension module to
expose these constructors or functions to Python?
<p>
Unfortunately, what seems natural needs a little work in both the
std_vector and the statistics module.
<p>
In the std_vector extension module, std::vector&lt;double&gt; needs to be
exposed to Python with the x_class_builder&lt;&gt; template instead of the
regular class_builder&lt;&gt;. For example:
<pre>
x_class_builder&lt;std::vector&lt;double&gt; &gt; v_double(std_vector_module, &quot;double&quot;);
</pre>
In the extension module that wraps class xy we need to use
the import_class_builder&lt;&gt; template:
<pre>
import_class_builder&lt;std::vector&lt;double&gt; &gt; v_double(&quot;std_vector&quot;, &quot;double&quot;);
</pre>
That is all. All the properties that are defined for std_vector.double
in the std_vector Boost.Python module will be available for the
returned objects of xy.x() and xy.y(). Similarly, the constructor for
xy will accept objects that were created by the std_vector module.
<h1>Non-copyable types</h1>
The x_class_builder&lt;T&gt; instantiates template functions that invoke the
copy constructor of T. For a T that is non-copyable this will result in
compile-time error messages. In such a case, another variety of the
class_builder&lt;&gt;, the xptr_class_builder&lt;&gt; must be used.
For example:
<pre>
xptr_class_builder&lt;store&gt; py_store(your_module, &quot;store&quot;);
</pre>
The corresponding import_class_builder&lt;&gt; does not need any special
attention:
<pre>
import_class_builder&lt;store&gt; py_store(&quot;noncopyable_export&quot;, &quot;store&quot;);
</pre>
<h1>Python module search path</h1>
The std_vector and statistics modules can now be used in the following
way:
<pre>
import std_vector
import statistics
x = std_vector.double([1, 2, 3, 4])
y = std_vector.double([2, 4, 6, 8])
xy = statistics.xy(x, y)
xy.correlation()
</pre>
In this example it is clear that Python has to be able to find both the
std_vector and the statistics extension module. In other words, both
extension modules need to be in the Python module search path
(sys.path).
<p>
The situation is not always that obvious. Suppose the statistics
module has a random function that returns a vector of random
numbers with a given length:
<pre>
import statistics
x = statistics.random(5)
y = statistics.random(5)
xy = statistics.xy(x, y)
xy.correlation()
</pre>
A naive user will not easily anticipate that the std_vector module is
used to pass the x and y vectors around. If the std_vector module is in
the Python module search path, this form of ignorance is of no harm.
On the contrary, we are glad that we do not have to bother the user
with details like this.
<p>
If the std_vector module is not in the Python module search path, a
Python exception will be raised:
<pre>
Traceback (innermost last):
File &quot;foo.py&quot;, line 2, in ?
x = statistics.random(5)
ImportError: No module named std_vector
</pre>
As is the case with any system of a non-trivial complexity, it is
important that the setup is consistent and complete.
<h1>Two-way module dependencies</h1>
Boost.Python supports two-way module dependencies. This is best
illustrated by a simple example.
<p>
Suppose there is a module ivect that implements vectors of integers,
and a similar module dvect that implements vectors of doubles. We want
to be able do convert an integer vector to a double vector and vice
versa. For example:
<pre>
import ivect
iv = ivect.ivect((1,2,3,4,5))
dv = iv.as_dvect()
</pre>
The last expression will implicitly import the dvect module in order to
enable the conversion of the C++ representation of dvect to a Python
object. The analogous is possible for a dvect:
<pre>
import dvect
dv = dvect.dvect((1,2,3,4,5))
iv = dv.as_ivect()
</pre>
Now the ivect module is imported implicitly.
<p>
Note that the two-way dependencies are possible because the
dependencies are resolved only when needed. This is, the initialization
of the ivect module does not rely on the dvect module, and vice versa.
Only if as_dvect() or as_ivect() is actually invoked will the
corresponding module be implicitly imported. This also means that, for
example, the dvect module does not have to be available at all if
as_dvect() is never used.
<h1>Clarification of compile-time and link-time dependencies</h1>
Boost.Python's support for resolving cross-module dependencies at
runtime does not imply that compile-time dependencies are eliminated.
For example, the statistics extension module in the example above will
need to #include &lt;vector&gt;. This is immediately obvious from the
definition of class xy.
<p>
If a library is wrapped that consists of both header files and compiled
components (e.g. libdvect.a, dvect.lib, etc.), both the Boost.Python
extension module with the x_class_wrapper&lt;&gt; and the module with the
import_class_wrapper&lt;&gt; need to be linked against the object library.
Ideally one would build a shared library (e.g. libdvect.so, dvect.dll,
etc.). However, this introduces the issue of getting the search path
for the dynamic loading configured correctly. For small libraries it is
therefore often more convenient to ignore the fact that the object
files are loaded into memory more than once.
<p>
The main purpose of Boost.Python's support for resolving cross-module
dependencies at runtime is to allow for a modular system layout. With
this support it is straightforward to reflect C++ code organization at
the Python level. Without the cross-module support, a multi-purpose
module like std_vector would be impractical because the entire wrapper
code would somehow have to be duplicated in all extension modules that
use it, making them harder to maintain and harder to build.
<p>
Finally, there is an important psychological component. If a group of
classes is lumped together with many others in a huge module, the
authors will have difficulties in being identified with their work.
The situation is much more transparent if the work is represented by
a module with a recognizable name. This is not just a question of
strong egos, but also of getting credit and funding.
<h1>Why not use the x_class_builder universally?</h1>
There is some overhead associated with the Boost.Python cross-module
support. Depending on the platform, the code generated by
x_class_builder&lt;&gt; is roughly 10%-20% larger than that generated by
class_builder&lt;&gt;. For a large extension module with many wrapped
classes, this could mean a significant difference. Therefore the
general recommendation is to use x_class_wrapper&lt;&gt; only for classes
that are likely to be used as function arguments or return values in
other modules.
<hr>
<address>
Author: Ralf W. Grosse-Kunstleve, March 2001
</address>
</html>