mirror of
https://github.com/boostorg/python.git
synced 2026-01-22 05:22:45 +00:00
254 lines
8.6 KiB
HTML
254 lines
8.6 KiB
HTML
<html>
|
|
<head>
|
|
<title>Cross-extension-module dependencies</title>
|
|
</head>
|
|
<body>
|
|
|
|
<img src="../../../c++boost.gif"
|
|
alt="c++boost.gif (8819 bytes)"
|
|
align="center"
|
|
width="277" height="86">
|
|
|
|
</body>
|
|
<hr>
|
|
<h1>Cross-extension-module dependencies</h1>
|
|
|
|
It is good programming practice to organize large projects as modules
|
|
that interact with each other via well defined interfaces. With
|
|
Boost.Python it is possible to reflect this organization at the C++
|
|
level at the Python level. This is, each logical C++ module can be
|
|
organized as a separate Python extension module.
|
|
|
|
<p>
|
|
At first sight this might seem natural and straightforward. However, it
|
|
is a fairly complex problem to establish cross-extension-module
|
|
dependencies while maintaining the same ease of use Boost.Python
|
|
provides for classes that are wrapped in the same extension module. To
|
|
a large extent this complexity can be hidden from the author of a
|
|
Boost.Python extension module, but not entirely.
|
|
|
|
<h1>The recipe</h1>
|
|
|
|
Suppose there is an extension module that exposes certain instances of
|
|
the C++ std::vector template library such that it can be used from
|
|
Python in the following manner:
|
|
|
|
<pre>
|
|
import std_vector
|
|
v = std_vector.double([1, 2, 3, 4])
|
|
v.push_back(5)
|
|
v.size()
|
|
</pre>
|
|
|
|
Suppose the std_vector module is done well and reflects all C++
|
|
functions that are useful at the Python level, for all C++ built-in
|
|
data types (std_vector.int, std_vector.long, etc.).
|
|
|
|
<p>
|
|
Suppose further that there is statistic module with a C++ class that
|
|
has constructors or member functions that use or return a std::vector.
|
|
For example:
|
|
|
|
<pre>
|
|
class xy {
|
|
private:
|
|
std::vector<double> m_x;
|
|
std::vector<double> m_y;
|
|
public:
|
|
xy(const std::vector<double>& x, const std::vector<double>& y) : m_x(x), m_y(y) {}
|
|
const std::vector<double>& x() const { return m_x; }
|
|
const std::vector<double>& y() const { return m_y; }
|
|
double correlation();
|
|
}
|
|
</pre>
|
|
|
|
What is more natural then reusing the std_vector extension module to
|
|
expose these constructors or functions to Python?
|
|
|
|
<p>
|
|
Unfortunately, what seems natural needs a little work in both the
|
|
std_vector and the statistics module.
|
|
|
|
<p>
|
|
In the std_vector extension module, std::vector<double> needs to be
|
|
exposed to Python with the x_class_builder<> template instead of the
|
|
regular class_builder<>. For example:
|
|
|
|
<pre>
|
|
x_class_builder<std::vector<double> > v_double(std_vector_module, "double");
|
|
</pre>
|
|
|
|
In the extension module that wraps class xy we need to use
|
|
the import_class_builder<> template:
|
|
|
|
<pre>
|
|
import_class_builder<std::vector<double> > v_double("std_vector", "double");
|
|
</pre>
|
|
|
|
That is all. All the properties that are defined for std_vector.double
|
|
in the std_vector Boost.Python module will be available for the
|
|
returned objects of xy.x() and xy.y(). Similarly, the constructor for
|
|
xy will accept objects that were created by the std_vector module.
|
|
|
|
<h1>Non-copyable types</h1>
|
|
|
|
The x_class_builder<T> instantiates template functions that invoke the
|
|
copy constructor of T. For a T that is non-copyable this will result in
|
|
compile-time error messages. In such a case, another variety of the
|
|
class_builder<>, the xptr_class_builder<> must be used.
|
|
For example:
|
|
|
|
<pre>
|
|
xptr_class_builder<store> py_store(your_module, "store");
|
|
</pre>
|
|
|
|
The corresponding import_class_builder<> does not need any special
|
|
attention:
|
|
|
|
<pre>
|
|
import_class_builder<store> py_store("noncopyable_export", "store");
|
|
</pre>
|
|
|
|
<h1>Python module search path</h1>
|
|
|
|
The std_vector and statistics modules can now be used in the following
|
|
way:
|
|
|
|
<pre>
|
|
import std_vector
|
|
import statistics
|
|
x = std_vector.double([1, 2, 3, 4])
|
|
y = std_vector.double([2, 4, 6, 8])
|
|
xy = statistics.xy(x, y)
|
|
xy.correlation()
|
|
</pre>
|
|
|
|
In this example it is clear that Python has to be able to find both the
|
|
std_vector and the statistics extension module. In other words, both
|
|
extension modules need to be in the Python module search path
|
|
(sys.path).
|
|
|
|
<p>
|
|
The situation is not always that obvious. Suppose the statistics
|
|
module has a random function that returns a vector of random
|
|
numbers with a given length:
|
|
|
|
<pre>
|
|
import statistics
|
|
x = statistics.random(5)
|
|
y = statistics.random(5)
|
|
xy = statistics.xy(x, y)
|
|
xy.correlation()
|
|
</pre>
|
|
|
|
A naive user will not easily anticipate that the std_vector module is
|
|
used to pass the x and y vectors around. If the std_vector module is in
|
|
the Python module search path, this form of ignorance is of no harm.
|
|
On the contrary, we are glad that we do not have to bother the user
|
|
with details like this.
|
|
|
|
<p>
|
|
If the std_vector module is not in the Python module search path, a
|
|
Python exception will be raised:
|
|
|
|
<pre>
|
|
Traceback (innermost last):
|
|
File "foo.py", line 2, in ?
|
|
x = statistics.random(5)
|
|
ImportError: No module named std_vector
|
|
</pre>
|
|
|
|
As is the case with any system of a non-trivial complexity, it is
|
|
important that the setup is consistent and complete.
|
|
|
|
<h1>Two-way module dependencies</h1>
|
|
|
|
Boost.Python supports two-way module dependencies. This is best
|
|
illustrated by a simple example.
|
|
|
|
<p>
|
|
Suppose there is a module ivect that implements vectors of integers,
|
|
and a similar module dvect that implements vectors of doubles. We want
|
|
to be able do convert an integer vector to a double vector and vice
|
|
versa. For example:
|
|
|
|
<pre>
|
|
import ivect
|
|
iv = ivect.ivect((1,2,3,4,5))
|
|
dv = iv.as_dvect()
|
|
</pre>
|
|
|
|
The last expression will implicitly import the dvect module in order to
|
|
enable the conversion of the C++ representation of dvect to a Python
|
|
object. The analogous is possible for a dvect:
|
|
|
|
<pre>
|
|
import dvect
|
|
dv = dvect.dvect((1,2,3,4,5))
|
|
iv = dv.as_ivect()
|
|
</pre>
|
|
|
|
Now the ivect module is imported implicitly.
|
|
|
|
<p>
|
|
Note that the two-way dependencies are possible because the
|
|
dependencies are resolved only when needed. This is, the initialization
|
|
of the ivect module does not rely on the dvect module, and vice versa.
|
|
Only if as_dvect() or as_ivect() is actually invoked will the
|
|
corresponding module be implicitly imported. This also means that, for
|
|
example, the dvect module does not have to be available at all if
|
|
as_dvect() is never used.
|
|
|
|
<h1>Clarification of compile-time and link-time dependencies</h1>
|
|
|
|
Boost.Python's support for resolving cross-module dependencies at
|
|
runtime does not imply that compile-time dependencies are eliminated.
|
|
For example, the statistics extension module in the example above will
|
|
need to #include <vector>. This is immediately obvious from the
|
|
definition of class xy.
|
|
|
|
<p>
|
|
If a library is wrapped that consists of both header files and compiled
|
|
components (e.g. libdvect.a, dvect.lib, etc.), both the Boost.Python
|
|
extension module with the x_class_wrapper<> and the module with the
|
|
import_class_wrapper<> need to be linked against the object library.
|
|
Ideally one would build a shared library (e.g. libdvect.so, dvect.dll,
|
|
etc.). However, this introduces the issue of getting the search path
|
|
for the dynamic loading configured correctly. For small libraries it is
|
|
therefore often more convenient to ignore the fact that the object
|
|
files are loaded into memory more than once.
|
|
|
|
<p>
|
|
The main purpose of Boost.Python's support for resolving cross-module
|
|
dependencies at runtime is to allow for a modular system layout. With
|
|
this support it is straightforward to reflect C++ code organization at
|
|
the Python level. Without the cross-module support, a multi-purpose
|
|
module like std_vector would be impractical because the entire wrapper
|
|
code would somehow have to be duplicated in all extension modules that
|
|
use it, making them harder to maintain and harder to build.
|
|
|
|
<p>
|
|
Finally, there is an important psychological component. If a group of
|
|
classes is lumped together with many others in a huge module, the
|
|
authors will have difficulties in being identified with their work.
|
|
The situation is much more transparent if the work is represented by
|
|
a module with a recognizable name. This is not just a question of
|
|
strong egos, but also of getting credit and funding.
|
|
|
|
<h1>Why not use the x_class_builder universally?</h1>
|
|
|
|
There is some overhead associated with the Boost.Python cross-module
|
|
support. Depending on the platform, the code generated by
|
|
x_class_builder<> is roughly 10%-20% larger than that generated by
|
|
class_builder<>. For a large extension module with many wrapped
|
|
classes, this could mean a significant difference. Therefore the
|
|
general recommendation is to use x_class_wrapper<> only for classes
|
|
that are likely to be used as function arguments or return values in
|
|
other modules.
|
|
|
|
<hr>
|
|
<address>
|
|
Author: Ralf W. Grosse-Kunstleve, March 2001
|
|
</address>
|
|
</html>
|