diff --git a/doc/cross_module_dependencies.html b/doc/cross_module_dependencies.html
new file mode 100644
index 00000000..3d416bf5
--- /dev/null
+++ b/doc/cross_module_dependencies.html
@@ -0,0 +1,253 @@
+
+
+ Cross-extension-module dependencies
+
+
+
+
+
+
+
+Cross-extension-module dependencies
+
+It is good programming practice to organize large projects as modules
+that interact with each other via well defined interfaces. With
+Boost.Python it is possible to reflect this organization at the C++
+level at the Python level. This is, each logical C++ module can be
+organized as a separate Python extension module.
+
+
+At first sight this might seem natural and straightforward. However, it
+is a fairly complex problem to establish cross-extension-module
+dependencies while maintaining the same ease of use Boost.Python
+provides for classes that are wrapped in the same extension module. To
+a large extent this complexity can be hidden from the author of a
+Boost.Python extension module, but not entirely.
+
+
The recipe
+
+Suppose there is an extension module that exposes certain instances of
+the C++ std::vector template library such that it can be used from
+Python in the following manner:
+
+
+import std_vector
+v = std_vector.double([1, 2, 3, 4])
+v.push_back(5)
+v.size()
+
+
+Suppose the std_vector module is done well and reflects all C++
+functions that are useful at the Python level, for all C++ built-in
+data types (std_vector.int, std_vector.long, etc.).
+
+
+Suppose further that there is statistic module with a C++ class that
+has constructors or member functions that use or return a std::vector.
+For example:
+
+
+class xy {
+ private:
+ std::vector<double> m_x;
+ std::vector<double> m_y;
+ public:
+ xy(const std::vector<double>& x, const std::vector<double>& y) : m_x(x), m_y(y) {}
+ const std::vector<double>& x() const { return m_x; }
+ const std::vector<double>& y() const { return m_y; }
+ double correlation();
+}
+
+
+What is more natural then reusing the std_vector extension module to
+expose these constructors or functions to Python?
+
+
+Unfortunately, what seems natural needs a little work in both the
+std_vector and the statistics module.
+
+
+In the std_vector extension module, std::vector<double> needs to be
+exposed to Python with the x_class_builder<> template instead of the
+regular class_builder<>. For example:
+
+
+ x_class_builder<std::vector<double> > v_double(std_vector_module, "double");
+
+
+In the extension module that wraps class xy we need to use
+the import_class_builder<> template:
+
+
+ import_class_builder<std::vector<double> > v_double("std_vector", "double");
+
+
+That is all. All the properties that are defined for std_vector.double
+in the std_vector Boost.Python module will be available for the
+returned objects of xy.x() and xy.y(). Similarly, the constructor for
+xy will accept objects that were created by the std_vector module.
+
+Non-copyable types
+
+The x_class_builder<T> instantiates template functions that invoke the
+copy constructor of T. For a T that is non-copyable this will result in
+compile-time error messages. In such a case, another variety of the
+class_builder<>, the xptr_class_builder<> must be used.
+For example:
+
+
+xptr_class_builder<store> py_store(your_module, "store");
+
+
+The corresponding import_class_builder<> does not need any special
+attention:
+
+
+import_class_builder<store> py_store("noncopyable_export", "store");
+
+
+Python module search path
+
+The std_vector and statistics modules can now be used in the following
+way:
+
+
+import std_vector
+import statistics
+x = std_vector.double([1, 2, 3, 4])
+y = std_vector.double([2, 4, 6, 8])
+xy = statistics.xy(x, y)
+xy.correlation()
+
+
+In this example it is clear that Python has to be able to find both the
+std_vector and the statistics extension module. In other words, both
+extension modules need to be in the Python module search path
+(sys.path).
+
+
+The situation is not always that obvious. Suppose the statistics
+module has a random function that returns a vector of random
+numbers with a given length:
+
+
+import statistics
+x = statistics.random(5)
+y = statistics.random(5)
+xy = statistics.xy(x, y)
+xy.correlation()
+
+
+A naive user will not easily anticipate that the std_vector module is
+used to pass the x and y vectors around. If the std_vector module is in
+the Python module search path, this form of ignorance is of no harm.
+On the contrary, we are glad that we do not have to bother the user
+with details like this.
+
+
+If the std_vector module is not in the Python module search path, a
+Python exception will be raised:
+
+
+Traceback (innermost last):
+ File "foo.py", line 2, in ?
+ x = statistics.random(5)
+ImportError: No module named std_vector
+
+
+As is the case with any system of a non-trivial complexity, it is
+important that the setup is consistent and complete.
+
+Two-way module dependencies
+
+Boost.Python supports two-way module dependencies. This is best
+illustrated by a simple example.
+
+
+Suppose there is a module ivect that implements vectors of integers,
+and a similar module dvect that implements vectors of doubles. We want
+to be able do convert an integer vector to a double vector and vice
+versa. For example:
+
+
+import ivect
+iv = ivect.ivect((1,2,3,4,5))
+dv = iv.as_dvect()
+
+
+The last expression will implicitly import the dvect module in order to
+enable the conversion of the C++ representation of dvect to a Python
+object. The analogous is possible for a dvect:
+
+
+import dvect
+dv = dvect.dvect((1,2,3,4,5))
+iv = dv.as_ivect()
+
+
+Now the ivect module is imported implicitly.
+
+
+Note that the two-way dependencies are possible because the
+dependencies are resolved only when needed. This is, the initialization
+of the ivect module does not rely on the dvect module, and vice versa.
+Only if as_dvect() or as_ivect() is actually invoked will the
+corresponding module be implicitly imported. This also means that, for
+example, the dvect module does not have to be available at all if
+as_dvect() is never used.
+
+
Clarification of compile-time and link-time dependencies
+
+Boost.Python's support for resolving cross-module dependencies at
+runtime does not imply that compile-time dependencies are eliminated.
+For example, the statistics extension module in the example above will
+need to #include <vector>. This is immediately obvious from the
+definition of class xy.
+
+
+If a library is wrapped that consists of both header files and compiled
+components (e.g. libdvect.a, dvect.lib, etc.), both the Boost.Python
+extension module with the x_class_wrapper<> and the module with the
+import_class_wrapper<> need to be linked against the object library.
+Ideally one would build a shared library (e.g. libdvect.so, dvect.dll,
+etc.). However, this introduces the issue of getting the search path
+for the dynamic loading configured correctly. For small libraries it is
+therefore often more convenient to ignore the fact that the object
+files are loaded into memory more than once.
+
+
+The main purpose of Boost.Python's support for resolving cross-module
+dependencies at runtime is to allow for a modular system layout. With
+this support it is straightforward to reflect C++ code organization at
+the Python level. Without the cross-module support, a multi-purpose
+module like std_vector would be impractical because the entire wrapper
+code would somehow have to be duplicated in all extension modules that
+use it, making them harder to maintain and harder to build.
+
+
+Finally, there is an important psychological component. If a group of
+classes is lumped together with many others in a huge module, the
+authors will have difficulties in being identified with their work.
+The situation is much more transparent if the work is represented by
+a module with a recognizable name. This is not just a question of
+strong egos, but also of getting credit and funding.
+
+
Why not use the x_class_builder universally?
+
+There is some overhead associated with the Boost.Python cross-module
+support. Depending on the platform, the code generated by
+x_class_builder<> is roughly 10%-20% larger than that generated by
+class_builder<>. For a large extension module with many wrapped
+classes, this could mean a significant difference. Therefore the
+general recommendation is to use x_class_wrapper<> only for classes
+that are likely to be used as function arguments or return values in
+other modules.
+
+
+
+Author: Ralf W. Grosse-Kunstleve, March 2001
+
+
diff --git a/doc/pickle.html b/doc/pickle.html
new file mode 100644
index 00000000..842112d3
--- /dev/null
+++ b/doc/pickle.html
@@ -0,0 +1,245 @@
+
+
+Boost.Python Pickle Support
+
+
+
+
+
+
+
Boost.Python Pickle Support
+
+Pickle is a Python module for object serialization, also known
+as persistence, marshalling, or flattening.
+
+
+It is often necessary to save and restore the contents of an object to
+a file. One approach to this problem is to write a pair of functions
+that read and write data from a file in a special format. A powerful
+alternative approach is to use Python's pickle module. Exploiting
+Python's ability for introspection, the pickle module recursively
+converts nearly arbitrary Python objects into a stream of bytes that
+can be written to a file.
+
+
+The Boost Python Library supports the pickle module by emulating the
+interface implemented by Jim Fulton's ExtensionClass module that is
+included in the
+ZOPE
+distribution.
+This interface is similar to that for regular Python classes as
+described in detail in the
+Python Library Reference for pickle.
+
+
+
The Boost.Python Pickle Interface
+
+At the user level, the Boost.Python pickle interface involves three special
+methods:
+
+
+-
+__getinitargs__
+
-
+ When an instance of a Boost.Python extension class is pickled, the
+ pickler tests if the instance has a __getinitargs__ method.
+ This method must return a Python tuple (it is most convenient to use
+ a boost::python::tuple). When the instance is restored by the
+ unpickler, the contents of this tuple are used as the arguments for
+ the class constructor.
+
+
+ If __getinitargs__ is not defined, the class constructor
+ will be called without arguments.
+
+
+
-
+__getstate__
+
+
-
+ When an instance of a Boost.Python extension class is pickled, the
+ pickler tests if the instance has a __getstate__ method.
+ This method should return a Python object representing the state of
+ the instance.
+
+
+ If __getstate__ is not defined, the instance's
+ __dict__ is pickled (if it is not empty).
+
+
+
-
+__setstate__
+
+
-
+ When an instance of a Boost.Python extension class is restored by the
+ unpickler, it is first constructed using the result of
+ __getinitargs__ as arguments (see above). Subsequently the
+ unpickler tests if the new instance has a __setstate__
+ method. If so, this method is called with the result of
+ __getstate__ (a Python object) as the argument.
+
+
+ If __setstate__ is not defined, the result of
+ __getstate__ must be a Python dictionary. The items of this
+ dictionary are added to the instance's __dict__.
+
+
+
+If both
__getstate__ and
__setstate__ are defined,
+the Python object returned by
__getstate__ need not be a
+dictionary. The
__getstate__ and
__setstate__ methods
+can do what they want.
+
+
+
Pitfalls and Safety Guards
+
+In Boost.Python extension modules with many extension classes,
+providing complete pickle support for all classes would be a
+significant overhead. In general complete pickle support should only be
+implemented for extension classes that will eventually be pickled.
+However, the author of a Boost.Python extension module might not
+anticipate correctly which classes need support for pickle.
+Unfortunately, the pickle protocol described above has two important
+pitfalls that the end user of a Boost.Python extension module might not
+be aware of:
+
+
+-
+Pitfall 1:
+Both __getinitargs__ and __getstate__ are not defined.
+
+
-
+ In this situation the unpickler calls the class constructor without
+ arguments and then adds the __dict__ that was pickled by
+ default to that of the new instance.
+
+
+ However, most C++ classes wrapped with Boost.Python will have member
+ data that are not restored correctly by this procedure. To alert the
+ user to this problem, a safety guard is provided. If both
+ __getinitargs__ and __getstate__ are not defined,
+ Boost.Python tests if the class has an attribute
+ __dict_defines_state__. An exception is raised if this
+ attribute is not defined:
+
+
+ RuntimeError: Incomplete pickle support (__dict_defines_state__ not set)
+
+
+ In the rare cases where this is not the desired behavior, the safety
+ guard can deliberately be disabled. The corresponding C++ code for
+ this is, e.g.:
+
+
+ class_builder<your_class> py_your_class(your_module, "your_class");
+ py_your_class.dict_defines_state();
+
+
+ It is also possible to override the safety guard at the Python level.
+ E.g.:
+
+
+ import your_bpl_module
+ class your_class(your_bpl_module.your_class):
+ __dict_defines_state__ = 1
+
+
+
+
-
+Pitfall 2:
+__getstate__ is defined and the instance's __dict__ is not empty.
+
+
-
+ The author of a Boost.Python extension class might provide a
+ __getstate__ method without considering the possibilities
+ that:
+
+
+
+ -
+ his class is used in Python as a base class. Most likely the
+ __dict__ of instances of the derived class needs to be
+ pickled in order to restore the instances correctly.
+
+
+
-
+ the user adds items to the instance's __dict__ directly.
+ Again, the __dict__ of the instance then needs to be
+ pickled.
+
+
+
+
+ To alert the user to this highly unobvious problem, a safety guard is
+ provided. If __getstate__ is defined and the instance's
+ __dict__ is not empty, Boost.Python tests if the class has
+ an attribute __getstate_manages_dict__. An exception is
+ raised if this attribute is not defined:
+
+
+ RuntimeError: Incomplete pickle support (__getstate_manages_dict__ not set)
+
+
+ To resolve this problem, it should first be established that the
+ __getstate__ and __setstate__ methods manage the
+ instances's __dict__ correctly. Note that this can be done
+ both at the C++ and the Python level. Finally, the safety guard
+ should intentionally be overridden. E.g. in C++:
+
+
+ class_builder<your_class> py_your_class(your_module, "your_class");
+ py_your_class.getstate_manages_dict();
+
+
+ In Python:
+
+
+ import your_bpl_module
+ class your_class(your_bpl_module.your_class):
+ __getstate_manages_dict__ = 1
+ def __getstate__(self):
+ # your code here
+ def __setstate__(self, state):
+ # your code here
+
+
+
+
+
Practical Advice
+
+
+-
+ Avoid using __getstate__ if the instance can also be
+ reconstructed by way of __getinitargs__. This automatically
+ avoids Pitfall 2.
+
+
+
-
+ If __getstate__ is required, include the instance's
+ __dict__ in the Python object that is returned.
+
+
+
+
+
Example
+
+An example that shows how to configure pickle support is available in the
+
boost/lib/python/example directory
+(
getting_started3.cpp).
+
+
+© Copyright Ralf W. Grosse-Kunstleve 2001. Permission to copy,
+use, modify, sell and distribute this document is granted provided this
+copyright notice appears in all copies. This document is provided "as
+is" without express or implied warranty, and with no claim as to its
+suitability for any purpose.
+
+
+Updated: March 10, 2001
+