mirror of
https://github.com/boostorg/python.git
synced 2026-01-19 04:22:16 +00:00
281 lines
10 KiB
HTML
281 lines
10 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
|
|
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
|
|
|
|
<html xmlns="http://www.w3.org/1999/xhtml">
|
|
<head>
|
|
<title>Boost.Python Pickle Support</title>
|
|
</head>
|
|
|
|
<body>
|
|
<div>
|
|
<img src="../../../../boost.png" alt="boost.png (6897 bytes)" align=
|
|
"center" width="277" height="86" />
|
|
<hr />
|
|
|
|
<h1>Boost.Python Pickle Support</h1>Pickle is a Python module for object
|
|
serialization, also known as persistence, marshalling, or flattening.
|
|
|
|
<p>It is often necessary to save and restore the contents of an object to
|
|
a file. One approach to this problem is to write a pair of functions that
|
|
read and write data from a file in a special format. A powerful
|
|
alternative approach is to use Python's pickle module. Exploiting
|
|
Python's ability for introspection, the pickle module recursively
|
|
converts nearly arbitrary Python objects into a stream of bytes that can
|
|
be written to a file.</p>
|
|
|
|
<p>The Boost Python Library supports the pickle module through the
|
|
interface as described in detail in the <a href=
|
|
"http://www.python.org/doc/current/lib/module-pickle.html">Python Library
|
|
Reference for pickle.</a> This interface involves the special methods
|
|
<tt>__getinitargs__</tt>, <tt>__getstate__</tt> and <tt>__setstate__</tt>
|
|
as described in the following. Note that Boost.Python is also fully
|
|
compatible with Python's cPickle module.</p>
|
|
<hr />
|
|
|
|
<h2>The Boost.Python Pickle Interface</h2>At the user level, the
|
|
Boost.Python pickle interface involves three special methods:
|
|
|
|
<dl>
|
|
<dt><strong><tt>__getinitargs__</tt></strong></dt>
|
|
|
|
<dd>
|
|
When an instance of a Boost.Python extension class is pickled, the
|
|
pickler tests if the instance has a <tt>__getinitargs__</tt> method.
|
|
This method must return a Python tuple (it is most convenient to use
|
|
a boost::python::tuple). When the instance is restored by the
|
|
unpickler, the contents of this tuple are used as the arguments for
|
|
the class constructor.
|
|
|
|
<p>If <tt>__getinitargs__</tt> is not defined, <tt>pickle.load</tt>
|
|
will call the constructor (<tt>__init__</tt>) without arguments;
|
|
i.e., the object must be default-constructible.</p>
|
|
</dd>
|
|
|
|
<dt><strong><tt>__getstate__</tt></strong></dt>
|
|
|
|
<dd>When an instance of a Boost.Python extension class is pickled, the
|
|
pickler tests if the instance has a <tt>__getstate__</tt> method. This
|
|
method should return a Python object representing the state of the
|
|
instance.</dd>
|
|
|
|
<dt><strong><tt>__setstate__</tt></strong></dt>
|
|
|
|
<dd>When an instance of a Boost.Python extension class is restored by
|
|
the unpickler (<tt>pickle.load</tt>), it is first constructed using the
|
|
result of <tt>__getinitargs__</tt> as arguments (see above).
|
|
Subsequently the unpickler tests if the new instance has a
|
|
<tt>__setstate__</tt> method. If so, this method is called with the
|
|
result of <tt>__getstate__</tt> (a Python object) as the argument.</dd>
|
|
</dl>The three special methods described above may be <tt>.def()</tt>'ed
|
|
individually by the user. However, Boost.Python provides an easy to use
|
|
high-level interface via the
|
|
<strong><tt>boost::python::pickle_suite</tt></strong> class that also
|
|
enforces consistency: <tt>__getstate__</tt> and <tt>__setstate__</tt>
|
|
must be defined as pairs. Use of this interface is demonstrated by the
|
|
following examples.
|
|
<hr />
|
|
|
|
<h2>Examples</h2>There are three files in <tt>boost/libs/python/test</tt>
|
|
that show how to provide pickle support.
|
|
<hr />
|
|
|
|
<h3><a href="../../test/pickle1.cpp"><tt>pickle1.cpp</tt></a></h3>The C++
|
|
class in this example can be fully restored by passing the appropriate
|
|
argument to the constructor. Therefore it is sufficient to define the
|
|
pickle interface method <tt>__getinitargs__</tt>. This is done in the
|
|
following way:
|
|
|
|
<ul>
|
|
<li>1. Definition of the C++ pickle function:
|
|
<pre>
|
|
struct world_pickle_suite : boost::python::pickle_suite
|
|
{
|
|
static
|
|
boost::python::tuple
|
|
getinitargs(world const& w)
|
|
{
|
|
return boost::python::make_tuple(w.get_country());
|
|
}
|
|
};
|
|
</pre>
|
|
</li>
|
|
|
|
<li>2. Establishing the Python binding:
|
|
<pre>
|
|
class_<world>("world", args<const std::string&>())
|
|
// ...
|
|
.def_pickle(world_pickle_suite())
|
|
// ...
|
|
</pre>
|
|
</li>
|
|
</ul>
|
|
<hr />
|
|
|
|
<h3><a href="../../test/pickle2.cpp"><tt>pickle2.cpp</tt></a></h3>The C++
|
|
class in this example contains member data that cannot be restored by any
|
|
of the constructors. Therefore it is necessary to provide the
|
|
<tt>__getstate__</tt>/<tt>__setstate__</tt> pair of pickle interface
|
|
methods:
|
|
|
|
<ul>
|
|
<li>1. Definition of the C++ pickle functions:
|
|
<pre>
|
|
struct world_pickle_suite : boost::python::pickle_suite
|
|
{
|
|
static
|
|
boost::python::tuple
|
|
getinitargs(const world& w)
|
|
{
|
|
// ...
|
|
}
|
|
|
|
static
|
|
boost::python::tuple
|
|
getstate(const world& w)
|
|
{
|
|
// ...
|
|
}
|
|
|
|
static
|
|
void
|
|
setstate(world& w, boost::python::tuple state)
|
|
{
|
|
// ...
|
|
}
|
|
};
|
|
</pre>
|
|
</li>
|
|
|
|
<li>2. Establishing the Python bindings for the entire suite:
|
|
<pre>
|
|
class_<world>("world", args<const std::string&>())
|
|
// ...
|
|
.def_pickle(world_pickle_suite())
|
|
// ...
|
|
</pre>
|
|
</li>
|
|
</ul>
|
|
|
|
<p>For simplicity, the <tt>__dict__</tt> is not included in the result of
|
|
<tt>__getstate__</tt>. This is not generally recommended, but a valid
|
|
approach if it is anticipated that the object's <tt>__dict__</tt> will
|
|
always be empty. Note that the safety guard described below will catch
|
|
the cases where this assumption is violated.</p>
|
|
<hr />
|
|
|
|
<h3><a href="../../test/pickle3.cpp"><tt>pickle3.cpp</tt></a></h3>This
|
|
example is similar to <a href=
|
|
"../../test/pickle2.cpp"><tt>pickle2.cpp</tt></a>. However, the object's
|
|
<tt>__dict__</tt> is included in the result of <tt>__getstate__</tt>.
|
|
This requires a little more code but is unavoidable if the object's
|
|
<tt>__dict__</tt> is not always empty.
|
|
<hr />
|
|
|
|
<h2>Pitfall and Safety Guard</h2>The pickle protocol described above has
|
|
an important pitfall that the end user of a Boost.Python extension module
|
|
might not be aware of:
|
|
|
|
<p><strong><tt>__getstate__</tt> is defined and the instance's
|
|
<tt>__dict__</tt> is not empty.</strong></p>
|
|
|
|
<p>The author of a Boost.Python extension class might provide a
|
|
<tt>__getstate__</tt> method without considering the possibilities
|
|
that:</p>
|
|
|
|
<ul>
|
|
<li>his class is used in Python as a base class. Most likely the
|
|
<tt>__dict__</tt> of instances of the derived class needs to be pickled
|
|
in order to restore the instances correctly.</li>
|
|
|
|
<li>the user adds items to the instance's <tt>__dict__</tt> directly.
|
|
Again, the <tt>__dict__</tt> of the instance then needs to be
|
|
pickled.</li>
|
|
</ul>
|
|
|
|
<p>To alert the user to this highly unobvious problem, a safety guard is
|
|
provided. If <tt>__getstate__</tt> is defined and the instance's
|
|
<tt>__dict__</tt> is not empty, Boost.Python tests if the class has an
|
|
attribute <tt>__getstate_manages_dict__</tt>. An exception is raised if
|
|
this attribute is not defined:</p>
|
|
<pre>
|
|
RuntimeError: Incomplete pickle support (__getstate_manages_dict__ not set)
|
|
</pre>To resolve this problem, it should first be established that the <tt>
|
|
__getstate__</tt> and <tt>__setstate__</tt> methods manage the
|
|
instances's <tt>__dict__</tt> correctly. Note that this can be done
|
|
either at the C++ or the Python level. Finally, the safety guard should
|
|
intentionally be overridden. E.g. in C++ (from <a href=
|
|
"../../test/pickle3.cpp"><tt>pickle3.cpp</tt></a>):
|
|
<pre>
|
|
struct world_pickle_suite : boost::python::pickle_suite
|
|
{
|
|
// ...
|
|
|
|
static bool getstate_manages_dict() { return true; }
|
|
};
|
|
</pre>Alternatively in Python:
|
|
<pre>
|
|
import your_bpl_module
|
|
class your_class(your_bpl_module.your_class):
|
|
__getstate_manages_dict__ = 1
|
|
def __getstate__(self):
|
|
# your code here
|
|
def __setstate__(self, state):
|
|
# your code here
|
|
</pre>
|
|
<hr />
|
|
|
|
<h2>Practical Advice</h2>
|
|
|
|
<ul>
|
|
<li>In Boost.Python extension modules with many extension classes,
|
|
providing complete pickle support for all classes would be a
|
|
significant overhead. In general complete pickle support should only be
|
|
implemented for extension classes that will eventually be pickled.</li>
|
|
|
|
<li>Avoid using <tt>__getstate__</tt> if the instance can also be
|
|
reconstructed by way of <tt>__getinitargs__</tt>. This automatically
|
|
avoids the pitfall described above.</li>
|
|
|
|
<li>If <tt>__getstate__</tt> is required, include the instance's
|
|
<tt>__dict__</tt> in the Python object that is returned.</li>
|
|
</ul>
|
|
<hr />
|
|
|
|
<h2>Light-weight alternative: pickle support implemented in Python</h2>
|
|
|
|
<h3><a href="../../test/pickle4.cpp"><tt>pickle4.cpp</tt></a></h3>The
|
|
<tt>pickle4.cpp</tt> example demonstrates an alternative technique for
|
|
implementing pickle support. First we direct Boost.Python via the
|
|
<tt>class_::enable_pickling()</tt> member function to define only the
|
|
basic attributes required for pickling:
|
|
<pre>
|
|
class_<world>("world", args<const std::string&>())
|
|
// ...
|
|
.enable_pickling()
|
|
// ...
|
|
</pre>This enables the standard Python pickle interface as described in the
|
|
Python documentation. By "injecting" a <tt>__getinitargs__</tt> method into
|
|
the definition of the wrapped class we make all instances pickleable:
|
|
<pre>
|
|
# import the wrapped world class
|
|
from pickle4_ext import world
|
|
|
|
# definition of __getinitargs__
|
|
def world_getinitargs(self):
|
|
return (self.get_country(),)
|
|
|
|
# now inject __getinitargs__ (Python is a dynamic language!)
|
|
world.__getinitargs__ = world_getinitargs
|
|
</pre>See also the <a href=
|
|
"../tutorial/doc/html/python/techniques.html#python.extending_wrapped_objects_in_python">
|
|
tutorial section</a> on injecting additional methods from Python.
|
|
<hr />
|
|
© Copyright Ralf W. Grosse-Kunstleve 2001-2004. Distributed under the
|
|
Boost Software License, Version 1.0. (See accompanying file
|
|
LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)
|
|
|
|
<p>Updated: Feb 2004.</p>
|
|
</div>
|
|
</body>
|
|
</html>
|