mirror of
https://github.com/boostorg/python.git
synced 2026-01-21 17:12:22 +00:00
224 lines
6.6 KiB
HTML
224 lines
6.6 KiB
HTML
<html>
|
|
<head>
|
|
<title>BPL Pickle Support</title>
|
|
</head>
|
|
<body>
|
|
|
|
<img src="../../../c++boost.gif"
|
|
alt="c++boost.gif (8819 bytes)"
|
|
align="center"
|
|
width="277" height="86">
|
|
|
|
</body>
|
|
<hr>
|
|
<h1>BPL Pickle Support</h1>
|
|
|
|
Pickle is a Python module for object serialization, also known
|
|
as persistence, marshalling, or flattening.
|
|
|
|
<p>
|
|
It is often necessary to save and restore the contents of an object to
|
|
a file. One approach to this problem is to write a pair of functions
|
|
that read and write data from a file in a special format. A powerful
|
|
alternative approach is to use Python's pickle module. Exploiting
|
|
Python's ability for introspection, the pickle module recursively
|
|
converts nearly arbitrary Python objects into a stream of bytes that
|
|
can be written to a file.
|
|
|
|
<p>
|
|
The Boost Python Library supports the pickle module by emulating the
|
|
interface implemented by Jim Fulton's ExtensionClass module that is
|
|
included in the ZOPE distribution
|
|
(<a href="http://www.zope.org/">http://www.zope.org/</a>).
|
|
This interface is similar to that for regular Python classes as
|
|
described in detail in the Python Library Reference for pickle:
|
|
|
|
<blockquote>
|
|
<a href="http://www.python.org/doc/current/lib/module-pickle.html"
|
|
>http://www.python.org/doc/current/lib/module-pickle.html</a>
|
|
</blockquote>
|
|
|
|
<hr>
|
|
<h1>The BPL Pickle Interface</h1>
|
|
|
|
At the user level, the BPL pickle interface involves three special
|
|
methods:
|
|
|
|
<dl>
|
|
<dt>
|
|
<strong>__getinitargs__</strong>
|
|
<dd>
|
|
When an instance of a BPL extension class is pickled, the pickler
|
|
tests if the instance has a __getinitargs__ method. This method must
|
|
return a Python tuple. When the instance is restored by the
|
|
unpickler, the contents of this tuple are used as the arguments for
|
|
the class constructor.
|
|
|
|
<p>
|
|
If __getinitargs__ is not defined, the class constructor will be
|
|
called without arguments.
|
|
|
|
<p>
|
|
<dt>
|
|
<strong>__getstate__</strong>
|
|
|
|
<dd>
|
|
When an instance of a BPL extension class is pickled, the pickler
|
|
tests if the instance has a __getstate__ method. This method should
|
|
return a Python object representing the state of the instance.
|
|
|
|
<p>
|
|
If __getstate__ is not defined, the instance's __dict__ is pickled
|
|
(if it is not empty).
|
|
|
|
<p>
|
|
<dt>
|
|
<strong>__setstate__</strong>
|
|
|
|
<dd>
|
|
When an instance of a BPL extension class is restored by the
|
|
unpickler, it is first constructed using the result of
|
|
__getinitargs__ as arguments (see above). Subsequently the unpickler
|
|
tests if the new instance has a __setstate__ method. If so, this
|
|
method is called with the result of __getstate__ (a Python object) as
|
|
the argument.
|
|
|
|
<p>
|
|
If __setstate__ is not defined, the result of __getstate__ must be
|
|
a Python dictionary. The items of this dictionary are added to
|
|
the instance's __dict__.
|
|
</dl>
|
|
|
|
If both __getstate__ and __setstate__ are defined, the Python object
|
|
returned by __getstate__ need not be a dictionary. The __getstate__ and
|
|
__setstate__ methods can do what they want.
|
|
|
|
<hr>
|
|
<h1>Pitfalls and Safety Guards</h1>
|
|
|
|
In BPL extension modules with many extension classes, providing
|
|
complete pickle support for all classes would be a significant
|
|
overhead. In general complete pickle support should only be implemented
|
|
for extension classes that will eventually be pickled. However, the
|
|
author of a BPL extension module might not anticipate correctly which
|
|
classes need support for pickle. Unfortunately, the pickle protocol
|
|
described above has two important pitfalls that the end user of a BPL
|
|
extension module might not be aware of:
|
|
|
|
<dl>
|
|
<dt>
|
|
<strong>Pitfall 1:</strong>
|
|
Both __getinitargs__ and __getstate__ are not defined.
|
|
|
|
<dd>
|
|
In this situation the unpickler calls the class constructor without
|
|
arguments and then adds the __dict__ that was pickled by default to
|
|
that of the new instance.
|
|
|
|
<p>
|
|
However, most C++ classes wrapped with the BPL will have member data
|
|
that are not restored correctly by this procedure. To alert the user
|
|
to this problem, a safety guard is provided. If both __getinitargs__
|
|
and __getstate__ are not defined, the BPL tests if the class has an
|
|
attribute __dict_defines_state__. An exception is raised if this
|
|
attribute is not defined:
|
|
|
|
<pre>
|
|
RuntimeError: Incomplete pickle support (__dict_defines_state__ not set)
|
|
</pre>
|
|
|
|
In the rare cases where this is not the desired behavior, the safety
|
|
guard can deliberately be disabled. The corresponding C++ code for
|
|
this is, e.g.:
|
|
|
|
<pre>
|
|
class_builder<your_class> py_your_class(your_module, "your_class");
|
|
py_your_class.dict_defines_state();
|
|
</pre>
|
|
|
|
It is also possible to override the safety guard at the Python level.
|
|
E.g.:
|
|
|
|
<pre>
|
|
import your_bpl_module
|
|
class your_class(your_bpl_module.your_class):
|
|
__dict_defines_state__ = 1
|
|
</pre>
|
|
|
|
<p>
|
|
<dt>
|
|
<strong>Pitfall 2:</strong>
|
|
__getstate__ is defined and the instance's __dict__ is not empty.
|
|
|
|
<dd>
|
|
The author of a BPL extension class might provide a __getstate__
|
|
method without considering the possibilities that:
|
|
|
|
<p>
|
|
<ul>
|
|
<li>
|
|
his class is used as a base class. Most likely the __dict__ of
|
|
instances of the derived class needs to be pickled in order to
|
|
restore the instances correctly.
|
|
|
|
<p>
|
|
<li>
|
|
the user adds items to the instance's __dict__ directly. Again,
|
|
the __dict__ of the instance then needs to be pickled.
|
|
</ul>
|
|
<p>
|
|
|
|
To alert the user to this highly unobvious problem, a safety guard is
|
|
provided. If __getstate__ is defined and the instance's __dict__ is
|
|
not empty, the BPL tests if the class has an attribute
|
|
__getstate_manages_dict__. An exception is raised if this attribute
|
|
is not defined:
|
|
|
|
<pre>
|
|
RuntimeError: Incomplete pickle support (__getstate_manages_dict__ not set)
|
|
</pre>
|
|
|
|
To resolve this problem, it should first be established that the
|
|
__getstate__ and __setstate__ methods manage the instances's __dict__
|
|
correctly. Note that this can be done both at the C++ and the Python
|
|
level. Finally, the safety guard should intentionally be overridden.
|
|
E.g. in C++:
|
|
|
|
<pre>
|
|
class_builder<your_class> py_your_class(your_module, "your_class");
|
|
py_your_class.getstate_manages_dict();
|
|
</pre>
|
|
|
|
In Python:
|
|
|
|
<pre>
|
|
import your_bpl_module
|
|
class your_class(your_bpl_module.your_class):
|
|
__getstate_manages_dict__ = 1
|
|
def __getstate__(self):
|
|
# your code here
|
|
def __setstate__(self, state):
|
|
# your code here
|
|
</pre>
|
|
</dl>
|
|
|
|
<hr>
|
|
<h1>Practical Advice</h1>
|
|
|
|
<ul>
|
|
<li>
|
|
Avoid using __getstate__ if the instance can also be reconstructed
|
|
by way of __getinitargs__. This automatically avoids Pitfall 2.
|
|
|
|
<p>
|
|
<li>
|
|
If __getstate__ is required, include the instance's __dict__ in the
|
|
Python object that is returned.
|
|
</ul>
|
|
|
|
<hr>
|
|
<address>
|
|
Author: Ralf W. Grosse-Kunstleve, March 2001
|
|
</address>
|
|
</html>
|