diff --git a/doc/index.html b/doc/index.html index 754ef53f..52e9083c 100644 --- a/doc/index.html +++ b/doc/index.html @@ -88,6 +88,8 @@
Pyste (Boost.Python code generator)
+
Internals Documentation
+
News/Change Log
LLNL Progress Reports
diff --git a/doc/internals.html b/doc/internals.html new file mode 100755 index 00000000..d777bee9 --- /dev/null +++ b/doc/internals.html @@ -0,0 +1,184 @@ + + + + + + +Boost Python Internals + + + + +
+

Boost Python Internals

+

A conversation between Brett Calcott and David Abrahams

+ +++ + + + +
Copyright:Copyright Brett Calcott and David Abrahams 2003. All +rights reserved.
+

In both of these cases, I'm quite capable of reading code - but the +thing I don't get from scanning the source is a sense of the +architecture, both structurally, and temporally (er, I mean in what +order things go on).

+
    +
  1. What happens when you do the following:

    +
    +struct boring {};
    +...etc...
    +class_<boring>("boring")
    +    ;
    +
    +
  2. +
+

There seems to be a fair bit going on.

+
+ +
+

Can you gesture in the general direction where these things are done?

+
+

I only have time for a "off-the-top-of-my-head" answer at the moment; +I suggest you step through the code with a debugger after reading this +to see how it works, fill in details, and make sure I didn't forget +anything.

+
+

A new (Python) subclass of Boost.Python.Instance (see +libs/python/src/object/class.cpp) is created by invoking +Boost.Python.class, the metatype:

+
+>>> boring = Boost.Python.class(
+...     'boring'
+...   , bases_tuple       # in this case, just ()
+...   , { 
+...         '__module__' : module_name
+...       , '__doc__' : doc_string # optional
+...     }
+... )
+
+

A handle to this object is stuck in the m_class_object field +of the registration associated with typeid(boring). The +registry will keep that object alive forever, even if you +wipe out the 'boring' attribute of the extension module +(probably not a good thing).

+

Because you didn't specify class<boring, non_copyable, +...>, a to-python converter for boring is registered which +copies its argument into a value_holder held by the the +Python boring object.

+

Because you didn't specify class<boring ...>(no_init), +an __init__ function object is added to the class +dictionary which default-constructs a boring in a +value_holder (because you didn't specify some smart pointer +or derived wrapper class as a holder) held by the Python +boring object.

+

register_class_from_python is used to register a +from-python converter for shared_ptr<boring>. +boost::shared_ptrs are special among smart pointers +because their Deleter argument can be made to manage the +whole Python object, not just the C++ object it contains, no +matter how the C++ object is held.

+

If there were any bases<>, we'd also be registering the +relationship between these base classes and boring in the +up/down cast graph (inheritance.[hpp/cpp]).

+

In earlier versions of the code, we'd be registering lvalue +from-python converters for the class here, but now +from-python conversion for wrapped classes is handled as a +special case, before consulting the registry, if the source +Python object's metaclass is the Boost.Python metaclass.

+

Hmm, that from-python converter probably ought to be handled +the way class converters are, with no explicit conversions +registered.

+
+
+
    +
  1. Can you give a brief overview of the data structures that are +present in the registry

    +
    +

    The registry is simple: it's just a map from typeid -> +registration (see boost/python/converter/registrations.hpp). +lvalue_chain and rvalue_chain are simple endogenous +linked lists.

    +

    If you want to know more, just ask.

    +

    If you want to know about the cast graph, ask me something specific in +a separate message.

    +
    +

    and an overview of the process that happens as a type makes its +way from c++ to python and back again.

    +
  2. +
+
+

Big subject. I suggest some background reading: look for relevant +info in the LLNL progress reports and the messages they link to. +Also,

+
+http://mail.python.org/pipermail/c++-sig/2002-May/001023.html +http://mail.python.org/pipermail/c++-sig/2002-December/003115.html +http://aspn.activestate.com/ASPN/Mail/Message/1280898 +http://mail.python.org/pipermail/c++-sig/2002-July/001755.html
+

from c++ to python:

+
+

It depends on the type and the call policies in use or, for +call<>(...), call_method<>(...), or object(...), if +ref or ptr is used. There are also two basic +categories to to-python conversion, "return value" conversion +(for Python->C++ calls) and "argument" conversion (for +C++->Python calls and explicit object() conversions). The +behavior of these two categories differs subtly in various ways +whose details I forget at the moment. You can probably find +the answers in the above references, and certainly in the code.

+

The "default" case is by-value (copying) conversion, which uses +to_python_value as a to-python converter.

+
+

Since there can sensibly be only one way to convert any type +to python (disregarding the idea of scoped registries for the +moment), it makes sense that to-python conversions can be +handled by specializing a template. If the type is one of +the types handled by a built-in conversion +(builtin_converters.hpp), the corresponding template +specialization of to_python_value gets used.

+

Otherwise, to_python_value uses the m_to_python +function in the registration for the C++ type.

+
+

Other conversions, like by-reference conversions, are only +available for wrapped classes, and are requested explicitly by +using ref(...), ptr(...), or by specifying different +CallPolicies for a call, which can cause a different to-python +converter to be used. These conversions are never registered +anywhere, though they do need to use the registration to find +the Python class corresponding to the C++ type being referred +to. They just build a new Python instance and stick the +appropriate Holder instance in it.

+
+

from python to C++:

+
+

Once again I think there is a distinction between "return value" +and "argument" conversions, and I forget exactly what that is.

+

What happens depends on whether an lvalue conversion is needed +(see http://mail.python.org/pipermail/c++-sig/2002-May/001023.html) +All lvalue conversions are also registered in a type's rvalue +conversion chain, since when an rvalue will do, an lvalue is +certainly good enough.

+

An lvalue conversion can be done in one step (just get me the +pointer to the object - it can be NULL if no conversion is +possible) while an rvalue conversion requires two steps to +support wrapped function overloading and multiple converters for +a given C++ target type: first tell me if a conversion is +possible, then construct the converted object as a second step.

+
+
+
+ + + + diff --git a/doc/internals.rst b/doc/internals.rst new file mode 100755 index 00000000..a0b028b9 --- /dev/null +++ b/doc/internals.rst @@ -0,0 +1,168 @@ +======================== + Boost Python Internals +======================== + +------------------------------------------------------- +A conversation between Brett Calcott and David Abrahams +------------------------------------------------------- + +:copyright: Copyright Brett Calcott and David Abrahams 2003. All + rights reserved. + +In both of these cases, I'm quite capable of reading code - but the +thing I don't get from scanning the source is a sense of the +architecture, both structurally, and temporally (er, I mean in what +order things go on). + +1) What happens when you do the following:: + + struct boring {}; + ...etc... + class_("boring") + ; + +There seems to be a fair bit going on. + + - Python needs a new ClassType to be registered. + - We need to construct a new type that can hold our boring struct. + - Inward and outward converters need to be registered for the type. + +Can you gesture in the general direction where these things are done? + + I only have time for a "off-the-top-of-my-head" answer at the moment; + I suggest you step through the code with a debugger after reading this + to see how it works, fill in details, and make sure I didn't forget + anything. + + A new (Python) subclass of Boost.Python.Instance (see + libs/python/src/object/class.cpp) is created by invoking + Boost.Python.class, the metatype:: + + >>> boring = Boost.Python.class( + ... 'boring' + ... , bases_tuple # in this case, just () + ... , { + ... '__module__' : module_name + ... , '__doc__' : doc_string # optional + ... } + ... ) + + A handle to this object is stuck in the m_class_object field + of the registration associated with ``typeid(boring)``. The + registry will keep that object alive forever, even if you + wipe out the 'boring' attribute of the extension module + (probably not a good thing). + + Because you didn't specify ``class``, a to-python converter for boring is registered which + copies its argument into a value_holder held by the the + Python boring object. + + Because you didn't specify ``class(no_init)``, + an ``__init__`` function object is added to the class + dictionary which default-constructs a boring in a + value_holder (because you didn't specify some smart pointer + or derived wrapper class as a holder) held by the Python + boring object. + + ``register_class_from_python`` is used to register a + from-python converter for ``shared_ptr``. + ``boost::shared_ptr``\ s are special among smart pointers + because their Deleter argument can be made to manage the + whole Python object, not just the C++ object it contains, no + matter how the C++ object is held. + + If there were any ``bases<>``, we'd also be registering the + relationship between these base classes and boring in the + up/down cast graph (``inheritance.[hpp/cpp]``). + + In earlier versions of the code, we'd be registering lvalue + from-python converters for the class here, but now + from-python conversion for wrapped classes is handled as a + special case, before consulting the registry, if the source + Python object's metaclass is the Boost.Python metaclass. + + Hmm, that from-python converter probably ought to be handled + the way class converters are, with no explicit conversions + registered. + +2) Can you give a brief overview of the data structures that are + present in the registry + + The registry is simple: it's just a map from typeid -> + registration (see boost/python/converter/registrations.hpp). + ``lvalue_chain`` and ``rvalue_chain`` are simple endogenous + linked lists. + + If you want to know more, just ask. + + If you want to know about the cast graph, ask me something specific in + a separate message. + + and an overview of the process that happens as a type makes its + way from c++ to python and back again. + + Big subject. I suggest some background reading: look for relevant + info in the LLNL progress reports and the messages they link to. + Also, + + http://mail.python.org/pipermail/c++-sig/2002-May/001023.html + http://mail.python.org/pipermail/c++-sig/2002-December/003115.html + http://aspn.activestate.com/ASPN/Mail/Message/1280898 + http://mail.python.org/pipermail/c++-sig/2002-July/001755.html + + from c++ to python: + + It depends on the type and the call policies in use or, for + ``call<>(...)``, ``call_method<>(...)``, or ``object(...)``, if + ``ref`` or ``ptr`` is used. There are also two basic + categories to to-python conversion, "return value" conversion + (for Python->C++ calls) and "argument" conversion (for + C++->Python calls and explicit ``object()`` conversions). The + behavior of these two categories differs subtly in various ways + whose details I forget at the moment. You can probably find + the answers in the above references, and certainly in the code. + + The "default" case is by-value (copying) conversion, which uses + to_python_value as a to-python converter. + + Since there can sensibly be only one way to convert any type + to python (disregarding the idea of scoped registries for the + moment), it makes sense that to-python conversions can be + handled by specializing a template. If the type is one of + the types handled by a built-in conversion + (builtin_converters.hpp), the corresponding template + specialization of to_python_value gets used. + + Otherwise, to_python_value uses the ``m_to_python`` + function in the registration for the C++ type. + + Other conversions, like by-reference conversions, are only + available for wrapped classes, and are requested explicitly by + using ``ref(...)``, ``ptr(...)``, or by specifying different + CallPolicies for a call, which can cause a different to-python + converter to be used. These conversions are never registered + anywhere, though they do need to use the registration to find + the Python class corresponding to the C++ type being referred + to. They just build a new Python instance and stick the + appropriate Holder instance in it. + + + from python to C++: + + Once again I think there is a distinction between "return value" + and "argument" conversions, and I forget exactly what that is. + + What happens depends on whether an lvalue conversion is needed + (see http://mail.python.org/pipermail/c++-sig/2002-May/001023.html) + All lvalue conversions are also registered in a type's rvalue + conversion chain, since when an rvalue will do, an lvalue is + certainly good enough. + + An lvalue conversion can be done in one step (just get me the + pointer to the object - it can be ``NULL`` if no conversion is + possible) while an rvalue conversion requires two steps to + support wrapped function overloading and multiple converters for + a given C++ target type: first tell me if a conversion is + possible, then construct the converted object as a second step. +