mirror of
https://github.com/boostorg/python.git
synced 2026-01-19 04:22:16 +00:00
e70bbe4791
Also port some "glaringly obvious" bugfixes from HEAD. Hope it doesn't cause problems. [SVN r35237]
368 lines
14 KiB
HTML
368 lines
14 KiB
HTML
<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
|
|
<meta http-equiv="Content-Type" content=
|
|
"text/html; charset=iso-8859-1">
|
|
<link rel="stylesheet" type="text/css" href="../boost.css">
|
|
|
|
<title>Boost.Python - February 2002 Progress Report</title>
|
|
<style type="text/css">
|
|
:link { color: #0000ff }
|
|
:visited { color: #800080 }
|
|
p.c3 {font-style: italic}
|
|
h2.c2 {text-align: center}
|
|
h1.c1 {text-align: center}
|
|
</style>
|
|
|
|
<table border="0" cellpadding="7" cellspacing="0" width=
|
|
"100%" summary="header">
|
|
<tr>
|
|
<td valign="top" width="300">
|
|
<h3><a href="../../../../index.htm"><img height="86" width="277" alt=
|
|
"C++ Boost" src="../../../../boost.png" border="0"></a></h3>
|
|
|
|
<td valign="top">
|
|
<h1 class="c1"><a href="../index.html">Boost.Python</a></h1>
|
|
|
|
<h2 class="c2">February 2002 Progress Report</h2>
|
|
</table>
|
|
<hr>
|
|
|
|
<h2>Contents</h2>
|
|
|
|
<dl class="index">
|
|
<dt><a href="#Python10">Python10 Conference Report</a>
|
|
|
|
<dt><a href="#progress">Boost.Python v2 Progress</a>
|
|
|
|
<dd>
|
|
<dl class="index">
|
|
<dt><a href="#documentation">Documentation</a>
|
|
|
|
<dt><a href="#conversion">Overhaul of
|
|
<code>to_python</code>/<code>from_python</code>
|
|
conversion mechanism</a>
|
|
|
|
<dt><a href="#miscellaneous">Miscellaneous</a>
|
|
</dl>
|
|
</dl>
|
|
|
|
<h2><a name="Python10">Python10 Conference Report</a></h2>
|
|
I spent the first week of February at the Python10 conference
|
|
in Alexandria, VA. I'm including this experience report
|
|
for two reasons: firstly, it documents where my time was
|
|
used. Secondly, a public presence for Boost.Python and
|
|
interaction between the Python and C++ communities is
|
|
important to the future of Boost.Python, which in turn is
|
|
important to the Kull Project.
|
|
|
|
<p>Andy Koenig, of all people, was the keynote speaker of
|
|
this year's opening plenary session. He presented his
|
|
"impressions of a polyglot outsider", which
|
|
studiously avoided any mention of C++ until the end of his
|
|
talk, when he was asked about standardization. I was
|
|
surprised to learn that the C++ community at large wanted a
|
|
few more years before beginning but when ANSI accepted
|
|
HP's request for a standard, the process was forced to
|
|
start: it was a matter of participating or having
|
|
standardization proceed without one's input. Andy managed
|
|
to highlight very effectively the balance of strengths in
|
|
Python, one of the most important being its support for
|
|
extension via libraries. In many ways that makes Python a
|
|
good analogue for C++ in the interpreted world
|
|
|
|
<p>There were several kind mentions of the Boost.Python
|
|
library from people who found it indispensable. I was
|
|
particularly happy that Karl MacMillan, Michael Droettboom,
|
|
and Ichiro Fujinaga from Johns Hopkins is using it to do OCR
|
|
on a vast library of music notation, since in a previous life
|
|
I was an author of music notation software. These guys are
|
|
also drawing on Ullrich Koethe's VIGRA library for image
|
|
manipulation (Ullrich has been a major contributor to
|
|
Boost.Python). They also have a system for writing the
|
|
Boost.Python wrapper code in C++ comments, which allows them
|
|
to keep all of the code in one place. I've asked them to
|
|
send me some information on that.
|
|
|
|
<p>The development of Swig has been gaining momentum again
|
|
(the basic description at
|
|
www.boost.org/libs/python/doc/comparisons.html still
|
|
applies). The talk given about it by David Beazly was very
|
|
well-attended, and they appear to have quite a few users.
|
|
Swig's strengths (coverage of many langauages) and
|
|
weaknesses (incomplete C++ language support) haven't
|
|
changed, although the C++ support seems to have improved
|
|
considerably - they now claim to have a complete model of the
|
|
C++ type system. It seems to be mostly geared at wrapping
|
|
what Walter Landry calls "C-Tran": C++ code which
|
|
traffics in built-in types with little use of abstraction.
|
|
I'm not knocking that, either: I'm sure a lot of that
|
|
code exists, so it's a valuable service. One feature Swig
|
|
has which I'd like to steal is the ability to unwrap a
|
|
single Python argument into multiple C++ arguments, for
|
|
example, by converting a Python string into a pointer and
|
|
length. When his talk was over, David approached me about a
|
|
possible joint workshop on language binding, which sounds
|
|
like a fun idea to me.
|
|
|
|
<p>I spent some considerable time talking with Steven Knight,
|
|
the leader of the Scons build tool effort. We had a lot to
|
|
share with one another, and I gained a much better
|
|
appreciation for many of the Scons design decisions. Scons
|
|
seems to be concentrating on being the ultimate build system
|
|
substrate, and Steve seemed to think that we were on the
|
|
right track with our high-level design. We both hope that the
|
|
Boost.Build V2 high-level architecture can eventually be
|
|
ported to run on top of Scons.
|
|
|
|
<p>They also have a highly-refined and successful development
|
|
procedure which I'd like to emulate for Boost.Build V2.
|
|
Among many other things they do, their source-control system
|
|
automatically ensures that when you check in a new test, it
|
|
is automatically run on the currently checked-in state of the
|
|
code, and is expected to fail -- a relatively obvious good
|
|
idea which I've never heard before.
|
|
|
|
<p>Guido Van Rossum's "State of the Python
|
|
Union" address was full of questions for the community
|
|
about what should be done next, but the one idea Guido seemed
|
|
to stress was that core language stability and continuing
|
|
library development would be a good idea (sound familiar?) I
|
|
mentioned the Boost model as a counterpoint to the idea of
|
|
something like CPAN (the massive Perl library archives), and
|
|
it seemed to generate some significant interest. I've
|
|
offered to work with anyone from the Python community who
|
|
wants to set up something like Boost.
|
|
|
|
<p>There was some discussion of "string
|
|
interpolation" (variable substitution in strings), and
|
|
Guido mentioned that he had some thoughts about the
|
|
strengths/weaknesses of Python's formatting interface. It
|
|
might be useful for those working on formatting for boost to
|
|
contact him and find out what he has to say.
|
|
|
|
<p>Ka-Ping Yee demoed a Mailman discussion thread weaver.
|
|
This tool weaves the various messages in a discussion thread
|
|
into a single document so you can follow the entire
|
|
conversation. Since we're looking very seriously at
|
|
moving Boost to Mailman, this could be a really useful thing
|
|
for us to have. If we do this, we'll move the yahoogroups
|
|
discussions into the mailman archive so old discussions can
|
|
be easily accessed in the same fashion.
|
|
|
|
<p>And, just because it's cool, though perhaps not
|
|
relevant: http://homepages.ulb.ac.be/~arigo/psyco/ is a
|
|
promising effort to accelerate the execution of Python code
|
|
to speeds approaching those of compiled languages. It
|
|
reminded me a lot of Todd Veldhuizen's research into
|
|
moving parts of C++ template compilation to runtime, only
|
|
coming from the opposite end of things.
|
|
|
|
<h2><a name="progress">Boost.Python v2 Progress</a></h2>
|
|
Here's what actually got accomplished.
|
|
|
|
<h3><a name="documentation">Documentation</a></h3>
|
|
|
|
<p>My first priority upon returning from Python10 was to get
|
|
some documentation in place. After wasting an unfortunate
|
|
amount of time looking at automatic documentation tools which
|
|
don't quite work, I settled down to use Bill Kempf's
|
|
HTML templates designed to be a boost standard. While they
|
|
are working well, it is highly labor-intensive.
|
|
|
|
<p>I decided to begin with the high-level reference material,
|
|
as opposed to tutorial, narrative, or nitty-gritty details of
|
|
the framework. It seemed more important to have a precise
|
|
description of the way the commonly-used components work than
|
|
to have examples in HTML (since we already have some test
|
|
modules), and since the low-level details are much
|
|
less-frequently needed by users it made sense for me to
|
|
simply respond to support requests for the time being.
|
|
|
|
<p>After completing approximately 60% of the high-level docs
|
|
(currently checked in to libs/python/doc/v2), I found myself
|
|
ready to start documenting the mechanisms for creating
|
|
to-/from-python converters. This caused a dilemma: I had
|
|
realized during the previous week that a much simpler,
|
|
more-efficient, and easier-to-use implementation was
|
|
possible, but I hadn't planned on implementing it right
|
|
away, since what was already in place worked adequately. I
|
|
had also received my first query on the C++-sig about how to
|
|
write such a converter
|
|
|
|
<p>Given the labor-intensive nature of documentation writing,
|
|
I decided it would be a bad idea to document the conversion
|
|
mechanism if I was just going to rewrite it. Often the best
|
|
impetus for simplifying a design is the realization that
|
|
understandably documenting its current state would be too
|
|
difficult, and this was no exception.
|
|
|
|
<h3><a name="conversion">Overhaul of
|
|
<code>to_python</code>/<code>from_python</code> conversion
|
|
mechanism</a></h3>
|
|
|
|
<p>There were two basic realizations involved here:
|
|
|
|
<ol>
|
|
<li><code>to_python</code> conversion could be a one-step
|
|
process, once an appropriate conversion function is found.
|
|
This allows elimination of the separate indirect
|
|
convertibility check
|
|
|
|
<li>There are basically two categories of from_python
|
|
conversions: those which lvalues stored within or held by
|
|
the Python object (essentially extractions), like what
|
|
happens when an instance of a C++ class exposed with class_
|
|
is used as the target of a wrapped member function), and
|
|
those in which a new rvalue gets created, as when a Python
|
|
Float is converted to a C++
|
|
<code>complex<double></code> or a Python tuple is
|
|
converted to a C++ <code>std::vector<></code>. From
|
|
the client side, there are two corresponding categories of
|
|
conversion: those which demand an lvalue conversion and
|
|
those which can accept an lvalue or an rvalue conversion.
|
|
</ol>
|
|
The latter realization allowed the following collapse, which
|
|
considerably simplified things:
|
|
|
|
<blockquote>
|
|
<table border="1" summary="Conversion protocol">
|
|
<tr>
|
|
<th>Target Type
|
|
|
|
<th>Eligible Converters
|
|
|
|
<tr>
|
|
<td><code>T</code>
|
|
|
|
<td rowspan="5"><code>T</code> rvalue or lvalue
|
|
|
|
<tr>
|
|
<td><code>T const</code>
|
|
|
|
<tr>
|
|
<td><code>T volatile</code>
|
|
|
|
<tr>
|
|
<td><code>T const volatile</code>
|
|
|
|
<tr>
|
|
<td><code>T const&</code>
|
|
|
|
<tr>
|
|
<td><code>T const*</code>
|
|
|
|
<td rowspan="9"><code>T</code> lvalue
|
|
|
|
<tr>
|
|
<td><code>T volatile*</code>
|
|
|
|
<tr>
|
|
<td><code>T const volatile*</code>
|
|
|
|
<tr>
|
|
<td><code>T&</code>
|
|
|
|
<tr>
|
|
<td><code>T volatile&</code>
|
|
|
|
<tr>
|
|
<td><code>T const volatile&</code>
|
|
|
|
<tr>
|
|
<td><code>T* const&</code>
|
|
|
|
<tr>
|
|
<td><code>T const* const&</code>
|
|
|
|
<tr>
|
|
<td><code>T volatile*const&</code>
|
|
|
|
<tr>
|
|
<td><code>T const volatile*const&</code>
|
|
</table>
|
|
</blockquote>
|
|
This job included the following additional enhancements:
|
|
|
|
<ul>
|
|
<li>Elimination of virtual functions, which cause object
|
|
code bloat
|
|
|
|
<li>Registration of a single converter function for all
|
|
lvalue conversions, two for all rvalue conversions
|
|
|
|
<li>Killed lots of unneeded code
|
|
|
|
<li>Increased opacity of registry interface
|
|
|
|
<li>Eliminated all need for decorated runtime type
|
|
identifiers
|
|
|
|
<li>Updated test modules to reflect new interface
|
|
|
|
<li>Eliminated the need for users to worry about converter
|
|
lifetime issues Additional Builtin Conversion Enhancements
|
|
|
|
<li>Support for complex<float>,
|
|
complex<double>, and complex<long double>
|
|
conversions
|
|
|
|
<li>Support for bool conversions
|
|
|
|
<li>NULL pointers representable by None in Python
|
|
|
|
<li>Support for conversion of Python classic classes to
|
|
numeric types
|
|
</ul>
|
|
|
|
<h3><a name="miscellaneous">Miscellaneous</a></h3>
|
|
These don't fit easily under a large heading:
|
|
|
|
<ul>
|
|
<li>Support CallPolicies for class member functions
|
|
|
|
<li>from_python_data.hpp: revamped type alignment
|
|
metaprogram so that it's fast enough for KCC
|
|
|
|
<li>classfwd.hpp header forward-declares class_<T>
|
|
|
|
<li>indirect_traits.hpp:
|
|
|
|
<li>added is_pointer_to_reference
|
|
|
|
<li>fixed bugs
|
|
|
|
<li>Reduced recompilation dependencies
|
|
|
|
<li>msvc_typeinfo works around broken MS/Intel typeid()
|
|
implementation
|
|
|
|
<li>Many fixes and improvements to the type_traits library
|
|
in order to work around compiler bugs and suppress warnings
|
|
|
|
<li>Eliminated the need for explicit acquisition of
|
|
converter registrations
|
|
|
|
<li>Expanded constructor support to 6 arguments
|
|
|
|
<li>Implemented generalized pointer lifetime support
|
|
|
|
<li>Updated code generation for returning.hpp
|
|
|
|
<li>Tracked down and fixed cycle GC bugs
|
|
|
|
<li>Added comprehensive unit tests for destroy_reference,
|
|
pointer_type_id, select_from_python, complex<T>,
|
|
bool, and classic class instance conversions
|
|
</ul>
|
|
|
|
<p>Revised
|
|
<!--webbot bot="Timestamp" S-Type="EDITED" S-Format="%d %B, %Y" startspan -->
|
|
13 November, 2002
|
|
<!--webbot bot="Timestamp" endspan i-checksum="39359" -->
|
|
|
|
|
|
<p class="c3">© Copyright <a href=
|
|
"../../../../people/dave_abrahams.htm">Dave Abrahams</a> 2002. Distributed
|
|
under the Boost Software License, Version 1.0. (See accompanying file
|
|
LICENSE_1_0.txt or copy at http://www.boost.org/LICENSE_1_0.txt)</p>
|
|
|