From 9227ba0108adecaa89979181b661bcd0131662c9 Mon Sep 17 00:00:00 2001 From: Dave Abrahams Date: Mon, 16 Oct 2000 01:37:07 +0000 Subject: [PATCH] Add documentation [SVN r7959] --- comparisons.html | 176 ++++++++++++++++++++++++++++++++++++++++++++ example1.html | 127 ++++++++++++++++++++++++++++++++ extending.html | 73 ++++++++++++++++++ overloading.html | 139 ++++++++++++++++++++++++++++++++++ overriding.html | 175 +++++++++++++++++++++++++++++++++++++++++++ py_cpp.html | 120 ++++++++++++++++++++++++++++++ under-the-hood.html | 58 +++++++++++++++ 7 files changed, 868 insertions(+) create mode 100644 comparisons.html create mode 100644 example1.html create mode 100644 extending.html create mode 100644 overloading.html create mode 100644 overriding.html create mode 100644 py_cpp.html create mode 100644 under-the-hood.html diff --git a/comparisons.html b/comparisons.html new file mode 100644 index 00000000..565e8bad --- /dev/null +++ b/comparisons.html @@ -0,0 +1,176 @@ + + + Comparisons with Other Systems + +
+

+ c++boost.gif (8819 bytes)Comparisons with + Other Systems +

+ +

CXX

+

+ Like py_cpp, CXX attempts to + provide a C++-oriented interface to Python. In most cases, like py_cpp, + it relieves the user from worrying about reference-counts. As far as I + can tell, there is no support for subclassing C++ extension types in + Python. An even more-significant difference is that a user's C++ code is + still basically "dealing with Python objects", though they are wrapped + in C++ classes. This means such jobs as argument parsing and conversion + are still left to be done explicitly by the user. This is not entirely a + bad thing, as you can do some Pythonic things with CXX (e.g. variable + and keyword arguments) that I haven't yet figured out how to enable with + py_cpp. As far as I can tell, also CXX enables one to write what is + essentially idiomatic Python code in C++, manipulating Python objects + through the same fully-generic interfaces we use in Python. That use is + also supported (less-well) by the py_cpp object wrappers. Paul F. Dubois, the CXX maintainer, + has told me that what I've described is only half of the picture with + CXX, but I never understood his explanation well-enough to fill in the + other half. I hope that you, dear reader, may be able to help me + complete my comparitive analysis of CXX. + + +

SWIG

+

+ SWIG is an impressively mature tool + for exporting an existing ANSI 'C' interface into various scripting + languages. Swig relies on a parser to read your source code and produce + additional source code files which can be compiled into a Python (or + Perl or Tcl) extension module. It has been successfully used to create + many Python extension modules. Like py_cpp, SWIG is trying to allow an + existing interface to be wrapped with little or no change to the + existing code. The documentation says "SWIG parses a form of ANSI C + syntax that has been extended with a number of special directives. As a + result, interfaces are usually built by grabbing a header file and + tweaking it a little bit." For C++ interfaces, the tweaking has often + proven to amount to more than just a little bit. One user + writes:

"The problem with swig (when I used it) is that it + couldnt handle templates, didnt do func overloading properly etc. For + ANSI C libraries this was fine. But for usual C++ code this was a + problem. Simple things work. But for anything very complicated (or + realistic), one had to write code by hand. I believe py_cpp doesnt have + this problem[sic]... IMHO overloaded functions are very important to + wrap correctly."
-Prabhu Ramachandran +
+ +

+ By contrast, py_cpp doesn't attempt to parse C++ - the problem is simply + too complex to do correctly. Technically, one does write code by hand to + use py_cpp. The goal, however, has been to make that code nearly as + simple as listing the names of the classes and member functions you want + to expose in Python. + +

SIP

+

+ SIP + is a system similar to SWIG, though seemingly more + C++-oriented. The author says that like py_cpp, SIP supports overriding + extension class member functions in Python subclasses. It appears to + have been designed specifically to directly support some features of + PyQt/PyKDE, which is its primary client. Documentation is almost + entirely missing at the time of this writing, so a detailed comparison + is difficult. + +

ILU

+

+ ILU + is a very ambitious project which tries to describe a module's interface + (types and functions) in terms of an Interface + Specification Language (ISL) so that it can be uniformly interfaced + to a wide range of computer languages, including Common Lisp, C++, C, + Modula-3, and Python. ILU can parse the ISL to generate a C++ language + header file describing the interface, of which the user is expected to + provide an implementation. Unlike py_cpp, this means that the system + imposes implementation details on your C++ code at the deepest level. It + is worth noting that some of the C++ names generated by ILU are supposed + to be reserved to the C++ implementation. It is unclear from the + documentation whether ILU supports overriding C++ virtual functions in Python. + +

Zope ExtensionClasses

+

+ + ExtensionClasses in Zope use the same underlying mechanism as py_cpp + to support subclassing of extension types in Python, including + multiple-inheritance. Both systems rely on the same "Don + Beaudry Hack" that also inspired Don's MESS System. +

+ The major differences are: +

+

+ Also, the Zope docs say: "The first superclass listed in the class + statement defining an extension subclass must be either a base + extension class or an extension subclass. This restriction will be + removed in Python-1.5." I believe that this promise was made + prematurely. I have looked at the Python 1.5.2 source code and I don't + believe it is possible to deliver on it. +

+ Previous: A Brief Introduction to writing Python Extension Modules + Next: A Simple Example Using py_cpp + Up: Top +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability + for any purpose. +

+ Updated: Oct 15, 2000 +

+ diff --git a/example1.html b/example1.html new file mode 100644 index 00000000..9a4a868d --- /dev/null +++ b/example1.html @@ -0,0 +1,127 @@ + + + A Simple Example Using py_cpp + +
+

+ + +

+

+ A Simple Example Using py_cpp +

+

+ Suppose we have the following C++ API which we want to expose in + Python: +

+
+namespace hello {
+  class world
+  {
+   public:
+      world(int);
+      ~world();
+      const char* get() const { return "hi, world"; }
+    ...
+  };
+  void length(const world& x) { return std::strlen(x.get()); }
+}
+
+
+
+

+ Here is the C++ code for a python module called hello + which exposes the API using py_cpp: +

+
+#include <py_cpp/class_wrapper.h>
+// Python requires an exported function called init<module-name> in every
+// extension module. This is where we build the module contents.
+extern "C"
+#ifdef _WIN32
+__declspec(dllexport)
+#endif
+void inithello()
+{
+    try
+    {
+       // create an object representing this extension module
+       py::Module hello("hello");
+       // Create the Python type object for our extension class
+       py::ClassWrapper<hello::world> world_class(hello, "world");
+       // Add the __init__ function
+       world_class.def(py::Constructor<int>());
+       // Add a regular member function
+       world_class.def(&hello::world::get, "get");
+       // Add a regular function to the module
+       hello.def(hello::length, "length");
+    }
+    catch(...)
+    {
+       py::handle_exception();    // Deal with the exception for Python
+    }
+}
+// Win32 DLL boilerplate
+#if defined(_WIN32)
+#include <windows.h>
+extern "C" BOOL WINAPI DllMain(HINSTANCE, DWORD, LPVOID)
+{
+    return 1;
+}
+#endif // _WIN32
+
+
+

+ That's it! If we build this shared library and put it on our + PYTHONPATH we can now access our C++ class and function from + Python. +

+
+>>> import hello
+>>> hi_world = hello.world(3)
+>>> hi_world.get()
+'hi, world'
+>>> hello.length(hi_world)
+9
+
+
+

+ We can even make a subclass of hello.world: +

+
+>>> class my_subclass(hello.world):
+...     def get(self):
+...         return 'hello, world'
+...
+>>> y = my_subclass(4)
+>>> y.get()
+'hello, world'
+
+
+

+ Pretty cool! You can't do that with an ordinary Python extension type! +

+
+>>> hello.length(y)
+9
+
+
+

+ Of course, you may now have a slightly empty feeling in the pit of + your little pythonic stomach. Perhaps you feel your subclass deserves + to have a length() of 12? If so, read on... +

+ Previous: Comparisons with other systems Next: Overridable virtual functions Up: + Top +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability + for any purpose. +

+ Updated: Oct 15, 2000 +

+ diff --git a/extending.html b/extending.html new file mode 100644 index 00000000..a3a2da98 --- /dev/null +++ b/extending.html @@ -0,0 +1,73 @@ + + + + A Brief Introduction to writing Python extension modules + +

+ c++boost.gif (8819 bytes) +

+

+ A Brief Introduction to writing Python extension modules +

+

+ Interfacing any language to Python involves building a module which can + be loaded by the Python interpreter, but which isn't written in Python. + This is known as an extension module. Many of the built-in Python + libraries are constructed in 'C' this way; Python even supplies its + fundamental + types using the same mechanism. An extension module can be statically + linked with the Python interpreter, but it more commonly resides in a + shared library or DLL. +

+ As you can see from The Python Extending + and Embedding Tutorial, writing an extension module normally means + worrying about +

+ This last item typically occupies a great deal of code in an extension + module. Remember that Python is a completely dynamic language. A callable + object receives its arguments in a tuple; it is up to that object to + extract those arguments from the tuple, check their types, and raise + appropriate exceptions. There are numerous other tedious details that need + to be managed; too many to mention here. Py_cpp is designed to lift most of + that burden.
+
+ +

+ Another obstacle that most people run into eventually when extending + Python is that there's no way to make a true Python class in an extension + module. The typical solution is to create a new Python type in the + extension module, and then write an additional module in 100% Python. The + Python module defines a Python class which dispatches to an instance of + the extension type, which it contains. This allows users to write + subclasses of the class in the Python module, almost as though they were + sublcassing the extension type. Aside from being tedious, it's not really + the same as having a true class, because there's no way for the user to + override a method of the extension type which is called from the + extension module. Py_cpp solves this problem by taking advantage of Python's metaclass + feature to provide objects which look, walk, and hiss almost exactly + like regular Python classes. Py_cpp classes are actually cleaner than + Python classes in some subtle ways; a more detailed discussion will + follow (someday).

+

Next: Comparisons with Other Systems Up: Top

+

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability for + any purpose.

+ diff --git a/overloading.html b/overloading.html new file mode 100644 index 00000000..9857c529 --- /dev/null +++ b/overloading.html @@ -0,0 +1,139 @@ + + + Function Overloading + +
+

+ c++boost.gif (8819 bytes)Function Overloading +

+ +

An Example

+

+ To expose overloaded functions in Python, simply def() each + one with the same Python name: +

+
+inline int f1() { return 3; }
+inline int f2(int x) { return x + 1; }
+
+class X {
+public:
+    X() : m_value(0) {}
+    X(int n) : m_value(n) {}
+    int value() const { return m_value; }
+    void value(int v) { m_value = v; }
+private:
+    int m_value;
+};
+  ...
+
+void initoverload_demo()
+{
+    try
+    {
+        py::Module overload_demo("overload_demo");
+        // Overloaded functions at module scope
+        overload_demo.def(f1, "f");
+        overload_demo.def(f2, "f");
+
+        py::ClassWrapper<X> x_class(overload_demo, "X");
+        // Overloaded constructors
+        x_class.def(py::Constructor<>());
+        x_class.def(py::Constructor<int>());
+
+        // Overloaded member functions
+        x_class.def((int (X::*)() const)&X::value, "value");
+        x_class.def((void (X::*)(int))&X::value, "value");
+  ...
+
+
+ +

+ Now in Python: +

+
+>>> from overload_demo import *
+>>> x0 = X()
+>>> x1 = X(1)
+>>> x0.value()
+0
+>>> x1.value()
+1
+>>> x0.value(3)
+>>> x0.value()
+3
+>>> X('hello')
+TypeError: No overloaded functions match (X, string). Candidates are:
+void (*)()
+void (*)(int)
+>>> f()
+3
+>>> f(4)
+5
+
+
+ +

Discussion

+

+ Notice that overloading in the Python module was produced three ways:

    +
  1. by combining the non-overloaded C++ functions int f1() + and int f2(int) and exposing them as f in Python. +
  2. by exposing the overloaded constructors of class X +
  3. by exposing the overloaded member functions X::value. +
+

+ Techniques 1. and 3. above are really alternatives. In case 3, you need + to form a pointer to each of the overloaded functions. The casting + syntax shown above is one way to do that in C++. Case 1 does not require + complicated-looking casts, but may not be viable if you can't change + your C++ interface. N.B. There's really nothing unsafe about casting an + overloaded (member) function address this way: the compiler won't let + you write it at all unless you get it right. + +

An Alternative to Casting

+

+ This approach is not neccessarily better, but may be preferable for some + people who have trouble writing out the types of (member) function + pointers or simply prefer to avoid all casts as a matter of principle: +

+
+// Forwarding functions for X::value
+inline void set_x_value(X& self, int v) { self.value(v); }
+inline int get_x_value(X& self) { return self.value(); }
+   ...
+        // Overloaded member functions
+        x_class.def(set_x_value, "value");
+        x_class.def(get_x_value, "value");
+
+
+

Here we are taking advantage of the ability to expose C++ functions at +namespace scope functions as Python member functions. + +

Overload Resolution

+

+ The function overload resolution mechanism in py_cpp works as + follows:

+

+ Prev: Function Overloading + Next: A Peek Under the Hood + Up: Top +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability + for any purpose. +

+ Updated: Oct 15, 2000 +

+ diff --git a/overriding.html b/overriding.html new file mode 100644 index 00000000..82cb442d --- /dev/null +++ b/overriding.html @@ -0,0 +1,175 @@ + + + + Overridable Virtual Functions + +

+ c++boost.gif (8819 bytes) +

+

+ Overridable Virtual Functions +

+

+ In the previous example we exposed a simple + C++ class in Python and showed that we could write a subclass. We even + redefined one of the functions in our derived class. Now we will learn + how to make the function behave virtually. Of course, the first + step if we want it to act like a virtual function when called from our + C++ code, is to make it virtual: +

+
+  class world
+  {
+    ...
+      virtual const char* get() const { return "hi, world"; }
+    ...
+  };
+
+
+

+ Then we'll need a derived class* to help us + dispatch the call to Python: +

+
+struct world_callback : hello::world
+{
+    // The first argument must by a PyObject* (the corresponding Python object)
+    // The rest of the argument list should match the base class constructor
+    world_callback(PyObject* self, int x)
+        : world(x),            // dispatch to base object
+          m_self(self) {}      // hang onto the Python object
+
+    // Dispatch the call to Python
+    const char* get() const
+    {
+        // Any function arguments would go at the end of the argument list
+        // The return type is a template parameter
+        return py::Callback<const char*>::call_method(m_self, "get");
+    }
+
+    // Something Python can call in case there is no override of get()
+    const char* default_get() const
+        { return this->hello::world::get(); }
+ private:
+    PyObject* m_self; // A way to hold onto the Python object
+};
+
+
+

+ Finally, we add world_callback to the + ClassWrapper<> declaration in our module initialization + function: +

+
+// Create the Python type object for our extension class
+py::ClassWrapper<hello::world,world_callback> world_class(hello, "world");
+       ...
+
+
+

+ ...and when we define the function, we must tell py_cpp about the default + implementation: +

+
+// Add a virtual member function
+world_class.def(&hello::world::get, "get", &world_callback::default_get);
+
+
+

+ Now our subclass of hello.world behaves as expected: +

+

+
+>>> class my_subclass(hello.world):
+...     def get(self):
+...         return 'hello, world'
+...
+>>> hello.length(my_subclass())
+12
+
+
+

+ *You may ask, "Why do we need this derived + class? This could have been designed so that everything gets done right + inside of hello::world." One of the goals of py_cpp is to be + minimally intrusive on an existing C++ design. In principle, it should be + possible to expose the interface for a 3rd party library without changing + it. +

+ Pure Virtual Functions +

+

+ A pure virtual function with no implementation is actually a lot easier + to deal with than a virtual function with a default implementation. First + of all, you obviously don't need to + supply a default implementation. Secondly, you don't need to call + def() on the ExtensionClass<> instance + for the virtual function. In fact, you wouldn't want to: if the + corresponding attribute on the Python class stays undefined, you'll get + an AttributeError in Python when you try to call the + function, indicating that it should have been implemented. For example: +

+
+struct baz
+{
+    virtual void pure(int) = 0;
+};
+
+struct baz_callback
+{
+    void pure(int x) { py::Callback<void>::call_method(m_self, "pure", x); }
+};
+
+extern "C"
+#ifdef _WIN32
+__declspec(dllexport)
+#endif
+initfoobar()
+{
+    try
+    {
+       py::Module foobar("foobar");
+       py::ClassWrapper<baz,baz_callback> baz_class("baz");
+       baz_class.def(&baz::pure, "pure");
+    }
+    catch(...)
+    {
+       py::handle_exception();    // Deal with the exception for Python
+    }
+}
+
+
+

+ Now in Python: +

+
+>>> from foobar import baz
+>>> x = baz()
+>>> x.pure()
+Traceback (innermost last):
+  File "<stdin>", line 1, in ?
+AttributeError: pure
+>>> class mumble(baz):
+...    def pure(self, z): pass
+...
+>>> y = mumble()
+>>> y.pure()
+>>> 
+
+
+

+ Prev: A Simple Example Using py_cpp Next: A Peek Under the Hood Up: Top +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability for + any purpose. +

+ Updated: Sept 30, 2000 + diff --git a/py_cpp.html b/py_cpp.html new file mode 100644 index 00000000..b9b28f02 --- /dev/null +++ b/py_cpp.html @@ -0,0 +1,120 @@ + + + + py_cpp Python/C++ binding documentation + +

+ c++boost.gif (8819 bytes) py_cpp* +

+

+ Py_cpp is a system for quickly and easily interfacing C++ code with Python such that the Python interface is + very similar to the C++ interface. It is designed to be minimally + intrusive on your C++ design. In most cases, you should not have to alter + your C++ classes in any way in order to use them with py_cpp. The system + should simply "reflect" your C++ classes and functions into + Python. +

+ The source code for py_cpp, including a MSVC demo project is available + here. It has been tested against Python + 1.5.2 with GCC 2.95.2, Metrowerks CodeWarrior Pro6 and with Microsoft + Visual C++ 6 sp4 using both the + STLport standard library implementation and the library + implementation which ships with the compiler. It has also been tested + against Python 2.0c1 with an unknown version of MSVC++ by Alex Martelli. + Py_cpp requires the Boost libraries, + and is currently under formal review on the boost mailing list for + acceptance into boost. +

+ py_cpp was originally written by David Abrahams. Ullrich Koethe supplied + an early version of the overloading support and wrote the support for + implicit conversions of arguments that have a C++ inheritance + relationship. Alex Martelli supplied the first tests against Python 2.0. + The members of the boost mailing list and the Python community supplied + invaluable early feedback. The development of py_cpp wouldn't have been + possible without the generous support of Dragon Systems/Lernout and + Hauspie, Inc. +

+ Table of Contents +

+
    +
  1. + A Brief Introduction to writing Python + extension modules +
  2. + Comparisons between py_cpp and other systems + for extending Python +
  3. + A Simple Example Using py_cpp +
  4. + Overridable Virtual Functions +
  5. + Function Overloading +
  6. + A Peek Under the Hood +
+

+ More sophisticated examples, including examples which demonstrate that + these ExtensionClasses support some of Python's "special" member + functions (e.g. __getattr__(self, name)), are given in + extclass_demo.cpp, extclass_demo.h, and + test_extclass.py in the source code + archive. There's much more here, and much more documentation to + come... +

+ Questions should be directed to the boost mailing list or to David Abrahams, the primary + author and maintainer. +

+ Naming Contest +

+

+ Yes, I know py_cpp is a lousy name. Problem is, the best names my puny + imagination can muster (IDLE and GRAIL) are taken, so I'm holding a + naming contest. First prize? You get to pick the name<0.2wink> and + you will be credited in the documentation. Names that have been suggested + so far include: +

+ Please post or send me your suggestions!
+
+ +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability for + any purpose. +

+ Updated: Oct 15, 2000 + diff --git a/under-the-hood.html b/under-the-hood.html new file mode 100644 index 00000000..fb17539f --- /dev/null +++ b/under-the-hood.html @@ -0,0 +1,58 @@ + + + + A Peek Under the Hood + +

+ c++boost.gif (8819 bytes) +

+

+ A Peek Under the Hood +

+

+ ExtensionClass<T> is a subclass of + PyTypeObject, the struct which Python's 'C' API uses + to describe a type. An instance of + the ExtensionClass<> becomes the Python type + object corresponding to hello::world. When we add it to the module it goes into the + module's dictionary to be looked up under the name "world". +

+ Py_cpp uses C++'s template argument deduction mechanism to determine the + types of arguments to functions (except constructors, for which we must + provide an argument list + because they can't be named in C++). Then, it calls the appropriate + overloaded functions PyObject* + to_python(S) and + S'from_python(PyObject*, + Type<S>) which convert between any C++ + type S and a PyObject*, the type which represents a + reference to any Python object in its 'C' API. The ExtensionClass<T> + template defines a whole raft of these conversions (for T, T*, + T&, std::auto_ptr<T>, etc.), using the same inline + friend function technique employed by the boost operators + library. +

+ Because the to_python and from_python functions + for a user-defined class are defined by + ExtensionClass<T>, it is important that an instantiation of + ExtensionClass<T> is visible to any code which wraps + a C++ function with a T, T*, const T&, etc. parameter or + return value. In particular, you may want to create all of the classes at + the top of your module's init function, then def the member + functions later to avoid problems with inter-class dependencies. +

+ Previous: Function Overloading Up: Top +

+ © Copyright David Abrahams 2000. Permission to copy, use, modify, + sell and distribute this document is granted provided this copyright + notice appears in all copies. This document is provided "as is" without + express or implied warranty, and with no claim as to its suitability for + any purpose. +

+ Updated: Sept 30, 2000 +