From 511a6e84db5d6f62121c254377abcfcd4ecb6285 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Ullrich=20K=C3=B6the?= Date: Tue, 21 Nov 2000 23:12:22 +0000 Subject: [PATCH] updated documentation of special name support, added documentation of automatic operator export and coercion [SVN r8283] --- overloading.html | 12 +- py_cpp.html | 9 +- special.html | 661 ++++++++++++++++++++++++++++++++--------------- 3 files changed, 469 insertions(+), 213 deletions(-) diff --git a/overloading.html b/overloading.html index 868a9861..66fff494 100644 --- a/overloading.html +++ b/overloading.html @@ -126,15 +126,23 @@ namespace scope as Python member functions. this sense, overload resolution mirrors the C++ mechanism, where a name in a derived class "hides" all functions with the same name from a base class. +

  • Within a name-space context (extension class or module), overloaded functions are tried in the same order they were def()ed. The first function whose signature can be made to match each argument passed is the one which is ultimately called. +
    + This means in particular that you cannot overload the same function on + both "int" and "float" because Python + automatically converts either of the two types into the other one. + Thus, if the "float"-overload is found first, it is used + also used for arguments of type "int" as well, and the + "int"-Version of the function is never invoked.

    - Prev: Function Overloading + Prev: Overridable Virtual Functions Next: Special Method Names Up: Top

    @@ -144,6 +152,6 @@ namespace scope as Python member functions. express or implied warranty, and with no claim as to its suitability for any purpose.

    - Updated: Oct 30, 2000 + Updated: Nov 21, 2000 diff --git a/py_cpp.html b/py_cpp.html index c1381eae..22495875 100644 --- a/py_cpp.html +++ b/py_cpp.html @@ -77,8 +77,9 @@ href="overloading.html">function overloading and wrote the support for reflecting C++ inheritance relationships. He has helped to improve error-reporting from both - Python and C++, and is currently doing valuable research into the best approach for numeric coercion. + Python and C++, and has designed an extremely easy-to-use approach to + export numeric operators including + the possibility to avoid explicit coercion by means of overloading.

  • The members of the boost mailing list and the Python community supplied invaluable early feedback. In particular, Ron Clarke, Mark Evans, Anton @@ -108,7 +109,7 @@
  • Inheritance -
  • Special Method Name Support +
  • Special Method and Operator Support
  • A Peek Under the Hood @@ -199,5 +200,5 @@ express or implied warranty, and with no claim as to its suitability for any purpose.

    - Updated: Oct 18, 2000 + Updated: Nov 21, 2000 diff --git a/special.html b/special.html index d225451c..fbdd301a 100644 --- a/special.html +++ b/special.html @@ -1,13 +1,13 @@ - Special Method Name Support + Special Method and Operator Support

    c++boost.gif (8819 bytes)Special Method Name - Support + "c++boost.gif" alt="c++boost.gif (8819 bytes)">Special Method + and Operator Support

    Overview @@ -18,121 +18,515 @@ "http://www.pythonlabs.com/pub/www.python.org/doc/current/ref/specialnames.html"> special method names supported by real Python class instances except __complex__ (more on the reasons below). + Supported operators include general, + numeric, and sequence and + mapping operators. In addition, py_cpp provides a simple way to export member + variables and define attributes by means of getters and + setters. -

    Numeric Operators

    + +

    General Operators

    +     -There are two fundamental ways to define numeric operators within py_cpp: automatic wrapping -and manual wrapping. Suppose, C++ defines an addition operator for type Rational, so that we can write: +Python provides a number of special operatos for basic customization of a class: + +
    +
    __repr__: +
    create a string representation from which the object can be reconstructed +

    +

    __str__: +
    create a string representation which is suitable for printing +

    +

    __cmp__: +
    three-way compare function, used to implement comparison operators (< etc.) +

    +

    __hash__: +
    needed to use the object as a dictionary key (only allowed if __cmp__ is also defined) +

    +

    __nonzero__: +
    called if the object is used as a truth value (e.g. in an if statement) +

    +

    __call__: +
    make instances of the class callable like a function +

    +

    + +If we have a suitable C++ function that supports any of these features, we can export it like any other function, using its Python special name. For example, suppose that class Foo provides a string conversion function: 
    -    Rational a, b, c;
+    std::string to_string(Foo const & f)
+    {
+        std::ostringstream s;
+        s << f;
+        return s.str();
+    }
+
    + +This function would be wrapped like this: + +
    +    py::ClassWrapper<:Foo> foo_class(my_module, "Foo");
+    foo_class.def(&to_string, "__str__");
+
    + +Note that py_cpp also supports automatic wrapping in case of __str__ and __cmp__. This is explained in the next section and the table of numeric operators. + + +

    Numeric Operators

    +     + +There are two fundamental ways to define numeric operators within py_cpp: manual wrapping (as is done with general operators) and automatic wrapping. Lets start with the second possibility. Suppose, C++ defines a class Int (which might represent an infinite-precision integer) which support addition, so that we can write (in C++): + +
    +    Int a, b, c;
     ...
     c = a + b;
    -To enable the same functionality in Python, we first wrap the Rational class as usual: +To enable the same functionality in Python, we first wrap the Int class as usual: 
    -    py::ClassWrapper<Rational> rational_class(my_module, "Rational");
-    rational_class.def(py::Constructor<>());
+    py::ClassWrapper<Int> int_class(my_module, "Int");
+    int_class.def(py::Constructor<>());
     ...
    Then we export the addition operator like this: 
    -    rational_class.def(py::operators<py::op_add>());
+    int_class.def(py::operators<py::op_add>());
    -Since Rational also supports subtraction, multiplication, adn division, we want to export those also. This can be done in a single command by 'or'ing the operator identifiers together (a complete list of these identifiers and the corresponding operators can be found in the table): +Since Int also supports subtraction, multiplication, adn division, we want to export those also. This can be done in a single command by 'or'ing the operator identifiers together (a complete list of these identifiers and the corresponding operators can be found in the table): 
    -    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>());
+    int_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>());
    -Note that the or-expression must be enclosed in parentheses. This form of operator definition will wrap homogeneous operators, that is operators whose left and right operand have the same type. Now, suppose that our C++ library also supports addition of Rationals and integers: +Note that the or-expression must be enclosed in parentheses. This form of operator definition will wrap homogeneous operators, i.e. operators whose left and right operand have the same type. Now, suppose that our C++ library also supports addition of Ints and plain integers: 
    -    Rational a, b;
+    Int a, b;
     int i;
     ...
     a = b + i;
     a = i + b;
    -To wrap these heterogeneous operators (left and right hand side have different types), we need a possibility to specify a different operand type. This is done using the right_operand and left_operand templates: +To wrap these heterogeneous operators (left and right hand side have different types), we need a possibility to specify a different type for one of the operands. This is done using the right_operand and left_operand templates: 
    -    rational_class.def(py::operators<py::op_add>(), py::right_operand<int>());
-    rational_class.def(py::operators<py::op_add>(), py::left_operand<int>());
+    int_class.def(py::operators<py::op_add>(), py::right_operand<int>());
+    int_class.def(py::operators<py::op_add>(), py::left_operand<int>());
    Py_cpp uses overloading to register several variants of the same operation (more on this in the context of coercion). Again, several operators can be exported at once: 
    -    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(),
+    int_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(),
                        py::right_operand<int>());
-    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(), 
+    int_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(), 
                        py::left_operand<int>());
    -The type of the operand not mentioned is taken from the class object. In our example, the class object is rational_class, and thus the other operand's type is `Rational const &'. You can override this default by explicitly specifying a type in the operators template: +The type of the operand not mentioned is taken from the class object. In our example, the class object is int_class, and thus the other operand's type is `Int const &'. You can override this default by explicitly specifying a type in the operators template: 
    -    rational_class.def(py::operators<py::op_add, Rational>(), py::right_operand<int>());
+    int_class.def(py::operators<py::op_add, Int>(), py::right_operand<int>());
    -Here, `Rational' would be used instead of `Rational const &'. +Here, `Int' would be used instead of `Int const &'.

    -Note that automatic wrapping doesn't need any specific form of operator+() (or any other operator), but rather wraps the expression `left + right'. That is, this mechanism can be used for any definition of operator+(), such as a free function `Rational operator+(Rational, Rational)' or a member function `Rational Rational::operator+(Rational)'. +Note that automatic wrapping doesn't need any specific form of operator+() (or one of the other operators), but rather wraps the expression `left + right'. That is, this mechanism can be used for any definition of operator+(), such as a free function `Int operator+(Int, Int)' or a member function `Int Int::operator+(Int)'. +

    +For the Python operators pow() and abs(), there is no corresponding C++ operator. Instead, automatic wrapping attempts to wrap C++ functions of the same name. This only works if those functions are known in namespace py::detail. Thus it might be necessary to add a using declaration prior to wrapping: + +

    +    namespace py { 
+      namespace detail {
+        using my_namespace::pow;
+        using my_namespace::abs;
+    }}
+
    +

    -In some cases, automatic wrapping of operators is not possible or not desirable. Suppose, for example, that the power operation for Rationals is defined by a set of functions pow(): +In some cases, automatic wrapping of operators is not possible or not desirable. Suppose, for example, that the modulo operation for Ints is defined by a set of functions mod() (for automatic wrapping, we would need operator%()): 

    -    Rational pow(Rational const & left, Rational const & right);
-    Rational pow(Rational const & left, int right);
-    Rational pow(int left, Rational const & right);
+    Int mod(Int const & left, Int const & right);
+    Int mod(Int const & left, int right);
+    Int mod(int left, Int const & right);
    -In order to create the Python operator "pow" from these functions, we have to wrap them manually: +In order to create the Python operator "__mod__" from these functions, we have to wrap them manually: 
    -    rational_class.def((Rational (*)(Rational const &, Rational const &))&pow, "__pow__");
-    rational_class.def((Rational (*)(Rational const &, int))&pow, "__pow__");
+    int_class.def((Int (*)(Int const &, Int const &))&mod, "__mod__");
+    int_class.def((Int (*)(Int const &, int))&mod, "__mod__");
    -The third form (with int as left operand) cannot be wrapped this way. We must first create a function rpow() with the operands reversed: +The third form (with int as left operand) cannot be wrapped this way. We must first create a function rmod() with the operands reversed: 
    -    Rational rpow(Rational const & right, int left)
+    Int rmod(Int const & right, int left)
     {
-        return pow(left, right);
+        return mod(left, right);
     }
    -This function must be wrapped under the name "__rpow__": +This function must be wrapped under the name "__rmod__": 
    -    rational_class.def(&rpow,  "__rpow__");
+    int_class.def(&rmod,  "__rmod__");
    A list of the possible operator names is also found in the table. -Special treatment is necessary to define the ternary pow. +Special treatment is necessary to export the ternary pow operator.

    -Automatic and manual wrapping can be mixed arbitrarily. +Automatic and manual wrapping can be mixed arbitrarily. Note that you cannot overload the same operator for a given extension class on both `int' and `float', because Python implicitly converts these types into each other. Thus, the overloaded variant found first (be it `int' or `float') will be used for either of the two types.

    Coercion

    +Plain Python can only execute operators with identical types on the left and right hand side. If it encounters an expression where the types of the left and right operand differ, it tries to coerce these type to a common type before invoking the actual operator. Implementing good coercion functions can be difficult if many type combinations must be supported. +

    +In contrast, py_cpp provides overloading. By means of overloading, operator calling can be simplyfied drastically: you just register operators for all desired type combinations, and py_cpp automatically ensures that the correct function is called in each case. User defined coercion functions are not necessary. To enable operator overloading, py_cpp provides a standard coercion which is implicitly registered whenever automatic operator wrapping is used. +

    +If you wrap all operator functions manually, but still want to use operator overloading, you have to register the standard coercion function explicitly: + +

    +    // this is not necessary if automatic operator wrapping is used
+    int_class.def_standard_coerce();
+
    + +In case you encounter a situation where you absolutely need a customized coercion, you can overload the "__coerce__" operator itself. The signature of a coercion function must look like this: + +
    +    py::Tuple custom_coerce(PyObject * left, PyObject * right);
+
    + +The resulting Tuple must contain two elements which represent the values of left and right converted to the same type. Such a function is wrapped as usual: + +
    +    some_class.def(&custom_coerce, "__coerce__");
+
    + +Note that the custom coercion function is only used if it is defined before any automatic operator wrapping on the given class or a call to `some_class.def_standard_coerce()'. + + + +

    The Ternary pow() Operator

     + +In addition to the usual binary pow()-operator (meaning x^y), Python also provides a ternary variant that implements (x^y) % z (presumably using a more efficient algorithm than concatenation of power and modulo operators). Automatic operator wrapping can only be used with the binary variant. Ternary pow() must always be wrapped manually. For a homgeneous ternary pow(), this is done as usual: + +
    +    Int power(Int const & first, Int const & second, Int const & module);
+    typedef Int (ternary_function1)(const Int&, const Int&, const Int&);
+    ...
+    int_class.def((ternary_function1)&power,  "__pow__");
+
    + +In case you want to support this function with non-uniform argument types, wrapping is a little more involved. Suppose, you have to wrap: + +
    +    Int power(Int const & first, int second, int module);
+    Int power(int first, Int const & second, int module);
+    Int power(int first, int second, Int const & module);
+
    + +The first variant can be wrapped as usual: + +
    +    typedef Int (ternary_function2)(const Int&, int, int);
+    int_class.def((ternary_function2)&power,  "__pow__");
+
    + +In the second variant, however, Int appears only as second argument, and in the last one it is the third argument. Therefor we must first provide functions where the argumant order is changed so that Int appears in first place: + +
    +    Int rpower(Int const & second, int first, int module)
+    {
+        return power(first, second, third);
+    }
+    Int rrpower(Int const & third, int first, int second)
+    {
+        return power(first, second, third);
+    }
+
    + +These functions must be wrapped under the names "__rpow__" and "__rrpow__" respectively: + +
    +    int_class.def((ternary_function2)&rpower,  "__rpow__");
+    int_class.def((ternary_function2)&rrpower,  "__rrpow__");
+
    + +Note that "__rrpow__" is an extension not present in plain Python. + + +

    Table of Numeric Operators

     + +

    + Py_cpp supports the + Python operators listed in the following table. Note that comparison (__cmp__) and + string conversion (__str__) operators are included + in the list, although they are not strictly "numeric". +

    + + + + + + + + + + + + + + + + + + + + + + + + + + + +
    + Python Operator Name + + Python Expression + + C++ Operator Id + + C++ Expression Used For Automatic Wrapping
    + with cpp_left = from_python(left, Type<Left>()),
    + cpp_right = from_python(right, Type<Right>()),
    + and cpp_oper = from_python(oper, Type<Oper>()) +
    + __add__, __radd__ + + left + right + + py::op_add + + cpp_left + cpp_right +
    + __sub__, __rsub__ + + left - right + + py::op_sub + + cpp_left - cpp_right +
    + __mul__, __rmul__ + + left * right + + py::op_mul + + cpp_left * cpp_right +
    + __div__, __rdiv__ + + left / right + + py::op_div + + cpp_left / cpp_right +
    + __mod__, __rmod__ + + left % right + + py::op_mod + + cpp_left % cpp_right +
    + __divmod__, __rdivmod__ + + (quotient, remainder)
    = divmod(left, right)     +     		+ py::op_divmod + + cpp_left / cpp_right  and +  cpp_left % cpp_right +
    + __pow__, __rpow__ + + pow(left, right)
    + (binary power) +     		+ py::op_pow + + pow(cpp_left, cpp_right) +
    + __pow__ + + pow(left, right, modulo)
    + (ternary power modulo) +     		+ no automatic wrapping, special treatment required +
    + __lshift__, __rlshift__ + + left << right + + py::op_lshift + + cpp_left << cpp_right +
    + __rshift__, __rrshift__ + + left >> right + + py::op_rshift + + cpp_left >> cpp_right +
    + __and__, __rand__ + + left & right + + py::op_and + + cpp_left & cpp_right +
    + __xor__, __rxor__ + + left ^ right + + py::op_xor + + cpp_left ^ cpp_right +
    + __or__, __ror__ + + left | right + + py::op_or + + cpp_left | cpp_right +
    + __cmp__, __rcmp__ + + cmp(left, right) (3-way compare)
    + left < right
    + left <= right
    + left > right
    + left >= right
    + left == right +     		+ py::op_cmp + + cpp_left < cpp_right  and  cpp_right < cpp_left +
    + __neg__ + + -oper  (unary negation) + + py::op_neg + + -cpp_oper +
    + __pos__ + + +oper  (identity) + + py::op_pos + + +cpp_oper +
    + __abs__ + + abs(oper)  (absolute value) + + py::op_abs + + abs(cpp_oper) +
    + __invert__ + + ~oper  (bitwise inversion) + + py::op_invert + + ~cpp_oper +
    + __int__ + + int(oper)  (integer conversion) + + py::op_int + + long(cpp_oper) +
    + __long__ + + long(oper) 
    (infinite precision integer conversion) +     		+ py::op_long + + PyLong_FromLong(cpp_oper) +
    + __float__ + + float(oper)  (float conversion) + + py::op_float + + double(cpp_oper) +
    + __oct__ + + oct(oper)  (octal conversion) + + must be wrapped manually (wrapped function should return a string) +
    + __hex__ + + hex(oper)  (hex conversion) + + must be wrapped manually (wrapped function should return a string) +
    + __str__ + + str(oper)  (string conversion) + + py::op_str + + std::ostringstream s; s << oper; +
    + __coerce__ + + coerce(left, right) + + usually defined automatically, otherwise special + treatment required +
    + + +

    Sequence and Mapping Operators

    + + +Sequence and mapping operators let wrapped objects behave in accordance to Python's iteration and access protocols. These protocols differ considerably from the ones found in C++. For example, Python's typically iteration idiom looks like  "for i in S:" , while in C++ one uses  "for(iterator i = S.begin(); i != S.end(); ++i)". One could try to wrap C++ iterators in order to carry the C++ idiom into Python. However, this does not work very well because (1) it leads to non-uniform Python code (wrapped types must be used in a different way than Python built-in types) and (2) iterators are often implemented as plain C++ pointers which cannot be wrapped easily because py_cpp is designed to handle objects only. +

    +Thus, it is a good idea to provide sequence and mapping operators for your wrapped containers. These operators have to be wrapped manually because there are no corresponding C++ operators that could be used for automatic wrapping. The Python documentation lists the relevant container operators. In particular, expose __getitem__, __setitem__ and remember to throw the PyExc_IndexError when the index is out-of-range in order to enable the  "for i in S:"  idiom. +

    +Here is an example. Suppose you, we want to wrap a std::map<std::size_t,std::string>. This is done as follows as follows: - So, for example, we can wrap a - std::map<std::size_t,std::string> as follows: -

    - Example - 

     typedef std::map<std::size_t, std::string> StringMap;
@@ -207,9 +601,19 @@ Traceback (innermost last):
 KeyError: 2
 >>> len(m)
 0
->>> m[3] = 'farther'
+>>> m[0] = 'zero'
+>>> m[1] = 'one'
+>>> m[2] = 'two'
+>>> m[3] = 'three'
 >>> len(m)
-1
+4
+>>> for i in m:
+...    print i
+...
+zero
+one
+two
+three

    @@ -247,9 +651,9 @@ KeyError: 2 6 -

    +

    Direct Access to Data Members -

    +

    Py_cpp uses the special __xxxattr__<name>__ functionality described above @@ -317,165 +721,8 @@ pair_int_long.def_read_write(&Pil::second, "second"); 8 -

    - Numeric Method Support -

    -

    - Py_cpp supports the following - Python special numeric method names: -

    - - - - - - - - - - - - - - - - - - - - - - - - - -
    - Name - - Notes -
    - __add__(self, other) - - operator+(const T&, const T&) -
    - __sub__(self, other) - - operator-(const T&, const T&) -
    - __mul__(self, other) - - operator*(const T&, const T&) -
    - __div__(self, other) - - operator/(const T&, const T&) -
    - __mod__(self, other) - - operator%(const T&, const T&) -
    - __divmod__(self, other) - - return a py::Tuple initialized with - (quotient, - remainder). -
    - __pow__(self, other [, modulo]) - - use overloading to support both - forms of __pow__ -
    - __lshift__(self, other) - - operator<<(const T&, const T&) -
    - __rshift__(self, other) - - operator>>(const T&, const T&) -
    - __and__(self, other) - - operator&(const T&, const T&) -
    - __xor__(self, other) - - operator^(const T&, const T&) -
    - __or__(self, other) - - operator|(const T&, const T&) -
    - __neg__(self) - - operator-(const T&) (unary negation) -
    - __pos__(self) - - operator+(const T&) (identity) -
    - __abs__(self) - - Called to implement the built-in function abs() -
    - __invert__(self) - - operator~(const T&) -
    - __int__(self) - - operator long() const -
    - __long__(self) - - Should return a Python long object. Can be - implemented with PyLong_FromLong(value), - for example. -
    - __float__(self) - - operator double() const -
    - __oct__(self) - - Called to implement the built-in function oct(). Should return a - string value. -
    - __hex__(self) - - Called to implement the built-in function hex(). Should return a - string value. -
    - __coerce__(self, other) - - Should return a Python 2-tuple (C++ code may return a - py::Tuple) where the elements represent the values - of self and other converted to the - same type. -
    -

    Where are the __r<name>__ -functions?

    - -

    - At first we thought that supporting __radd__ and its ilk would be - impossible, since Python doesn't supply any direct support and in fact - implements a special case for its built-in class instances. This - article gives a pretty good overview of the direct support for numerics - that Python supplies for extension types. We've since discovered that it can - be done, but there are some pretty convincing arguments - out there that this arrangement is less-than-ideal. Instead of supplying a - sub-optimal solution for the sake of compatibility with built-in Python - classes, we're doing the neccessary research so we can "do it right". This - will also give us a little time to hear from users about what they want. The - direction we're headed in is based on the idea of multimethods - rather than on trying to find a coercion function bound to one of the - arguments. - -

    And what about __complex__?

    +

    And what about __complex__?

    That, dear reader, is one problem we don't know how to solve. The Python source contains the following fragment, indicating the special-case code really is hardwired: @@ -490,12 +737,12 @@ if (PyInstance_Check(r)) { ... href="under-the-hood.html">A Peek Under the Hood Up: Top

    - © Copyright David Abrahams 2000. Permission to copy, use, modify, + © Copyright David Abrahams and Ullrich Köthe 2000. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.

    - Updated: Oct 19, 2000 + Updated: Nov 21, 2000