Special Method Name Support

Overview

Py_cpp is able to wrap suitable C++ functions and C++ operators into Python operators. It supports all of the standard special method names supported by real Python class instances except __complex__ (more on the reasons below).

Numeric Operators

There are two fundamental ways to define numeric operators within py_cpp: automatic wrapping and manual wrapping. Suppose, C++ defines an addition operator for type Rational, so that we can write:

    Rational a, b, c;
    ...
    c = a + b;

To enable the same functionality in Python, we first wrap the Rational class as usual:

    py::ClassWrapper<Rational> rational_class(my_module, "Rational");
    rational_class.def(py::Constructor<>());
    ...

Then we export the addition operator like this:

    rational_class.def(py::operators<py::op_add>());

Since Rational also supports subtraction, multiplication, adn division, we want to export those also. This can be done in a single command by 'or'ing the operator identifiers together (a complete list of these identifiers and the corresponding operators can be found in the table):

    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>());

Note that the or-expression must be enclosed in parentheses. This form of operator definition will wrap homogeneous operators, that is operators whose left and right operand have the same type. Now, suppose that our C++ library also supports addition of Rationals and integers:

    Rational a, b;
    int i;
    ...
    a = b + i;
    a = i + b;

To wrap these heterogeneous operators (left and right hand side have different types), we need a possibility to specify a different operand type. This is done using the right_operand and left_operand templates:

    rational_class.def(py::operators<py::op_add>(), py::right_operand<int>());
    rational_class.def(py::operators<py::op_add>(), py::left_operand<int>());

Py_cpp uses overloading to register several variants of the same operation (more on this in the context of coercion). Again, several operators can be exported at once:

    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(),
                       py::right_operand<int>());
    rational_class.def(py::operators<(py::op_sub | py::op_mul | py::op_div)>(), 
                       py::left_operand<int>());

The type of the operand not mentioned is taken from the class object. In our example, the class object is rational_class, and thus the other operand's type is `Rational const &'. You can override this default by explicitly specifying a type in the operators template:

    rational_class.def(py::operators<py::op_add, Rational>(), py::right_operand<int>());

Here, `Rational' would be used instead of `Rational const &'.

Note that automatic wrapping doesn't need any specific form of operator+() (or any other operator), but rather wraps the expression `left + right'. That is, this mechanism can be used for any definition of operator+(), such as a free function `Rational operator+(Rational, Rational)' or a member function `Rational Rational::operator+(Rational)'.

In some cases, automatic wrapping of operators is not possible or not desirable. Suppose, for example, that the power operation for Rationals is defined by a set of functions pow():

    Rational pow(Rational const & left, Rational const & right);
    Rational pow(Rational const & left, int right);
    Rational pow(int left, Rational const & right);

In order to create the Python operator "pow" from these functions, we have to wrap them manually:

    rational_class.def((Rational (*)(Rational const &, Rational const &))&pow, "__pow__");
    rational_class.def((Rational (*)(Rational const &, int))&pow, "__pow__");

The third form (with int as left operand) cannot be wrapped this way. We must first create a function rpow() with the operands reversed:

    Rational rpow(Rational const & right, int left)
    {
        return pow(left, right);
    }

This function must be wrapped under the name "__rpow__":

    rational_class.def(&rpow,  "__rpow__");

A list of the possible operator names is also found in the table. Special treatment is necessary to define the ternary pow.

Automatic and manual wrapping can be mixed arbitrarily.

Coercion

So, for example, we can wrap a std::map<std::size_t,std::string> as follows:

Example

typedef std::map<std::size_t, std::string> StringMap;

// A helper function for dealing with errors. Throw a Python exception
// if p == m.end().
void throw_key_error_if_end(
        const StringMap& m, 
        StringMap::const_iterator p, 
        std::size_t key)
{
    if (p == m.end())
    {
        PyErr_SetObject(PyExc_KeyError, py::converters::to_python(key));
        throw py::ErrorAlreadySet();
    }
}

// Define some simple wrapper functions which match the Python  protocol
// for __getitem__, __setitem__, and __delitem__.  Just as in Python, a
// free function with a "self" first parameter makes a fine class method.

const std::string& get_item(const StringMap& self, std::size_t key)
{
    const StringMap::const_iterator p = self.find(key);
    throw_key_error_if_end(self, p, key);
    return p->second;
}

// Sets the item corresponding to key in the map.
void StringMapPythonClass::set_item(StringMap& self, std::size_t key, const std::string& value)
{
    self[key] = value;
}

// Deletes the item corresponding to key from the map.
void StringMapPythonClass::del_item(StringMap& self, std::size_t key)
{
    const StringMap::iterator p = self.find(key);
    throw_key_error_if_end(self, p, key);
    self.erase(p);
}

ClassWrapper<StringMap> string_map(my_module, "StringMap");
string_map.def(py::Constructor<>());
string_map.def(&StringMap::size, "__len__");
string_map.def(get_item, "__getitem__");
string_map.def(set_item, "__setitem__");
string_map.def(del_item, "__delitem__");

Then in Python:

>>> m = StringMap()
>>> m[1]
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 1
>>> m[1] = 'hello'
>>> m[1]
'hello'
>>> del m[1]
>>> m[1]            # prove that it's gone
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 1
>>> del m[2]
Traceback (innermost last):
  File "<stdin>", line 1, in ?
KeyError: 2
>>> len(m)
0
>>> m[3] = 'farther'
>>> len(m)
1

Getters and Setters

Py_cpp extension classes support some additional "special method" protocols not supported by built-in Python classes. Because writing __getattr__, __setattr__, and __delattr__ functions can be tedious in the common case where the attributes being accessed are known statically, py_cpp checks the special names

__getattr__<name>__
__setattr__<name>__
__delattr__<name>__

to provide functional access to the attribute <name>. This facility can be used from C++ or entirely from Python. For example, the following shows how we can implement a "computed attribute" in Python:

>>> class Range(AnyPy_cppExtensionClass):
...    def __init__(self, start, end):
...        self.start = start
...        self.end = end
...    def __getattr__length__(self):
...        return self.end - self.start
...
>>> x = Range(3, 9)
>>> x.length
6

Direct Access to Data Members

Py_cpp uses the special __xxxattr__<name>__ functionality described above to allow direct access to data members through the following special functions on ClassWrapper<> and ExtensionClass<>:

def_getter(pointer-to-member, name) // read access to the member via attribute name
def_setter(pointer-to-member, name) // write access to the member via attribute name
def_readonly(pointer-to-member, name) // read-only access to the member via attribute name
def_read_write(pointer-to-member, name) // read/write access to the member via attribute name

Note that the first two functions, used alone, may produce surprising behavior. For example, when def_getter() is used, the default functionality for setattr() and delattr() remains in effect, operating on items in the extension instance's name-space (i.e., its __dict__). For that reason, you'll usually want to stick with def_readonly and def_read_write.

For example, to expose a std::pair<int,long> we might write:

typedef std::pair<int,long> Pil;
int first(const Pil& x) { return x.first; }
long second(const Pil& x) { return x.second; }
   ...
my_module.def(first, "first");
my_module.def(second, "second");

ClassWrapper<Pil> pair_int_long(my_module, "Pair");
pair_int_long.def(py::Constructor<>());
pair_int_long.def(py::Constructor<int,long>());
pair_int_long.def_read_write(&Pil::first, "first");
pair_int_long.def_read_write(&Pil::second, "second");

Now your Python class has attributes first and second which, when accessed, actually modify or reflect the values of corresponding data members of the underlying C++ object. Now in Python:

>>> x = Pair(3,5)
>>> x.first
3
>>> x.second
5
>>> x.second = 8
>>> x.second
8
>>> second(x) # Prove that we're not just changing the instance __dict__
8

Numeric Method Support

Py_cpp supports the following Python special numeric method names:

Name	Notes
`__add__(self, other)`	`operator+(const T&, const T&)`
`__sub__(self, other)`	`operator-(const T&, const T&)`
`__mul__(self, other)`	`operator*(const T&, const T&)`
`__div__(self, other)`	`operator/(const T&, const T&)`
`__mod__(self, other)`	`operator%(const T&, const T&)`
`__divmod__(self, other)`	return a `py::Tuple` initialized with `(`quotient`,` remainder`)`.
`__pow__(self, other [, modulo])`	use overloading to support both forms of __pow__
`__lshift__(self, other)`	`operator<<(const T&, const T&)`
`__rshift__(self, other)`	`operator>>(const T&, const T&)`
`__and__(self, other)`	`operator&(const T&, const T&)`
`__xor__(self, other)`	`operator^(const T&, const T&)`
`__or__(self, other)`	`operator\|(const T&, const T&)`
`__neg__(self)`	`operator-(const T&)` (unary negation)
`__pos__(self)`	`operator+(const T&)` (identity)
`__abs__(self)`	Called to implement the built-in function abs()
`__invert__(self)`	`operator~(const T&)`
`__int__(self)`	`operator long() const`
`__long__(self)`	Should return a Python `long` object. Can be implemented with `PyLong_FromLong(value)`, for example.
`__float__(self)`	`operator double() const`
`__oct__(self)`	Called to implement the built-in function oct(). Should return a string value.
`__hex__(self)`	Called to implement the built-in function hex(). Should return a string value.
`__coerce__(self, other)`	Should return a Python 2-tuple (C++ code may return a `py::Tuple`) where the elements represent the values of `self` and `other` converted to the same type.

Where are the `r`<name>`` functions?

At first we thought that supporting __radd__ and its ilk would be impossible, since Python doesn't supply any direct support and in fact implements a special case for its built-in class instances. This article gives a pretty good overview of the direct support for numerics that Python supplies for extension types. We've since discovered that it can be done, but there are some pretty convincing arguments out there that this arrangement is less-than-ideal. Instead of supplying a sub-optimal solution for the sake of compatibility with built-in Python classes, we're doing the neccessary research so we can "do it right". This will also give us a little time to hear from users about what they want. The direction we're headed in is based on the idea of multimethods rather than on trying to find a coercion function bound to one of the arguments.

And what about `complex`?

That, dear reader, is one problem we don't know how to solve. The Python source contains the following fragment, indicating the special-case code really is hardwired:

/* XXX Hack to support classes with __complex__ method */
if (PyInstance_Check(r)) { ...

Previous: Inheritance Next: A Peek Under the Hood Up: Top

© Copyright David Abrahams 2000. Permission to copy, use, modify, sell and distribute this document is granted provided this copyright notice appears in all copies. This document is provided "as is" without express or implied warranty, and with no claim as to its suitability for any purpose.

Updated: Oct 19, 2000