Extension Types

Cython 0.19.1

previous page next page

Extension Types

Normal Python as well as extension type classes can be defined.
Extension types:

Are considered by Python as “built-in” types.

Can be used to wrap arbitrary C-data structures, and provide a Python-like interface to them from Python.

Attributes and methods can be called from Python or Cython code

Are defined by the cdef class statement.

cdef class Shrubbery:

    cdef int width, height

    def __init__(self, w, h):
        self.width = w
        self.height = h

    def describe(self):
        print "This shrubbery is", self.width, \
            "by", self.height, "cubits."

Attributes

Are stored directly in the object’s C struct.
Are fixed at compile time.

You can’t add attributes to an extension type instance at run time like in normal Python.

You can sub-class the extenstion type in Python to add attributes at run-time.

There are two ways to access extension type attributes:

By Python look-up.

Python code’s only method of access.

By direct access to the C struct from Cython code.

Cython code can use either method of access, though.

By default, extension type attributes are:

Only accessible by direct access.

Not accessible from Python code.

To make attributes accessible to Python, they must be declared public or readonly:

cdef class Shrubbery:
    cdef public int width, height
    cdef readonly float depth

The width and height attributes are readable and writable from Python code.

The depth attribute is readable but not writable.

Note

You can only expose simple C types, such as ints, floats, and strings, for Python access. You can also expose Python-valued attributes.

Note

The public and readonly options apply only to Python access, not direct access. All the attributes of an extension type are always readable and writable by C-level access.

Methods

self is used in extension type methods just like it normally is in Python.
See Functions and Methods; all of which applies here.

Properties

Cython provides a special syntax:

cdef class Spam:

    property cheese:

        "A doc string can go here."

        def __get__(self):
            # This is called when the property is read.
            ...

        def __set__(self, value):
            # This is called when the property is written.
            ...

        def __del__(self):
            # This is called when the property is deleted.

The __get__(), __set__(), and __del__() methods are all optional.

If they are ommitted, An exception is raised when an access attempt is made.

Below, is a full example that defines a property which can..

Add to a list each time it is written to ("__set__").

Return the list when it is read ("__get__").

Empty the list when it is deleted ("__del__").

# cheesy.pyx
cdef class CheeseShop:

    cdef object cheeses

    def __cinit__(self):
        self.cheeses = []

    property cheese:

        def __get__(self):
            return "We don't have: %s" % self.cheeses

        def __set__(self, value):
            self.cheeses.append(value)

        def __del__(self):
            del self.cheeses[:]

# Test input
from cheesy import CheeseShop

shop = CheeseShop()
print shop.cheese

shop.cheese = "camembert"
print shop.cheese

shop.cheese = "cheddar"
print shop.cheese

del shop.cheese
print shop.cheese

# Test output
We don't have: []
We don't have: ['camembert']
We don't have: ['camembert', 'cheddar']
We don't have: []

Special Methods

Note

The semantics of Cython’s special methods are similar in principle to that of Python’s.
There are substantial differences in some behavior.
Some Cython special methods have no Python counter-part.

See the Special Methods Table for the many that are available.

Declaration

Must be declared with def and cannot be declared with cdef.
Performance is not affected by the def declaration because of special calling conventions

Docstrings

Docstrings are not supported yet for some special method types.
They can be included in the source, but may not appear in the corresponding __doc__ attribute at run-time.

This a Python library limitation because the PyTypeObject data structure is limited

Initialization: `cinit()` and `init()`

Any arguments passed to the extension type’s constructor, will be passed to both initialization methods.
__cinit__() is where you should perform C-level initialization of the object

This includes any allocation of C data structures.

Caution is warranted as to what you do in this method.

The object may not be fully valid Python object when it is called.

Calling Python objects, including the extensions own methods, may be hazardous.

By the time __cinit__() is called...

Memory has been allocated for the object.

All C-level attributes have been initialized to 0 or null.

Python have been initialized to None, but you can not rely on that for each occasion.

This initialization method is guaranteed to be called exactly once.

For Extensions types that inherit a base type:

The __cinit__() method of the base type is automatically called before this one.

The inherited __cinit__() method can not be called explicitly.

Passing modified argument lists to the base type must be done through __init__().

It may be wise to give the __cinit__() method both "*" and "**" arguments.

Allows the method to accept or ignore additional arguments.

Eliminates the need for a Python level sub-class, that changes the __init__() method’s signature, to have to override both the __new__() and __init__() methods.

If __cinit__() is declared to take no arguments except self, it will ignore any extra arguments passed to the constructor without complaining about a signature mis-match

__init__() is for higher-level initialization and is safer for Python access.

By the time this method is called, the extension type is a fully valid Python object.

All operations are safe.

This method may sometimes be called more than once, or possibly not at all.

Take this into consideration to make sure the design of your other methods are robust of this fact.

Finalization: `dealloc()`

This method is the counter-part to __cinit__().
Any C-data that was explicitly allocated in the __cinit__() method should be freed here.
Use caution in this method:

The Python object to which this method belongs may not be completely intact at this point.

Avoid invoking any Python operations that may touch the object.

Don’t call any of this object’s methods.

It’s best to just deallocate C-data structures here.

All Python attributes of your extension type object are deallocated by Cython after the __dealloc__() method returns.

Arithmetic Methods

Note

Most of these methods behave differently than in Python

There are not “reversed” versions of these methods... there is no __radd__() for instance.
If the first operand cannot perform the operation, the same method of the second operand is called, with the operands in the same order.
Do not rely on the first parameter of these methods, being "self" or the right type.
The types of both operands should be tested before deciding what to do.
Return NotImplemented for unhandled, mis-matched operand types.
The previously mentioned points..

Also apply to ‘in-place’ method __ipow__().

Do not apply to other ‘in-place’ methods like __iadd__(), in that these always take self as the first argument.

Rich Comparisons

Note

There are no separate methods for individual rich comparison operations.

A single special method called __richcmp__() replaces all the individual rich compare, special method types.
__richcmp__() takes an integer argument, indicating which operation is to be performed as shown in the table below.

< 0

== 2

> 4

<= 1

!= 3

>= 5

The `next()` Method

Extension types used to expose an iterator interface should define a __next__() method.
Do not explicitly supply a next() method, because Python does that for you automatically.

Subclassing

An extension type may inherit from a built-in type or another extension type:

cdef class Parrot:
    ...

cdef class Norwegian(Parrot):
    ...

A complete definition of the base type must be available to Cython

If the base type is a built-in type, it must have been previously declared as an extern extension type.

cimport can be used to import the base type, if the extern declared base type is in a .pxd definition file.

In Cython, multiple inheritance is not permitted.. singlular inheritance only

Cython extenstion types can also be sub-classed in Python.

Here multiple inhertance is permissible as is normal for Python.

Even multiple extension types may be inherited, but C-layout of all the base classes must be compatible.

Forward Declarations

Extension types can be “forward-declared”.

This is necessary when two extension types refer to each other:

cdef class Shrubbery # forward declaration

cdef class Shrubber:
    cdef Shrubbery work_in_progress

cdef class Shrubbery:
    cdef Shrubber creator

An extension type that has a base-class, requires that both forward-declarations be specified:
```
cdef class A(B)

...

cdef class A(B):
    # attributes and methods
```

Extension Types and None

Parameters and C-variables declared as an Extension type, may take the value of None.
This is analogous to the way a C-pointer can take the value of NULL.

Note

Exercise caution when using None
Read this section carefully.

There is no problem as long as you are performing Python operations on it.

This is because full dynamic type checking is applied

When accessing an extension type’s C-attributes, make sure it is not None.

Cython does not check this for reasons of efficency.

Be very aware of exposing Python functions that take extension types as arguments:

def widen_shrubbery(Shrubbery sh, extra_width): # This is
sh.width = sh.width + extra_width

* Users could **crash** the program by passing ``None`` for the ``sh`` parameter.
* This could be avoided by::

    def widen_shrubbery(Shrubbery sh, extra_width):
        if sh is None:
            raise TypeError
        sh.width = sh.width + extra_width

* Cython provides a more convenient way with a ``not None`` clause::

    def widen_shrubbery(Shrubbery sh not None, extra_width):
        sh.width = sh.width + extra_width

* Now this function automatically checks that ``sh`` is not ``None``, as well as that is the right type.

not None can only be used in Python functions (declared with def not cdef).
For cdef functions, you will have to provide the check yourself.
The self parameter of an extension type is guaranteed to never be None.
When comparing a value x with None, and x is a Python object, note the following:

x is None and x is not None are very efficient.

They translate directly to C-pointer comparisons.

x == None and x != None or if x: ... (a boolean condition), will invoke Python operations and will therefore be much slower.

Weak Referencing

By default, weak references are not supported.

It can be enabled by declaring a C attribute of the object type called __weakref__():

cdef class ExplodingAnimal:
    """This animal will self-destruct when it is
    no longer strongly referenced."""

    cdef object __weakref__

External and Public Types

Public

When an extention type is declared public, Cython will generate a C-header (”.h”) file.
The header file will contain the declarations for it’s object-struct and it’s type-object.
External C-code can now access the attributes of the extension type.

External

An extern extension type allows you to gain access to the internals of:

Python objects defined in the Python core.

Non-Cython extension modules

The following example lets you get at the C-level members of Python’s built-in “complex” object:

cdef extern from "complexobject.h":

    struct Py_complex:
        double real
        double imag

    ctypedef class __builtin__.complex [object PyComplexObject]:
        cdef Py_complex cval

# A function which uses the above type
def spam(complex c):
    print "Real:", c.cval.real
    print "Imag:", c.cval.imag

Note

Some important things in the example: #. ctypedef has been used because because Python’s header file has the struct decalared with:

ctypedef struct {
...
} PyComplexObject;

The module of where this type object can be found is specified along side the name of the extension type. See Implicit Importing.
When declaring an external extension type...

Don’t declare any methods, because they are Python method class the are not needed.

Similiar to structs and unions, extension classes declared inside a cdef extern from block only need to declare the C members which you will actually need to access in your module.

Name Specification Clause

Note

Only available to public and extern extension types.

Example:

[object object_struct_name, type type_object_name ]

object_struct_name is the name to assume for the type’s C-struct.
type_object_name is the name to assume for the type’s statically declared type-object.
The object and type clauses can be written in any order.
For cdef extern from declarations, This clause is required.

The object clause is required because Cython must generate code that is compatible with the declarations in the header file.

Otherwise the object clause is optional.

For public extension types, both the object and type clauses are required for Cython to generate code that is compatible with external C-code.

Type Names vs. Constructor Names

In a Cython module, the name of an extension type serves two distinct purposes:

When used in an expression, it refers to a “module-level” global variable holding the type’s constructor (i.e. it’s type-object)

It can also be used as a C-type name to declare a “type” for variables, arguments, and return values.

Example:

cdef extern class MyModule.Spam:
    ...

The name “Spam” serves both of these roles.

Only “Spam” can be used as the type-name.

The constructor can be referred to by other names.

Upon an explicit import of “MyModule”...

MyModule.Spam() could be used as the constructor call.

MyModule.Spam could not be used as a type-name

When an “as” clause is used, the name specified takes over both roles:
```
cdef extern class MyModule.Spam as Yummy:
    ...
```

Yummy becomes both type-name and a name for the constructor.

There other ways of course, to get hold of the constructor, but Yummy is the only usable type-name.

previous page start next page

Extension Types

Cython 0.19.1

Navigation

Extension Types

Attributes

Methods

Properties

Special Methods

Declaration

Docstrings

Initialization: __cinit__() and __init__()

Finalization: __dealloc__()

Arithmetic Methods

Rich Comparisons

The __next__() Method

Subclassing

Forward Declarations

Extension Types and None

Weak Referencing

External and Public Types

Public

External

Name Specification Clause

Type Names vs. Constructor Names

Navigation

Initialization: `cinit()` and `init()`

Finalization: `dealloc()`

The `next()` Method