Created originally on Bitbucket by shmuller (Stefan Muller)
Changesets for this Pull Request have not been imported, because it had been already declined on Bitbucket. Marked as closed by the import user.
This pull request adds support for the most important PyArray_* routines of the Numpy C API in the cpyext module, interfacing with the PyPy numpypy module. This extension to cpyext allows one to build many C extensions that invoke the Numpy C API (#include <numpy/arrayobject.h>), in a straightforward way with distutils, using "pypy-c setup.py install".
The widespread adoption of the Numpy C API in scientific modules and wrapper code comes from 3 factors:
- It is the officially supported API of Numpy, with the associated guarantees of long-term stability.
- It is well thought out and very similar to the API's of other important scientific array libraries/programs such as HDF5 and MATLAB, which facilitated the translation of already existing wrapper code.
- It is the fastest way to interface C code with Numpy for CPython, adding almost no interface overhead (other solutions are hampered by the inefficiency to obtain a pointer to the memory of a numpy array on the Python level, which requires dict and tuple lookups:
ptr = arr.__array_interface__['data'][0])
Also, automatic interface generators such as Cython generate code for the Numpy C API.
Previously, such extensions could not be used with PyPy, which was a major blocking factor for the adoption of PyPy in scientific computing: Rewriting C extensions in cffi or ctypes is practical only in the simplest of cases, and many extensions wrap huge libraries such as h5py (HDF5), matplotlib and VTK. These are usually not performance critical, but essential for the scientific workflow.
This implementation of the Numpy C API adds a new file cpyext/ndarrayobject.py and a corresponding test suite in cpyext/test/test_ndarrayobject.py. Also a header file cpyext/include/arrayobject.h is added. In order not to preclude the usage of another Numpy implementation with PyPy, should one become available in the future, the API functions are name-mangled as _PyArray_*
in ndarrayobject.py and mapped to the official PyArray_*
names in arrayobject.h.
Two other changes are made to PyPy to integrate this new functionality:
-
The behavior of lib_pypy/numpy.py is altered from "raise ImportError()" to a warnings.warn(). The previous behavior required user code that would otherwise work to be modified to catch the ImportError, which was simply annoying. This change of behavior was agreed to in a discussion on IRC.
-
cpyext now provides aliases for PyBoolObject and PyComplexObject, which were previously undefined symbols. In many cases, code generators like Cython simply generate type casts between PyObject and these types, without actually accessing any structure fields. Such code works now.