Discrepancy with CPython on handling dynamic/heap types' `tp_name`
It seems that PyPy's behavior diverges from CPython on handling the tp_name
for so-called heap types. For example, extending the standard example from the CPython docs to also dynamically allocate and create a heap type:
#define PY_SSIZE_T_CLEAN
#include <Python.h>
typedef struct {
PyObject_HEAD
/* Type-specific fields go here. */
} CustomObject;
static PyTypeObject CustomType = {
PyVarObject_HEAD_INIT(NULL, 0)
.tp_name = "custom.Custom",
.tp_doc = "Custom objects",
.tp_basicsize = sizeof(CustomObject),
.tp_itemsize = 0,
.tp_flags = Py_TPFLAGS_DEFAULT,
.tp_new = PyType_GenericNew,
};
static PyModuleDef custommodule = {
PyModuleDef_HEAD_INIT,
.m_name = "custom",
.m_doc = "Example module that creates an extension type.",
.m_size = -1,
};
PyMODINIT_FUNC
PyInit_custom(void)
{
PyObject *m;
if (PyType_Ready(&CustomType) < 0)
return NULL;
m = PyModule_Create(&custommodule);
if (m == NULL)
return NULL;
Py_INCREF(&CustomType);
if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) {
Py_DECREF(&CustomType);
Py_DECREF(m);
return NULL;
}
PyType_Slot CustomHeap_Type_slots[] = {
{0, 0},
};
PyType_Spec CustomHeap_Type_spec = {
"custom.CustomHeap",
sizeof(CustomObject),
0,
Py_TPFLAGS_DEFAULT,
&CustomHeap_Type_slots
};
PyObject *CustomHeapType = PyType_FromSpec(&CustomHeap_Type_spec);
PyModule_AddObject(m, "CustomHeap", (PyObject *) CustomHeapType);
return m;
}
In PyPy3, this results in
$ pypy3
Python 3.6.9 (bef50b0f3fe7, Sep 15 2020, 19:53:56)
[PyPy 7.3.2 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import custom
>>>> custom.Custom
<class 'custom.Custom'>
>>>> custom.Custom.__name__
'Custom'
>>>> custom.CustomHeap
<class '_frozen_importlib.custom.CustomHeap'>
>>>> custom.CustomHeap.__name__
'custom.CustomHeap'
While CPython 3 handles both types in the same way:
$ python3
Python 3.6.9 (default, Jul 17 2020, 12:50:27)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import custom
>>> custom.Custom
<class 'custom.Custom'>
>>> custom.Custom.__name__
'Custom'
>>> custom.CustomHeap
<class 'custom.CustomHeap'>
>>> custom.CustomHeap.__name__
'CustomHeap'
Admittedly, the Python docs aren't exactly clear on what should happen (https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_name), and seem almost contradictory:
For types that are accessible as module globals, the string should be the full module name, followed by a dot, followed by the type name
vs.
For dynamically allocated type objects, this should just be the type name, and the module name explicitly stored in the type dict as the value for key 'module'.
But it does seem as if there's an important difference in practice between CPython and PyPy.
pybind11 has had a workaround for this for a long time, but after trying to increase the consistency between CPython and PyPy, we get some other inconsistencies where e.g. list
becomes builtin.list
in examples.
I'm happy to try contributing a solution, if someone can confirm this to be an issue and if you give me a few pointers on where to start looking! :-)