Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 674
    • Issues 674
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • PyPy
  • pypypypy
  • Issues
  • #3318
Closed
Open
Created Oct 04, 2020 by Yannick Jadoul@YannickJadoulMaintainer

Discrepancy with CPython on handling dynamic/heap types' `tp_name`

It seems that PyPy's behavior diverges from CPython on handling the tp_name for so-called heap types. For example, extending the standard example from the CPython docs to also dynamically allocate and create a heap type:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

typedef struct {
    PyObject_HEAD
    /* Type-specific fields go here. */
} CustomObject;

static PyTypeObject CustomType = {
    PyVarObject_HEAD_INIT(NULL, 0)
    .tp_name = "custom.Custom",
    .tp_doc = "Custom objects",
    .tp_basicsize = sizeof(CustomObject),
    .tp_itemsize = 0,
    .tp_flags = Py_TPFLAGS_DEFAULT,
    .tp_new = PyType_GenericNew,
};

static PyModuleDef custommodule = {
    PyModuleDef_HEAD_INIT,
    .m_name = "custom",
    .m_doc = "Example module that creates an extension type.",
    .m_size = -1,
};

PyMODINIT_FUNC
PyInit_custom(void)
{
    PyObject *m;
    if (PyType_Ready(&CustomType) < 0)
        return NULL;

    m = PyModule_Create(&custommodule);
    if (m == NULL)
        return NULL;

    Py_INCREF(&CustomType);
    if (PyModule_AddObject(m, "Custom", (PyObject *) &CustomType) < 0) {
        Py_DECREF(&CustomType);
        Py_DECREF(m);
        return NULL;
    }


    PyType_Slot CustomHeap_Type_slots[] = {
        {0, 0},
    };

    PyType_Spec CustomHeap_Type_spec = {
        "custom.CustomHeap",
        sizeof(CustomObject),
        0,
        Py_TPFLAGS_DEFAULT,
        &CustomHeap_Type_slots
    };

    PyObject *CustomHeapType = PyType_FromSpec(&CustomHeap_Type_spec);
    PyModule_AddObject(m, "CustomHeap", (PyObject *) CustomHeapType);

    return m;
}

In PyPy3, this results in

$ pypy3
Python 3.6.9 (bef50b0f3fe7, Sep 15 2020, 19:53:56)
[PyPy 7.3.2 with GCC 7.3.1 20180303 (Red Hat 7.3.1-5)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>> import custom
>>>> custom.Custom
<class 'custom.Custom'>
>>>> custom.Custom.__name__
'Custom'
>>>> custom.CustomHeap
<class '_frozen_importlib.custom.CustomHeap'>
>>>> custom.CustomHeap.__name__
'custom.CustomHeap'

While CPython 3 handles both types in the same way:

$ python3
Python 3.6.9 (default, Jul 17 2020, 12:50:27) 
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import custom
>>> custom.Custom
<class 'custom.Custom'>
>>> custom.Custom.__name__
'Custom'
>>> custom.CustomHeap
<class 'custom.CustomHeap'>
>>> custom.CustomHeap.__name__
'CustomHeap'

Admittedly, the Python docs aren't exactly clear on what should happen (https://docs.python.org/3/c-api/typeobj.html#c.PyTypeObject.tp_name), and seem almost contradictory:

For types that are accessible as module globals, the string should be the full module name, followed by a dot, followed by the type name

vs.

For dynamically allocated type objects, this should just be the type name, and the module name explicitly stored in the type dict as the value for key 'module'.

But it does seem as if there's an important difference in practice between CPython and PyPy.

pybind11 has had a workaround for this for a long time, but after trying to increase the consistency between CPython and PyPy, we get some other inconsistencies where e.g. list becomes builtin.list in examples.

I'm happy to try contributing a solution, if someone can confirm this to be an issue and if you give me a few pointers on where to start looking! :-)

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking