Skip to content
GitLab
  • Menu
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 674
    • Issues 674
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • PyPy
  • pypypypy
  • Issues
  • #3542
Closed
Open
Created Aug 28, 2021 by Antonio Cuni@antocuniOwner

The implementation of W_HPyObject is sub-optimal

The implementation of W_HPyObject is sub-optimal for various reasons:

  1. to allocate an instace (by calling _create_instance) we do two mallocs: one is a GC-managed malloc for W_HPyObject itself, the other is a raw-malloc to hpy_data.

  2. We are using both lightweight and "heavy" finalizers. Ideally, we would like to call tp_destroy from the del, but probably it doesn't work out of the box because RPython actively checks that we don't call anything "weird". However we know that tp_destroy cannot do much, so we should find a workaround to convince RPython to allow it.

More info on how to solve point (1).

Ideally, we would like to allocate the memory for hpy_data together with W_HPyObject, but we think that currently RPython doesn't have the machinery to do that, i.e. to allocate an instance of a given class + some arbitrary extra amount of memory at the end.

The second problem is about moving vs non-moving memory. Ideally we would like this solution:

  1. we allocate W_HPyObject+extra_memory in the nursery
  2. we pin the memory before returning from HPy_New
  3. we unpin the memory when we close the handle.

However, the current pinning strategy of the RPython GC doesn't work well because it might refuse to pin memory after a while. We probably need to fix the GC in order to allow an arbitrary amount of pinned memory.

At the moment of writing, we can see the problems in the test_allocate_obj microbenchmark. Here are the results on CPython and PyPy:

                                                hpy
                                       -------------
CPython: TestType::test_allocate_obj        673.72 us 

PyPY:   TestType::test_allocate_obj        1492.93 us 
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking