Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 676
    • Issues 676
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 13
    • Merge requests 13
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

April 05-07 - Mercurial Paris Conference 2023 - One day of workshop dedicated to Heptapod and Mercurial usage and workflow. There are still places available for both workshops and talks!

  • PyPyPyPy
  • pypypypy
  • Merge requests
  • !358

Fix the cpyext tests on OS X by linking with -flat_namespace.

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Bitbucket Importer requested to merge bitbucket/merged-pr-358 into branch/default Nov 19, 2015
  • Overview 1
  • Commits 1
  • Pipelines 0
  • Changes 1

Created originally on Bitbucket by jamadden (Jason Madden)

Was already merged in Bitbucket before import, marked as merged by the import user

This adds -flat_namespace to the linker args provided in rpython.translator.platform.darwin:Darwin._args_for_shared. Without this argument, many of the cpyext tests fail with runtime link errors like this one:

    def test_pass_ndarray_object_to_c(self):
            from _numpypy.multiarray import ndarray
            mod = self.import_extension('foo', [
                    ("check_array", "METH_VARARGS",
                    '''
                        PyObject* obj;
                        if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &obj))
                            return NULL;
                        Py_INCREF(obj);
                        return obj;
                    '''),
>                   ], prologue='#include <numpy/arrayobject.h>')
E    (application-level) ImportError: unable to load extension module '/var/.../T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so': dlopen(/var//...T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so, 6): Symbol not found: _cpyexttestArray_Type
E   Referenced from: /var/.../T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so
E   Expected in: /var/.../T/usession-osx-flat-namespace-1/shared_cache/externmod.dylib

On OS X, the linker by default uses a two-level namespace, in which all compile-time references resolved to a dynamic library record the library to which they were resolved. At runtime, that symbol is only looked for in that specific dynamic library, not any of the other libraries loaded by the process. This is faster since the search scope is limited, and also prevents name collisions. However, this can be a compatibility issue when porting code from other systems (it was really bad in the early days of OS X, IIRC).

In this case, the missing symbols are actually in pypyapi.dylib, which is dynamically loaded into the running interpreter (the one that dlopens foo), but that library is not dynamically linked as a direct on indirect library to either externod.dylib or foo.pypy-26i.dylib so it's not in the list of libraries to search.

The solution here is the use of the -flat_namespace linker argument. This omits the library name from the resolved symbol and thus forces the runtime to search all loaded libraries for the symbol. (This is the way most *ix linkers work.)

An alternate solution, I think, would be to explicitly link to pypyapi.dylib, but I quit trying to make that happen after I figured this out. Also, I have no idea why the linker recorded that externmod.dylib has the PyPy API symbols; again, I quit digging once I remembered this flag.

Note: The OS X buildbots do not currently run the cpyext tests.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: bitbucket/merged-pr-358