Created originally on Bitbucket by jamadden (Jason Madden)
Was already merged in Bitbucket before import, marked as merged by the import user
This adds -flat_namespace
to the linker args provided in rpython.translator.platform.darwin:Darwin._args_for_shared
. Without this argument, many of the cpyext tests fail with runtime link errors like this one:
def test_pass_ndarray_object_to_c(self):
from _numpypy.multiarray import ndarray
mod = self.import_extension('foo', [
("check_array", "METH_VARARGS",
'''
PyObject* obj;
if (!PyArg_ParseTuple(args, "O!", &PyArray_Type, &obj))
return NULL;
Py_INCREF(obj);
return obj;
'''),
> ], prologue='#include <numpy/arrayobject.h>')
E (application-level) ImportError: unable to load extension module '/var/.../T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so': dlopen(/var//...T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so, 6): Symbol not found: _cpyexttestArray_Type
E Referenced from: /var/.../T/usession-osx-flat-namespace-1/module_2/foo.pypy-26i.so
E Expected in: /var/.../T/usession-osx-flat-namespace-1/shared_cache/externmod.dylib
On OS X, the linker by default uses a two-level namespace, in which all compile-time references resolved to a dynamic library record the library to which they were resolved. At runtime, that symbol is only looked for in that specific dynamic library, not any of the other libraries loaded by the process. This is faster since the search scope is limited, and also prevents name collisions. However, this can be a compatibility issue when porting code from other systems (it was really bad in the early days of OS X, IIRC).
In this case, the missing symbols are actually in pypyapi.dylib
, which is dynamically loaded into the running interpreter (the one that dlopens
foo
), but that library is not dynamically linked as a direct on indirect library to either externod.dylib
or foo.pypy-26i.dylib
so it's not in the list of libraries to search.
The solution here is the use of the -flat_namespace
linker argument. This omits the library name from the resolved symbol and thus forces the runtime to search all loaded libraries for the symbol. (This is the way most *ix linkers work.)
An alternate solution, I think, would be to explicitly link to pypyapi.dylib
, but I quit trying to make that happen after I figured this out. Also, I have no idea why the linker recorded that externmod.dylib
has the PyPy API symbols; again, I quit digging once I remembered this flag.
Note: The OS X buildbots do not currently run the cpyext tests.