Unable to install libraries with c modules with custom rpath on certain OS (e.g. Arch)
Created originally on Bitbucket by Yusuke Tsutsumi
Steps To Reproduce
- create a virtualenv for pypy (bin/virtualenv-pypy)
$ pip install pyzmq --global-option="build_ext" --global-option="--rpath=./Lib"
The output will probably look like:
#!python creating build/lib.linux-x86_64-3.5/zmq/backend/cffi cc -pthread -shared -Wunused-parameter -L/home/tsutsumi/Downloads/pypy3/env/lib -Wl,-rpath,/home/tsutsumi/Downloads/pypy3/env/lib build/temp.linux-x86_64-3.5/zmq/backend/cffi/__pycache__/_cffi_ext.o -R./Lib -lzmq -o build/lib.linux-x86_64-3.5/zmq/backend/cffi/_cffi_ext.pypy3-510-x86_64-linux-gnu.so cc: error: unrecognized command line option ‘-R’ error: command 'cc' failed with exit status 1 ---------------------------------------- Command "/home/tsutsumi/Downloads/pypy3/env/bin/pypy -u -c "import setuptools, tokenize;__file__='/tmp/pip-install-ncfdiwxd/pyzmq/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" build_ext --rpath=./Lib install --record /tmp/pip-record-ky03ulyu/install-record.txt --single-version-externally-managed --compile --install-headers /home/tsutsumi/Downloads/pypy3/env/include/site/python3.5/pyzmq" failed with error code 1 in /tmp/pip-install-ncfdiwxd/pyzmq/
Investigation / Details
The --rpath parameter passed into the setup.py specifies additional parameters to be passed into GCC. Those parameters are chosen by the unixcompiler.py module (which seems to be the same for both pypy and cpython):
Which is choosing the erroneous flags -R in lieu of -Wl for gcc.
It looks like this isn't encountered during cpython, as a result of gcc being noted as the compiler, and that forces the correct values:
Running the sysconfig.get_config_var for CC gives nothing on pypy3:
$ pypy3 -c "import sysconfig; print(sysconfig.get_config_var('CC'))" None
Digging a bit deeper, it looks like these variables are autopopulated by consuming the Makefile used to compile Python. In pypy3, this file does not exist.
The function responsible for populating the CC config var:
The function reading the makefile, and creating the subsequent file that is consumed to construct the config var needed to resolve to the right cc variable:
It seems like the solution could be providing a faux-makefile, or potentially patching or creating the final build file from which the variables are consumed.
I'm happy to do some work in the area and send a PR, just not sure what the best direction would be to resolve this.