Libraries like Pikepdf fail to build because of Py_EnterRecursiveCall
https://github.com/pikepdf/pikepdf/blob/master/src/qpdf/pikepdf.h#L177
Because of lines like the above, there is a problem while building these packages in pypy.
The error is
In file included from src/qpdf/annotation.cpp:21:
src/qpdf/pikepdf.h: In constructor ‘StackGuard::StackGuard(const char*)’:
src/qpdf/pikepdf.h:177:31: error: invalid conversion from ‘const char*’ to ‘char*’ [-fpermissive]
177 | Py_EnterRecursiveCall(where);
| ^~~~~
| |
| const char*
In file included from /usr/local/include/Python.h:144,
from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/detail/common.h:112,
from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/pytypes.h:12,
from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/cast.h:13,
from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/attr.h:13,
from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/pybind11.h:44,
from src/qpdf/annotation.cpp:18:
/usr/local/include/pypy_decl.h:962:45: note: initializing argument 1 of ‘int PyPy_EnterRecursiveCall(char*)’
962 | PyAPI_FUNC(int) Py_EnterRecursiveCall(char *arg0);
| ~~~~~~^~~~
error: command 'gcc' failed with exit status 1
I'm pasting a Dockerfile which will reproduce the issue for you (takes 45 minutes to build)
FROM buildpack-deps:sid
# ensure local pypy is preferred over distribution pypy
ENV PATH /usr/local/bin:$PATH
# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8
# runtime dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
tcl \
tk \
&& rm -rf /var/lib/apt/lists/*
ENV PYPY_VERSION 7.3.1
RUN set -ex; \
\
# this "case" statement is generated via "update.sh"
dpkgArch="$(dpkg --print-architecture)"; \
case "${dpkgArch##*-}" in \
# amd64
amd64) pypyArch='linux64'; sha256='f67cf1664a336a3e939b58b3cabfe47d893356bdc01f2e17bc912aaa6605db12' ;; \
# arm64v8
arm64) pypyArch='aarch64'; sha256='b900241bca7152254c107a632767f49edede99ca6360b9a064141267b47ef598' ;; \
# i386
i386) pypyArch='linux32'; sha256='7045b295d38ba0b5ee65bd3f078ca249fcf1de73fedeaab2d6ad78de2eab0f0e' ;; \
# ppc64le
ppc64el) pypyArch='ppc64le'; sha256='d6f3b701313df69483b43ebdd21b9652ae5e808b2eea5fbffe3b74b82d2e7433' ;; \
# s390x
s390x) pypyArch='s390x'; sha256='0fe2f7bbf42ea88b40954d7de773a43179a44f40656f2f58201524be70699544' ;; \
*) echo >&2 "error: current architecture ($dpkgArch) does not have a corresponding PyPy $PYPY_VERSION binary release"; exit 1 ;; \
esac; \
\
savedAptMark="$(apt-mark showmanual)"; \
apt-get update; \
apt-get install -y --no-install-recommends \
# sometimes "pypy3" itself is linked against libexpat1 / libncurses5, sometimes they're ".so" files in "/usr/local/lib_pypy"
libexpat1 \
libncurses5 \
# (so we'll add them temporarily, then use "ldd" later to determine which to keep based on usage per architecture)
; \
\
wget -O pypy.tar.bz2 "https://bitbucket.org/pypy/pypy/downloads/pypy3.6-v${PYPY_VERSION}-${pypyArch}.tar.bz2" --progress=dot:giga; \
echo "$sha256 *pypy.tar.bz2" | sha256sum -c; \
tar -xjC /usr/local --strip-components=1 -f pypy.tar.bz2; \
find /usr/local/lib-python -depth -type d -a \( -name test -o -name tests \) -exec rm -rf '{}' +; \
rm pypy.tar.bz2; \
\
# smoke test
pypy3 --version; \
\
if [ -f /usr/local/lib_pypy/_ssl_build.py ]; then \
# on pypy3, rebuild ffi bits for compatibility with Debian Stretch+ (https://github.com/docker-library/pypy/issues/24#issuecomment-409408657)
cd /usr/local/lib_pypy; \
pypy3 _ssl_build.py; \
# TODO rebuild other cffi modules here too? (other _*_build.py files)
fi; \
\
apt-mark auto '.*' > /dev/null; \
[ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; \
find /usr/local -type f -executable -exec ldd '{}' ';' \
| awk '/=>/ { print $(NF-1) }' \
| sort -u \
| xargs -r dpkg-query --search \
| cut -d: -f1 \
| sort -u \
| xargs -r apt-mark manual \
; \
apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
rm -rf /var/lib/apt/lists/*; \
# smoke test again, to be sure
pypy3 --version; \
\
find /usr/local -depth \
\( \
\( -type d -a \( -name test -o -name tests \) \) \
-o \
\( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
\) -exec rm -rf '{}' +
# if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'"
ENV PYTHON_PIP_VERSION 20.0.2
# https://github.com/pypa/get-pip
ENV PYTHON_GET_PIP_URL https://github.com/pypa/get-pip/raw/d59197a3c169cef378a22428a3fa99d33e080a5d/get-pip.py
ENV PYTHON_GET_PIP_SHA256 421ac1d44c0cf9730a088e337867d974b91bdce4ea2636099275071878cc189e
RUN set -ex; \
\
wget -O get-pip.py "$PYTHON_GET_PIP_URL"; \
echo "$PYTHON_GET_PIP_SHA256 *get-pip.py" | sha256sum --check --strict -; \
\
pypy3 get-pip.py \
--disable-pip-version-check \
--no-cache-dir \
"pip==$PYTHON_PIP_VERSION" \
; \
# smoke test
pip --version; \
\
find /usr/local -depth \
\( \
\( -type d -a \( -name test -o -name tests \) \) \
-o \
\( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
\) -exec rm -rf '{}' +; \
rm -f get-pip.py
# CMD ["pypy3"]
WORKDIR /usr/src/app
# Tell Python not to recreate the bytecode files. Since this is a docker image,
# these will be recreated every time, writing them just uses unnecessary disk
# space.
ENV PYTHONDONTWRITEBYTECODE=true
RUN apt-get update && apt-get install -y build-essential
#4 opencv3
ENV OPENCV_VERSION 3.4.2
RUN apt-get install --no-install-recommends -y openssh-client git cmake make
################## shared packages for chromium headless ######################
RUN apt-get install -y libx11-xcb1 libxcb1 libx11-6 libxcomposite1 libxcursor1 \
libxdamage1 libxext6 libxi6 libxtst6 libgtk-3-0 libnss3 libxss1 libasound2
# RUN apt-get --no-install-recommends -y install libatlas-base-dev
RUN apt-get install --no-install-recommends -y libopenblas-base libopenblas-dev
# RUN pypy -m ensurepip
RUN pip download scipy
RUN apt-get install -y gfortran
RUN pip install --no-cache-dir --only-binary cython,numpy numpy cython
RUN pypy3 -m pip install --upgrade pip
# RUN pip install --no-cache-dir pybind11
RUN apt-get install -y libopenblas-dev liblapack-dev
RUN pip install git+https://github.com/scipy/scipy@master#egg=scipy
RUN pip install --no-cache-dir pandas dask
# RUN pip install --no-cache-dir pandas dask
RUN apt-get install -y llvm-8-dev llvm-8 pypy-dev
RUN update-alternatives --install /usr/local/bin/llvm-config llvm-config /usr/bin/llvm-config-8 40 && \
# update-alternatives --install /usr/local/bin/clang clang /usr/bin/clang-8 40 && \
update-alternatives --install /usr/local/bin/opt opt /usr/bin/opt-8 40 && \
update-alternatives --install /usr/local/bin/llvm-link llvm-link /usr/bin/llvm-link-8 40
RUN pip install --no-cache-dir llvmlite
RUN pip install --no-cache-dir bokeh scikit-learn
RUN pip install --no-cache-dir reportlab
RUN pip install --no-cache-dir pendulum==2.0.3
RUN pip install --no-cache-dir psycopg2-binary
RUN pip install --no-cache-dir cryptography
RUN apt-get install -y libqpdf-dev
RUN pip install --no-cache-dir pikepdf