Skip to content
GitLab
Projects Groups Topics Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Register
  • Sign in
  • pypy pypy
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributor statistics
    • Graph
    • Compare revisions
  • Issues 698
    • Issues 698
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 14
    • Merge requests 14
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Container Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar

Due to a massive storm of spam, registration on this instance now requires explicit administrator approval. Sorry for the added friction, we're also looking into automatic filtering options.

  • PyPyPyPy
  • pypypypy
  • Issues
  • #3232
Closed
Open
Issue created May 21, 2020 by Sandeep Srinivasa@sandys

Libraries like Pikepdf fail to build because of Py_EnterRecursiveCall

https://github.com/pikepdf/pikepdf/blob/master/src/qpdf/pikepdf.h#L177

Because of lines like the above, there is a problem while building these packages in pypy.

The error is

  In file included from src/qpdf/annotation.cpp:21:
  src/qpdf/pikepdf.h: In constructor ‘StackGuard::StackGuard(const char*)’:
  src/qpdf/pikepdf.h:177:31: error: invalid conversion from ‘const char*’ to ‘char*’ [-fpermissive]
    177 |         Py_EnterRecursiveCall(where);
        |                               ^~~~~
        |                               |
        |                               const char*
  In file included from /usr/local/include/Python.h:144,
                   from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/detail/common.h:112,
                   from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/pytypes.h:12,
                   from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/cast.h:13,
                   from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/attr.h:13,
                   from /tmp/pip-build-env-nw8gtmor/overlay/site-packages/pybind11/include/pybind11/pybind11.h:44,
                   from src/qpdf/annotation.cpp:18:
  /usr/local/include/pypy_decl.h:962:45: note:   initializing argument 1 of ‘int PyPy_EnterRecursiveCall(char*)’
    962 | PyAPI_FUNC(int) Py_EnterRecursiveCall(char *arg0);
        |                                       ~~~~~~^~~~
  error: command 'gcc' failed with exit status 1

I'm pasting a Dockerfile which will reproduce the issue for you (takes 45 minutes to build)

FROM buildpack-deps:sid

# ensure local pypy is preferred over distribution pypy
ENV PATH /usr/local/bin:$PATH

# http://bugs.python.org/issue19846
# > At the moment, setting "LANG=C" on a Linux system *fundamentally breaks Python 3*, and that's not OK.
ENV LANG C.UTF-8

# runtime dependencies
RUN apt-get update && apt-get install -y --no-install-recommends \
    tcl \
    tk \
    && rm -rf /var/lib/apt/lists/*

ENV PYPY_VERSION 7.3.1

RUN set -ex; \
    \
    # this "case" statement is generated via "update.sh"
    dpkgArch="$(dpkg --print-architecture)"; \
    case "${dpkgArch##*-}" in \
    # amd64
    amd64) pypyArch='linux64'; sha256='f67cf1664a336a3e939b58b3cabfe47d893356bdc01f2e17bc912aaa6605db12' ;; \
    # arm64v8
    arm64) pypyArch='aarch64'; sha256='b900241bca7152254c107a632767f49edede99ca6360b9a064141267b47ef598' ;; \
    # i386
    i386) pypyArch='linux32'; sha256='7045b295d38ba0b5ee65bd3f078ca249fcf1de73fedeaab2d6ad78de2eab0f0e' ;; \
    # ppc64le
    ppc64el) pypyArch='ppc64le'; sha256='d6f3b701313df69483b43ebdd21b9652ae5e808b2eea5fbffe3b74b82d2e7433' ;; \
    # s390x
    s390x) pypyArch='s390x'; sha256='0fe2f7bbf42ea88b40954d7de773a43179a44f40656f2f58201524be70699544' ;; \
    *) echo >&2 "error: current architecture ($dpkgArch) does not have a corresponding PyPy $PYPY_VERSION binary release"; exit 1 ;; \
    esac; \
    \
    savedAptMark="$(apt-mark showmanual)"; \
    apt-get update; \
    apt-get install -y --no-install-recommends \
    # sometimes "pypy3" itself is linked against libexpat1 / libncurses5, sometimes they're ".so" files in "/usr/local/lib_pypy"
    libexpat1 \
    libncurses5 \
    # (so we'll add them temporarily, then use "ldd" later to determine which to keep based on usage per architecture)
    ; \
    \
    wget -O pypy.tar.bz2 "https://bitbucket.org/pypy/pypy/downloads/pypy3.6-v${PYPY_VERSION}-${pypyArch}.tar.bz2" --progress=dot:giga; \
    echo "$sha256 *pypy.tar.bz2" | sha256sum -c; \
    tar -xjC /usr/local --strip-components=1 -f pypy.tar.bz2; \
    find /usr/local/lib-python -depth -type d -a \( -name test -o -name tests \) -exec rm -rf '{}' +; \
    rm pypy.tar.bz2; \
    \
    # smoke test
    pypy3 --version; \
    \
    if [ -f /usr/local/lib_pypy/_ssl_build.py ]; then \
    # on pypy3, rebuild ffi bits for compatibility with Debian Stretch+ (https://github.com/docker-library/pypy/issues/24#issuecomment-409408657)
    cd /usr/local/lib_pypy; \
    pypy3 _ssl_build.py; \
    # TODO rebuild other cffi modules here too? (other _*_build.py files)
    fi; \
    \
    apt-mark auto '.*' > /dev/null; \
    [ -z "$savedAptMark" ] || apt-mark manual $savedAptMark > /dev/null; \
    find /usr/local -type f -executable -exec ldd '{}' ';' \
    | awk '/=>/ { print $(NF-1) }' \
    | sort -u \
    | xargs -r dpkg-query --search \
    | cut -d: -f1 \
    | sort -u \
    | xargs -r apt-mark manual \
    ; \
    apt-get purge -y --auto-remove -o APT::AutoRemove::RecommendsImportant=false; \
    rm -rf /var/lib/apt/lists/*; \
    # smoke test again, to be sure
    pypy3 --version; \
    \
    find /usr/local -depth \
    \( \
    \( -type d -a \( -name test -o -name tests \) \) \
    -o \
    \( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
    \) -exec rm -rf '{}' +

# if this is called "PIP_VERSION", pip explodes with "ValueError: invalid truth value '<VERSION>'"
ENV PYTHON_PIP_VERSION 20.0.2
# https://github.com/pypa/get-pip
ENV PYTHON_GET_PIP_URL https://github.com/pypa/get-pip/raw/d59197a3c169cef378a22428a3fa99d33e080a5d/get-pip.py
ENV PYTHON_GET_PIP_SHA256 421ac1d44c0cf9730a088e337867d974b91bdce4ea2636099275071878cc189e

RUN set -ex; \
    \
    wget -O get-pip.py "$PYTHON_GET_PIP_URL"; \
    echo "$PYTHON_GET_PIP_SHA256 *get-pip.py" | sha256sum --check --strict -; \
    \
    pypy3 get-pip.py \
    --disable-pip-version-check \
    --no-cache-dir \
    "pip==$PYTHON_PIP_VERSION" \
    ; \
    # smoke test
    pip --version; \
    \
    find /usr/local -depth \
    \( \
    \( -type d -a \( -name test -o -name tests \) \) \
    -o \
    \( -type f -a \( -name '*.pyc' -o -name '*.pyo' \) \) \
    \) -exec rm -rf '{}' +; \
    rm -f get-pip.py

# CMD ["pypy3"]


WORKDIR /usr/src/app


# Tell Python not to recreate the bytecode files. Since this is a docker image,
# these will be recreated every time, writing them just uses unnecessary disk
# space.
ENV PYTHONDONTWRITEBYTECODE=true

RUN apt-get  update &&  apt-get install -y build-essential 

#4 opencv3
ENV OPENCV_VERSION 3.4.2


RUN apt-get install --no-install-recommends -y openssh-client git cmake make

################## shared packages for chromium headless ######################
RUN apt-get install -y libx11-xcb1 libxcb1 libx11-6 libxcomposite1 libxcursor1 \
    libxdamage1 libxext6 libxi6 libxtst6 libgtk-3-0 libnss3 libxss1 libasound2

# RUN  apt-get --no-install-recommends -y install libatlas-base-dev
RUN apt-get install --no-install-recommends -y libopenblas-base  libopenblas-dev 

# RUN pypy -m ensurepip

RUN pip download  scipy
RUN apt-get install -y gfortran
RUN pip install --no-cache-dir --only-binary cython,numpy numpy cython 
RUN pypy3 -m pip install --upgrade pip
# RUN pip install --no-cache-dir pybind11 
RUN apt-get install -y libopenblas-dev liblapack-dev 
RUN  pip install git+https://github.com/scipy/scipy@master#egg=scipy

RUN pip install --no-cache-dir  pandas dask 

# RUN pip install --no-cache-dir  pandas dask 

RUN apt-get install -y llvm-8-dev  llvm-8 pypy-dev
RUN update-alternatives --install /usr/local/bin/llvm-config llvm-config /usr/bin/llvm-config-8 40 && \
    # update-alternatives --install /usr/local/bin/clang clang /usr/bin/clang-8 40 && \
    update-alternatives --install /usr/local/bin/opt opt /usr/bin/opt-8 40 && \
    update-alternatives --install /usr/local/bin/llvm-link llvm-link /usr/bin/llvm-link-8 40
RUN pip install --no-cache-dir  llvmlite

RUN pip install --no-cache-dir  bokeh scikit-learn
RUN pip install --no-cache-dir  reportlab
RUN pip install --no-cache-dir   pendulum==2.0.3
RUN pip install --no-cache-dir  psycopg2-binary
RUN pip install --no-cache-dir  cryptography
RUN apt-get install -y libqpdf-dev
RUN pip install --no-cache-dir pikepdf
Edited May 21, 2020 by Sandeep Srinivasa
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information
Assignee
Assign to
Time tracking