Skip to content

Draft: packaging: use modern setuptools for pep 517 / isolated build

Pierre Augier requested to merge topic/default/setup.py-less-imports-distutils into branch/default

I'm trying to improve Mercurial's setup.py. This will be a long work and I'm going step by step.

distutils has been removed from Python in 3.12.

For >=3.12, imports from distutils use a copy provided by setuptools. distutils distributed by setuptools is not compatible with pep 517 ("A build-system independent format for source trees").

Even with Python 3.11, I get that

python -m build --sdist
* Creating isolated environment: venv+pip...
* Installing packages in isolated environment:
  - setuptools
  - wheel
* Getting build dependencies for sdist...
/tmp/build-env-rxp79i75/lib/python3.11/site-packages/setuptools/command/sdist.py:121: SetuptoolsDeprecationWarning: `build_py` command does not inherit from setuptools' `build_py`.
!!

        ********************************************************************************
        Custom 'build_py' does not implement 'get_data_files_without_manifest'.
        Please extend command classes from setuptools instead of distutils.

        See https://peps.python.org/pep-0632/ for details.
        ********************************************************************************

!!

Moreover, calling directly setup.py is deprecated and one should use "pep 517 frontends" (pip, python -m build, pdm, etc.). The frontends interacts with the backend (for us setuptools.build_meta), prepares an isolated environment so that the backend can build/install. setuptools.build_meta (https://github.com/pypa/setuptools/blob/main/setuptools/build_meta.py) still supports setup.py. In practice, it calls the command dist_info, sdist and bdist_wheel.

With the current state of this MR, modern workflows mostly work (with different pep 517 frontends like pip, build, pdm, ...). For example:

# install mercurial in editable mode in a virtual env created by pdm (in .venv)
pdm install
pdm run hg version -v
pdm run hg debuginstall

# install in the environment (no editable) 
pdm run pip install . -C --global-option=--rust -v
pdm run hg debuginstall
# rust extension available and working

Note that the Rust extension is not compatible with editable installs (--pure and C ext are compatible).

I guess it would be good if I document this somewhere in the repo but I don't know where.

Note: the Makefile will have to be cleaned up and adapted but I hope it can be done in another MR (?)

A question: do we still want to continue supporting installation without setuptools? In setup.py there is written:

# We have issues with setuptools on some platforms and builders. Until
# those are resolved, setuptools is opt-in except for platforms where
# we don't have issues.

However, it is not easy to know what are these issues and if they are now resolved.

Edited by Pierre Augier

Merge request reports