ci-images issueshttps://foss.heptapod.net/mercurial/ci-images/-/issues2022-10-05T16:40:55Zhttps://foss.heptapod.net/mercurial/ci-images/-/issues/1Normalizing and sharing CI images for Mercurial extensions2022-10-05T16:40:55ZGeorges RacinetNormalizing and sharing CI images for Mercurial extensionsThere are currently different Docker images for CI in use by Mercurial related projects hosted here, representing different stages of evolution (also according to capabilites of GitLab/Heptapod available at the time they were created)
I...There are currently different Docker images for CI in use by Mercurial related projects hosted here, representing different stages of evolution (also according to capabilites of GitLab/Heptapod available at the time they were created)
I think it would be profitable to create a common ground, hence minimizing rebuild costs, storage and most importantly bandwidth. Note that foss.heptapod.net is hosted in France, while many of our CI servers are in the USA.
## Catalog
In chronological order, starting from the oldest all paths relative to `registry.heptapod.net`"
### `mercurial/ci-images/py2-hgext3rd` and `mercurial/ci-images/py3-hgext3rd`
Their `Dockerfiles` are in in [octobus/ci-dockerfiles](https://foss.heptapod.net/octobus/ci-dockerfiles).
These are used notably by Evolve and other extensions managed by Octobus.
They were formerly hosted on DockerHub as `octobus/ci-py2-hgext3rd` and `/ci-py3hgext3rd` and I simply pushed them to get over the Docker Hub rate limits ASAP
They come with full clones of the Mercurial source repository and are built manually. We've been wanting to improve on that for a long time. Most builds start by updating the sources and building Mercurial.
They also provide the standard Mercurial `run-tests.py`. It's policy of most of these extensions not to vendor their own copy
### `heptapod/ci-images/mercurial`
Used by [py-heptapod](https://foss.heptapod.net/heptapod/py-heptapod] and [HGitaly](https://foss.heptapod.net/heptapod/hgitaly).
Dockerfiles are in the project with the same path from the Heptapod root.
These have either a fixed installed version of Mercurial (only one current version, the one used by Heptapod) from PyPi or from tarball archives for the stable/default development versions.
They come with small additional dependencies to run tests with [mercurial_testhelpers](https://foss.heptapod.net/mercurial/testhelpers): `pytest`, `pytest-cov` etc.
They are automatically rebuilt daily, and based on a common `fixed-pydeps` image which is rebuilt weekly. This is how the Mercurial stable/default versions stay up to date (given our reactivity, one day is good enough)
### `mercurial/hg-git/ci`
Provided very recently by @dan in hg-git!56. These are based on the Alpine development image provided by the Python project, with Mercurial stable/default installed directly from PyPI or from a tarball archive of stable/default branches (downloaded from the mercurial-scm.org server).
They are rebuilt weekly, and are declined for many versions, on Python 2 and Python 3. Thanks to Alpine (and the care in Dockerfile definition), they are pretty slim.
An additional image is provided with a Mercurial version packaged by Ubuntu.
They just obsoleted the images I had previously made from Octobus' `ci-pyX-hg3rd` that were almost never updated.
## The plan
I had a chat earlier today with @marmoute, who seems to agree that the images created by @dan would probably work for Evolve and extensions following the same testing process, if they included `run-tests.py`, which should be in the Mercurial source tarball.
Adding Git, pytest/tox and mercurial_testhelpers wouldn't make a huge difference in total image size.
It seems pretty realistic to me that we could host the Dockerfiles in the present project, and have images that could serve in all scenarios. We could split in several layers later on if that becomes too big anyway
I think we don't need Rust variants right now, but they could be added later as separate layers.
Any thoughts?