heptapod issueshttps://foss.heptapod.net/groups/heptapod/-/issues2020-09-11T12:44:03Zhttps://foss.heptapod.net/heptapod/heptapod/-/issues/344Displaying Mercurial Node IDs in the repository graph2020-09-11T12:44:03ZGeorges RacinetDisplaying Mercurial Node IDs in the repository graphThe Repository Graph page displays contains full commit details which are displayed upon hovering over the dot that materializes the commit within the graph.
Currently, that includes the full Git SHA, we want it to display the Mercurial...The Repository Graph page displays contains full commit details which are displayed upon hovering over the dot that materializes the commit within the graph.
Currently, that includes the full Git SHA, we want it to display the Mercurial Node ID instead.![graph-hovering](/uploads/c77edeb87c815a97c8141ae5b642a636/graph-hovering.png)Heptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod/-/issues/342Git SHAs: Markdown rendering including MRs additional commits2020-09-24T16:54:16ZGeorges RacinetGit SHAs: Markdown rendering including MRs additional commitsThe Markdown rendering subsystem of GitLab is able to link automatically to Git commits.
While doing so, it normalizes the link text to the preferred form, so that, e.g, `ca1234fe56789de01234` becomes just `ca1234fe56`. We could take a...The Markdown rendering subsystem of GitLab is able to link automatically to Git commits.
While doing so, it normalizes the link text to the preferred form, so that, e.g, `ca1234fe56789de01234` becomes just `ca1234fe56`. We could take advantage of that to display the Mercurial short node id (truncated hash) instead of the Git commit short id.
This would still not allow people to directly link to Mercurial changesets, but it would immediately apply to "system notes", for instance the additional commits messages in Merge Requests.
Part of #6Heptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod/-/issues/339Tarballs generation for the Rails app2020-09-08T13:27:20ZGeorges RacinetTarballs generation for the Rails appPushing tarballs for this project to a permanent download site will be a step for omnibus-heptapod#1 (hence #338).
It is also probably useful for source installations: don't need to install an arbitrary version of Mercurial before the [...Pushing tarballs for this project to a permanent download site will be a step for omnibus-heptapod#1 (hence #338).
It is also probably useful for source installations: don't need to install an arbitrary version of Mercurial before the [exact specification](python/requirements.txt) is available and various downstream packaging efforts.
Of course we'd like to do this in a CI job, using `hg archive` and running automatically for Heptapod tags.
Because we want to be able to push intermediate testing Docker images, we also need to produce tarballs for arbitrary revisions, perhaps with a manual job.Heptapod 0.16.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/338Packaging of frontend assets2020-09-24T11:28:13ZGeorges RacinetPackaging of frontend assetsSince the inception of Heptapod, we've had no consistent way of packaging changes in frontend assets, because these are precompiled in the Debian package on which the upstream Docker image is based. This includes notably various images a...Since the inception of Heptapod, we've had no consistent way of packaging changes in frontend assets, because these are precompiled in the Debian package on which the upstream Docker image is based. This includes notably various images and JavaScript code. Only source installations could take advantage of frontend changes, and obviously we didn't want to create differences between installation types.
This currently blocks many features (all issues bearing the `Front` label`). Also, with many UI GitLab elements being rendered in JavaScript, this problem is getting automatically worse over time.
Here's a plan to overcome that in the short run:
- omnibus-heptapod#1: create packages including Heptapod Rails app sources instead of GitLab Rails but otherwise identical to the upstream package.
- omnibus-heptapod#2: new intermediate base Heptapod Docker image, based on the package produced by previous step.
- heptapod-docker#10: use this new Docker base image instead of the upstream GitLab image.
(writing this on the main project instead of heptapod-docker because the involvement of the latter will be minimal and the visible consequences are very much about the Web application)Heptapod 0.16.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod-docker/-/issues/10Use the base image produced by omnibus-heptapod2020-09-24T11:28:13ZGeorges RacinetUse the base image produced by omnibus-heptapodOnce omnibus-heptapod#2 is ready, we can use the new base image instead of `gitlab/gitlab-ce`.
Potentially, `heptapod_revisions.json` would already be sitting in the filesystem, and obviously we wouldn't need to take care of the Rails a...Once omnibus-heptapod#2 is ready, we can use the new base image instead of `gitlab/gitlab-ce`.
Potentially, `heptapod_revisions.json` would already be sitting in the filesystem, and obviously we wouldn't need to take care of the Rails application, since it's included already, hence automatically closing #9.Heptapod 0.16.0https://foss.heptapod.net/heptapod/omnibus-heptapod/-/issues/2Publish an Heptapod base Docker image2020-09-08T08:43:15ZGeorges RacinetPublish an Heptapod base Docker imageThis is the logical follow-up to #1: once we are able to produce a Debian package from the Heptapod Rails sources, use it to produce a Docker image and upload it to Docker Hub as. e.g., `heptapod/build-base`.
This image would not be via...This is the logical follow-up to #1: once we are able to produce a Debian package from the Heptapod Rails sources, use it to produce a Docker image and upload it to Docker Hub as. e.g., `heptapod/build-base`.
This image would not be viable for deployments, as it would not even have Mercurial, but it would be usable as a replacement for the GitLab image in the Heptapod Docker builds.Heptapod 0.16.0https://foss.heptapod.net/heptapod/omnibus-heptapod/-/issues/1Replace gitlab-rails sources by heptapod-rails2020-09-08T08:59:46ZGeorges RacinetReplace gitlab-rails sources by heptapod-railsThis is enough to produce packages with Heptapod frontend assets properly compiled (thumbnails and other images with discriminating hashes in their names, minified JavaScript code etc).
This is harder than it looks, because `omnibus-git...This is enough to produce packages with Heptapod frontend assets properly compiled (thumbnails and other images with discriminating hashes in their names, minified JavaScript code etc).
This is harder than it looks, because `omnibus-gitlab` very much wants to fetch the sources with Git and its own upstream Omnibus doesn't have a `HgFetcher`.
An intermediate step might be to switch to a manually prefetched directory in `/var/cache/omnibus`.
Then perhaps, we could just reuse parts of [install_heptapod.py](https://foss.heptapod.net/heptapod/heptapod-docker/-/blob/branch/default/heptapod/assets/install_heptapod.py) to do that fetching for us (and move `heptapod_revisions.json` to the `omnibus-heptapod` project).Heptapod 0.16.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/336GitLab 13.32020-08-30T16:47:18ZGeorges RacinetGitLab 13.3GitLab 13.3 was released as expected on 2020-08-22. Merging it in Heptapod will make us current until late November, and will make downstream packaging easier, notably the work done by @arrowd for FreeBSDGitLab 13.3 was released as expected on 2020-08-22. Merging it in Heptapod will make us current until late November, and will make downstream packaging easier, notably the work done by @arrowd for FreeBSDHeptapod 0.16.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/335Bump Mercurial to 5.5 and extensions2020-09-22T17:09:50ZGeorges RacinetBump Mercurial to 5.5 and extensionsGiven that the tests for py-heptapod work for Mercurial 5.5, using it in Heptapod (instead of the current 5.4.2) should be straightforward, implying only to bump Evolve and hg-git as wellGiven that the tests for py-heptapod work for Mercurial 5.5, using it in Heptapod (instead of the current 5.4.2) should be straightforward, implying only to bump Evolve and hg-git as wellHeptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod/-/issues/334Low level enforcement of Mercurial permissions from the Rails app2020-08-27T18:19:07ZGeorges RacinetLow level enforcement of Mercurial permissions from the Rails appThis is the follow-up to #332: when performing Mercurial actions, the web app must pass down the needed information so that Mercurial can perform the necessary checks (publication, notably).
This will duplicate the a priori checks that ...This is the follow-up to #332: when performing Mercurial actions, the web app must pass down the needed information so that Mercurial can perform the necessary checks (publication, notably).
This will duplicate the a priori checks that the web application already does but that's a necessary evil:
- it is the only way not to be more permissive in the Web app than in direct pushes done with a client.
- it does not have the same atomicity as protections in the Rails app, meaning we need to keep the latter as well. The typical example is the merge with explicit changeset sequence: with low level protections only, Mercurial will not refuse to perform the merge commit (as a draft), but it will refuse the later publication. This leaves a situation that a Maintainer can clean up (e.g., with a prune), but is still suboptimal.Heptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod/-/issues/276Replace favicon with heptapod logo (replacements attached)2020-09-26T10:06:58ZSietse BrouwerReplace favicon with heptapod logo (replacements attached)Heptapod's favicon is still the Gitlab tanuki logo. I have created replacement favicon files and attached them to this issue.
- [favicon](/uploads/67d42be104055dc6e2517660125b18ce/favicon.png) ![favicon](/uploads/67d42be104055dc6e251766...Heptapod's favicon is still the Gitlab tanuki logo. I have created replacement favicon files and attached them to this issue.
- [favicon](/uploads/67d42be104055dc6e2517660125b18ce/favicon.png) ![favicon](/uploads/67d42be104055dc6e2517660125b18ce/favicon.png) A Heptapod logo replacement for `app/assets/images/favicon.png`.
- [canary.zip](/uploads/eaac7301a6f07c44985fc70376905bf1/canary.zip) Heptapod logo replacements for the icons in `app/assets/images/ci_favicons/canary`. That folder contains favicons with a pipeline status badge overlaid, like this. ![favicon_status_warning](/uploads/412d68b43db51f6741ee6ff68daca233/favicon_status_warning.ico) ![favicon_status_success](/uploads/21608e84487a79c78c4661d032f96309/favicon_status_success.ico)
I regret that I haven't been able to test this fix, because I couldn't get the container to build. Instead, I have checked the source code etc. to get as sure as possible. Hopefully this will be easy to test for somebody with a working build.Heptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod-docker/-/issues/9Reduce the size of the layer for the Rails application2020-09-24T09:02:02ZGeorges RacinetReduce the size of the layer for the Rails applicationWith the work done in !12 and a few follow-ups, the size of our main image is about 300MB on top of the base GitLab image.
The bulk of that improvement is #5: loading the [Rails application](https://foss.heptapod.net/heptapod/heptapod) ...With the work done in !12 and a few follow-ups, the size of our main image is about 300MB on top of the base GitLab image.
The bulk of that improvement is #5: loading the [Rails application](https://foss.heptapod.net/heptapod/heptapod) from a pre-generated tarball instead of making it a full repository. But the corresponding layer (for Dockerfile `ADD` command) weighs 137MB whereas our diff from GitLab 12.3.9 to Heptapod 0.13.0rc1 is about 300K only. That means it should be possible to gain another 130MB with little effort.
Some ideas:
1. Have the Dockerfile `ADD` be smarter and not touch unchanged files (could be just a matter of metadata)
2. Instead of a full tarball, generate a diff and apply it with `patch` during the build
3. Pre-generate a "delta tarball", with only those files that actually differ between GitLab upstream and Heptapod
Note that at some point in the future, we'll rebuild everything from the base OS image (currently Ubuntu 16.04) or from new "cloud native" GitLab images. This could happen in a few months, and that will make current optimization efforts obsolete,
Gaining 130MB with at most a few hours of work seems right in that perspective.Heptapod 0.16.0https://foss.heptapod.net/heptapod/heptapod/-/issues/362GitLab 13.42020-11-02T16:02:47ZGeorges RacinetGitLab 13.4End of support for the current 13.3 version should be on November 22.
Bumping the upstream version is also a good opportunity to introduce the new [versioning policy for components](#352).
Since we are in the middle of important founda...End of support for the current 13.3 version should be on November 22.
Bumping the upstream version is also a good opportunity to introduce the new [versioning policy for components](#352).
Since we are in the middle of important foundational changes (Git support, first HGitaly inclusion, packaging), it's best to keep it minimal and not jump right away to GitLab 13.5Heptapod 0.17.0https://foss.heptapod.net/heptapod/heptapod/-/issues/359Project and Namespace environment variables on hashed storage2020-10-31T16:42:52ZGeorges RacinetProject and Namespace environment variables on hashed storageWhen called from the Rails application, the Mercurial process receives information about the current project, passed as environment variables which can in turn be useful to hooks.
We have notably `HEPTAPOD_PROJECT_PATH` and `HEPTAPOD_PR...When called from the Rails application, the Mercurial process receives information about the current project, passed as environment variables which can in turn be useful to hooks.
We have notably `HEPTAPOD_PROJECT_PATH` and `HEPTAPOD_PROJECT_NAMESPACE_FULL_PATH`, the concatenation of which is the relative URI of the project from the server base URL.
The initial implementation was done for heptapod#72, at a time when these paths could be inferred in a very straightforward way from the repository path on the file system. With the hashed storage, the repository file system path has nothing to do with the URI path of the project.Heptapod 0.17.0https://foss.heptapod.net/heptapod/hgitaly/-/issues/11Bump protocol, at least to GitLab 13.32022-01-26T15:18:33ZGeorges RacinetBump protocol, at least to GitLab 13.3Currently the main development branch of Heptapod is based on GitLab 13.3, and it's quite possible that we bump to GitLab 13.4 for Heptapod 0.17.
But the Gitaly protocol we're using is older, dating back to GitLab 13.1 or even 12.x.Currently the main development branch of Heptapod is based on GitLab 13.3, and it's quite possible that we bump to GitLab 13.4 for Heptapod 0.17.
But the Gitaly protocol we're using is older, dating back to GitLab 13.1 or even 12.x.Heptapod 0.17.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/hgitaly/-/issues/8Tests: bring coverage up to 100% and enforce it2022-01-26T15:18:32ZGeorges RacinetTests: bring coverage up to 100% and enforce itThis kind of thing is waaaay easier to do when the project is young.This kind of thing is waaaay easier to do when the project is young.Heptapod 0.17.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/hgitaly/-/issues/6RepositoryService: implement GetArchive2022-01-26T15:18:31ZGeorges RacinetRepositoryService: implement GetArchiveThis has a direct application to heptapod#204, without going through the Rails application.This has a direct application to heptapod#204, without going through the Rails application.Heptapod 0.17.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/omnibus-heptapod/-/issues/12HGitaly service2020-10-15T11:28:51ZGeorges RacinetHGitaly serviceShould be easy, just a variation on work done for #10.
Could perhaps be backported to the 0.16 series, if the timing makes sense, but that's a stretch: it would make us rebuild Mercurial entirely for the Python 3.7 built by Omnibus (se...Should be easy, just a variation on work done for #10.
Could perhaps be backported to the 0.16 series, if the timing makes sense, but that's a stretch: it would make us rebuild Mercurial entirely for the Python 3.7 built by Omnibus (see #4).Heptapod 0.17.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod-tests/-/issues/13Running functional tests against an Omnibus instance2020-10-05T23:04:36ZGeorges RacinetRunning functional tests against an Omnibus instanceWe currently don't support Omnibus installations at all, but it's technically not far from working and there's immediate value to provide the base fixture class for them.
The GitLab monolithic Docker images is nothing but an Omnibus ins...We currently don't support Omnibus installations at all, but it's technically not far from working and there's immediate value to provide the base fixture class for them.
The GitLab monolithic Docker images is nothing but an Omnibus instance with a fixed Linux distribution providing the base user land (currently Ubuntu 16.04) and a general starter script. Heptapod Docker images add more software and configuration on top of that, but we are close to the point where there's nothing to add.
Once we have an `OmnibusHeptapod` base fixture class, we can launch the tests from the inside of a container, and we can also do it for the current stable series (Heptapod 0.16). In turn, this means that we can have a CI job that launches the tests that are skipped with the `--heptapod-remote` option we use on CI, without resorting to Docker in Docker.
No more manual testing would be required for intermediate Docker images and releases. As an extra bonus, once the Docker image is the one provided by Omnibus Heptapod, we can trigger everything from the Omnibus (or even Rails) tag push, saving many hours of work. It would be very nice to produce Heptapod 0.17.0rc1 this way.
Mostly what has to be done is a middle ground between the current `SourceHeptapod` and `DockerHeptapod` classes:
* using `gitlab-ctl` and `gitlab-rake` as in `DockerHeptapod`
* direct file system access and subprocess spawning as in `SourceHeptapod`
* HTTP and SSH management as in `SourceHeptapod`
* file system paths as in `DockerHeptapod` (hence hardcoded to the default Omnibus ones). Later on we could also read them from configuration.Heptapod 0.17.0https://foss.heptapod.net/heptapod/heptapod/-/issues/353Future 0.17: drop Python 2 support2020-11-26T11:26:08ZGeorges RacinetFuture 0.17: drop Python 2 supportSince the landing of omnibus-heptapod!3, the `octobus/heptapod:testing-py2` Docker image is actually running Python 3 code, even though it has all the Python 2 versions available.
This is due to a mix of explicit configuration done by O...Since the landing of omnibus-heptapod!3, the `octobus/heptapod:testing-py2` Docker image is actually running Python 3 code, even though it has all the Python 2 versions available.
This is due to a mix of explicit configuration done by Omnibus (Rails application and Heptapod Shell) and PATH precedence (for the `hgserve` service).
Of course we could fix it in various ways, but I'd prefer not even have to worry about it and the accompanying uncertainty.
We are also close to the point where Omnibus Heptapod can build a fully working image, hence the only added value of the heptapod-docker project is to provide a Python 2 version (see omnibus-heptapod!7 and omnibus-heptapod!5).
We've had dual support since %"Heptapod 0.13.0", about 4 months ago, and only had very minor glitches reported to us about it. Maybe it's simply time to just drop Python 2 support in Heptapod 0.17?
Task list:
- [x] push Python 3 only `octobus/heptapod:testing` image
- [x] remove `testing-py3` and `testing-py2` tags
- [x] update [INSTALL_HEPTAPOD](INSTALL_HEPTAPOD.md)
- [x] upon Heptapod 0.17.0rc1 release, remove the `latest-py3` and `latest-py2` tags of the Docker imageHeptapod 0.17.0