heptapod issueshttps://foss.heptapod.net/heptapod/heptapod/-/issues2024-01-25T17:20:56Zhttps://foss.heptapod.net/heptapod/heptapod/-/issues/1367Automate releases triggered by upstream GitLab patch versions2024-01-25T17:20:56ZGeorges RacinetAutomate releases triggered by upstream GitLab patch versionsAs #1366 can testify, the release watcher process is still in working order.
Since the last time it has been actually useful, lots of progress has been made, and we now have an automatic merge system running regularly (both are actually...As #1366 can testify, the release watcher process is still in working order.
Since the last time it has been actually useful, lots of progress has been made, and we now have an automatic merge system running regularly (both are actually tied, so that the release watcher does not miss a tag because the auto-merge would have been the one to pull it).
By reusing logic done for the auto-merge, it should be possible to fully automate creating an Heptapod release from an upstream patch release.
Some ideas:
- abort and file an issue if there is a bump of GitLab Shell version, or a significant Gitaly change (protocol, backups)
- resolve conflicts as in auto-merge (for now just trivial ones such as in issue templates, most unlikely in upstream stable branches, but it could become smarter)
- merge Omnibus (not yet handled by upstream auto-merge process), filing issues upon problems.
- perform MR, monitor it and set the appropriate tag
- ultimately, perform Docker image tagging and pushing
All of this will take some time, because it is very hard to test separately, but it would give us eventually:
- less latency in producing security releases (down to a couple of hours)
- more developer time available for other endeavors.https://foss.heptapod.net/heptapod/heptapod/-/issues/765Backups: accomodate remove-all at restoration2023-07-21T23:57:48ZGeorges RacinetBackups: accomodate remove-all at restorationThe `-remove-all-repositories` of `gitaly-backup` is finally put to use in 76b74f63c8bf (see also hgitaly#139).
However, we cannot use it in Heptapod right away, because it will also remove the namespaces hierarchy that we are using for...The `-remove-all-repositories` of `gitaly-backup` is finally put to use in 76b74f63c8bf (see also hgitaly#139).
However, we cannot use it in Heptapod right away, because it will also remove the namespaces hierarchy that we are using for group-level Mercurial configuration (caught by our functional tests). Hence in Heptapod 0.38, we will have to filter out those storages that have a HGitaly address. Upstream concern is about the race in repositories created between the restoration trigger and actual restoration, probably not so much a concert for us (for the time being).
To get back to upstream we will need either:
- to find another storage solution for group-level configuration
- to make HGitaly servers completely separate from Gitaly servers, but that is very far away and cannot accomodate any legacy (hg-git based) project.https://foss.heptapod.net/heptapod/heptapod/-/issues/760Implement change of storage for Mercurial projects2024-03-21T13:42:51ZGeorges RacinetImplement change of storage for Mercurial projectsHeptapod still supports having only one Mercurial-enabled repository storage (i.e Gitaly and HGitaly services). However, it will be in the not-so-far future.
At this point, `Projects::UpdateRepositoryStorageService` will have to support...Heptapod still supports having only one Mercurial-enabled repository storage (i.e Gitaly and HGitaly services). However, it will be in the not-so-far future.
At this point, `Projects::UpdateRepositoryStorageService` will have to support Mercurial projects, its point being to move repositories from one storage to the other. The already are some provisions toward that, but it's obviously incomplete (see in particular how `Concerns::updateRepositoryStorageMethods` handles the Project repository).Heptapod 1.4https://foss.heptapod.net/heptapod/heptapod/-/issues/758Capability to upgrade a GitLab instance to Heptapod2024-03-21T13:42:50ZGeorges RacinetCapability to upgrade a GitLab instance to HeptapodIt should be possible to upgrade a GitLab CE instance to Heptapod, and it certainly has been possible for many Heptapod versions in the past.
However, some of our data migrations were breaking rules that once were soft and have since be...It should be possible to upgrade a GitLab CE instance to Heptapod, and it certainly has been possible for many Heptapod versions in the past.
However, some of our data migrations were breaking rules that once were soft and have since become mandatory. It was reasonable because many of these rules are geared towards high volumetry (as can be found on gitlab.com) which was not a concern to us at the time yet have become enforced upstream.
The typical case would be `AddHgAsDefaultVcs` as it was until the merge of GitLab 15.8: it changes the schema (default value on `application_settings`) and updates a column on the `namespaces` table: it wouldn't have passed upstream review at the time, but there is a strict enforcement nowadays that migrations have to use either Data Definition Language (DDL) or Data Manipulation Language (DML). It makes the migration fail if running anew on recent Heptapod versions (which won't happen unless upgrading from an upstream GitLab instance because it dates way back). Also it breaks all the RSpec test that contain *any* migration test.
The problem in RSpec tests will of course be solved with the [merge of GitLab 15.8](!491), yet in a minimal way that is not suitable to upgrade an upstream GitLab CE instance.
This issue to keep track of the work need for such upgrades to be supported:
- [ ] `application_settings.vcs_types` should be introduced by a single migration, with the correct default set in the schema (single row table)
- [ ] `namespaces.default_vcs_type` does not need a migration of the value if starting afresh on current Heptapod versions. A `NULL` value means to default to Application Settings, so we don't need any backfill.
- [ ] actual testing of an upgrade from GitLab (how to automate that?)Heptapod 1.4