heptapod issueshttps://foss.heptapod.net/groups/heptapod/-/issues2023-07-19T15:19:46Zhttps://foss.heptapod.net/heptapod/hgitaly/-/issues/142Commit gRPC message: encoding field2023-07-19T15:19:46ZGeorges RacinetCommit gRPC message: encoding fieldGitaly v15.10 adds the `encoding` field to the `GitCommit` message:
Extract from the Git commit in Gitaly that introduced it:
```diff
~/heptapod/gitaly $ git log -n1 -p cf24112527 -- proto/*.proto
commit cf24112527408773814956ae6aaf94a...Gitaly v15.10 adds the `encoding` field to the `GitCommit` message:
Extract from the Git commit in Gitaly that introduced it:
```diff
~/heptapod/gitaly $ git log -n1 -p cf24112527 -- proto/*.proto
commit cf24112527408773814956ae6aaf94a8f4a41ad9
Author: arkn98 <2424696-arkn98@users.noreply.gitlab.com>
Date: Tue Mar 14 22:58:41 2023 -0400
catfile: Add encoding information to GitCommit
Currently, Gitaly doesn't return the `encoding` header, which is
populated by `git cat-file` when the commit message was encoded using a
non-UTF-8 encoding and the `i18n.commitEncoding` was set to that
encoding when the commit was made. This can cause clients to show
corrupted commit messages, as seen in [1]. This commit adds the
`encoding` field to the `GitCommit` proto.
1. https://gitlab.com/gitlab-org/gitlab/-/issues/20058
Fixes: https://gitlab.com/gitlab-org/gitaly/-/issues/836
Changelog: added
Signed-off-by: arkn98 <2424696-arkn98@users.noreply.gitlab.com>
diff --git a/proto/shared.proto b/proto/shared.proto
index e2c2a41b7..7ca3e1ee4 100644
--- a/proto/shared.proto
+++ b/proto/shared.proto
@@ -128,6 +128,11 @@ message GitCommit {
// referenced_by contains fully-qualified reference names (e.g refs/heads/main)
// that point to the commit.
repeated bytes referenced_by = 12; // protolint:disable:this REPEATED_FIELD_NAMES_PLURALIZED
+ // The encoding of the commit message. This field will only be present if
+ // `i18n.commitEncoding` was set to a value other than "UTF-8" at the time
+ // this commit was made.
+ // See: https://git-scm.com/docs/git-commit#_discussion
+ string encoding = 13;
}
// This comment is left unintentionally blank.
```
I'm not sure if that kind of information is available in Mercurial.
The comment suggests that the situation could be no worse than it already is if we don't fill it, but perhaps some fallbacks will be removed in the Rails app or other clients because the decoding has become more reliable (yet obviously not 100% guaranteed).https://foss.heptapod.net/heptapod/heptapod/-/issues/764VCS type alignment on home panel2024-03-21T13:27:26ZGeorges RacinetVCS type alignment on home panelWith the move to upstream GitLab v15.9, the short VCS type in project home page is now in exponent position.
![home-panel-v15.9](/uploads/c6bba4a6c1060f863041978ecc06fe2d/home-panel-v15.9.png)
This should just be a matter of using the ...With the move to upstream GitLab v15.9, the short VCS type in project home page is now in exponent position.
![home-panel-v15.9](/uploads/c6bba4a6c1060f863041978ecc06fe2d/home-panel-v15.9.png)
This should just be a matter of using the appropriate CSS class in app/views/projects/_home_panel.html.hamlHeptapod 1.3.2https://foss.heptapod.net/heptapod/heptapod/-/issues/763Document RHGitaly source installation2023-07-06T10:34:33ZGeorges RacinetDocument RHGitaly source installationGenerally speaking, our [install from source](INSTALL_HEPTAPOD.md) doc is probably in need of a serious overhaulGenerally speaking, our [install from source](INSTALL_HEPTAPOD.md) doc is probably in need of a serious overhaulhttps://foss.heptapod.net/heptapod/heptapod/-/issues/762Freeze requests python dependency2023-07-04T14:07:04ZGeorges RacinetFreeze requests python dependencyWhile upgrading to 0.36.3, we got new versions of the `urllib3` and `requests` libraries, and they were incompatible.
Freeze to the latest compatible set of versions in the release branchWhile upgrading to 0.36.3, we got new versions of the `urllib3` and `requests` libraries, and they were incompatible.
Freeze to the latest compatible set of versions in the release branchhttps://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/225OSUOSL: experiment without the Cellar cache2023-06-30T16:42:58ZGeorges RacinetOSUOSL: experiment without the Cellar cacheWhen we started using our own Container Registry, `registry.heptapod.net`, we quickly had to work around bandwidth issues between Clever Cloud services (in this case the S3-compatible Cellar, which actually holds container layers) and OS...When we started using our own Container Registry, `registry.heptapod.net`, we quickly had to work around bandwidth issues between Clever Cloud services (in this case the S3-compatible Cellar, which actually holds container layers) and OSUOSL. It was by then around 2-3 MB/s.
At the time, the solution was to introduce a caching layer at OSUOSL. In theory, the configuration should not have tampered with uploads, but in practice it does, which makes for long debugging sessions that we cannot really afford. The consequence was the introduction of the `container-registry-push` tag, to avoid building images on runners that don't have the capability to upload them.
After some changes in our swarm of Runners, we only have two instance runners available for container builds: `osci-01-docker-x86_64` and `clever-cloud`. The OSCI runner is currently [in trouble](#224) and the Clever runner cannot be used [in some cases](#218). This is bad.
I've just took some fresh bandwidth measurements and got 17 MB/s to get a 1 GB file from Cellar to OSUOSL. This looks acceptable enough to at least experiment removing this caching layer from OSUOSL runners. In turn, this should make them also able to use shared CI caches (less efficiently than the `clever-cloud` runner, obviously).
## Task list
- [x] Detach `osuosl-01` and `osuosl-02` from the Cellar cache, tag them `container-registry-push`
- [x] Test Docker image production on instance runners
- [x] Configure S3 shared CI caches on Omnibus Heptapod dedicated runners (`osuosl-openstack-01-omnibus-heptapod` and `osuosl-openstack-02-omnibus-heptapod`, same systems actually, with different configurations)
- [x] Confirm proper cache behaviour on the Omnibus runners with the upcoming builds for Heptapod 0.36.3
- [x] Generalizehttps://foss.heptapod.net/heptapod/heptapod/-/issues/760Implement change of storage for Mercurial projects2024-03-21T13:42:51ZGeorges RacinetImplement change of storage for Mercurial projectsHeptapod still supports having only one Mercurial-enabled repository storage (i.e Gitaly and HGitaly services). However, it will be in the not-so-far future.
At this point, `Projects::UpdateRepositoryStorageService` will have to support...Heptapod still supports having only one Mercurial-enabled repository storage (i.e Gitaly and HGitaly services). However, it will be in the not-so-far future.
At this point, `Projects::UpdateRepositoryStorageService` will have to support Mercurial projects, its point being to move repositories from one storage to the other. The already are some provisions toward that, but it's obviously incomplete (see in particular how `Concerns::updateRepositoryStorageMethods` handles the Project repository).Heptapod 1.4https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/48Error 403 when sending logs about job not launched2023-06-26T13:02:23ZGeorges RacinetError 403 when sending logs about job not launchedI got a report today of a 403 (Forbidden) error, and actually what happened is:
```
Traceback (most recent call last):
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/paas_dispatcher.py", line 393,...I got a report today of a 403 (Forbidden) error, and actually what happened is:
```
Traceback (most recent call last):
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/paas_dispatcher.py", line 393, in launch_job
if not runner.job_wait_trace(
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/runner.py", line 405, in job_wait_trace
raise JobLaunchTimeout(job_handle, timeout)
```
and the 403 error is in the error treatment:
```
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/runner.py", line 359, in job_append_trace
target=lambda: self.launch_job(job_handle, job_data))
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/paas_dispatcher.py", line 405, in launch_job
new_range = self.gl_job_send_trace(
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/runner.py", line 337, in gl_job_send_trace
runner.job_append_trace(
File "/home/paas-runner/paas-runner-venv/lib/python3.9/site-packages/heptapod_runner/runner.py", line 337, in gl_job_send_trace
raise GitLabUnexpectedError(status_code=resp.status_code,
```
It is possible that the job was already considered to be failed by the coordinator. In any case, this is too much log pollution, and it diverts investigation from the orginal cause.https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/224System upgrade for OSCI Runner2023-09-13T08:12:33ZGeorges RacinetSystem upgrade for OSCI RunnerThe OSCI Runner is running an older OS version than the rest of the swarm.
Several issues depend on its upgrade, even if just to move this cause of problems out of the way: #221, #222 and perhaps more.
~RunnersThe OSCI Runner is running an older OS version than the rest of the swarm.
Several issues depend on its upgrade, even if just to move this cause of problems out of the way: #221, #222 and perhaps more.
~RunnersGeorges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/223Project for hg-fastimport2023-07-12T12:40:03ZRoy MarplesProject for hg-fastimport## Description of the projects to be hosted
- Project Name: hg-fastimport
- Project URL: https://roy.marples.name/projects/hg-fastimport, https://wiki.mercurial-scm.org/FastImportExtension
- Project Maintainers: Roy Marples
- Project Li...## Description of the projects to be hosted
- Project Name: hg-fastimport
- Project URL: https://roy.marples.name/projects/hg-fastimport, https://wiki.mercurial-scm.org/FastImportExtension
- Project Maintainers: Roy Marples
- Project License: GPL2
Please explain here how they meet our current criteria.
### Main criteria
- [X] Free / Open Source
- [X] main development repositories
- [X] you are an official maintainer of the project
- [X] agreement to acknowledge where appropriate the help provided by Octobus
and Clever Cloud by hosting on foss.heptapod.net
### Additional information
Knowing this can help in case there are constraints on resources and/or
administrators availability at the time of the request. It can also help us
suggest the right solution for the needs of the project.
- is the project officially released? In downstream distributions or
packaging systems?
Yes, in pkgsrc at least.
https://pkgsrc.se/devel/py-hg-fastimport
- estimation of the user base if possible, or number of downloads,
reverse dependencies…
No idea, but likely small.
- number of maintainers and regular contributors
1
- where is the project currently hosted? Are there issue trackers,
review systems, CI/CD to hook to or to replace somehow?
I used to self host but I can no longer do this.
I am looking for a new host, there are no trackers or CI to replace, just a place to host the code is currently needed.
### Hosting type
What would you like to have?
- [ ] a top-level group
- [ ] a project in an existing group
- [X] just a project in your personal namespace
## Post approval maintainer task list
See [README](README.md#maintainer-task-list-for-the-import-to-happen)
for explanations.
- [ ] Reviewing [Heptapod workflow](https://heptapod.net/pages/faq.html#workflow)
## Post import maintainer task list
- [ ] Add the Octobus and Clever cloud acknowledgement at the selected place.
- [ ] Communicate about the migration to help others to aware of the options
(for example on mastodon, twitter, bitbucket thread, mailing list, …)https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/222Many CI jobs fail randomly for "shell not found"2023-06-26T09:53:36ZCédric Kriercedric.krier@b2ck.comMany CI jobs fail randomly for "shell not found"We have many CI jobs like https://foss.heptapod.net/tryton/tryton/-/jobs/1966078 that are failing randomly with this message: `shell not found` just after using the docker image.We have many CI jobs like https://foss.heptapod.net/tryton/tryton/-/jobs/1966078 that are failing randomly with this message: `shell not found` just after using the docker image.Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/221Running node on CI fails2023-06-26T10:22:50ZCédric Kriercedric.krier@b2ck.comRunning node on CI failsSuch [job](https://foss.heptapod.net/tryton/tryton-docker/-/jobs/1957699) is now failing. This seems to be related to thread.\
I got recently similar issue with memcached, this was because the docker version was too old.Such [job](https://foss.heptapod.net/tryton/tryton-docker/-/jobs/1957699) is now failing. This seems to be related to thread.\
I got recently similar issue with memcached, this was because the docker version was too old.Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/759Document how to enable Largefile extension2023-07-25T06:45:09ZnekopepDocument how to enable Largefile extension### Summary
We are migrating form rhodecode to heptapod and we are stuck with repos using largefile extension.
- We have tried to disable extension while migrating but as it change changeset sha1 into the repos it is not possible for re...### Summary
We are migrating form rhodecode to heptapod and we are stuck with repos using largefile extension.
- We have tried to disable extension while migrating but as it change changeset sha1 into the repos it is not possible for repos used as subrepos.
I've seen that heptapod seems to support lfs (see bug #118: https://foss.heptapod.net/heptapod/heptapod/-/issues/118)
-> lib/gitlab/hg_access.rb looks correclty modified on heptapod 0.35 (the one I use)
It looks like I need to modify `config/mercurial/repos.hgrc` but I can't find this file on my instance.
Can you update the doc to explain how to migrate largefile extension enabled repos to help switching to heptapod?
I can also help with the doc provided you explain what is missing :smile:https://foss.heptapod.net/heptapod/heptapod/-/issues/758Capability to upgrade a GitLab instance to Heptapod2024-03-21T13:42:50ZGeorges RacinetCapability to upgrade a GitLab instance to HeptapodIt should be possible to upgrade a GitLab CE instance to Heptapod, and it certainly has been possible for many Heptapod versions in the past.
However, some of our data migrations were breaking rules that once were soft and have since be...It should be possible to upgrade a GitLab CE instance to Heptapod, and it certainly has been possible for many Heptapod versions in the past.
However, some of our data migrations were breaking rules that once were soft and have since become mandatory. It was reasonable because many of these rules are geared towards high volumetry (as can be found on gitlab.com) which was not a concern to us at the time yet have become enforced upstream.
The typical case would be `AddHgAsDefaultVcs` as it was until the merge of GitLab 15.8: it changes the schema (default value on `application_settings`) and updates a column on the `namespaces` table: it wouldn't have passed upstream review at the time, but there is a strict enforcement nowadays that migrations have to use either Data Definition Language (DDL) or Data Manipulation Language (DML). It makes the migration fail if running anew on recent Heptapod versions (which won't happen unless upgrading from an upstream GitLab instance because it dates way back). Also it breaks all the RSpec test that contain *any* migration test.
The problem in RSpec tests will of course be solved with the [merge of GitLab 15.8](!491), yet in a minimal way that is not suitable to upgrade an upstream GitLab CE instance.
This issue to keep track of the work need for such upgrades to be supported:
- [ ] `application_settings.vcs_types` should be introduced by a single migration, with the correct default set in the schema (single row table)
- [ ] `namespaces.default_vcs_type` does not need a migration of the value if starting afresh on current Heptapod versions. A `NULL` value means to default to Application Settings, so we don't need any backfill.
- [ ] actual testing of an upgrade from GitLab (how to automate that?)Heptapod 1.4https://foss.heptapod.net/heptapod/heptapod/-/issues/757Not possible to close protected branch2023-06-10T07:42:29ZCédric Kriercedric.krier@b2ck.comNot possible to close protected branchIt is not possible to closed a protected branch:
> No push is allowed to delete protected branches on Mercurial project. If that was your intent, you will have to unprotect it first
This is a little bit annoying because you have to unp...It is not possible to closed a protected branch:
> No push is allowed to delete protected branches on Mercurial project. If that was your intent, you will have to unprotect it first
This is a little bit annoying because you have to unprotect it so any one with mercurial publish right will be allowed to push during the time we unprotect the branch.
It is worse if using wildcard pattern to protect many branches at once like `branch/*`.https://foss.heptapod.net/heptapod/heptapod/-/issues/756Topic branches are marked as merged2023-06-05T13:09:28ZCédric Kriercedric.krier@b2ck.comTopic branches are marked as mergedWhen having a large project with many contributors creating topics, looking at the stale branches to find abandoned topics can be useful to make some cleaning. But all topics are labelled as `merged`.
Instead of this label, it will be b...When having a large project with many contributors creating topics, looking at the stale branches to find abandoned topics can be useful to make some cleaning. But all topics are labelled as `merged`.
Instead of this label, it will be better to have a link to the merge request if there are one.https://foss.heptapod.net/heptapod/heptapod/-/issues/755Can not find merge request linked to a topic2023-06-03T06:25:40ZCédric Kriercedric.krier@b2ck.comCan not find merge request linked to a topicOn large project we may have quite a lot of topics opened by contributors. But it is difficult to find the related merge request in heptapod.
I think we need a keyword to search by topic name.On large project we may have quite a lot of topics opened by contributors. But it is difficult to find the related merge request in heptapod.
I think we need a keyword to search by topic name.https://foss.heptapod.net/heptapod/heptapod/-/issues/751Hashed storage migration: robustness if wiki repo is missing2023-05-26T12:24:22ZGeorges RacinetHashed storage migration: robustness if wiki repo is missingA fix for an old version that will help migrate to the newer ones!
As seen in #742, in Heptapod 0.23, if a Project lacks its wiki repository, then the hashed storage migration fails. It should not.
I don't have on mind whether the wiki...A fix for an old version that will help migrate to the newer ones!
As seen in #742, in Heptapod 0.23, if a Project lacks its wiki repository, then the hashed storage migration fails. It should not.
I don't have on mind whether the wiki repo should exist if the feature is disabled in the Project settings, this has to be checked. If not, that is also an easy recommendation to administrators of the affected instance: just disable it on the offending Projects.Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/750Robustness if hgrc is missing2023-05-26T12:20:17ZGeorges RacinetRobustness if hgrc is missingNowadays, it'd be a major inconsistency for `.hg/hgrc` not to exist in a server-side Mercurial repository on Heptapod.
However, as was highlighted by #742, it can happen in long-standing instances of our early adopters, and we should ma...Nowadays, it'd be a major inconsistency for `.hg/hgrc` not to exist in a server-side Mercurial repository on Heptapod.
However, as was highlighted by #742, it can happen in long-standing instances of our early adopters, and we should make at least `mv_repository` more robust about that, as it can block the hashed storage migration.
Task list:
- [ ] make `mv_repository` create the `hgrc` file with default content if missing (perhaps with inheritance set to `False` in order not to change any behavioural aspect)
- [ ] chase down parts in the code base where the existence of the main `hgrc` is simply assumed (methods to change configuration, notably)
- [ ] backport to Heptapod 0.23Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/747Topics of closed merge request are still visible in local clone2024-02-10T10:07:37ZCédric Kriercedric.krier@b2ck.comTopics of closed merge request are still visible in local cloneWhen a merge request has been closed from heptapod UI, after pulling a local repository (which had the topic), the topic is still there.\
I would expect that the topic would be cleaned like when a merge request is merged.When a merge request has been closed from heptapod UI, after pulling a local repository (which had the topic), the topic is still there.\
I would expect that the topic would be cleaned like when a merge request is merged.https://foss.heptapod.net/heptapod/hgext-loggingmod/-/issues/5Drop Python 2 support2023-05-12T07:47:43ZGeorges RacinetDrop Python 2 supportIt's now just a waste of resources. This means supporting only Mercurial versions that require Python 3, which is fine.It's now just a waste of resources. This means supporting only Mercurial versions that require Python 3, which is fine.