heptapod issueshttps://foss.heptapod.net/groups/heptapod/-/issues2020-04-04T20:18:44Zhttps://foss.heptapod.net/heptapod/heptapod/-/issues/235Project transfers on legacy storage2020-04-04T20:18:44ZGeorges RacinetProject transfers on legacy storageThe project transfer does not work on legacy (not hashed) storage with Heptapod 0.12.0 RCs.
This will be fixed in the 0.12.0 final release.
The hashed storage becomes the default in GitLab 12, that's why it went under the radar.The project transfer does not work on legacy (not hashed) storage with Heptapod 0.12.0 RCs.
This will be fixed in the 0.12.0 final release.
The hashed storage becomes the default in GitLab 12, that's why it went under the radar.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/9"unrelated" repository requires force pull2022-01-18T13:54:04ZPierre-Yves David"unrelated" repository requires force pullIn some case (eg: the root have been rewritten), the previous clone will be considered unrelated to the new clone. We should ignore that and always use `hg pull --force`In some case (eg: the root have been rewritten), the previous clone will be considered unrelated to the new clone. We should ignore that and always use `hg pull --force`Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/234Remove GitHub and GitLab Export import sources2020-04-05T20:29:04ZGeorges RacinetRemove GitHub and GitLab Export import sourcesThat was the case in Heptapod before 0.12, it's probably just a bad merge on this template file.
They can be disabled from the application settings, but since they can't work anyway, it's better just to remove them.That was the case in Heptapod before 0.12, it's probably just a bad merge on this template file.
They can be disabled from the application settings, but since they can't work anyway, it's better just to remove them.Heptapod 0.12.0https://foss.heptapod.net/heptapod/foss.heptapod.net/-/issues/50Upgrade to Heptapod 0.122020-04-20T12:17:51ZGeorges RacinetUpgrade to Heptapod 0.12We are nearing the point where Heptapod 0.12.0 RCs are good enough to handle foss.heptapod.net
This should bring very substantial performance improvements for large projects.
The migration procedure itself being heavier than usual, thi...We are nearing the point where Heptapod 0.12.0 RCs are good enough to handle foss.heptapod.net
This should bring very substantial performance improvements for large projects.
The migration procedure itself being heavier than usual, this will create a downtime of one to two hours.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/229Fix Backups for GitLab 122020-04-05T10:58:56ZGeorges RacinetFix Backups for GitLab 12The way GitLab backup system has changed since 10.5 (to accomodate the fact that Gitaly is now owning the Git repos), has made it to completely bypass the Heptapod specific backup steps.
This must be fixed for the 0.12.0 final release.The way GitLab backup system has changed since 10.5 (to accomodate the fact that Gitaly is now owning the Git repos), has made it to completely bypass the Heptapod specific backup steps.
This must be fixed for the 0.12.0 final release.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/228Adaptations for repo migration to hashed storage2020-04-04T20:18:44ZGeorges RacinetAdaptations for repo migration to hashed storageIn Gitlab 12, repositories can be stored in a hierarchy with directory names that are hashes, but it's not systematic, especially for repositories predating the migration from Heptapod 0.8.
There's a rake task to migrate the repositorie...In Gitlab 12, repositories can be stored in a hierarchy with directory names that are hashes, but it's not systematic, especially for repositories predating the migration from Heptapod 0.8.
There's a rake task to migrate the repositories to the hashed storage. I don't expect it to work at all for Heptapod 0.12.0rc4.
It *must* work in Heptapod 0.12.0
The hashed storage is the one used by default on GDK based development setups, and Heptapod 0.12.0 RCs work on it with only one problem, the group HGRC issue mentioned in the changelog.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/226bitbucket importer can create content-divergence from PR2020-04-05T19:46:14ZPierre-Yves Davidbitbucket importer can create content-divergence from PRIF the same changeset are part of two differents PR (eg: because one is based on the other), they will get rewritting to the PR's topic twice, leading to content divergenceIF the same changeset are part of two differents PR (eg: because one is based on the other), they will get rewritting to the PR's topic twice, leading to content divergenceHeptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/224Initialising a repository with a README fails2020-05-14T08:53:44ZvomoutInitialising a repository with a README failsCreating a repository from the web interface (0.12.0rc3) with the "create initial README.md" option checked does not work: The interface afterwards fails to load the README and the logs have errors:
```
Completed 500 Internal Server Erro...Creating a repository from the web interface (0.12.0rc3) with the "create initial README.md" option checked does not work: The interface afterwards fails to load the README and the logs have errors:
```
Completed 500 Internal Server Error in 156ms (ActiveRecord: 40.3ms)
Errno::ENOENT (No such file or directory @ rb_sysopen - /var/opt/gitlab/git-data/repositories/tuomov/secondtest.hg/.hg/git-mapfile):
lib/gitlab/git/repository.rb:377:in `initialize'
lib/gitlab/git/repository.rb:377:in `open'
lib/gitlab/git/repository.rb:377:in `hg_sha_map'
lib/gitlab/git/repository.rb:401:in `hgsha_from_sha'
```
Moreover, the README.md is not to be found when pulling the Mercurial repository.
Probably this is due to the one-way hg-git bridge, and it might be simplest to disable the README-creation option for now?
/label ~Bug Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/223NoMethodError when viewing commit contents2020-03-25T22:27:16ZvomoutNoMethodError when viewing commit contentsThis worked in 0.8 but when trying to view commit contents in 0.12.0rc3:
````
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/username/test/commit/f115ef2252d976f396749c3a85cb3ccc08975e57" for 93.205.209.41 at 2020-03-...This worked in 0.8 but when trying to view commit contents in 0.12.0rc3:
````
==> /var/log/gitlab/gitlab-rails/production.log <==
Started GET "/username/test/commit/f115ef2252d976f396749c3a85cb3ccc08975e57" for 93.205.209.41 at 2020-03-23 16:16:39 +0000
Processing by Projects::CommitController#show as HTML
Parameters: {"namespace_id"=>"username", "project_id"=>"test", "id"=>"f115ef2252d976f396749c3a85cb3ccc08975e57"}
Completed 500 Internal Server Error in 145ms (ActiveRecord: 28.4ms)
NoMethodError (undefined method `first' for nil:NilClass):
lib/gitlab/git/commit.rb:262:in `parent_hgid'
app/models/commit.rb:282:in `method_missing'
app/models/commit.rb:297:in `diff_refs'
lib/gitlab/diff/file_collection/commit.rb:11:in `initialize'
app/models/commit.rb:443:in `new'
app/models/commit.rb:443:in `diffs'
app/controllers/projects/commit_controller.rb:150:in `define_commit_vars'
lib/gitlab/session.rb:11:in `with_session'
app/controllers/application_controller.rb:461:in `set_session_storage'
lib/gitlab/i18n.rb:55:in `with_locale'
lib/gitlab/i18n.rb:61:in `with_user_locale'
app/controllers/application_controller.rb:455:in `set_locale'
lib/gitlab/middleware/multipart.rb:103:in `call'
lib/gitlab/request_profiler/middleware.rb:17:in `call'
lib/gitlab/middleware/go.rb:20:in `call'
lib/gitlab/etag_caching/middleware.rb:13:in `call'
lib/gitlab/middleware/correlation_id.rb:16:in `block in call'
lib/gitlab/middleware/correlation_id.rb:15:in `call'
lib/gitlab/middleware/read_only/controller.rb:40:in `call'
lib/gitlab/middleware/read_only.rb:18:in `call'
lib/gitlab/middleware/basic_health_check.rb:25:in `call'
lib/gitlab/request_context.rb:26:in `call'
lib/gitlab/metrics/requests_rack_middleware.rb:29:in `call'
lib/gitlab/middleware/release_env.rb:12:in `call'
```
/label ~Bug
Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/8Evolve and Topic2022-01-18T13:54:05ZGeorges RacinetEvolve and TopicIn the context of heptapod#221, we'll need the `topic` extension to be active on the runner. For clarity, and since they are packaged together, I'll prefer to activate `evolve` as well. We should:
- [x] add to our install instructions t...In the context of heptapod#221, we'll need the `topic` extension to be active on the runner. For clarity, and since they are packaged together, I'll prefer to activate `evolve` as well. We should:
- [x] add to our install instructions that `topic` and `evolve` should be installed and activated
- [x] install `hg-evolve` in the helper Docker image. In Alpine, `pip` can be installed as `pip install py-pip` and that doesn't pull the whole jungle of C build toolchains
- [x] activate `evolve` and `topic` in the helper Docker image. Via `/etc/mercurial/hgrc.d` should be fine.
- [x] add options to the runner's pull in order to avoid some unnecessary and heavy operations such as obsmarker exchangeHeptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/7Official Docker image2022-01-18T13:54:05ZGeorges RacinetOfficial Docker imageSome of our users are happy with the runner itself packaged as a Docker image. This is also the easiest distribution channel we can target.
Not to be confused with the helper images `octobus/heptapod-runner-helper` who are already avail...Some of our users are happy with the runner itself packaged as a Docker image. This is also the easiest distribution channel we can target.
Not to be confused with the helper images `octobus/heptapod-runner-helper` who are already available on Docker hub.Heptapod 0.12.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/221do not serve changeset with topic to people without the extensions2020-05-14T10:15:29ZPierre-Yves Daviddo not serve changeset with topic to people without the extensionsSince topic `0.18.0` (shipped with evolve `9.3.0`), one can use:
```
[experimental]
topic.server-gate-topic-changesets = yes
```
To prevent client without topic to clone/pull topic by default.
We need to adjust the runner to still pul...Since topic `0.18.0` (shipped with evolve `9.3.0`), one can use:
```
[experimental]
topic.server-gate-topic-changesets = yes
```
To prevent client without topic to clone/pull topic by default.
We need to adjust the runner to still pull the topic (installing the topic extensions should be enough.Heptapod 0.12.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/220gitlab_shell_ssh_port2020-04-05T12:34:30Zvomoutgitlab_shell_ssh_portThe `gitlab_rails['gitlab_shell_ssh_port']` setting `gitlab.rb` does not seem to be respected: the SSH URL displayed in the web interface does not indicate non-standard ports. (Heptapod docker image id 57a2500f2ccd.)The `gitlab_rails['gitlab_shell_ssh_port']` setting `gitlab.rb` does not seem to be respected: the SSH URL displayed in the web interface does not indicate non-standard ports. (Heptapod docker image id 57a2500f2ccd.)Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/219Hide/Fix the "revert" option on merge request2020-04-05T20:29:04ZPierre-Yves DavidHide/Fix the "revert" option on merge requestA new "revert" button appears for merge request after they are merged.
I don't expect this option to work. So we should hide it for now.A new "revert" button appears for merge request after they are merged.
I don't expect this option to work. So we should hide it for now.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/218Hide/Fix cherry pick option2020-04-05T20:29:04ZPierre-Yves DavidHide/Fix cherry pick optionA new "Cherry Pick" appears for merge request once it is merged.
I don't expect this option to work. So we should hide it for now.A new "Cherry Pick" appears for merge request once it is merged.
I don't expect this option to work. So we should hide it for now.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/213Group HGRCs won't work on hashed projects2020-04-05T11:07:31ZGeorges RacinetGroup HGRCs won't work on hashed projectsThis is a know issue in Heptapod 0.12.0rc1. With the new hashing strategy used in GitLab 12, our system for group HGRCs just falls apart.
We'll need to design a new one. That probably means that transfers will need to maintain the incl...This is a know issue in Heptapod 0.12.0rc1. With the new hashing strategy used in GitLab 12, our system for group HGRCs just falls apart.
We'll need to design a new one. That probably means that transfers will need to maintain the inclusion of the group HGRC if any.Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod-runner/-/issues/5Gitlab Runner v122022-01-18T13:54:04ZGeorges RacinetGitlab Runner v12With Heptapod rebase on top GitLab 12 now around the corner, we will need a compatible runnerWith Heptapod rebase on top GitLab 12 now around the corner, we will need a compatible runnerHeptapod 0.12.0Georges RacinetGeorges Racinethttps://foss.heptapod.net/heptapod/heptapod/-/issues/212GitLab CE 12.2.x2020-04-14T12:11:47ZGeorges RacinetGitLab CE 12.2.xHeptapod 0.12 will feature a double bump of GitLab versions, from 10.5 to 12.2
The current main development branch is now based on GitLab 12.2.0.pre.
What remains is
- [x] bump to 12.2.12 or any stable released 12.2.x version
- [x] mi...Heptapod 0.12 will feature a double bump of GitLab versions, from 10.5 to 12.2
The current main development branch is now based on GitLab 12.2.0.pre.
What remains is
- [x] bump to 12.2.12 or any stable released 12.2.x version
- [x] migration steps based on GitLab 10.8, 11.0, and 11.11Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/192optimize repository after import2020-07-29T14:08:32ZPierre-Yves Davidoptimize repository after importMercurial had various improvement to the way it store delta. They make the repository smaller and less CPU hungry to look at.
It is worth running the following commands aftern the initial import to make sure we are getting the best poss...Mercurial had various improvement to the way it store delta. They make the repository smaller and less CPU hungry to look at.
It is worth running the following commands aftern the initial import to make sure we are getting the best possible repository format
`hg debugupgraderepo -o re-delta-all --run --no-backup`Heptapod 0.12.0https://foss.heptapod.net/heptapod/heptapod/-/issues/135Performance of pushes2020-03-20T21:24:44ZGeorges RacinetPerformance of pushesThe inner push to the auxiliary Git repository is the major bottleneck in Heptapod taking pushes. This can be cut into two parts:
1. exporting Mercurial changesets as Git changesets
2. pushing to the Git repository seen by GitLab
For b...The inner push to the auxiliary Git repository is the major bottleneck in Heptapod taking pushes. This can be cut into two parts:
1. exporting Mercurial changesets as Git changesets
2. pushing to the Git repository seen by GitLab
For both, the correct long-term fix is to stop using this auxiliary Git repo, in favor of the hgitaly project.
We've been aware of the first for a long time. On my development workstation, the speed varies between 10 changesets per second (empty repo) and 3-5 per second (end of PyPy import). It's obviously at least linear with respect to the repo size.
We haven't looked at the second until recently. Here's a log extract, taken on this instance, on `octobus/mercurial-devel`:
```
[2019-12-01 19:22:55 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] GitHandler.push_gitlab_branches, pushing {'refs/heads/topic/default/fix-last-uncheck-background-process': 'ea6bac45ae265cdece17e84eb329a7d1fdf91966'} to remote 'git+ssh://git@localhost:octobus/mercurial-devel.git'
[2019-12-01 19:23:48 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] 2 commits found
[2019-12-01 19:24:01 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] GitLab 'post-receive' hook wrapper: sending changes {'refs/heads/topic/default/fix-last-uncheck-background-process': ('0000000000000000000000000000000000000000', 'ea6bac45ae265cdece17e84eb329a7d1fdf91966')}
```
The time between the two first log lines seem to be Git discovery process, the time between the two last is a bit more than the conclusion of the push (we don't have a precise log line for that). But it's telling, and it dominates largely the export time (under one second). We have up to three such pushes, but in that case the others were faster.
I don't want to spend too much time on this, because that would be time taken from the hgitaly project, unless it becomes unbearable for our users.
For the slowness of the Git push itself, if someone wants to tackle it, it would be interesting to try and use the Git binary provided with GitLab, instead of attempting to improve dulwich performance.Heptapod 0.12.0