Performance of pushes
The inner push to the auxiliary Git repository is the major bottleneck in Heptapod taking pushes. This can be cut into two parts:
- exporting Mercurial changesets as Git changesets
- pushing to the Git repository seen by GitLab
For both, the correct long-term fix is to stop using this auxiliary Git repo, in favor of the hgitaly project.
We've been aware of the first for a long time. On my development workstation, the speed varies between 10 changesets per second (empty repo) and 3-5 per second (end of PyPy import). It's obviously at least linear with respect to the repo size.
We haven't looked at the second until recently. Here's a log extract, taken on this instance, on octobus/mercurial-devel
:
[2019-12-01 19:22:55 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] GitHandler.push_gitlab_branches, pushing {'refs/heads/topic/default/fix-last-uncheck-background-process': 'ea6bac45ae265cdece17e84eb329a7d1fdf91966'} to remote 'git+ssh://git@localhost:octobus/mercurial-devel.git'
[2019-12-01 19:23:48 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] 2 commits found
[2019-12-01 19:24:01 +0000] [24208] [INFO] repo:/var/opt/gitlab/git-data/repositories/octobus/mercurial-devel.hg [hg] GitLab 'post-receive' hook wrapper: sending changes {'refs/heads/topic/default/fix-last-uncheck-background-process': ('0000000000000000000000000000000000000000', 'ea6bac45ae265cdece17e84eb329a7d1fdf91966')}
The time between the two first log lines seem to be Git discovery process, the time between the two last is a bit more than the conclusion of the push (we don't have a precise log line for that). But it's telling, and it dominates largely the export time (under one second). We have up to three such pushes, but in that case the others were faster.
I don't want to spend too much time on this, because that would be time taken from the hgitaly project, unless it becomes unbearable for our users.
For the slowness of the Git push itself, if someone wants to tackle it, it would be interesting to try and use the Git binary provided with GitLab, instead of attempting to improve dulwich performance.