CommitService.FindCommits: combinatoric explosion with `follow`
HGitaly logs on this very instance reveal a problem with FindCommits
:
[2022-10-21 04:51:22,339] [3398712] [DEBUG] [hgitaly.service.commit] Processing FindCommitsRequest { repository { storage_name: "default" relative_path: "@hashed/ef/2d/ef2d127de37b942baad06145e54b0c619a1f22327b2ebbcfbec78f5564afe39d.git" gl_repository: "project-5" gl_project_path: "heptapod/heptapod" } revision: "branch/heptapod" limit: 40 paths: "README.md" follow: true disable_walk: true global_options { literal_pathspecs: true } }
[2022-10-21 04:52:51,688] [3398702] [WARNING] [hgitaly.server.worker] Worker 2 (pid: 3398712) over limit, will restart: RSS=4230MiB. Full mem stats: pmem(rss=4436221952, vms=4857106432, shared=2400256, text=4096, lib=0, data=4482277376, dirty=0)
Actually memory consumption was so fast that the process was killed by the kernel before its graceful restart occurred.
This is narrowed down from a list of requests as part of a reproduction attempt on my workstation. It is actually not a memory leak, rather a loop explosion in the revset generated by the follow
option (passing follow=False
instead does not exhibit the issue).
On such a large repository, it is quite severe, putting a CPU core at 100% and eating more than 100MB of resident memory per second. The reproduction ended up freezing my workstation despite its 32GB total RAM.