Freshly created containers are on the wrong version of Heptapod
(currently testing a fix for this)
Our Docker image has a preclone of Heptapod, while the first startup replaces the Gitlab CE code at
/opt/gitlab/embedded/services/gitlab-rails
by pulling from this clone.
However this ends up with all draft heads of the Heptapod clone being published within gitlab-rails
and the subsequent update to the heptapod
branch currently ends up on the wrong one (giving 502 errors on pushes due to an internal 401 from hgserve). This is because there's a topic head that's older than the public head.
While working, developers update/revert to specific revisions, that's why their running containers are unaffected. Later on, trying to understand why the neighbour's freshly created container doesn't work, they would revert to the latest public revision, hence won't reproduce.