Commit 3a5971dc authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Heptapod CI: match HGitaly build/spawn with Gitaly

This creates and preserves the `tmp/tests/hgitaly` subdir
and separates for CI the install (build) and startup (spawn)

By putting `hgitaly.socket` in that preserved directory,
it also fixes test runs on developer machines involving
several spec files (previously, the socket was destroyed
at the end of each spec file)

This is heavier than strictly needed for HGitaly, but
matching what is already done for Gitaly will help us
schedule differently (prepare phase, straight download), and
prebuilding just Gitaly, but with `TestEnv` that actually starts
HGitaly is an unmaintanable mess.

branch : heptapod-stable
parent 3cf6af5d98ac
......@@ -1352,7 +1352,7 @@ test:
path: tmp/tests/repositories/
gitaly_address: unix:tmp/tests/gitaly/gitaly.socket
hgitaly_address: unix:tmp/tests/gitaly/hgitaly.socket
hgitaly_address: unix:tmp/tests/hgitaly/hgitaly.socket
client_path: tmp/tests/gitaly
......@@ -69,6 +69,7 @@ rspec:
- config/gitlab.yml
- rm -rf tmp/tests/gitaly # stale files from previous builds can prevent proper installation
- run_timed_command "scripts/hgitaly-test-build"
- run_timed_command "bundle exec ruby -I. -e 'require \"config/environment\"; TestEnv.init'"
# copied from compile-assets-base. The day we want several jobs, this
# will be one prime candidate to do like upstream and just produce an
......@@ -79,15 +80,15 @@ rspec:
- run_timed_command "bin/rake gitlab:assets:compile > assets-compile.log 2>&1"
- run_timed_command "scripts/clean-old-cached-assets"
# end of assets production
- run_timed_command "python3 scripts/"
- ls -al tmp/tests/*
- run_timed_command "scripts/gitaly-test-build" # Do not use 'bundle exec' here
- rm tmp/tests/gitaly/.ruby-bundle # This file prevents gems from being installed even if vendor/gitaly-ruby is missing
- run_timed_command "scripts/gitaly-test-spawn"
- run_timed_command "scripts/hgitaly-test-spawn"
- grep -A3 'hgrc:' config/gitlab.yml
- cat $PWD/config/tests.hgrc
- cat $PWD/vendor/py-heptapod/heptapod/required.hgrc
- HGRCPATH=vendor/py-heptapod/heptapod/required.hgrc /usr/local/bin/hg hpd-versions; echo
- cat $PWD/hgitaly/py-heptapod/heptapod/required.hgrc
- HGRCPATH=hgitaly/py-heptapod/heptapod/required.hgrc /usr/local/bin/hg hpd-versions; echo
- scripts/ --format d
......@@ -10,9 +10,12 @@ import subprocess
INTERPRETER = 'python3.8'
CWD = os.getcwd()
BASE_DIR = os.path.join(CWD, 'tmp', 'tests', 'hgitaly')
# lazy, but I'm tired of reimplenting `mkdir -p` over and over
subprocess.check_call(['mkdir', '-p', BASE_DIR])
PY_HEPTAPOD_CLONE_PATH = os.path.join(CWD, 'vendor', 'py-heptapod')
HGITALY_CLONE_PATH = os.path.join(CWD, 'vendor', 'hgitaly')
PY_HEPTAPOD_CLONE_PATH = os.path.join(BASE_DIR, 'py-heptapod')
HGITALY_CLONE_PATH = os.path.join(BASE_DIR, 'hgitaly')
HG_BIN = '/usr/local/bin/hg'
PIP_BIN = 'pip3.8'
# This script is used only in CI
set -eu
# CI base image has Mercurial, installed system-wide by pip3 (Python 3.8)
export HGRCPATH=config/tests.hgrc:${PY_HEPTAPOD_CLONE_PATH}/heptapod/required.hgrc
echo "Starting HGitaly server"
set -x
$HG --config extensions.hgitaly= \
hgitaly-serve \
--repositories-root $PWD/tmp/tests/repositories \
--listen unix:tmp/tests/hgitaly/hgitaly.socket &
......@@ -178,7 +178,7 @@ def setup_gitaly
def setup_hgitaly
# as far as configuration goes, for now there's nothing to be done here:
# as far as configuration goes, for now there's not much to be done here:
# we'll assume that the
# GitLab configured HGRCs are fine, except for repositories root, which
# we'll pass on the command line.
......@@ -188,12 +188,19 @@ def setup_hgitaly
# component_timed setup is assuming there's an install dir, a rake task…
puts "\n==> Setting up HGitaly..."
start =
puts " HGitaly setup in #{ - start} seconds"
def start_hgitaly
# setup_timed
if ci?
# HGitaly has been spawned outside this process already
# this would less necessary in principle than in the case of
# Gitaly, but it is useful for pre-warmings to behave in the
# same way as for upstream GitLab
env = {'HGRCPATH' => Gitlab::Mercurial.hgrc_path}
address = Gitlab.config.repositories.storages[REPOS_STORAGE].hgitaly_address
......@@ -223,6 +230,10 @@ def gitaly_dir
def hgitaly_dir
def start_gitaly(gitaly_dir)
if ci?
# Gitaly has been spawned outside this process already
......@@ -462,6 +473,7 @@ def test_dirs
@test_dirs ||= %w[
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment