Commit 3a5971dc authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Heptapod CI: match HGitaly build/spawn with Gitaly

This creates and preserves the `tmp/tests/hgitaly` subdir
and separates for CI the install (build) and startup (spawn)
phases.

By putting `hgitaly.socket` in that preserved directory,
it also fixes test runs on developer machines involving
several spec files (previously, the socket was destroyed
at the end of each spec file)

This is heavier than strictly needed for HGitaly, but
matching what is already done for Gitaly will help us
schedule differently (prepare phase, straight download), and
prebuilding just Gitaly, but with `TestEnv` that actually starts
HGitaly is an unmaintanable mess.

--HG--
branch : heptapod-stable
parent 3cf6af5d98ac
......@@ -1352,7 +1352,7 @@ test:
default:
path: tmp/tests/repositories/
gitaly_address: unix:tmp/tests/gitaly/gitaly.socket
hgitaly_address: unix:tmp/tests/gitaly/hgitaly.socket
hgitaly_address: unix:tmp/tests/hgitaly/hgitaly.socket
gitaly:
client_path: tmp/tests/gitaly
......
......@@ -69,6 +69,7 @@ rspec:
- config/gitlab.yml
script:
- rm -rf tmp/tests/gitaly # stale files from previous builds can prevent proper installation
- run_timed_command "scripts/hgitaly-test-build"
- run_timed_command "bundle exec ruby -I. -e 'require \"config/environment\"; TestEnv.init'"
# copied from compile-assets-base. The day we want several jobs, this
# will be one prime candidate to do like upstream and just produce an
......@@ -79,15 +80,15 @@ rspec:
- run_timed_command "bin/rake gitlab:assets:compile > assets-compile.log 2>&1"
- run_timed_command "scripts/clean-old-cached-assets"
# end of assets production
- run_timed_command "python3 scripts/prepare_build_py_heptapod.py"
- ls -al tmp/tests/*
- run_timed_command "scripts/gitaly-test-build" # Do not use 'bundle exec' here
- rm tmp/tests/gitaly/.ruby-bundle # This file prevents gems from being installed even if vendor/gitaly-ruby is missing
- run_timed_command "scripts/gitaly-test-spawn"
- run_timed_command "scripts/hgitaly-test-spawn"
- grep -A3 'hgrc:' config/gitlab.yml
- cat $PWD/config/tests.hgrc
- cat $PWD/vendor/py-heptapod/heptapod/required.hgrc
- HGRCPATH=vendor/py-heptapod/heptapod/required.hgrc /usr/local/bin/hg hpd-versions; echo
- cat $PWD/hgitaly/py-heptapod/heptapod/required.hgrc
- HGRCPATH=hgitaly/py-heptapod/heptapod/required.hgrc /usr/local/bin/hg hpd-versions; echo
- scripts/heptapod_rspec.sh --format d
......
......@@ -10,9 +10,12 @@ import subprocess
INTERPRETER = 'python3.8'
CWD = os.getcwd()
BASE_DIR = os.path.join(CWD, 'tmp', 'tests', 'hgitaly')
# lazy, but I'm tired of reimplenting `mkdir -p` over and over
subprocess.check_call(['mkdir', '-p', BASE_DIR])
PY_HEPTAPOD_CLONE_PATH = os.path.join(CWD, 'vendor', 'py-heptapod')
HGITALY_CLONE_PATH = os.path.join(CWD, 'vendor', 'hgitaly')
PY_HEPTAPOD_CLONE_PATH = os.path.join(BASE_DIR, 'py-heptapod')
HGITALY_CLONE_PATH = os.path.join(BASE_DIR, 'hgitaly')
HG_BIN = '/usr/local/bin/hg'
PIP_BIN = 'pip3.8'
......
#!/bin/sh
# This script is used only in CI
set -eu
# CI base image has Mercurial, installed system-wide by pip3 (Python 3.8)
HG=/usr/local/bin/hg
PY_HEPTAPOD_CLONE_PATH=hgitaly/py-heptapod
export HGRCPATH=config/tests.hgrc:${PY_HEPTAPOD_CLONE_PATH}/heptapod/required.hgrc
echo "Starting HGitaly server"
set -x
$HG --config extensions.hgitaly= \
hgitaly-serve \
--repositories-root $PWD/tmp/tests/repositories \
--listen unix:tmp/tests/hgitaly/hgitaly.socket &
......@@ -178,7 +178,7 @@ def setup_gitaly
end
def setup_hgitaly
# as far as configuration goes, for now there's nothing to be done here:
# as far as configuration goes, for now there's not much to be done here:
# we'll assume that the
# GitLab configured HGRCs are fine, except for repositories root, which
# we'll pass on the command line.
......@@ -188,12 +188,19 @@ def setup_hgitaly
# component_timed setup is assuming there's an install dir, a rake task…
puts "\n==> Setting up HGitaly..."
start = Time.now
FileUtils.mkdir_p(hgitaly_dir)
start_hgitaly
puts " HGitaly setup in #{Time.now - start} seconds"
end
def start_hgitaly
# setup_timed
if ci?
# HGitaly has been spawned outside this process already
# this would less necessary in principle than in the case of
# Gitaly, but it is useful for pre-warmings to behave in the
# same way as for upstream GitLab
return
end
env = {'HGRCPATH' => Gitlab::Mercurial.hgrc_path}
address = Gitlab.config.repositories.storages[REPOS_STORAGE].hgitaly_address
......@@ -223,6 +230,10 @@ def gitaly_dir
File.dirname(gitaly_socket_path)
end
def hgitaly_dir
TMP_TEST_PATH.join('hgitaly')
end
def start_gitaly(gitaly_dir)
if ci?
# Gitaly has been spawned outside this process already
......@@ -462,6 +473,7 @@ def test_dirs
@test_dirs ||= %w[
frontend
gitaly
hgitaly
gitlab-shell
gitlab-test
gitlab-test_bare
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment