Commit 9609ae2d authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Heptapod CI: prebuilding Gitaly and other deps, pushed on download.h.n

if it works, we'll probably just keep the result and enable the job
only on GitLab version bumps (perhaps with the `changes` directive)

--HG--
branch : heptapod-stable
parent bf0b1d7eaa6c
......@@ -6,6 +6,7 @@
# to upstream, but we are just at the beginning of the CI process for Heptapod.
stages:
- prepare
- test
- upload
......@@ -44,7 +45,25 @@ include:
# Merge Requests (see `.shared:rules:update-cache`)
policy: pull-push
# inspired from rails.gitlab-ci.yml (setup-test-env notably)
# this plays the role of the setup-test-env of upstream GitLab
# the main difference is that it is "colder" a single tarball per
# GitLab version
gitlab-prebuilds:
extends:
- .use-pg11 # base image and services
- .default-before_script # shell functions and some environment vars
- .heptapod-rails-cache
stage: prepare
variables:
GITLAB_TEST_EAGER_LOAD: "0"
script:
- rm -rf tmp/tests/gitaly # stale files from previous builds can prevent proper installation
- run_timed_command "bundle exec ruby -I. -e 'require \"config/environment\"; TestEnv.init'"
- run_timed_command "scripts/gitaly-test-build" # Do not use 'bundle exec' here
- rm tmp/tests/gitaly/.ruby-bundle # This file prevents gems from being installed even if vendor/gitaly-ruby is missing
- run_timed_command "scripts/hgitaly-test-build"
- scripts/heptapod-make-push-gitlab-prebuilds.sh
rspec:
rules:
- changes:
......@@ -68,9 +87,9 @@ rspec:
- log/hgitaly-test.log
- config/gitlab.yml
script:
- rm -rf tmp/tests/gitaly # stale files from previous builds can prevent proper installation
- run_timed_command "scripts/hgitaly-test-build"
- run_timed_command "bundle exec ruby -I. -e 'require \"config/environment\"; TestEnv.init'"
# TODO put it in cellar and benefit from site-local caches
- wget --progress=dot:giga -O /tmp/gitlab-prebuilds.tar.gz https://download.heptapod.net/gitlab-prebuilds/ci-$(cat VERSION).tar.gz
- tar xzf /tmp/gitlab-prebuilds.tar.gz
# copied from compile-assets-base. The day we want several jobs, this
# will be one prime candidate to do like upstream and just produce an
# artifact instead
......@@ -82,8 +101,10 @@ rspec:
# end of assets production
- ls -al tmp/tests/*
- run_timed_command "scripts/gitaly-test-build" # Do not use 'bundle exec' here
- rm tmp/tests/gitaly/.ruby-bundle # This file prevents gems from being installed even if vendor/gitaly-ruby is missing
- run_timed_command "scripts/gitaly-test-spawn"
# python components already installed, but this will refresh them
# and update subsequent conf (gitlab.yml etc)
- run_timed_command "scripts/hgitaly-test-build"
- run_timed_command "scripts/hgitaly-test-spawn"
- grep -A3 'hgrc:' config/gitlab.yml
- cat $PWD/config/tests.hgrc
......
#!/bin/sh
set -e
# it's nicer for console output to add the extension right away
TARBALL=ci-$(cat VERSION).tar.gz
TARBALL_PATH=/tmp/${TARBALL}
echo "Generating ${TARBALL}"
# same paths as artifacts in upstream GitLab `setup-test-env` job
tar zcf ${TARBALL_PATH} \
config/secrets.yml \
tmp/tests/gitaly \
tmp/tests/hgitaly \
tmp/tests/gitlab-shell \
tmp/tests/gitlab-test-fork \
tmp/tests/gitlab-test-fork_bare \
tmp/tests/gitlab-test \
tmp/tests/gitlab-workhorse \
tmp/tests/repositories \
tmp/tests/second_storage
if [ -n "$HEPTAPOD_TARBALL_UPLOAD_KEY" ]; then
`dirname $0`/heptapod-sftp-push \
download.heptapod.net \
"$HEPTAPOD_TARBALL_UPLOAD_KEY" \
"${TARBALL_PATH}" \
"public/gitlab-prebuilds/"
fi
......@@ -3,6 +3,11 @@
Expected to be run from a clone of the Rails application, and assumes
that `config/gitlab.yml` has already been initialized.
In the prebuild phase, the relevant that it does is to prepare the
clones. The installation in the interpreter will be lost and redone in
the actual testing phase. (TODO add a flag to skip it, although it's
probably negligible, given than most deps are in the base image)
"""
import os
import re
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment