Read about our upcoming Code of Conduct on this issue

Commit e9f59edd authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Heptapod CI: building bootstrapped images with Kaniko

A later problem is that we will need to also produce the 'testing'
image (or maybe have omnibus produce it for tag builds only, or
just get rid of it entirely).

Attempts at building the two images in a single call so far
have failed. Notably, --cache will want to push its cached layers
to a local registry, defaulting to registry.h.n to which upload is very
slow. Also, it is not clear what the end step would be
after running Kaniko twice in the same filesystem (not what Kaniko is
designed for).

Perhaps we can hope for a new Kaniko feature to become available later on,
like pushing two build stages defined in a single Dockerfile? In the mean time
let's just accept the possibility of an unnecessary pull of the
bootstrapped image (not so grim).
parent 4b88739c923c
Pipeline #13696 canceled with stages
in 1 minute and 57 seconds
variables:
TARGET_HEPTAPOD_IMAGE: "testing"
HEPTAPOD_BASE_IMAGE: registry.heptapod.net/heptapod/omnibus-heptapod
HEPTAPOD_BOOTSTRAPPED_IMAGE: "$CI_REGISTRY_IMAGE/bootstrapped"
HEPTAPOD_CHROMIUM_IMAGE: "$CI_REGISTRY_IMAGE/bootstrapped-chromium"
stages:
- image-data
- image-webdriver
- lint
- test-heptapod
.build-image:
# TODO rules, of course. Probably only if trigger variable is there
# (wouldn't block web triggering)
image:
name: gcr.io/kaniko-project/executor:debug
entrypoint: [""]
script:
- ci/kaniko
bootstrapped-image:
stage: image-data
variables:
HEPTAPOD_BUILD_CONTEXT: bootstrapped
extends:
- .build-image
chromium-image:
stage: image-webdriver
variables:
HEPTAPOD_BUILD_CONTEXT: chromium-driver
extends:
- .build-image
flake8:
stage: lint
# overkill, but should already be on all our runners
......@@ -46,7 +72,7 @@ docker-inside:
stage: test-heptapod
extends:
- .rules:test-heptapod
image: $HEPTAPOD_BASE_IMAGE:$TARGET_HEPTAPOD_IMAGE
image: $HEPTAPOD_CHROMIUM_IMAGE:$TARGET_HEPTAPOD_IMAGE
script:
- apt-get update
- apt-get -y install sudo
......@@ -76,7 +102,7 @@ docker-remote-0:
services:
# unfortunately variables are not expanded in service aliases
# so we need to duplicate
- name: $HEPTAPOD_BASE_IMAGE:$TARGET_HEPTAPOD_IMAGE
- name: $HEPTAPOD_BOOTSTRAPPED_IMAGE:$TARGET_HEPTAPOD_IMAGE
alias: heptapod-0
- name: selenium/standalone-chrome
alias: selenium-0
......@@ -90,7 +116,7 @@ docker-remote-1:
services:
# unfortunately variables are not expanded in service aliases
# so we need to duplicate
- name: $HEPTAPOD_BASE_IMAGE:$TARGET_HEPTAPOD_IMAGE
- name: $HEPTAPOD_BOOTSTRAPPED_IMAGE:$TARGET_HEPTAPOD_IMAGE
alias: heptapod-1
- name: selenium/standalone-chrome
alias: selenium-1
......@@ -109,7 +135,7 @@ docker-native-remote-0:
services:
# unfortunately variables are not expanded in service aliases
# so we need to duplicate
- name: $HEPTAPOD_BASE_IMAGE:$TARGET_HEPTAPOD_IMAGE
- name: $HEPTAPOD_BOOTSTRAPPED_IMAGE:$TARGET_HEPTAPOD_IMAGE
alias: heptapod-native-0
- name: selenium/standalone-chrome
alias: selenium-native-0
......@@ -124,7 +150,7 @@ docker-native-remote-1:
services:
# unfortunately variables are not expanded in service aliases
# so we need to duplicate
- name: $HEPTAPOD_BASE_IMAGE:$TARGET_HEPTAPOD_IMAGE
- name: $HEPTAPOD_BOOTSTRAPPED_IMAGE:$TARGET_HEPTAPOD_IMAGE
alias: heptapod-native-1
- name: selenium/standalone-chrome
alias: selenium-native-1
......
ARG boostrapped_image
ARG heptapod_image
FROM $bootstrapped_image
FROM $heptapod_image
# installing tox here will in particular bring a version of virtualenv that's suited
# for Python 3.8 instead of one that can't use Python 3.8 in --python option
......
#!/busybox/sh
set -eu
mkdir -p /kaniko/.docker
echo "{\"auths\":{\"$CI_REGISTRY\":{\"username\":\"$CI_REGISTRY_USER\",\"password\":\"$CI_REGISTRY_PASSWORD\"}}}" > /kaniko/.docker/config.json
# clearer naming in that context
IMAGE_TAG=${TARGET_HEPTAPOD_IMAGE}
# dispatching done explictely with if/elif because cascading
# variable definitions in .gitlab-ci.yml don't work. The most downstream one
# (defined in the job), using a global one itself based on a predefined
# such as CI_IMAGE_REGISTRY would have $CI_IMAGE_REGISTRY
# contain the unexpanded form of the first. This is also shorter on yml side.
if [ "$HEPTAPOD_BUILD_CONTEXT" = 'bootstrapped' ]; then
HEPTAPOD_FROM_IMAGE=$HEPTAPOD_BASE_IMAGE
HEPTAPOD_DESTINATION_IMAGE=$HEPTAPOD_BOOTSTRAPPED_IMAGE
elif [ "$HEPTAPOD_BUILD_CONTEXT" = 'chromium-driver' ]; then
HEPTAPOD_FROM_IMAGE=$HEPTAPOD_BOOTSTRAPPED_IMAGE
HEPTAPOD_DESTINATION_IMAGE=$HEPTAPOD_CHROMIUM_IMAGE
fi
HEPTAPOD_FROM_IMAGE_FULL=${HEPTAPOD_FROM_IMAGE}:${IMAGE_TAG}
HEPTAPOD_DESTINATION_IMAGE_FULL=${HEPTAPOD_DESTINATION_IMAGE}:${IMAGE_TAG}
echo "Building with kaniko, FROM ${HEPTAPOD_FROM_IMAGE_FULL}, "
echo "pushing to ${HEPTAPOD_DESTINATION_IMAGE_FULL}"
/kaniko/executor \
--context $CI_PROJECT_DIR/ci/docker/$HEPTAPOD_BUILD_CONTEXT \
--dockerfile $CI_PROJECT_DIR/ci/docker/$HEPTAPOD_BUILD_CONTEXT/Dockerfile \
--build-arg heptapod_image=${HEPTAPOD_FROM_IMAGE_FULL} \
--single-snapshot \
--cleanup \
--reproducible \
--destination $HEPTAPOD_DESTINATION_IMAGE_FULL
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment