Commit 65aadec8 authored by Georges Racinet's avatar Georges Racinet 🦑
Browse files

Merge with GitLab v13.5.4

This result passed all RSpec and functional tests on my
dev workstation.

Here follow the most notable changes that will need further
attention.

The upstream backup code has been entirely refactored, with a
new separation of concerns that has the Repository class take
care of the actual repo bundle production and restoration.
This is a breeze for Heptapod and its hierarchy of repository
classes. Not only is our specific code clearly delimited and
short (45 lines, including comments), but it is also more
future proof because we don't rely on spaggheti-like conditionals
any more. For instance, the day native projects don't include
a Git repository any more, we can simply provide appropriate
methods in HGitalyRepository. First shot implementation passed
the functional tests right away. It will need RSpec tests, though,
which are by the way now much easier to write as well.

There's been refactoring around Wikis, probably related to the
introduction of the Group Wikis. It was enough to pass the
tests to just delegate the two Heptapod-specific methods needed
for the raw repository instantiation, but this needs more scrutinity,
and probably Mercurial options for Group Wikis have to be
considered.

--HG--
branch : heptapod
Pipeline #14212 skipped with stages
...@@ -96,4 +96,4 @@ apollo.config.js ...@@ -96,4 +96,4 @@ apollo.config.js
/tmp/matching_foss_tests.txt /tmp/matching_foss_tests.txt
/tmp/matching_tests.txt /tmp/matching_tests.txt
ee/changelogs/unreleased-ee ee/changelogs/unreleased-ee
/sitespeed-result
...@@ -17,7 +17,7 @@ stages: ...@@ -17,7 +17,7 @@ stages:
# in cases where jobs require Docker-in-Docker, the job # in cases where jobs require Docker-in-Docker, the job
# definition must be extended with `.use-docker-in-docker` # definition must be extended with `.use-docker-in-docker`
default: default:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-84-node-12.x-yarn-1.21-postgresql-11-graphicsmagick-1.3.34" image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-85-node-12.18-yarn-1.22-postgresql-11-graphicsmagick-1.3.34"
tags: tags:
- gitlab-org - gitlab-org
# All jobs are interruptible by default # All jobs are interruptible by default
......
...@@ -184,7 +184,7 @@ Dangerfile @gl-quality/eng-prod ...@@ -184,7 +184,7 @@ Dangerfile @gl-quality/eng-prod
/lib/gitlab/auth/ldap/ @dblessing @mkozono /lib/gitlab/auth/ldap/ @dblessing @mkozono
[Templates] [Templates]
/lib/gitlab/ci/templates/ @nolith @dosuken123 /lib/gitlab/ci/templates/ @nolith @shinya.maeda
/lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml @DylanGriffith @mayra-cabrera @tkuah /lib/gitlab/ci/templates/Auto-DevOps.gitlab-ci.yml @DylanGriffith @mayra-cabrera @tkuah
/lib/gitlab/ci/templates/Security/ @plafoucriere @gonzoyumo @twoodham @sethgitlab /lib/gitlab/ci/templates/Security/ @plafoucriere @gonzoyumo @twoodham @sethgitlab
...@@ -213,10 +213,10 @@ Dangerfile @gl-quality/eng-prod ...@@ -213,10 +213,10 @@ Dangerfile @gl-quality/eng-prod
/ee/spec/lib/gitlab/code_owners/ @reprazent @kerrizor @garyh /ee/spec/lib/gitlab/code_owners/ @reprazent @kerrizor @garyh
/doc/user/project/code_owners.md @reprazent @kerrizor @garyh /doc/user/project/code_owners.md @reprazent @kerrizor @garyh
[Telemetry] [Product Analytics]
/ee/lib/gitlab/usage_data_counters/ @gitlab-org/growth/telemetry/engineers /ee/lib/gitlab/usage_data_counters/ @gitlab-org/growth/product_analytics/engineers
/ee/lib/ee/gitlab/usage_data.rb @gitlab-org/growth/telemetry/engineers /ee/lib/ee/gitlab/usage_data.rb @gitlab-org/growth/product_analytics/engineers
/lib/gitlab/grafana_embed_usage_data.rb @gitlab-org/growth/telemetry/engineers /lib/gitlab/grafana_embed_usage_data.rb @gitlab-org/growth/product_analytics/engineers
/lib/gitlab/usage_data.rb @gitlab-org/growth/telemetry/engineers /lib/gitlab/usage_data.rb @gitlab-org/growth/product_analytics/engineers
/lib/gitlab/cycle_analytics/usage_data.rb @gitlab-org/growth/telemetry/engineers /lib/gitlab/cycle_analytics/usage_data.rb @gitlab-org/growth/product_analytics/engineers
/lib/gitlab/usage_data_counters/ @gitlab-org/growth/telemetry/engineers /lib/gitlab/usage_data_counters/ @gitlab-org/growth/product_analytics/engineers
...@@ -28,6 +28,8 @@ ...@@ -28,6 +28,8 @@
# Help pages are excluded from scan as they are static pages. # Help pages are excluded from scan as they are static pages.
# profile/two_factor_auth is excluded from scan to prevent 2FA from being turned on from user profile, which will reduce coverage. # profile/two_factor_auth is excluded from scan to prevent 2FA from being turned on from user profile, which will reduce coverage.
- 'export DAST_AUTH_EXCLUDE_URLS="${DAST_WEBSITE}/help/.*,${DAST_WEBSITE}/profile/two_factor_auth,${DAST_WEBSITE}/users/sign_out"' - 'export DAST_AUTH_EXCLUDE_URLS="${DAST_WEBSITE}/help/.*,${DAST_WEBSITE}/profile/two_factor_auth,${DAST_WEBSITE}/users/sign_out"'
# Exclude the automatically generated monitoring project from being tested due to https://gitlab.com/gitlab-org/gitlab/-/issues/260362
- 'DAST_AUTH_EXCLUDE_URLS="${DAST_AUTH_EXCLUDE_URLS},https://.*\.gitlab-review\.app/gitlab-instance-(administrators-)?[a-zA-Z0-9]{8}/.*"'
- enable_rule () { read all_rules; rule=$1; echo $all_rules | sed -r "s/(,)?$rule(,)?/\1-1\2/" ; } - enable_rule () { read all_rules; rule=$1; echo $all_rules | sed -r "s/(,)?$rule(,)?/\1-1\2/" ; }
# Sort ids in DAST_RULES ascendingly, which is required when using DAST_RULES as argument to enable_rule # Sort ids in DAST_RULES ascendingly, which is required when using DAST_RULES as argument to enable_rule
- 'DAST_RULES=$(echo $DAST_RULES | tr "," "\n" | sort -n | paste -sd ",")' - 'DAST_RULES=$(echo $DAST_RULES | tr "," "\n" | sort -n | paste -sd ",")'
......
...@@ -14,14 +14,17 @@ ...@@ -14,14 +14,17 @@
SIZE: 0 # number of external projects to fork, requires network connection SIZE: 0 # number of external projects to fork, requires network connection
# SEED_NESTED_GROUPS: "false" # requires network connection # SEED_NESTED_GROUPS: "false" # requires network connection
.run-dev-fixtures-script: &run-dev-fixtures-script
- run_timed_command "scripts/gitaly-test-build"
- run_timed_command "scripts/gitaly-test-spawn"
- run_timed_command "RAILS_ENV=test bundle exec rake db:seed_fu"
run-dev-fixtures: run-dev-fixtures:
extends: extends:
- .run-dev-fixtures - .run-dev-fixtures
- .dev-fixtures:rules:ee-and-foss - .dev-fixtures:rules:ee-and-foss
script: script:
- run_timed_command "scripts/gitaly-test-build" - *run-dev-fixtures-script
- run_timed_command "scripts/gitaly-test-spawn"
- run_timed_command "RAILS_ENV=test bundle exec rake db:seed_fu"
run-dev-fixtures-ee: run-dev-fixtures-ee:
extends: extends:
...@@ -29,7 +32,5 @@ run-dev-fixtures-ee: ...@@ -29,7 +32,5 @@ run-dev-fixtures-ee:
- .dev-fixtures:rules:ee-only - .dev-fixtures:rules:ee-only
- .use-pg11-ee - .use-pg11-ee
script: script:
- run_timed_command "scripts/gitaly-test-build"
- run_timed_command "scripts/gitaly-test-spawn"
- cp ee/db/fixtures/development/* $FIXTURE_PATH - cp ee/db/fixtures/development/* $FIXTURE_PATH
- run_timed_command "RAILS_ENV=test bundle exec rake db:seed_fu" - *run-dev-fixtures-script
...@@ -4,7 +4,7 @@ ...@@ -4,7 +4,7 @@
- .docs:rules:review-docs - .docs:rules:review-docs
image: ruby:2.6-alpine image: ruby:2.6-alpine
stage: review stage: review
dependencies: [] needs: []
variables: variables:
# We're cloning the repo instead of downloading the script for now # We're cloning the repo instead of downloading the script for now
# because some repos are private and CI_JOB_TOKEN cannot access files. # because some repos are private and CI_JOB_TOKEN cannot access files.
...@@ -42,7 +42,7 @@ docs lint: ...@@ -42,7 +42,7 @@ docs lint:
extends: extends:
- .default-retry - .default-retry
- .docs:rules:docs-lint - .docs:rules:docs-lint
image: "registry.gitlab.com/gitlab-org/gitlab-docs/lint:vale-2.3.4-markdownlint-0.23.2" image: "registry.gitlab.com/gitlab-org/gitlab-docs/lint:ruby-2.7.2-alpine-3.12-vale-2.4.3-markdownlint-0.24.0"
stage: test stage: test
needs: [] needs: []
script: script:
......
...@@ -7,19 +7,21 @@ ...@@ -7,19 +7,21 @@
# we override the max_old_space_size to prevent OOM errors # we override the max_old_space_size to prevent OOM errors
NODE_OPTIONS: --max_old_space_size=3584 NODE_OPTIONS: --max_old_space_size=3584
.yarn-install: &yarn-install
- source scripts/utils.sh
- run_timed_command "retry yarn install --frozen-lockfile"
.compile-assets-base: .compile-assets-base:
extends: extends:
- .frontend-base - .frontend-base
- .assets-compile-cache - .assets-compile-cache
image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-git-2.28-lfs-2.9-node-12.x-yarn-1.21-graphicsmagick-1.3.34 image: registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-git-2.28-lfs-2.9-node-12.18-yarn-1.22-graphicsmagick-1.3.34
variables: variables:
WEBPACK_VENDOR_DLL: "true" WEBPACK_VENDOR_DLL: "true"
stage: prepare stage: prepare
script: script:
- node --version - *yarn-install
- run_timed_command "retry yarn install --frozen-lockfile" - run_timed_command "bin/rake gitlab:assets:compile"
- free -m
- run_timed_command "bin/rake gitlab:assets:compile > assets-compile.log 2>&1"
- run_timed_command "scripts/clean-old-cached-assets" - run_timed_command "scripts/clean-old-cached-assets"
compile-production-assets: compile-production-assets:
...@@ -34,7 +36,6 @@ compile-production-assets: ...@@ -34,7 +36,6 @@ compile-production-assets:
name: webpack-report name: webpack-report
expire_in: 31d expire_in: 31d
paths: paths:
- assets-compile.log
# These assets are used in multiple locations: # These assets are used in multiple locations:
# - in `build-assets-image` job to create assets image for packaging systems # - in `build-assets-image` job to create assets image for packaging systems
# - GitLab UI for integration tests: https://gitlab.com/gitlab-org/gitlab-ui/-/blob/e88493b3c855aea30bf60baee692a64606b0eb1e/.storybook/preview-head.pug#L1 # - GitLab UI for integration tests: https://gitlab.com/gitlab-org/gitlab-ui/-/blob/e88493b3c855aea30bf60baee692a64606b0eb1e/.storybook/preview-head.pug#L1
...@@ -51,7 +52,6 @@ compile-test-assets: ...@@ -51,7 +52,6 @@ compile-test-assets:
artifacts: artifacts:
expire_in: 7d expire_in: 7d
paths: paths:
- assets-compile.log
- public/assets/ - public/assets/
- node_modules/@gitlab/svgs/dist/icons.json # app/helpers/icons_helper.rb uses this file - node_modules/@gitlab/svgs/dist/icons.json # app/helpers/icons_helper.rb uses this file
when: always when: always
...@@ -87,8 +87,7 @@ update-yarn-cache: ...@@ -87,8 +87,7 @@ update-yarn-cache:
- .shared:rules:update-cache - .shared:rules:update-cache
stage: prepare stage: prepare
script: script:
- source scripts/utils.sh - *yarn-install
- run_timed_command "retry yarn install --frozen-lockfile"
cache: cache:
policy: push policy: push
...@@ -139,14 +138,14 @@ eslint-as-if-foss: ...@@ -139,14 +138,14 @@ eslint-as-if-foss:
- .as-if-foss - .as-if-foss
needs: [] needs: []
script: script:
- run_timed_command "retry yarn install --frozen-lockfile" - *yarn-install
- yarn run eslint - run_timed_command "yarn run eslint"
.karma-base: .karma-base:
extends: .frontend-test-base extends: .frontend-test-base
script: script:
- export BABEL_ENV=coverage CHROME_LOG_FILE=chrome_debug.log - export BABEL_ENV=coverage CHROME_LOG_FILE=chrome_debug.log
- run_timed_command "retry yarn install --frozen-lockfile" - *yarn-install
- run_timed_command "yarn karma" - run_timed_command "yarn karma"
karma: karma:
...@@ -177,7 +176,7 @@ karma-as-if-foss: ...@@ -177,7 +176,7 @@ karma-as-if-foss:
.jest-base: .jest-base:
extends: .frontend-test-base extends: .frontend-test-base
script: script:
- run_timed_command "retry yarn install --frozen-lockfile" - *yarn-install
- run_timed_command "yarn jest --ci --coverage --testSequencer ./scripts/frontend/parallel_ci_sequencer.js" - run_timed_command "yarn jest --ci --coverage --testSequencer ./scripts/frontend/parallel_ci_sequencer.js"
jest: jest:
...@@ -202,7 +201,7 @@ jest-integration: ...@@ -202,7 +201,7 @@ jest-integration:
- .frontend-test-base - .frontend-test-base
- .frontend:rules:default-frontend-jobs - .frontend:rules:default-frontend-jobs
script: script:
- run_timed_command "retry yarn install --frozen-lockfile" - *yarn-install
- run_timed_command "yarn jest:integration --ci" - run_timed_command "yarn jest:integration --ci"
needs: ["frontend-fixtures"] needs: ["frontend-fixtures"]
...@@ -222,8 +221,7 @@ coverage-frontend: ...@@ -222,8 +221,7 @@ coverage-frontend:
needs: ["jest"] needs: ["jest"]
stage: post-test stage: post-test
before_script: before_script:
- source scripts/utils.sh - *yarn-install
- run_timed_command "retry yarn install --frozen-lockfile"
script: script:
- run_timed_command "yarn node scripts/frontend/merge_coverage_frontend.js" - run_timed_command "yarn node scripts/frontend/merge_coverage_frontend.js"
coverage: '/^Statements\s*:\s*?(\d+(?:\.\d+)?)%/' coverage: '/^Statements\s*:\s*?(\d+(?:\.\d+)?)%/'
...@@ -243,9 +241,8 @@ coverage-frontend: ...@@ -243,9 +241,8 @@ coverage-frontend:
stage: test stage: test
dependencies: [] dependencies: []
script: script:
- source scripts/utils.sh - *yarn-install
- run_timed_command "yarn install --frozen-lockfile" - run_timed_command "retry yarn run webpack-prod"
- run_timed_command "yarn run webpack-prod"
qa-frontend-node:10: qa-frontend-node:10:
extends: .qa-frontend-node extends: .qa-frontend-node
...@@ -268,8 +265,7 @@ webpack-dev-server: ...@@ -268,8 +265,7 @@ webpack-dev-server:
WEBPACK_MEMORY_TEST: "true" WEBPACK_MEMORY_TEST: "true"
WEBPACK_VENDOR_DLL: "true" WEBPACK_VENDOR_DLL: "true"
script: script:
- source scripts/utils.sh - *yarn-install
- run_timed_command "retry yarn install --frozen-lockfile"
- run_timed_command "retry yarn webpack-vendor" - run_timed_command "retry yarn webpack-vendor"
- run_timed_command "node --expose-gc node_modules/.bin/webpack-dev-server --config config/webpack.config.js" - run_timed_command "node --expose-gc node_modules/.bin/webpack-dev-server --config config/webpack.config.js"
artifacts: artifacts:
......
...@@ -38,6 +38,13 @@ ...@@ -38,6 +38,13 @@
- tmp/rubocop_cache/ - tmp/rubocop_cache/
policy: pull policy: pull
.coverage-cache:
cache:
key: "coverage-cache-v1"
paths:
- vendor/ruby/
policy: pull
.qa-cache: .qa-cache:
cache: cache:
key: "qa-v1" key: "qa-v1"
...@@ -77,7 +84,7 @@ ...@@ -77,7 +84,7 @@
POSTGRES_HOST_AUTH_METHOD: trust POSTGRES_HOST_AUTH_METHOD: trust
.use-pg12: .use-pg12:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-84-node-12.x-yarn-1.21-postgresql-12-graphicsmagick-1.3.34" image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-85-node-12.18-yarn-1.22-postgresql-12-graphicsmagick-1.3.34"
services: services:
- name: postgres:12 - name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
...@@ -86,22 +93,24 @@ ...@@ -86,22 +93,24 @@
POSTGRES_HOST_AUTH_METHOD: trust POSTGRES_HOST_AUTH_METHOD: trust
.use-pg11-ee: .use-pg11-ee:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-84-node-12.x-yarn-1.21-postgresql-11-graphicsmagick-1.3.34" image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-85-node-12.18-yarn-1.22-postgresql-11-graphicsmagick-1.3.34"
services: services:
- name: postgres:11.6 - name: postgres:11.6
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:4.0-alpine - name: redis:4.0-alpine
- name: elasticsearch:6.4.2 - name: elasticsearch:7.9.2
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables: variables:
POSTGRES_HOST_AUTH_METHOD: trust POSTGRES_HOST_AUTH_METHOD: trust
.use-pg12-ee: .use-pg12-ee:
image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-84-node-12.x-yarn-1.21-postgresql-12-graphicsmagick-1.3.34" image: "registry.gitlab.com/gitlab-org/gitlab-build-images:ruby-2.6.6-golang-1.14-git-2.28-lfs-2.9-chrome-85-node-12.18-yarn-1.22-postgresql-12-graphicsmagick-1.3.34"
services: services:
- name: postgres:12 - name: postgres:12
command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"] command: ["postgres", "-c", "fsync=off", "-c", "synchronous_commit=off", "-c", "full_page_writes=off"]
- name: redis:4.0-alpine - name: redis:4.0-alpine
- name: elasticsearch:6.4.2 - name: elasticsearch:7.9.2
command: ["elasticsearch", "-E", "discovery.type=single-node"]
variables: variables:
POSTGRES_HOST_AUTH_METHOD: trust POSTGRES_HOST_AUTH_METHOD: trust
......
...@@ -3,6 +3,8 @@ ...@@ -3,6 +3,8 @@
stage: notify stage: notify
dependencies: [] dependencies: []
cache: {} cache: {}
variables:
MERGE_REQUEST_URL: ${CI_MERGE_REQUEST_PROJECT_URL}/-/merge_requests/${CI_MERGE_REQUEST_IID}
before_script: before_script:
- apk update && apk add git curl bash - apk update && apk add git curl bash
...@@ -16,8 +18,19 @@ notify-update-gitaly: ...@@ -16,8 +18,19 @@ notify-update-gitaly:
variables: variables:
NOTIFY_CHANNEL: g_create_gitaly NOTIFY_CHANNEL: g_create_gitaly
GITALY_UPDATE_BRANCH: release-tools/update-gitaly GITALY_UPDATE_BRANCH: release-tools/update-gitaly
MERGE_REQUEST_URL: ${CI_MERGE_REQUEST_PROJECT_URL}/-/merge_requests/${CI_MERGE_REQUEST_IID}
script: script:
- echo "NOTIFY_CHANNEL is ${NOTIFY_CHANNEL}" - echo "NOTIFY_CHANNEL is ${NOTIFY_CHANNEL}"
- echo "CI_PIPELINE_URL is ${CI_PIPELINE_URL}" - echo "CI_PIPELINE_URL is ${CI_PIPELINE_URL}"
- scripts/slack ${NOTIFY_CHANNEL} "☠️ \`${GITALY_UPDATE_BRANCH}\` failed! ☠️ See ${CI_PIPELINE_URL} (triggered from ${MERGE_REQUEST_URL})" ci_failing - scripts/slack ${NOTIFY_CHANNEL} "☠️ \`${GITALY_UPDATE_BRANCH}\` failed! ☠️ See ${CI_PIPELINE_URL} (triggered from ${MERGE_REQUEST_URL})" ci_failing "GitLab QA Bot"
notify-security-pipeline:
extends:
- .notify-slack
- .delivery:rules:security-pipeline-merge-result-failure
variables:
NOTIFY_CHANNEL: f_upcoming_release
script:
- echo "NOTIFY_CHANNEL is ${NOTIFY_CHANNEL}"
- echo "CI_PIPELINE_URL is ${CI_PIPELINE_URL}"
# <!subteam^S0127FU8PDE> mentions the `@release-managers` group
- scripts/slack ${NOTIFY_CHANNEL} "<!subteam^S0127FU8PDE> ☠️ Pipeline for merged result failed! ☠️ See ${CI_PIPELINE_URL} (triggered from ${MERGE_REQUEST_URL})" ci_failing "GitLab Release Tools Bot"
...@@ -6,14 +6,23 @@ ...@@ -6,14 +6,23 @@
- .default-before_script - .default-before_script
- .rails-cache - .rails-cache
.base-script: &base-script
# Only install knapsack after bundle install! Otherwise oddly some native
# gems could not be found under some circumstance. No idea why, hours wasted.
- run_timed_command "gem install knapsack --no-document"
- run_timed_command "scripts/gitaly-test-build"
- run_timed_command "scripts/gitaly-test-spawn"
- source ./scripts/rspec_helpers.sh
.rspec-base: .rspec-base:
extends: .rails-job-base extends: .rails-job-base
stage: test stage: test
variables:
RUBY_GC_MALLOC_LIMIT: 67108864
RUBY_GC_MALLOC_LIMIT_MAX: 134217728
needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets"] needs: ["setup-test-env", "retrieve-tests-metadata", "compile-test-assets"]
script: script:
- run_timed_command "scripts/gitaly-test-build" - *base-script
- run_timed_command "scripts/gitaly-test-spawn"
- source scripts/rspec_helpers.sh
- rspec_paralellized_job "--tag ~quarantine --tag ~geo --tag ~level:migration" - rspec_paralellized_job "--tag ~quarantine --tag ~geo --tag ~level:migration"
artifacts: artifacts:
expire_in: 31d expire_in: 31d
...@@ -25,6 +34,7 @@ ...@@ -25,6 +34,7 @@
- rspec_profiling/ - rspec_profiling/
- tmp/capybara/ - tmp/capybara/
- tmp/memory_test/ - tmp/memory_test/
- tmp/feature_flags/
- log/*.log - log/*.log
reports: reports:
junit: junit_rspec.xml junit: junit_rspec.xml
...@@ -32,9 +42,7 @@ ...@@ -32,9 +42,7 @@
.rspec-base-migration: .rspec-base-migration:
extends: .rails:rules:ee-and-foss-migration extends: .rails:rules:ee-and-foss-migration
script: script:
- run_timed_command "scripts/gitaly-test-build" - *base-script
- run_timed_command "scripts/gitaly-test-spawn"
- source scripts/rspec_helpers.sh
- rspec_paralellized_job "--tag ~quarantine --tag ~geo --tag level:migration" - rspec_paralellized_job "--tag ~quarantine --tag ~geo --tag level:migration"
.rspec-base-pg11: .rspec-base-pg11:
...@@ -67,9 +75,7 @@ ...@@ -67,9 +75,7 @@
.rspec-ee-base-geo: .rspec-ee-base-geo:
extends: .rspec-base extends: .rspec-base
script: script:
- run_timed_command "scripts/gitaly-test-build" - *base-script
- run_timed_command "scripts/gitaly-test-spawn"
- source scripts/rspec_helpers.sh
- rspec_paralellized_job "--tag ~quarantine --tag geo" - rspec_paralellized_job "--tag ~quarantine --tag geo"
.rspec-ee-base-geo-pg11: .rspec-ee-base-geo-pg11:
...@@ -160,6 +166,25 @@ update-rails-cache: ...@@ -160,6 +166,25 @@ update-rails-cache:
cache: cache:
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up. policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.coverage-base:
extends:
- .default-retry
- .default-before_script
- .coverage-cache
variables:
SETUP_DB: "false"
USE_BUNDLE_INSTALL: "false"
update-coverage-cache:
extends:
- .coverage-base
- .shared:rules:update-cache
stage: prepare
script:
- run_timed_command "bundle install --jobs=$(nproc) --path=vendor --retry=3 --quiet --without default development test production puma unicorn kerberos metrics omnibus ed25519"
cache:
policy: push # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up.
.static-analysis-base: .static-analysis-base:
extends: extends:
- .default-retry - .default-retry
...@@ -178,7 +203,7 @@ update-static-analysis-cache: ...@@ -178,7 +203,7 @@ update-static-analysis-cache:
script: script:
- rm -rf ./node_modules # We remove node_modules because there's no mechanism to remove stall entries. - rm -rf ./node_modules # We remove node_modules because there's no mechanism to remove stall entries.
- run_timed_command "retry yarn install --frozen-lockfile" - run_timed_command "retry yarn install --frozen-lockfile"
- bundle exec rubocop --parallel # For the moment we only cache `vendor/ruby/`, `node_modules/`, and `tmp/rubocop_cache` so we don't need to run all the tasks, - run_timed_command "bundle exec rubocop --parallel" # For the moment we only cache `vendor/ruby/`, `node_modules/`, and `tmp/rubocop_cache` so we don't need to run all the tasks,
cache: cache:
# We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up but RuboCop has a mechanism # We want to rebuild the cache from scratch to ensure stale dependencies are cleaned up but RuboCop has a mechanism
# for keeping only the N latest cache files, so we take advantage of it with `pull-push` and removing `node_modules` at the start of the job. # for keeping only the N latest cache files, so we take advantage of it with `pull-push` and removing `node_modules` at the start of the job.
...@@ -287,8 +312,7 @@ gitlab:setup: ...@@ -287,8 +312,7 @@ gitlab:setup:
# db/fixtures/development/04_project.rb thanks to SIZE=1 below # db/fixtures/development/04_project.rb thanks to SIZE=1 below
- git clone https://gitlab.com/gitlab-org/gitlab-test.git - git clone https://gitlab.com/gitlab-org/gitlab-test.git
/home/git/repositories/gitlab-org/gitlab-test.git /home/git/repositories/gitlab-org/gitlab-test.git
- run_timed_command "scripts/gitaly-test-build" - *base-script
- run_timed_command "scripts/gitaly-test-spawn"
- force=yes SIZE=1 FIXTURE_PATH="db/fixtures/development" bundle exec rake gitlab:setup - force=yes SIZE=1 FIXTURE_PATH="db/fixtures/development" bundle exec rake gitlab:setup
artifacts: artifacts:
when: on_failure when: on_failure
...@@ -313,7 +337,7 @@ db:backup_and_restore: ...@@ -313,7 +337,7 @@ db:backup_and_restore:
rspec:coverage: rspec:coverage:
extends: extends:
- .rails-job-base - .coverage-base
- .rails:rules:rspec-coverage - .rails:rules:rspec-coverage
stage: post-test stage: post-test
# We cannot use needs since it would mean needing 84 jobs (since most are parallelized) # We cannot use needs since it would mean needing 84 jobs (since most are parallelized)
...@@ -333,11 +357,10 @@ rspec:coverage: ...@@ -333,11 +357,10 @@ rspec:coverage:
- rspec-ee system pg11 geo - rspec-ee system pg11 geo
- memory-static - memory-static
- memory-on-boot - memory-on-boot
variables:
SETUP_DB: "false"
script: script:
- bundle exec scripts/merge-simplecov - run_timed_command "bundle install --jobs=$(nproc) --path=vendor --retry=3 --quiet --without default development test production puma unicorn kerberos metrics omnibus ed25519"
- bundle exec scripts/gather-test-memory-data - run_timed_command "bundle exec scripts/merge-simplecov"
- run_timed_command "bundle exec scripts/gather-test-memory-data"
coverage: '/LOC \((\d+\.\d+%)\) covered.$/' coverage: '/LOC \((\d+\.\d+%)\) covered.$/'
artifacts: artifacts:
name: coverage name: coverage
...@@ -348,6 +371,32 @@ rspec:coverage: ...@@ -348,6 +371,32 @@ rspec:coverage:
- tmp/memory_test/ - tmp/memory_test/
reports: reports:
cobertura: coverage/coverage.xml cobertura: coverage/coverage.xml
rspec:feature-flags:
extends:
- .coverage-base
- .rails:rules:rspec-feature-flags
stage: post-test
# We cannot use needs since it would mean needing 84 jobs (since most are parallelized)
# so we use `dependencies` here.
dependencies:
- setup-test-env
- rspec migration pg11
- rspec unit pg11
- rspec integration pg11
- rspec system pg11
- rspec-ee migration pg11
- rspec-ee unit pg11