Skip to content
Snippets Groups Projects
Commit 978e6c6a authored by DJ Mountney's avatar DJ Mountney
Browse files

Merge branch 'jarv/sync-to-gcp-storage-bucket' into 'master'

Syncs auto-deploy packages to a GCP bucket for SaaS download

See merge request https://gitlab.com/gitlab-org/omnibus-gitlab/-/merge_requests/6567



Merged-by: default avatarDJ Mountney <dj@gitlab.com>
Approved-by: default avatarDJ Mountney <dj@gitlab.com>
Co-authored-by: default avatarJohn Jarvis <jarv@gitlab.com>
parents 616b7d4e 089defa5
No related branches found
No related tags found
2 merge requests!102heptapod#1237: making 0.38 the new oldstable,!88Merging upstream 15.7.0+ce.0
......@@ -107,6 +107,8 @@
| RELEASE_BUCKET_REGION | S3 bucket region. |
| RELEASE_BUCKET_S3_ENDPOINT | Specify S3 endpoint. Especially useful when S3 compatible storage service is adopted. |
| GITLAB_BUNDLE_GEMFILE | Set Gemfile path required by `gitlab-rails` bundle. Default is `Gemfile`. |
| GITLAB_COM_PKGS_BUCKET | GCS bucket where release packages are pushed for SaaS deployments. |
| GITLAB_COM_PKGS_SA_FILE | Service account key used for pushing release packages for SaaS deployments, it must have write access to the pkgs bucket. |
## Unknown/outdated variables
......
......@@ -165,6 +165,14 @@
Gitlab::Util.get_env('RELEASE_BUCKET_S3_ENDPOINT') || "s3.amazonaws.com"
end
def gcp_release_bucket
Gitlab::Util.get_env('GITLAB_COM_PKGS_BUCKET') || "gitlab-com-pkgs"
end
def gcp_release_bucket_sa_file
Gitlab::Util.get_env('GITLAB_COM_PKGS_SA_FILE')
end
def log_level
if Gitlab::Util.get_env('BUILD_LOG_LEVEL') && !Gitlab::Util.get_env('BUILD_LOG_LEVEL').empty?
Gitlab::Util.get_env('BUILD_LOG_LEVEL')
......
require 'fileutils'
require_relative "../build.rb"
require_relative "../build/check.rb"
require_relative "../build/info.rb"
require_relative '../build/facts'
require_relative "../ohai_helper.rb"
......@@ -67,6 +68,6 @@
end
end
desc "Sync packages to aws"
desc "Sync packages to aws and gcp"
task :sync do
Gitlab::Util.section('build:package:sync', collapsed: Build::Check.on_tag?) do
......@@ -71,5 +72,13 @@
task :sync do
Gitlab::Util.section('build:package:sync', collapsed: Build::Check.on_tag?) do
pkgs_sa_file = Build::Info.gcp_release_bucket_sa_file
if pkgs_sa_file && Build::Check.is_auto_deploy_tag?
pkgs_bucket = Build::Info.gcp_release_bucket
puts 'GCS Sync: Activating service account'
system(*%W[gcloud auth activate-service-account --key-file #{pkgs_sa_file}])
puts "GCS Sync: Copying pkg/ contents to #{pkgs_bucket}"
system(*%W[gsutil -qm rsync -r pkg/ gs://#{pkgs_bucket}])
end
release_bucket = Build::Info.release_bucket
release_bucket_region = Build::Info.release_bucket_region
release_bucket_s3_endpoint = Build::Info.release_bucket_s3_endpoint
......@@ -73,6 +82,7 @@
release_bucket = Build::Info.release_bucket
release_bucket_region = Build::Info.release_bucket_region
release_bucket_s3_endpoint = Build::Info.release_bucket_s3_endpoint
puts "AWS S3 Sync: Copying pkg/ contents to #{release_bucket_s3_endpoint}"
system(*%W[aws s3 --endpoint-url https://#{release_bucket_s3_endpoint} sync pkg/ s3://#{release_bucket} --no-progress --acl public-read --region #{release_bucket_region}])
files = Dir.glob('pkg/**/*').select { |f| File.file? f }
files.each do |file|
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment