diff --git a/doc/architecture/index.md b/doc/architecture/index.md index fec51c2c05f758db95f8d532b5d34dd255b881cf_ZG9jL2FyY2hpdGVjdHVyZS9pbmRleC5tZA==..32a577a2051dcdab76adc63dd8c1169b690a08c7_ZG9jL2FyY2hpdGVjdHVyZS9pbmRleC5tZA== 100644 --- a/doc/architecture/index.md +++ b/doc/architecture/index.md @@ -232,10 +232,6 @@ 1. Create a snapshot and commit it. 1. Push back the new build cache to the CI cache. -### What happens during `gitlab-ctl reconfigure` - -One of the commonly used commands while managing a GitLab instance is `gitlab-ctl reconfigure`. This command, in short, parses the config file and runs the recipes with the values supplied from it. The recipes to be run are defined in a file called `dna.json` present in the `embedded` folder inside the installation directory (This file is generated by a software dependency named `gitlab-cookbooks` that's defined in the software definitions). In the case of GitLab CE, the cookbook named `gitlab` will be selected as the master recipe, which in turn invokes all other necessary recipes, including runit. In short, reconfigure is a chef-client run that configures different files and services with the values provided in the configuration template. - ## Multiple databases Previously, the GitLab Rails application was the sole client connected to the diff --git a/doc/development/index.md b/doc/development/index.md index fec51c2c05f758db95f8d532b5d34dd255b881cf_ZG9jL2RldmVsb3BtZW50L2luZGV4Lm1k..32a577a2051dcdab76adc63dd8c1169b690a08c7_ZG9jL2RldmVsb3BtZW50L2luZGV4Lm1k 100644 --- a/doc/development/index.md +++ b/doc/development/index.md @@ -31,3 +31,4 @@ ## Technical notes - [AWS AMis and Marketplace Listing](aws_amis_and_marketplace_listings.md) +- [Behind the scenes of `gitlab-ctl reconfigure`](reconfigure_in_detail.md) diff --git a/doc/development/reconfigure_in_detail.md b/doc/development/reconfigure_in_detail.md new file mode 100644 index 0000000000000000000000000000000000000000..32a577a2051dcdab76adc63dd8c1169b690a08c7_ZG9jL2RldmVsb3BtZW50L3JlY29uZmlndXJlX2luX2RldGFpbC5tZA== --- /dev/null +++ b/doc/development/reconfigure_in_detail.md @@ -0,0 +1,191 @@ +--- +stage: Systems +group: Distribution +info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/product/ux/technical-writing/#assignments +--- + +# What happens when `gitlab-ctl reconfigure` is run? + +Omnibus GitLab uses [Cinc](https://cinc.sh/) under the hood, +which is a free-as-in-beer distribution of the open source software of [Chef Software Inc](https://docs.chef.io/). + +In very basic terms, a [Cinc client](https://cinc.sh/start/client/) run +happens when `gitlab-ctl reconfigure` is run. This document elaborates +the process and details the flow of control during a `gitlab-ctl reconfigure` +run. + +`gitlab-ctl reconfigure` is defined in the +[`omnibus-ctl` project](https://gitlab.com/gitlab-org/build/omnibus-mirror/omnibus-ctl/-/blob/0.6.0.1/lib/omnibus-ctl.rb#L517) +and as mentioned above, it performs a +[`cinc-client` run](https://gitlab.com/gitlab-org/build/omnibus-mirror/omnibus-ctl/-/blob/0.6.0.1/lib/omnibus-ctl.rb#L501) +under the hood in [the local mode](https://docs.chef.io/ctl_chef_client/#run-in-local-mode) (using the `-z` flag). This invocation takes +two files as inputs: + +- A configuration file named [`solo.rb`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/solo.rb). +- An attribute file named `dna.json`, which is created during build time and loads: + - For CE, the [`gitlab`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab) cookbook. + - For EE, the [`gitlab-ee`](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab-ee) cookbook. + +Cinc then follows its [two-pass model of execution](https://coderanger.net/two-pass/) for the selected cookbook. + +In the load phase, the main cookbook and its dependency cookbooks (mentioned in +the `metadata.rb` file) are loaded. The attributes mentioned in the default +attribute files of these cookbooks are loaded (thus populating the `node` object +with the default values specified in those attribute files) and the custom +resources are all made available for use. Control then moves to the +execution phase. In the `dna.json` file, we specify the cookbook name as the +`run_list`, which makes Cinc use the default recipe as the only entry in the run +list. + +The `gitlab-ee` cookbook extends the `gitlab` cookbook with EE-only +features. For explanation purposes, let's first look at the +[`gitlab` cookbook's default recipe](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/gitlab/recipes/default.rb). + +## The default recipe + +The functionality in the default recipe can be summarized as below: + +1. Call the `config` recipe to load the attributes from `gitlab.rb` and fully + populate the `node` object. +1. Check for any deprecations and exit early. +1. Check for any problematic settings in the run environment. For example, + if `LD_LIBRARY_PATH` is defined it can interfere with the included + libraries, how software links against them, and raise warnings. +1. Check for non-UTF-8 locales and raise warnings. +1. Create and configure necessary base directories like `/etc/gitlab` (unless + explicitly disabled), `/var/opt/gitlab`, and `/var/log/gitlab`. +1. Call other necessary helper recipes and enable or disable recipes for different + services, etc. + +Note that this summary is not complete. Check out the default recipe code to +learn more. + +### The config recipe + +The config recipe populates the `node` object with the +final values for various settings after merging static default values, +computed default values, and user values specified +in `/etc/gitlab/gitlab.rb` file. + +In the above statement, we mention two types of default values: + +- **Static:** Static default values are specified in various + attribute files in different cookbooks and are independently set. +- **Computed:** Computed default values are used in scenarios where the + default value for a setting depends on either the static default + value or user-specified value of another setting. + +For example, `gitlab_rails['gitlab_port']` defaults to the static +value `80`. It translates to `production.gitlab.port` in the rendered +`gitlab.yml` file that configures the GitLab Rails listener port. The +`gitlab_rails['pages_host']` and `gitlab_rails['pages-port']` values, +which inform GitLab Rails about GitLab Pages, depend on the user +specified value in from `pages_external_url`. The computation of these +default values may only happen **_after_** `gitlab.rb` gets parsed. + +#### What goes in to the `gitlab.rb` file? + +Omnibus GitLab uses a module named `Gitlab` to store the settings specified in +`gitlab.rb`. This module, which extends `Mixlib::Config` module, can work as a +configuration hash. In the +[definition of this module](https://gitlab.com/gitlab-org/omnibus-gitlab/-/blob/master/files/gitlab-cookbooks/package/libraries/config/gitlab.rb), +we register the various roles, attribute blocks, and top-level attributes that +can be specified in `gitlab.rb`. The code for this registration is specified in +the `SettingsDSL` module, and is extended by `GitLab` module. + +When `Gitlab.from_file` is called in the config recipe, the settings from +`gitlab.rb` are parsed and loaded to the `Gitlab` object, and are accessible via +`Gitlab['<setting_name>']`. + +Once `gitlab.rb` has been parsed and its values are available in `Gitlab` +object, we can compute the default values of settings dependent on other settings. + +#### Computation of default values + +Each component with attributes that require calculation specify a library file +at registration. One method in this file, `parse_variables`, validates user-provided +input and, in typical usage, sets default values if the user did not already specify +a value. It also detects bad configuration and raises errors. + +As mentioned above, `parse_variables` sets default values based on static +defaults or user-provided values in related settings. The default values +from these related settings are available in the `node` object after the load +phase of the Cinc run. This `node` object, while available in the recipe, is +not available in the libraries. To make the static default values available +in the libraries, we attach the `node` object to the `GitLab` object in +the config recipe with the code below: + +```ruby +Gitlab[:node] = node +``` + +With this, the static default values of attributes can be accessed in the libraries +using `Gitlab[:node]['<top-level-key>'][<setting>]`. + +It is important to note that: + +- The `Gitlab` object stores keys as they are mentioned in `gitlab.rb`. +- `node` stores them based on the defined nesting attribute-block-attribute hierarchy. + +So, `gitlab_rails` settings from +`gitlab.rb` are available as `Gitlab['gitlab_rails'][*]` while default values of +those settings from attribute files are available at +`Gitlab[:node]['gitlab']['gitlab_rails']`. The `gitlab_rails` key is specified +under the `gitlab` attribute block, so an extra layer of nesting is present +while accessing it via `node`. + +While the `Gitlab` object is technically only supposed to hold the settings +specified in `gitlab.rb`, when computing default values of settings based on +other settings, we usually put them under `Gitlab` key itself. +[Issue #3932](https://gitlab.com/gitlab-org/omnibus-gitlab/-/issues/3923) is open to change +this behavior. + +#### Handling of secrets + +Many attributes required for GitLab functionality are secrets that need to +persist across reconfigure runs and, in multi-node setups, across different +nodes. Moreover, if the user did not specify values for these secrets then +Omnibus GitLab must create them to function. For this purpose, each library file +specifies a `parse_secrets` method similar to the `parse_variables` method. This +method generates secrets, unless explicitly disabled, if none have been specified +in the `gitlab.rb` file. + +These secrets are written, unless explicitly disabled, to a file named +`/etc/gitlab/gitlab-secrets.json`. This file is read by subsequent reconfigure +runs and the secret persists across every reconfigure run. + +#### Update the node object with final attribute list + +After all libraries parse their respective attributes and secrets, the +final configuration is ready to merge with default attributes already +present in `node`. The `node.consume_attributes` method merges the +final configuration with the default configuration populated in the load +phase. Any configuration read from `gitlab.rb` or computed in the libraries +overwrite the values for keys matched in the `node` object. At this point, +the `node` object contains the final attribute list. + +#### `gitlab-cluster.json` file + +A user configures the system with `gitlab.rb` to match the requirement. In +certain scenarios, however, we need to perform alterations without changes +to the `gitlab.rb` file. Omnibus GitLab writes to a different file, +`/etc/gitlab/gitlab-cluster.json`, that overrides user-specified values in +`gitlab.rb`. The `gitlab-ctl` command or a reconfigure dynamically populates +this file and it gets read and merged over the `node` attributes at the end +of the config recipe. + +The `gitlab-ctl geo promote` command, when used on a multi-node PostgreSQL +instance with Patroni, must disable the Patroni standby server. In this +example, the standby server would normally be disabled via +`patroni['standby_cluster']['enable']` in `gitlab.rb`. The `gitlab.rb` file +should remain read-only for the duration of the Cinc run, so this setting +is changed in the `gitlab-cluster.json` file. Future reconfigure runs parse +the `gitlab-cluster.json` file at the end and `node['patroni']['standby_cluster']['enable']` +will evaluate `false`. + +The Cinc run executes helper and service-specific recipes after +the config recipe. Once these are complete, the node object is +fully populated and may be used in recipes and resources. + +The default recipe in EE cookbook essentially calls `gitlab::default` recipe, +and then handles the EE-specific components separately. diff --git a/files/gitlab-cookbooks/gitlab-ee/recipes/default.rb b/files/gitlab-cookbooks/gitlab-ee/recipes/default.rb index fec51c2c05f758db95f8d532b5d34dd255b881cf_ZmlsZXMvZ2l0bGFiLWNvb2tib29rcy9naXRsYWItZWUvcmVjaXBlcy9kZWZhdWx0LnJi..32a577a2051dcdab76adc63dd8c1169b690a08c7_ZmlsZXMvZ2l0bGFiLWNvb2tib29rcy9naXRsYWItZWUvcmVjaXBlcy9kZWZhdWx0LnJi 100644 --- a/files/gitlab-cookbooks/gitlab-ee/recipes/default.rb +++ b/files/gitlab-cookbooks/gitlab-ee/recipes/default.rb @@ -15,6 +15,8 @@ # limitations under the License. # +# Add the EE specific services to the services list. Common services will be +# added in the `gitlab::config` recipe. Services.add_services('gitlab-ee', Services::EEServices.list) include_recipe 'gitlab::default' diff --git a/files/gitlab-cookbooks/gitlab/recipes/config.rb b/files/gitlab-cookbooks/gitlab/recipes/config.rb index fec51c2c05f758db95f8d532b5d34dd255b881cf_ZmlsZXMvZ2l0bGFiLWNvb2tib29rcy9naXRsYWIvcmVjaXBlcy9jb25maWcucmI=..32a577a2051dcdab76adc63dd8c1169b690a08c7_ZmlsZXMvZ2l0bGFiLWNvb2tib29rcy9naXRsYWIvcmVjaXBlcy9jb25maWcucmI= 100644 --- a/files/gitlab-cookbooks/gitlab/recipes/config.rb +++ b/files/gitlab-cookbooks/gitlab/recipes/config.rb @@ -15,5 +15,7 @@ # limitations under the License. # +# Make node object available to libraries to access default values specified in +# attribute files Gitlab[:node] = node @@ -18,4 +20,7 @@ Gitlab[:node] = node +# Populate the service list. When using `roles`, services will get +# automatically enabled. For that, services should be already present in the +# list. Services.add_services('gitlab', Services::BaseServices.list) @@ -20,4 +25,5 @@ Services.add_services('gitlab', Services::BaseServices.list) +# Parse `/etc/gitlab/gitlab.rb` and populate Gitlab object Gitlab.from_file('/etc/gitlab/gitlab.rb') if File.exist?('/etc/gitlab/gitlab.rb') @@ -22,4 +28,9 @@ Gitlab.from_file('/etc/gitlab/gitlab.rb') if File.exist?('/etc/gitlab/gitlab.rb') -node.consume_attributes(Gitlab.generate_config(node['fqdn'])) +# Generate config hash with settings specified in gitlab.rb, computed default +# values for settings via parse_variable method, and secrets either loaded from +# gitlab-secrets.json file or created anew. After this point, `Gitlab` object +# will have values either read from `/etc/gitlab/gitlab.rb` or computed by the +# libraries. +generated_config = Gitlab.generate_config(node['fqdn']) @@ -25,2 +36,7 @@ +# Populate node objects with the config hash generated above. After this point, +# the node objects will have the final values to be used in recipes. +node.consume_attributes(generated_config) + +# Override configuration with the one loaded from cluster.json file NodeHelper.consume_cluster_attributes(node, GitlabCluster.config.all)