Skip to content
Snippets Groups Projects
Commit d791bffa authored by Ian Baum's avatar Ian Baum
Browse files

Merge branch 'docs/registry-garbage-collection' into 'master'

Refactor the Container Registry garbage collection docs

See merge request gitlab-org/omnibus-gitlab!3229
parents e28d8a5e 91157562
No related branches found
No related tags found
No related merge requests found
# Maintenance commands
## After installation
The following commands can be run after installation.
......@@ -4,5 +4,5 @@
### Get service status
## Get service status
Run `sudo gitlab-ctl status`; the output should look like this:
......@@ -14,7 +14,7 @@
run: unicorn: (pid 961) 7s; run: log: (pid 960) 7s
```
### Tail process logs
## Tail process logs
See [settings/logs.md.](../settings/logs.md)
......@@ -18,7 +18,7 @@
See [settings/logs.md.](../settings/logs.md)
### Starting and stopping
## Starting and stopping
After omnibus-gitlab is installed and configured, your server will have a Runit
service directory (`runsvdir`) process running that gets started at boot via
......@@ -58,7 +58,7 @@
Note that you cannot use a Unicorn reload to update the Ruby runtime.
### Invoking Rake tasks
## Invoking Rake tasks
To invoke a GitLab Rake task, use `gitlab-rake`. For example:
......@@ -72,7 +72,7 @@
the user or the `RAILS_ENV` environment variable; this is taken care of by the
`gitlab-rake` wrapper script.
### Starting a Rails console session
## Starting a Rails console session
If you need access to a Rails production console for your GitLab installation
you can start one with the command below. Please be warned that it is very easy
......@@ -85,7 +85,7 @@
This will only work after you have run `gitlab-ctl reconfigure` at least once.
### Starting a Postgres superuser psql session
## Starting a Postgres superuser psql session
If you need superuser access to the bundled Postgres service you can
use the `gitlab-psql` command. It takes the same arguments as the
......@@ -101,7 +101,7 @@
remote Postgres server, nor to connect to a local non-Omnibus Postgres
server.
#### Starting a Postgres superuser psql session in Geo tracking database
### Starting a Postgres superuser psql session in Geo tracking database
Similar to the previous command, if you need superuser access to the bundled
Geo tracking database (`geo-postgresql`), you can use the `gitlab-geo-psql`.
......@@ -112,5 +112,5 @@
sudo gitlab-geo-psql -d gitlabhq_geo_production
```
### Container registry garbage collection
## Container Registry garbage collection
......@@ -116,4 +116,4 @@
Container registry can use considerable amounts of disk space. To clear up
Container Registry can use considerable amounts of disk space. To clear up
some unused layers, registry includes a garbage collect command.
......@@ -118,17 +118,6 @@
some unused layers, registry includes a garbage collect command.
There are a couple of considerations you need to note before running the
built in command:
* The built in command will stop the registry before it starts garbage collect
* The garbage collect command takes some time to complete, depending on the
amount of data that exists
* If you changed the location of registry configuration file, you will need to
specify the path
* After the garbage collect is done, registry should start up automatically
**Warning** The command below will cause Container registry downtime.
If you did not change the default location of the configuration file, to do
garbage collection:
GitLab offers a set of APIs to manipulate the Container Registry and aid the process
of removing unused tags. Currently, this is exposed using the API, but in the future,
these controls will be migrated to the GitLab interface.
......@@ -134,10 +123,5 @@
```
sudo gitlab-ctl registry-garbage-collect
```
This command will take some time to complete, depending on the amount of
layers you have stored.
If you changed the location of the Container registry config.yml:
Project maintainers can
[delete Container Registry tags in bulk](https://docs.gitlab.com/ce/api/container_registry.html#delete-repository-tags-in-bulk)
periodically based on their own criteria.
......@@ -143,7 +127,5 @@
```
sudo gitlab-ctl registry-garbage-collect /path/to/config.yml
```
#### Removing unused layers not referenced by manifests
However, this alone does not recycle data, it only unlinks tags from manifests
and image blobs. To recycle the Container Registry data in the whole GitLab
instance, you can use the built-in command provided by `gitlab-ctl`.
......@@ -149,10 +131,6 @@
GitLab Container Registry follows the same default workflow as Docker Distribution:
retain all layers, even ones that are unreferenced directly to allow all content
to be accessed using context addressable identifiers.
#### Content-addressable layers
### Understanding the content-addressable layers
Consider the following example, where you first build the image:
```bash
......@@ -155,5 +133,6 @@
Consider the following example, where you first build the image:
```bash
# This builds a image with content of sha256:111111
docker build -t my.registry.com/my.group/my.project:latest .
......@@ -159,8 +138,7 @@
docker build -t my.registry.com/my.group/my.project:latest .
# this builds a image with content of sha256:111111
docker push my.registry.com/my.group/my.project:latest
```
Now, you do overwrite `:latest` with a new version:
```bash
......@@ -161,7 +139,8 @@
docker push my.registry.com/my.group/my.project:latest
```
Now, you do overwrite `:latest` with a new version:
```bash
# This builds a image with content of sha256:222222
docker build -t my.registry.com/my.group/my.project:latest .
......@@ -167,5 +146,4 @@
docker build -t my.registry.com/my.group/my.project:latest .
# this builds a image with content of sha256:222222
docker push my.registry.com/my.group/my.project:latest
```
......@@ -169,8 +147,8 @@
docker push my.registry.com/my.group/my.project:latest
```
Now, the `:latest` points to manifest of `sha256:222222`. However, due to architecture
of registry this data is still accessible via:
`docker pull my.registry.com/my.group/my.project@sha256:111111` even though it is
no longer directly accessible via `:latest` tag.
Now, the `:latest` tag points to manifest of `sha256:222222`. However, due to
the architecture of registry, this data is still accessible when pulling the
image `my.registry.com/my.group/my.project@sha256:111111`, even though it is
no longer directly accessible via the `:latest` tag.
......@@ -176,3 +154,6 @@
#### Recycle unreference manifests
### Recycling unused tags
There are a couple of considerations you need to note before running the
built-in command:
......@@ -178,6 +159,8 @@
However, in most of workflows you do not care about old layers, if they are not directly
referenced by registry tag. The `registry-garbage-collect` supports the `-m` switch
to allow you to remove all unreferenced manifests and layers, that are not directly
accessible via `tag`.
- The built-in command will stop the registry before it starts the garbage collection.
- The garbage collect command takes some time to complete, depending on the
amount of data that exists.
- If you changed the location of registry configuration file, you will need to
specify its path.
- After the garbage collection is done, the registry should start up automatically.
......@@ -183,5 +166,7 @@
Since this is a way more destrictive operation, this behavior is disabled by default.
You are likely expecting this way of operation, but before doing that ensure
that you backup all registry data to ensure that you do not use the data.
CAUTION: **Warning:**
By running the built-in garbage collection command, it will cause downtime to
the Container Registry. To avoid that, you can [use another method](#performing-garbage-collection-without-downtime).
If you did not change the default location of the configuration file, run:
......@@ -187,3 +172,8 @@
This will allow you to recycle all registry space with the [Container Registry API](#administratively-recycling-unused-tags).
```sh
sudo gitlab-ctl registry-garbage-collect
```
This command will take some time to complete, depending on the amount of
layers you have stored.
......@@ -189,3 +179,7 @@
#### Doing garbage collect without downtime
If you changed the location of the Container Registry `config.yml`:
```sh
sudo gitlab-ctl registry-garbage-collect /path/to/config.yml
```
......@@ -191,5 +185,5 @@
You can do a garbage collect without stopping the Container registry by setting
it into a read only mode. During this time, you will be able to pull from
the Container registry but you will not be able to push.
You may also [remove all unreferenced manifests](#removing-unused-layers-not-referenced-by-manifests),
although this is a way more destructive operation, and you should first
understand the implications.
......@@ -195,3 +189,6 @@
These are the steps you need to take in order to complete the garbage collection:
### Removing unused layers not referenced by manifests
> [Introduced](https://gitlab.com/gitlab-org/omnibus-gitlab/merge_requests/3097)
in Omnibus GitLab 11.10.
......@@ -197,3 +194,8 @@
In `/etc/gitlab/gitlab.rb` specify the read only mode:
DANGER: **Danger:**
This is a destructive operation.
The GitLab Container Registry follows the same default workflow as Docker Distribution:
retain all layers, even ones that are unreferenced directly to allow all content
to be accessed using context addressable identifiers.
......@@ -199,11 +201,10 @@
```ruby
registry['storage'] = {
'maintenance' => {
'readonly' => {
'enabled' => true
}
}
}
However, in most workflows, you don't care about old layers if they are not directly
referenced by the registry tag. The `registry-garbage-collect` command supports the
`-m` switch to allow you to remove all unreferenced manifests and layers that are
not directly accessible via `tag`:
```sh
sudo gitlab-ctl registry-garbage-collect -m
```
......@@ -208,5 +209,8 @@
```
Save and run `sudo gitlab-ctl reconfigure`. This will set the Container registry
into the read only mode.
Since this is a way more destructive operation, this behavior is disabled by default.
You are likely expecting this way of operation, but before doing that, ensure
that you have backed up all registry data.
### Performing garbage collection without downtime
......@@ -212,3 +216,8 @@
Next, trigger the garbage collect command:
You can perform a garbage collection without stopping the Container Registry by setting
it into a read-only mode and by not using the built-in command. During this time,
you will be able to pull from the Container Registry, but you will not be able to
push. To enable the read-only mode:
1. In `/etc/gitlab/gitlab.rb`, specify the read-only mode:
......@@ -214,5 +223,11 @@
```
sudo /opt/gitlab/embedded/bin/registry garbage-collect /var/opt/gitlab/registry/config.yml
```
```ruby
registry['storage'] = {
'maintenance' => {
'readonly' => {
'enabled' => true
}
}
}
```
......@@ -218,3 +233,3 @@
This will start the garbage collection. The command will take some time to complete.
1. Save and reconfigure GitLab:
......@@ -220,3 +235,7 @@
Once done, in `/etc/gitlab/gitlab.rb` change the configuration to:
```sh
sudo gitlab-ctl reconfigure
```
This will set the Container Registry into the read only mode.
......@@ -222,11 +241,11 @@
```ruby
registry['storage'] = {
'maintenance' => {
'readonly' => {
'enabled' => false
}
}
}
```
1. Next, trigger the garbage collect command:
```sh
sudo /opt/gitlab/embedded/bin/registry garbage-collect /var/opt/gitlab/registry/config.yml
```
This will start the garbage collection, which might take some time to complete.
1. Once done, in `/etc/gitlab/gitlab.rb` change it back to read-write mode:
......@@ -232,3 +251,11 @@
and run `sudo gitlab-ctl reconfigure`.
```ruby
registry['storage'] = {
'maintenance' => {
'readonly' => {
'enabled' => false
}
}
}
```
......@@ -234,3 +261,3 @@
### Running on schedule
1. Save and reconfigure GitLab:
......@@ -236,4 +263,7 @@
Ideally, you want to run garbage collect of registry regularly on weekly basis
during time when registry is not being in-use.
```sh
sudo gitlab-ctl reconfigure
```
### Running the garbage collection on schedule
......@@ -239,3 +269,5 @@
The simplest way is to add a new crontab job that it is gonna run periodically,
Ideally, you want to run the garbage collection of the registry regularly on a
weekly basis at a time when the registry is not being in-use.
The simplest way is to add a new crontab job that it will run periodically
once a week.
......@@ -241,4 +273,5 @@
once a week.
Create a file under `/etc/cron.d/registry-garbage-collect`:
```bash
......@@ -248,32 +281,3 @@
# Run every Sunday at 04:05am
5 4 * * 0 root gitlab-ctl registry-garbage-collect
```
### Administratively recycling unused tags
GitLab offers a set of APIs to manipulate Container Registry and aid the process
of removing unused tags. Currently, this is exposed using API, but in the future
these controls will be migrated to the GitLab Interface and be allowed by the
developer.
Project maintainers can
[delete container registry tags in bulk](https://docs.gitlab.com/ce/api/container_registry.html#delete-repository-tags-in-bulk)
periodically based on their own criteria.
However, this alone does not recycle data, it only unlinks tags from manifests
and image blobs. To recycle the container registry data in the whole GitLab instance run:
```sh
sudo gitlab-ctl registry-garbage-collect
```
You might also remove all unreferenced manifests with.
Since this is a way more destructive operation take a look
at [Recycle unreference manifests](#recycle-unreference-manifests)
to understand the implications.
```sh
sudo gitlab-ctl registry-garbage-collect -m
# or
sudo gitlab-ctl registry-garbage-collect /path/to/config.yml -m
```
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment