Skip to content
Snippets Groups Projects
troubleshooting.md 46 KiB
Newer Older

   ```shell
   pulp rpm repository create --retain-package-versions=1 --name "gitlab-ee"
   pulp rpm remote create --name gitlab-ee --url "http://mirror.example.com/repos/gitlab_gitlab-ee/" --policy immediate
   pulp rpm repository update --name gitlab-ee --remote gitlab-ee
   ```

1. Sync the repository:

   ```shell
   pulp rpm repository sync --name gitlab-ee
   ```

   This command must be run periodically to update the local mirror with changes
   to the GitLab repository.

After the repository is synced, you can create a publication and distribution to
Kati Paizee's avatar
Kati Paizee committed
make it available. See <https://docs.pulpproject.org/pulp_rpm/> for details.

## Error: `E: connection refused to d20rj4el6vkp4c.cloudfront.net 443`

When you install a package hosted on our package repository at `packages.gitlab.com`, your client will receive and follow a redirect to the CloudFront address `d20rj4el6vkp4c.cloudfront.net`. Servers in an air-gapped environment can receive the following errors:

```shell
E: connection refused to d20rj4el6vkp4c.cloudfront.net 443
```

```shell
Failed to connect to d20rj4el6vkp4c.cloudfront.net port 443: Connection refused
```

To resolve this issue, you have three options:

- If you can allowlist by domain, add the endpoint `d20rj4el6vkp4c.cloudfront.net` to your firewall settings.
- If you cannot allowlist by domain, add the [CloudFront IP address ranges](https://d7uri8nf7uskq.cloudfront.net/tools/list-cloudfront-ips) to your firewall settings. You must
  keep this list synced with your firewall settings because they can change.
- Manually download the package file and upload it to your server.

## Do I need to increase `net.core.somaxconn` ?

The following may assist in identifying if the value of `net.core.somaxconn`
is set too low:

```shell
$ netstat -ant | grep -c SYN_RECV
4
```

The return value from `netstat -ant | grep -c SYN_RECV` is the number of connections
waiting to be established. If the value is greater than `net.core.somaxconn`:

```shell
$ sysctl net.core.somaxconn
net.core.somaxconn = 1024
```

You may experience timeouts or HTTP 502 errors and is recommended to increase this
value by updating the `puma['somaxconn']` variable in your `gitlab.rb`.

## `exec request failed on channel 0` or `shell request failed on channel 0` errors

When pulling or pushing by using Git over SSH, you might see the following errors:

- `exec request failed on channel 0`
- `shell request failed on channel 0`

These errors can happen if the number of processes from the `git` user is above the limit.

To try and resolve this issue:

1. Increase the `nproc` setting for the `git` user in the `/etc/security/limits.conf` file on the nodes where `gitlab-shell` is running.
   Typically, `gitlab-shell` runs on GitLab Rails nodes.
1. Retry the pull or push Git command.

## Hung installation after SSH connection loss

If you're installing GitLab on a remote virtual machine and your SSH connection gets lost,
the installation could hang with a zombie `dpkg` process. To resume the installation:

1. Run `top` to find the process ID of the associated `apt` process, which is the parent of the `dpkg` process.
1. Kill the `apt` process by running `sudo kill <PROCESS_ID>`.
1. Only if doing a fresh install, run `sudo gitlab-ctl cleanse`. This step erases existing data, so must not be used on upgrades.
1. Run `sudo dpkg configure -a`.
1. Edit the `gitlab.rb` file to include the desired external URL and any other configuration that might be missing.
1. Run `sudo gitlab-ctl reconfigure`.

## Redis-related error when reconfiguring GitLab

You might encounter the following error when reconfiguring GitLab:

```plaintext
RuntimeError: redis_service[redis] (redis::enable line 19) had an error: RuntimeError: ruby_block[warn pending redis restart] (redis::enable line 77) had an error: RuntimeError: Execution of the command /opt/gitlab/embedded/bin/redis-cli -s /var/opt/gitlab/redis/redis.socket INFO failed with a non-zero exit code (1)
```

The error message indicates that Redis might have restarted or shut down while trying to establish a connection with `redis-cli`. Given that recipe runs
`gitlab-ctl restart redis` and tries to check the version right away, there might be a race condition that causes the error.

To resolve this problem, run the following command:

```shell
sudo gitlab-ctl reconfigure
```

If that fails, check the output of `gitlab-ctl tail redis` and try to run `redis-cli`.