Commit 639daa69 authored by Evan Read's avatar Evan Read
Browse files

Merge branch 'sh-update-gdk-runner-documentation' into 'master'

Update GDK runner documentation

See merge request gitlab-org/gitlab-development-kit!1788
......@@ -2,6 +2,6 @@
Most features of [GitLab CI/CD](http://docs.gitlab.com/ee/ci/) need a
[Runner](http://docs.gitlab.com/ee/ci/runners/README.html) to be registered with
the GitLab installation. This howto takes you through the necessary steps to
the GitLab installation. This how-to takes you through the necessary steps to
do so when GitLab is running under GDK.
......@@ -6,10 +6,4 @@
do so when GitLab is running under GDK.
## Set up GitLab
Start by [setting up the GDK](../../index.md) for your workstation.
In some configurations, GitLab Runner needs access to GitLab from inside a
Docker container, or even another machine, which isn't supported in the default
configuration.
Before setting up Runner, you need to have [set up the GDK](../../index.md) for your workstation.
......@@ -15,10 +9,4 @@
### Simple configuration
If you intend to just use the "shell" executor (fine for very
simple jobs), you can use GDK with its default settings and skip the Advanced
configuration below. If GDK is already running, you need to restart it after making
these changes.
With GDK running:
You can set up a runner to run directly on your workstation or you can set up a runner in Docker.
We will outline the steps for setting up each of these separately.
......@@ -24,10 +12,4 @@
1. Navigate to `http://localhost:3000/admin/runners` (log in as root)
1. Make note of the `Registration token`.
### Advanced configuration
Ensure you have Docker installed, then set up GitLab to bind to all
IPs on your machine by following [these instructions](local_network.md).
Without this step, builds fail with a 'connection refused' error.
- [Simple configuration](#simple-configuration)
- [Docker configuration](#docker-configuration) (recommended)
......@@ -33,14 +15,3 @@
The configured `hostname` needs to be set to an IP address that
*actually exists on the computer*.
1. Run `ipconfig` (Windows), `ifconfig` (Mac, BSD) or `ip addr show` (Linux) to find
your machine's network IP address. The IP address to use depends on your network,
and may change from time to time (via DHCP). An address like `10.x.x.x`,
`172.16.x.x`, or `192.168.x.x` is normally correct.
**Note**: If you are comfortable configuring your network, set a static IP for your
machine so it never changes.
1. In your `gdk.yml` add:
## Simple configuration
......@@ -46,7 +17,7 @@
```yaml
hostname: <IP address from previous step>
```
1. Reconfigure and restart GDK:
If you intend to just use the "shell" executor (fine for simple jobs), you can use the GDK with its default settings.
Builds will run directly on the host computer. If you choose this configuration, don't use random `.gitlab-ci.yml`
files from the internet unless you understand them fully as this could be a security risk. If you need a basic pipeline,
[here is example configuration from our documentation](https://docs.gitlab.com/ee/ci/environments/#configuring-manual-deployments) that
you can use.
......@@ -52,15 +23,3 @@
```shell
gdk reconfigure
gdk restart
```
Navigate to `http://<IP address>:3000/gitlab-org/gitlab-test`.
If the URL doesn't work, repeat the last step and pick a different IP.
Once there, ensure that the HTTP clone URL is `http://<ip>:3000/gitlab-org/gitlab-test.git`.
If it points to `localhost` instead, `gitlab/config/gitlab.yml` is incorrect.
Finally, navigate to `http://<ip>:3000/admin/runners` (log in as root) and make
a note of the `Registration token`.
### Download GitLab Runner
......@@ -66,9 +25,3 @@
## Download GitLab Runner
### Simple configuration
Runner can be installed using a pre-build binary or from source.
#### Install pre-built binary
The runner can be installed using a pre-built binary or from source.
......@@ -74,9 +27,4 @@
Unless you want to make changes to the Runner, it's easiest to install a binary
package. Follow the
[installation instructions](https://docs.gitlab.com/runner/install/)
for your operating system
([Linux](https://docs.gitlab.com/runner/install/linux-repository.html),
[OSX](https://docs.gitlab.com/runner/install/osx.html),
[Windows](https://docs.gitlab.com/runner/install/windows.html)).
To install from the binary, follow [the runner installation instructions](https://docs.gitlab.com/runner/install/)
for your specific operation system.
......@@ -82,10 +30,6 @@
#### Build from source
To build from source, you need to set up a development environment manually -
GDK doesn't manage it for you. The official GitLab Runner repository is
[here](https://gitlab.com/gitlab-org/gitlab-runner); just follow
[the development instructions](https://docs.gitlab.com/runner/development/).
To build from source, you'll need to follow [the runner development instructions](https://docs.gitlab.com/runner/development/).
The official GitLab Runner repository is [here](https://gitlab.com/gitlab-org/gitlab-runner).
To specify a custom `gitlab-runner` binary, add the following to `gdk.yml`:
......@@ -94,8 +38,5 @@
bin: <path_to_gitlab_runner_binary>/gitlab-runner-darwin-amd64
```
### Advanced configuration
If you followed the advanced configuration and want to install the runner as a Docker service,
follow the steps described in <https://docs.gitlab.com/runner/install/docker.html#docker-image-installation>.
### Setting up a Runner
......@@ -101,13 +42,9 @@
## Setting up the Runner
### Simple configuration
Run `gitlab-runner register --run-untagged --config <path-to-gdk>/gitlab-runner-config.toml`
(as your normal user), and follow the prompts. Use:
Run `gitlab-runner register --run-untagged --config <path-to-gdk>/gitlab-runner-config.toml` (as your normal user),
and follow the prompts. Use:
- **coordinator URL**
Use either:
- `http://localhost:3000/`
......@@ -108,11 +45,10 @@
- **coordinator URL**
Use either:
- `http://localhost:3000/`
- `http://<custom_IP_address>:3000/`, if you customized your IP address using
[Advanced Configuration](#advanced-configuration).
- `http://<custom_IP_address>:3000/`, if you customized your IP address.
- **token**
......@@ -116,7 +52,7 @@
- **token**
`Registration token` (copied from `admin/runners`)
`Registration token` (copied from `<coordinator-url>/admin/runners`)
- **description** (optional)
......@@ -126,6 +62,10 @@
Comma-separated tags. Jobs can be set up to use only Runners with specific tags.
- **executor**
Since we are running directy on the host computer in this simple configuration, choose `shell`.
The Runner writes its configuration file to `gitlab-runner-config.toml`,
which is in GDK's `.gitignore` file.
......@@ -138,11 +78,9 @@
token: <runner-token>
```
If Docker is installed and you followed the special setup instructions above,
choose `docker` as the executor. Otherwise, choose `shell` - but remember that
builds then run directly on the host computer! Don't use random
`.gitlab-ci.yml` files from the Internet unless you understand them fully, it
could be a security risk.
Finally, rebuild your `Procfile` with `gdk update` or un-comment
the line that starts with `runner:`. This allows you to manage the runner along with
your other GDK processes.
You can run the `register` command multiple times to set up additional Runners -
fuller documentation on the different types of executor and their requirements
......@@ -150,10 +88,6 @@
Each `register` invocation adds a section to the configuration file, so make
sure you're referencing the same one each time.
Finally, rebuild your `Procfile` with `rm Procfile; make Procfile` or un-comment
the line that starts with `runner:`. This allows you to manage it along with
your other GDK processes.
Alternately, run `gitlab-runner --log-level debug run --config <path-to-gdk>/gitlab-runner-config.toml`
to get a long-lived Runner process, using the configuration you created in the
last step. It stays in the foreground, outputting logs as it executes
......@@ -165,7 +99,5 @@
or clone an [example project](https://gitlab.com/groups/gitlab-examples), and
watch as the Runner processes the builds just as it would on a "real" install!
### Advanced configuration
Register your Docker-based runner by following the steps described in <https://docs.gitlab.com/runner/register/index.html#docker>.
## Docker configuration
......@@ -171,5 +103,5 @@
### Docker executor
#### Docker for Mac
Using runners in Docker allows you to set up a clean environment for your builds
each time. It is also safer than running directly on your computer, as the
runner will not have direct access to your computer.
......@@ -175,10 +107,9 @@
Docker for Mac [has some networking
limitations](https://docs.docker.com/docker-for-mac/networking/), but
you can still get CI runners working with a Docker executor by using a
hostname trick. Within a container, Docker maps the
`host.docker.internal` hostname to the host IP address
(e.g. 192.168.65.2). On the host network, you can map this hostname to
the local IP address (e.g. 127.0.0.1) and use this hostname in three
different configuration files:
### Set up Docker and GDK
Ensure you have Docker installed, then we will need to set up GitLab to bind to an IP on your machine
instead of `127.0.0.1`. Without this step, builds fail with a `connection refused` error.
The easiest and most universal way to set this up is by using an internal, dummy interface that can
be used by both the host and the Docker container.
......@@ -184,4 +115,5 @@
1. In `config/gitlab.yml`, set the `host` parameter to `host.docker.internal`.
1. In `/etc/hosts`, add an entry:
1. [Create a loopback interface](../index.md#create-loopback-interface-for-gdk) for a new private network.
1. In the GitLab Runner configuration (for example, `~/.gitlab-runner/config.toml`), set the coordinator
URL with an IP on this private network:
......@@ -187,5 +119,5 @@
```plaintext
127.0.0.1 host.docker.internal
```
```toml
url = "http://172.16.123.1:3000/"
```
......@@ -191,4 +123,6 @@
1. In the GitLab Runner config (e.g. `~/.gitlab-runner/config.toml`), set the coordinator
URL with this hostname and the port used by GDK (`3001` if `EE`):
This can also be combined with a custom hostname by following these
[instructions for setting up `gdk.test`](../index.md#set-up-gdktest-hostname) but using the `172.16.123.1`
instead of `127.0.0.1`. Then set up your runner to resolve the hostname by adding `runner.extra_hosts`
to your `gdk.yml`. For example, for `gdk.test`:
......@@ -194,5 +128,6 @@
```toml
url = "http://host.docker.internal:3000/"
```
```yaml
runner:
extra_hosts: ["gdk.test:172.16.123.1"]
```
......@@ -198,4 +133,3 @@
Note that all three settings must be set to ensure a number of items
work with the runner:
### Set up a runner
......@@ -201,6 +135,8 @@
1. Registering the runner
1. Polling for jobs on the host network
1. Making API requests (e.g. sending artifacts) inside the container during a CI job.
1. Cloning the repository ([`CI_REPOSITORY_URL`](https://docs.gitlab.com/ee/ci/variables/predefined_variables.html))
To set up a runner in Docker,
[follow the runner Docker image installation documentation](https://docs.gitlab.com/runner/install/docker.html#docker-image-installation).
When registering a new runner in Docker, the following prompts will appear:
- **coordinator URL**
......@@ -206,6 +142,3 @@
The `/etc/hosts` parameter is needed to make the first two items work,
since this maps `host.docker.internal` to `localhost`. The `config.toml`
changes allow the third item to work. The `gitlab.yml` changes are used
for the fourth item.
Use either:
......@@ -211,3 +144,4 @@
#### Using an internal, dummy interface
- `http://172.16.123.1:3000/`
- `http://gdk.test:3000/`, if you set up a custom hostname such as `gdk.test`.
......@@ -213,6 +147,5 @@
The trick described above is a bit of a hack and only works for Docker
for Mac, but the "proper" way to support a Docker executor is to use an
internal, dummy interface that can be used by both the host and the
container:
- **token**
`Registration token` (copied from `<coordinator-url>/admin/runners`)
......@@ -218,6 +151,5 @@
1. [Create a loopback interface](../index.md#create-loopback-interface-for-gdk) for a new private
network.
1. In the GitLab Runner configuration (for example, `~/.gitlab-runner/config.toml`), set the
coordinator URL with an IP on this private network:
- **description** (optional)
A description of the Runner. Defaults to the hostname of the machine.
......@@ -223,5 +155,3 @@
```toml
url = "http://172.16.123.1:3001/"
```
- **tags** (optional)
......@@ -227,3 +157,3 @@
##### Use custom hostname
Comma-separated tags. Jobs can be set up to use only Runners with specific tags.
......@@ -229,4 +159,3 @@
To access your GDK via a hostname that points to this dummy interface (for example
`gdk.test`):
- **executor**
......@@ -232,4 +161,3 @@
1. Set your runner to resolve the hostname by adding `runner.extra_hosts` to your `gdk.yml`.
For example, for `gdk.test`:
Since we are running our runner in Docker, choose `docker`.
......@@ -235,6 +163,3 @@
```yaml
runner:
extra_hosts: ["gdk.test:172.16.123.1"]
```
- **Docker image**
......@@ -240,3 +165,4 @@
1. Set up the custom hostname. Follow these [instructions for setting up `gdk.test`](../index.md#set-up-gdktest-hostname)
but use the `172.16.123.1` instead of `127.0.0.1`.
Choose which Docker image you would like to use for this runner. Common ones are `ruby:2.6`
and `node:latest` but you can find images using
[Docker's image hub](https://hub.docker.com/search?type=image).
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment