This instance will be upgraded to Heptapod 0.26.0rc1 on 2021-10-25 at 14:00 UTC+2 (a few minutes of down time)

README.md 8.79 KB
Newer Older
1
2
# Heptapod automated functional tests

3
4
5
6
WARNING: to test production instances, use the dedicated mode exclusively. Other
modes assume that you are ready to **throw all data away** after the test
run, and hence are suitable at most for preflight full testing of a fresh
production instance.
7

8
9
10
11
## Installation

### Client-side install

12
13
14
15
16
17
#### Mercurial

The tests need a working `hg` executable, available on `$PATH`, with the
following extensions available:

- evolve and topic: [hg-evolve](https://pypi.org/project/hg-evolve) ≥ 9.3.0
Georges Racinet's avatar
Georges Racinet committed
18
- configexpress: [hg-configexpress](https://pypi.org/project/hg-configexpress)
19
20
21

#### Test harness (Selenium)

22
- tox: `pip install --user tox`
23
24
25
- [ChromeDriver](https://sites.google.com/a/chromium.org/chromedriver/home)

  For direct installation on the system running the tests:
26

27
  + Fedora 29 to 33: `dnf install chromedriver`
28
29
30
31
32
33
34
35
  + Debian 9 to 10, Ubuntu < 20.04: `apt install chromium-driver`
  + Ubuntu ≥ 20.04: the `chromium-driver` package actually installs a snap,
    which can be problematic in some environments. See how it is done in the
    `docker-inside` job of our [CI pipelines](.gitlab-ci.yml).

  Another option is to use a Selenium RemoteWebDriver, which proxies the
  actual web browsing through a dedicated system. Selenium provides the
  `selenium/standalone-chrome` Docker image for these purposes.
36

37
All further dependencies will be installed by the first run.
38

39
40
41
42
43
44
45
### Heptapod server requirements

These tests can work against Heptapod servers provided either as

- (default) local Docker containers manageable by the system user running the
  tests, or
- installed from source and being run by the same user as the tests, or
46
47
48
- completely remote, skipping some of the tests, or
- production, relying on users with at most ownership of a dedicated projects
  group, and running a subset of suitable tests.
49

50
51
Except in production server mode, the Gitlab root password
will be initialized by the first test to run.
52
53
The tests will fail if the Gitlab root password is already set
and does not match the expected one.
54
55


56

57
### Default Docker setup
58

59
In the Docker case, the expected container name is by default `heptapod`.
60

61
62
63
64
65
66
67
By default, the tests expect to be running on the Docker host, and that the
container can initiate TCP connections to the host, identified as the main
IPv4 gateway of the container. Don't forget to *allow incoming TCP connections
from the container in your firewall*, if you have one. You can also pass a
reachable address explicitely with `--heptapod-reverse-call-host` or disable
such tests by passing an empty string as address.

68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
The container HTTP and SSH ports must be forwarded by default to `heptapod:81`
and `heptapod:2022`. This is usually done by making the `heptapod` host name
resolve to a loopback IP address, such as 127.0.0.2, and forwarding the
container ports like this:

```
docker run --publish 127.0.0.2:2022:22 --publish 127.0.0.2:81:22
```

Using a dedicated host name and IP address helps preventing confusion in
the user's `.ssh/known_hosts` file.

## Running the tests

`tox` is the executable launching the tests. It is also responsible to
setup Python libraries used in the tests. The tests themselves are
written with [pytest](https://docs.pytest.org).

All `tox` commands have to be run from the root of this repository.
87
88
89
90

It is possible to pass down options to the underlying `pytest` command:

```
91
    tox -- --heptapod-url URL -k push_basic
92
93
```

94
95
96
97
98
99
100
101
102
103
All Heptapod specific options take the `--heptapod-some-option` form. For
the full list, do

```
   tox -- --help
```

### Common network options

These are available in all cases
104
105

- `--heptapod-url` (default `http://heptapod:81`): HTTP URL of the tested
106
  Heptapod instance. It must use a resolvable host *name*, not an IP address.
107
  It does not have to be resolved through DNS, an `/etc/host` entry pointing
108
  to the loopback interface is fine.
109
110
111
112
113
- `--heptapod-ssh-port` (default 2022): SSH port of the tested Heptapod
  instance. The same host name will be used as for HTTP. If the host name
  resolves to the loopback interface, it is advised to tie it to a dedicated
  address, such  as `127.0.0.2`, to minimize risks with your SSH
  `known_hosts` file.
114
115
116
- `--heptapod-reverse-call-host`: address that the Heptapod server can use
  to reach the system running theses tests (necessary to test outbound
  connections, such as web hooks).
117
118
119
- `--heptapod-root-password` (default `5iveL!fe`). The password to use and maybe
  set for the `root` user. The default value is the same as with the GitLab
  Development Kit (GDK).
120

121
122
123
124
125
126
127
128
### Running the tests concurrently

Use the `---tests-per-worker` option only.

Do *NOT* use the `--workers` option: it would setup the `Heptapod` session
fixture several times, leading to problems with GitLab user tokens and other
shared data that are session-local.

129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
### Testing a Docker container

Being the default, this is the simplest. If you followed the default namings
and the current system user can managed Docker containers,
just running `tox` will launch the whole tests suite

Specific options:

- `--heptapod-container-name` (default `heptapod`)

### Testing a local install from source.

You will need to run the tests and the Heptapod server under the same user
and to pass some mandatory options:

Minimal example:

```
~/heptapod/heptapod-tests $ tox -- --heptapod-source-install\
    --heptapod--repositories-root ~/heptapod/gdk/repositories-root
```

### Testing a remote server
Mandatory reminder: **Never, ever run these tests on an
  Heptapod server if you're not prepared to throw all its data**

you'll have to provide the `--heptapod-remote` option, and probably be explicit
about all network options:

```
~/heptapod/heptapod-tests $ tox -- --heptapod-remote \
  --heptapod-ssh-port 22 \
  --heptapod-url https://heptapod.test \
  --heptapod-root-password SECRET
```

The root password option is listed because you probably don't want to have
an instance with the default root password available on the internet.

168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
### Testing a production instance

*New on 2021-02-18*: see !80

To run the tests suitable for production instances, you will need first to
prepare:

- a projects group entirely dedicated to these functional tests
- a dedicated user that owns the dedicated group (more users will probably be
  needed in the future).

The production mode is activated by an explicit command-line option. Another
option is used to pass the dedicated user credentials.

Example:

```
~/heptapod/heptapod-tests $ tox -- --heptapod-prod-server \
186
    --heptapod-prod-group-owner-credentials ID:USERNAME:EMAIL:PASSWORD \
187
188
189
190
191
192
193
194
    --heptapod-url https://foss.heptapod.net \
    --heptapod-ssh-port 22 \
    --heptapod-ssh-user hg
```

where ID is the numeric user id, and USERNAME is the user login name
(e.g `testfonct`).

195
196
197
198
To launch tests on an instance tied to the Clever Cloud SSO,
use additionally the `--heptapod-clever-cloud-sso` option.


199
200
201
202
203
204
205
206
207
Remarks and safety:

- The user password must be fully operational: the functional tests won't
  take care of the finalization sequence that occurs at first login.
- Do not give the dedicated user any rights outside of the dedicated groups.
- It is advisable to block the dedicated user when not in use.
- Be prepared to receive email for Merge Requests on the dedicated user address.
  Arguably, this is part of the testing.

208
209

### Docker: choosing the version to test
210

211
212
The versions installed in the Docker image you're using are specified by the
[heptapod_revisions.json](https://dev.heptapod.net/heptapod/omnibus/blob/branch/heptapod/heptapod_docker/heptapod_revisions.json) file.
213

214
To test your local clone of heptapod/heptapod:
215
216

- mount your local `heptapod` clone in the container (assuming below it's seen
217
  as `/home/heptapod-rails` from the container)
218
219
220
221
- execute the following in the container:

  ```
  cd /var/opt/gitlab/embedded/service/gitlab-rails
222
223
  hg pull --config phases.publish=False /home/heptapod-rails
  hg up -r WISHED_REVISION
224
225
  gitlab-ctl restart unicorn
  ```
226
227

If you want to try changes to other components (e.g., `hg-git`), do something similar.
228
The paths can be seen in the Docker build logs, or you can read them in the [install script](https://dev.heptapod.net/heptapod/docker/blob/branch/default/heptapod/assets/install_heptapod.py)
229
230
231
232
233
234
235
236
237

## Adding new tests

### Branch and topics rules

The convention is that the tests in the default branch should pass against
the current `octobus/heptapod:latest` Docker image, so

* if you want to share a bug reproduction, please open a new topic
238
239
* even if a bug fix in heptapod has landed, please wait for the Docker push
  before publishing the corresponding tests
240
241
242
* tests can be published before a Heptapod new release, but please have the
  corresponding fixes landed and pushed to Docker Hub first.

243
244
245
If there is an active stable branch (e.g. `heptapod-0-6-stable` or similar),
then the tests of that branch must pass against the latest release version
corresponding to that branch. The same conclusions follow.