Heptapod automated functional tests
WARNING: don't try and run these if you're not willing to sacrifice the server data.
This can be used for preflight validation of production servers, provided the data is fully wiped afterwards.
Installation
Client-side install
-
tox:
pip install --user tox
-
- Fedora 29:
dnf install chromedriver
- Debian 9:
apt install chromium-driver
- Fedora 29:
All further dependencies will be installed by the first run.
Heptapod server requirements
These tests can work against Heptapod servers provided either as
- (default) local Docker containers manageable by the system user running the tests, or
- installed from source and being run by the same user as the tests, or
- completely remote, skipping some of the tests.
The Gitlab root password will be initialized by the first test to run. The tests will fail if the Gitlab root password is already set and does not match the expected one.
Default Docker setup
In the Docker case, the expected container name is by default heptapod
.
By default, the tests expect to be running on the Docker host, and that the
container can initiate TCP connections to the host, identified as the main
IPv4 gateway of the container. Don't forget to allow incoming TCP connections
from the container in your firewall, if you have one. You can also pass a
reachable address explicitely with --heptapod-reverse-call-host
or disable
such tests by passing an empty string as address.
The container HTTP and SSH ports must be forwarded by default to heptapod:81
and heptapod:2022
. This is usually done by making the heptapod
host name
resolve to a loopback IP address, such as 127.0.0.2, and forwarding the
container ports like this:
docker run --publish 127.0.0.2:2022:22 --publish 127.0.0.2:81:22
Using a dedicated host name and IP address helps preventing confusion in
the user's .ssh/known_hosts
file.
Running the tests
tox
is the executable launching the tests. It is also responsible to
setup Python libraries used in the tests. The tests themselves are
written with pytest.
All tox
commands have to be run from the root of this repository.
It is possible to pass down options to the underlying pytest
command:
tox -- --heptapod-url URL -k push_basic
All Heptapod specific options take the --heptapod-some-option
form. For
the full list, do
tox -- --help
Common network options
These are available in all cases
-
--heptapod-url
(defaulthttp://heptapod:81
): HTTP URL of the tested Heptapod instance. It must use a resolvable host name, not an IP address. It does not have to be resolved through DNS, an/etc/host
entry pointing to the loopback interface is fine. -
--heptapod-ssh-port
(default 2022): SSH port of the tested Heptapod instance. The same host name will be used as for HTTP. If the host name resolves to the loopback interface, it is advised to tie it to a dedicated address, such as127.0.0.2
, to minimize risks with your SSHknown_hosts
file. -
--heptapod-reverse-call-host
: address that the Heptapod server can use to reach the system running theses tests (necessary to test outbound connections, such as web hooks). -
--heptapod-root-password
(default5iveL!fe
). The password to use and maybe set for theroot
user. The default value is the same as with the GitLab Development Kit (GDK).
Testing a Docker container
Being the default, this is the simplest. If you followed the default namings
and the current system user can managed Docker containers,
just running tox
will launch the whole tests suite
Specific options:
-
--heptapod-container-name
(defaultheptapod
)
Testing a local install from source.
You will need to run the tests and the Heptapod server under the same user and to pass some mandatory options:
Minimal example:
~/heptapod/heptapod-tests $ tox -- --heptapod-source-install\
--heptapod--repositories-root ~/heptapod/gdk/repositories-root
Testing a remote server
Mandatory reminder: Never, ever run these tests on an Heptapod server if you're not prepared to throw all its data
you'll have to provide the --heptapod-remote
option, and probably be explicit
about all network options:
~/heptapod/heptapod-tests $ tox -- --heptapod-remote \
--heptapod-ssh-port 22 \
--heptapod-url https://heptapod.test \
--heptapod-root-password SECRET
The root password option is listed because you probably don't want to have an instance with the default root password available on the internet.
Docker: choosing the version to test
The versions installed in the Docker image you're using are specified by the heptapod_revisions.json file.
To test your local clone of heptapod/heptapod:
-
mount your local
heptapod
clone in the container (assuming below it's seen as/home/heptapod-rails
from the container) -
execute the following in the container:
cd /var/opt/gitlab/embedded/service/gitlab-rails hg pull --config phases.publish=False /home/heptapod-rails hg up -r WISHED_REVISION gitlab-ctl restart unicorn
If you want to try changes to other components (e.g., hg-git
), do something similar.
The paths can be seen in the Docker build logs, or you can read them in the install script
Adding new tests
Branch and topics rules
The convention is that the tests in the default branch should pass against
the current octobus/heptapod:latest
Docker image, so
- if you want to share a bug reproduction, please open a new topic
- even if a bug fix in heptapod has landed, please wait for the Docker push before publishing the corresponding tests
- tests can be published before a Heptapod new release, but please have the corresponding fixes landed and pushed to Docker Hub first.
If there is an active stable branch (e.g. heptapod-0-6-stable
or similar),
then the tests of that branch must pass against the latest release version
corresponding to that branch. The same conclusions follow.