Skip to content
Snippets Groups Projects
README.md 2.06 KiB
Newer Older
Boris Feld's avatar
Boris Feld committed
# High-level mercurial performance testing

## Setup with Docker

With docker, you only need to launch this command to get the results: `docker-compose up --build`

The runner will start, test a bunch of commits and you will see results at [http://localhost:8080](http://localhost:8080)

## Setup manually

You can follow the Dockerfile content which is basically:

* Install the dependencies with `pip install -r requirements.txt`
* Prepare the repos, asv specifics file and launch the workers with: `./launch.sh`
* Get the webserver running for showing the results with: `asv preview`

For pull/push benchmark with ssh you have to setup a ssh-key without passphrase (or rely on ssh-agent).

```
mkdir ~/.ssh
ssh-keygen -P '' -f ~/.ssh/bighgperf
cat ~/.ssh/bighgperf.pub >> ~/.ssh/authorized_keys
ssh-keyscan -H localhost >> ~/.ssh/known_hosts
```

How to Setup
============

Make sure you have the environment setup

  $ virtualenv some/dir/venv-asv
  $ some/dir/venv-asv/bin/pip install -r requirements.txt

# Do not forget to enable the virtual env before running the benchmark

  $ source some/dir/venv-asv

Running benchmarks
==================

How to run all benchmarks on the default revisions
--------------------------------------------------

  $ ./launch.sh

How to run all benchmarks on specific revisions
-----------------------------------------------

  $ ./launch.sh "0b63a6743010+33ac6a72308a"

(any revset are supported)


How to run some specific benchmark
----------------------------------

use the '--bench' option to filter the test to run using a regular expression

Running one benchmark only

  $ ./launch.sh "0b63a6743010+33ac6a72308a" --bench track_commit

Running one pypy variant only

  $ ./launch.sh "0b63a6743010+33ac6a72308a" --bench "netbeans-2018"

Running one test with one variant

  $ ./launch.sh "0b63a6743010+33ac6a72308a" --bench "track_commit.*mozilla-central"

Comparing results
=================

  $ asv compare -s oldhash newhash

Checking where a run spent time
===============================

  $ ./asv_result_time.py results/<machine>/<result-file>.json