Skip to content
Snippets Groups Projects
Commit 71963760ac6d authored by Philippe Pepiot's avatar Philippe Pepiot
Browse files

Enable clone benchmarks for all reference repositories

This is a bit tricky because there are several constraints:

* we must cleanup after clone (in setup/teardown) to limit disk space usage
* we must limit benchmark duration (avoid to run too much time on big repositories)
* we must have more stable results for small repositories

The whole algorithm is in asv.benchmark.benchmark_timing().

To ensure setup/teardown is called before/after each clone we *must* have
number = 1 (so timer.timeit() will be called with 1 as argument).
Then it's important to have a proper sample_time set, because it will control
how long the benchmark will run. timeit() is called `repeat` (default 10) times
unless the whole benchmark (= one benchmark function with one set of parameter)
takes more than `repeat` * 1.3 * `sample_time` where `sample_time` has a default value of 0.1

This can produce very short times, for instance for a clone duration of 20s
this will only run one clone even if repeat is set 10...

So use sample_time = (max_time_we_want_the_benchmark_to_run  / (repeat * 1.3))
to control maximum whole benchmark duration and to ensure timeit() will be
called close to `repeat` times.
parent 38865651f487
No related branches found
No related tags found
Loading
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment