- Nov 02, 2018
-
-
Boris Feld authored
-
- Nov 06, 2018
-
-
Pierre-Yves David authored
We use different tests instead of variants because we expect them to have significantly different performance patterns.
-
- Nov 05, 2018
-
-
Pierre-Yves David authored
We introduce some single-element variants to make it easier to add more variation later if needed.
-
- Sep 06, 2018
-
-
Pierre-Yves David authored
-
- Sep 04, 2018
-
-
Pierre-Yves David authored
-
- Aug 30, 2018
-
-
Pierre-Yves David authored
We use the new options offered by upstream ASV. We use a 3 repeat minimum and 10 repeat max. Max time (before we give up on max repeat) is 60 seconds per variants.
-
Pierre-Yves David authored
Now that we made `./launch.sh` more developer friendly, we need another place to fit logic related to the server runners.
-
Pierre-Yves David authored
-
- Jun 15, 2018
-
-
Philippe Pepiot authored
So we can compare clone with --noupdate
-
- Jun 12, 2018
-
-
Philippe Pepiot authored
In a dedicated class since results cannot be correlated with clone benchmarks. Also unlike clone benchmarks, don't benchmark with a given revision since --stream will ignore it.
-
Philippe Pepiot authored
This is a bit tricky because there are several constraints: * we must cleanup after clone (in setup/teardown) to limit disk space usage * we must limit benchmark duration (avoid to run too much time on big repositories) * we must have more stable results for small repositories The whole algorithm is in asv.benchmark.benchmark_timing(). To ensure setup/teardown is called before/after each clone we *must* have number = 1 (so timer.timeit() will be called with 1 as argument). Then it's important to have a proper sample_time set, because it will control how long the benchmark will run. timeit() is called `repeat` (default 10) times unless the whole benchmark (= one benchmark function with one set of parameter) takes more than `repeat` * 1.3 * `sample_time` where `sample_time` has a default value of 0.1 This can produce very short times, for instance for a clone duration of 20s this will only run one clone even if repeat is set 10... So use sample_time = (max_time_we_want_the_benchmark_to_run / (repeat * 1.3)) to control maximum whole benchmark duration and to ensure timeit() will be called close to `repeat` times.
-
Philippe Pepiot authored
'strip' param was only taking one value 'same'. Drop this param by inheriting from BaseTestSuite instead of BaseExchangeTimeSuite.
-
Philippe Pepiot authored
In following changesets we will add exchanges benchmarks not using "strip" or "revset" params. Moving this outside of the BaseExchangeTimeSuite allow to re-use this logic without inheriting from it.
-
- May 17, 2018
-
-
Philippe Pepiot authored
This apply to files, archive and exchange benchmarks. Some reference repositories doesn't have a 'default' revision, or it relate to very old changesets that are not stripped by prepare_repos.py, so exchange benchmarks might be irrelevant.
-
- Jul 19, 2018
-
-
Philippe Pepiot authored
This display extra information like stderr / stdout of installation of the repo and display information about benchmark and variant combination currently running. Closes #478
-
- Aug 14, 2018
-
-
Pierre-Yves David authored
The extra "-" is invalid.
-
Pierre-Yves David authored
-
Pierre-Yves David authored
Archive is a fairly long benchmark in all tests. We fix the number of run to 3 to work around limitation of the current ASV scheduling.
-
Pierre-Yves David authored
The `time_archive` benchmark is a long and "files" tends to files on larger repository. We disable it for now. If in the future we start focusing on `hg archive` performance, we can start optimizing this tests.
-
Pierre-Yves David authored
-
Pierre-Yves David authored
This make for clearer command line.
-
Pierre-Yves David authored
this was overlooked when introducing the test.
-
- Aug 13, 2018
-
-
Pierre-Yves David authored
Copying have the advantage of keeping the content strictly indentical, including caches and configuration. This is important when comparing format variants. (However copying will be slower and take more disk space)
-
Pierre-Yves David authored
The test and associated output were wrong.
-
Pierre-Yves David authored
This will make test simpler.
-
Pierre-Yves David authored
-
- Aug 14, 2018
-
-
Pierre-Yves David authored
-
- Jul 06, 2018
-
-
Philippe Pepiot authored
-
Philippe Pepiot authored
-
Philippe Pepiot authored
-
- Aug 13, 2018
-
-
Pierre-Yves David authored
This is an attempt to have a clearer root directory
-
Martijn Pieters authored
-
Martijn Pieters authored
-
Martijn Pieters authored
-
Pierre-Yves David authored
There are know issue with the current clone tests, so we disable them entirely until they get fixed.
-
Pierre-Yves David authored
Setting sample_time to 10 means each variant will be run for 10s and 10 time at mininum. For test with many fast variant the impact on runtime gets very problementation. Dropping this move a full run back to a bit less than 2 hours (from about 18 hours). In the future, we might carefully reintroduce more run in the case where this seems problematic not to.
-
Pierre-Yves David authored
This is overall bad for our number, however, the current setup poorly exploit them and create issue, we disable them until this get fixed.
-
Pierre-Yves David authored
This will help to update the code in a later changeset.
-
- Aug 02, 2018
-
-
Pierre-Yves David authored
This should avoid doing network exchange for each run. We now use a cache directory in `/tmp/.scm-perf-test/` to cache repository we need to pull. The use of this cache is automatically managed by the `share` extension. The directory is also used to cache wheels used by pip.
-
Pierre-Yves David authored
One more step toward the new world.
-