Skip to content
Snippets Groups Projects
Commit 47fb6cf1 authored by Raphaël Gomès's avatar Raphaël Gomès
Browse files

Update the benchmarks docs

parent c998a787
No related tags found
No related merge requests found
......@@ -2,8 +2,8 @@
Details on benchmarks in Poulpe
===============================
A benchmark describe "something" we want to get performance data on. It is
defined in a "poulpe-benchmark-definition" file (`.pbd`). It will use code that
exists in a poulpe's "binary-environment" to benchmark that "something" on the
data present in a poulpe's "binary-environment".
A benchmark describes "something" we want to get performance information about.
It is defined in a "poulpe-benchmark-definition" file (`.pbd`).
It will use code that exists in a binary-environment to benchmark
that "something" on the data present in a data-environment.
......@@ -9,4 +9,4 @@
The `.pbd` files includes multiple metadata and definition. Some are common to
all type of benchmarks, some are specific to each type of benchmarks.
The `.pbd` files include multiple kinds of metadata and definitions.
Some are common to all types of benchmarks, some are specific.
......@@ -12,6 +12,6 @@
Generic metata data
===================
Generic metadata data
=====================
section: "meta"
---------------
......@@ -19,10 +19,10 @@
format
~~~~~~
The version of this benchmark definition. Currently known version are:
The version of this benchmark definition. Currently known versions are:
:0: The very first, quite volatile, version.
name
~~~~
......@@ -23,12 +23,12 @@
:0: The very first, quite volatile, version.
name
~~~~
identifier of the benchmark. This will be stored in result.
identifier of the benchmark. This will be stored in results.
method
~~~~~~
The way this benchmark is expected to be run. It will also define most of the
......@@ -30,7 +30,7 @@
method
~~~~~~
The way this benchmark is expected to be run. It will also define most of the
other value we expect to find to configure this benchmark.
other values we expect to find to configure this benchmark.
......@@ -36,5 +36,5 @@
The current list of supported method is :
The current list of supported methods is:
:simple-command: benchmarking the run of a single shell command.
:mercurial-perf-extension: using Mercurial perf extensions to run benchmarks
......@@ -51,9 +51,9 @@
section: environment
--------------------
A section that allow to set environment variable to be used when running the
command. This is useful to replace some of the command argument with. Each
sub-key is a an environment variable to replace.
A section that allows to set environment variables to be used when running the
command. This is useful to replace some of the command's arguments with. Each
sub-key is an environment variable to replace.
Value supports the `DATA-VARS:` notation if necessary
......@@ -68,8 +68,8 @@
The command to be run.
It can we extended with variants (see next section)
It can be extended with variants (see next section)
cwd
~~~
......@@ -72,6 +72,6 @@
cwd
~~~
The directory to run the command in.
The directory in which to run the command.
......@@ -77,5 +77,5 @@
Value can be substituded for data-env variable using the
Values can be substituted for data-env variables using the
`"DATA-VARS:path.to.variable.in.the.data.env"` syntax.
For example `"DATA-VARS:mercurial.status.large.all"`
......@@ -87,8 +87,11 @@
prepare-run
~~~~~~~~~~~
A command (or series of command) to run before each command run.
A command (or series of commands) to run before each command run.
Note: this command will be run with bash and prefixed with `set -eEuo pipefail`.
You can opt out of the prefix by specifying "simple-command.no-set-builtin=true"
variants
--------
......@@ -91,7 +94,7 @@
variants
--------
Variants allow for slight variation of a benchmark. For example, they allow to change the
input data, of to turn some feature on and off.
Variants allow for slight variations of a benchmark. For example,
they allow for changing the input data, or for turning some feature on and off.
......@@ -97,5 +100,5 @@
Variants are organised within independant "dimensions".
Variants are organized within independent "dimensions".
Each dimension has a set of values, each value can modify the benchmark. Each
dimension must have a default value.
......@@ -107,7 +110,7 @@
simple-command.variants.dimensions.<dimension-name>.<variant-key>
The `dimension-name` and `variant-key` are used as identifier in the benchmark
The `dimension-name` and `variant-key` are used as identifiers in the benchmark
result and for selecting variant at run time.
variants variables
......@@ -118,9 +121,9 @@
Exactly one variant must have it set to `true` per dimension.
When set and no variant have been explicitly selected for this dimension, this
variant will be used.
When set and no variant has been explicitly selected for this dimension during
a benchmark run, this variant will be used.
extend-command
~~~~~~~~~~~~~~
......@@ -123,9 +126,9 @@
extend-command
~~~~~~~~~~~~~~
A string to append to the commend when this variable is selected.
A string to append to the command when this variable is selected.
cwd
~~~
......@@ -128,6 +131,6 @@
cwd
~~~
overwrite the global `simple-command.cwd` variable. Behave the same.
overwrites the global `simple-command.cwd` variable. Behaves the same.
......@@ -133,6 +136,6 @@
Mercurial perf extensions configuration
=======================================
Mercurial perf extension configuration
======================================
section: "hg-perf-ext"
----------------------
......@@ -151,8 +154,8 @@
args
~~~~
Arguments to pass to the command (a list of string).
Arguments to pass to the command (a list of strings).
setup-script
~~~~~~~~~~~~
......@@ -155,9 +158,12 @@
setup-script
~~~~~~~~~~~~
a shell script to run before running the perf-command.
A shell script to run before running the perf-command.
Note: this command will be run with bash and prefixed with `set -eEuo pipefail`.
You can opt out of the prefix by specifying "hg-perf-ext.no-set-builtin=true"
constraints
~~~~~~~~~~~
......@@ -160,10 +166,10 @@
constraints
~~~~~~~~~~~
Theses variables control when this benchmark can be run.
These variables control when this benchmark can be run.
Here is the list of currently supported constraints:
- `graph.visible-revision-count.min=<integer>`:
......@@ -165,10 +171,10 @@
Here is the list of currently supported constraints:
- `graph.visible-revision-count.min=<integer>`:
Target repository must have at least that many visible revision for this
Target repository must have at least that many visible revisions for this
benchmark to run.
- `compatible-revision-range=<revset>`:
......@@ -171,10 +177,10 @@
benchmark to run.
- `compatible-revision-range=<revset>`:
The range of revision compatible with this changes.
The range of revisions compatible with the command being run.
variants
--------
......@@ -176,8 +182,8 @@
variants
--------
Variants allow for slight variation of a benchmark. For example, they allow to change the
input data, of to turn some feature on and off.
Variants allow for slight variations of a benchmark. For example,
they allow for changing the input data, of for turning some feature on and off.
......@@ -183,5 +189,5 @@
Variants are organised within independant "dimensions".
Variants are organized within independent "dimensions".
Each dimension has a set of values, each value can modify the benchmark. Each
dimension must have a default value.
......@@ -189,7 +195,7 @@
defining a variant
~~~~~~~~~~~~~~~~~~
Variants are defined under keys following this patterns :
Variants are defined under keys following this pattern:
hg-perf-ext.variants.dimensions.<dimension-name>.<variant-key>
......@@ -193,8 +199,8 @@
hg-perf-ext.variants.dimensions.<dimension-name>.<variant-key>
The `dimension-name` and `variant-key` are used as identifier in the benchmark
result and for selecting variant at run time.
The `dimension-name` and `variant-key` are used as identifiers in the benchmark
result and for selecting variants at run time.
variants variables
~~~~~~~~~~~~~~~~~~
......@@ -204,7 +210,7 @@
Exactly one variant must have it set to `true` per dimension.
When set and no variant have been explicitly selected for this dimension, this
When set and no variant has been explicitly selected for this dimension, this
variant will be used.
args
......@@ -208,10 +214,10 @@
variant will be used.
args
~~~~~~~~~~~~~~
~~~~
A list of args to add to the command
cwd
~~~
......@@ -212,11 +218,11 @@
A list of args to add to the command
cwd
~~~
overwrite the global `hg-perf-ext.cwd` variable. Behave the same.
Overwrites the global `hg-perf-ext.cwd` variable. Behaves the same.
setup-script
~~~~~~~~~~~~
......@@ -219,5 +225,5 @@
setup-script
~~~~~~~~~~~~
overwrite the global `hg-perf-ext.setup-script` variable. Behave the same.
Overwrites the global `hg-perf-ext.setup-script` variable. Behaves the same.
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment