SCMPerf v2 : the empire strikes back
Write a new version of SCMPerf comprised of different modules
Modules
- Something that creates a benchmarking environment
- Something that can run benchmarks in said environment and save to format (CBOR?)
- Something to read and diff the results format (CLI)
- Something to draw graphs in a WebUI
- Something to find regressions
- Something to schedule
- Some kind of database backend
First milestone
-
Wrapper around hyperfine to write the format (Megafine?) -
Needs to record which env it is in (also, detect whether inside a megafence) -
Run the benchmark (process-level granularity) -
Record the results in a file
-
-
Define a format that is semver (probably something with CBOR) -
Something to read and diff the results format (CLI)