Skip to content
Snippets Groups Projects
Commit 9b61422e authored by Pierre-Yves David's avatar Pierre-Yves David :octopus:
Browse files

test: add a smoke test for the whole process

This check the overall run process from scratch works fine
parent 7e4a958a
No related branches found
No related tags found
No related merge requests found
......@@ -10,8 +10,8 @@
BASEDIR = os.path.abspath(os.path.join(os.path.dirname(__file__), os.path.pardir))
STRIP_VARIANTS_PATH = os.path.join(BASEDIR, "partial-sets.yaml")
REPOS_DIR = os.path.join(BASEDIR, "repos")
REPOS = [d for d in os.listdir(REPOS_DIR)
if os.path.isdir(os.path.join(REPOS_DIR, d, ".hg"))]
REPOS = sorted(d for d in os.listdir(REPOS_DIR)
if os.path.isdir(os.path.join(REPOS_DIR, d, ".hg")))
# TODO fix
# with open(STRIP_VARIANTS_PATH) as f:
# STRIP_VARIANTS = yaml.load(f.read())["partial-sets"]
......
......@@ -31,7 +31,7 @@
shift
# Launch asv
time taskset -c 2,3 asv run --show-stderr --skip-existing-successful ${target} $@
time taskset -c 2,3 asv run --show-stderr --skip-existing-successful ${target} "$@"
# Publish results
asv publish --no-pull
......
=============================
Make sure basic ASV run works
=============================
#require slow
initialize reference
$ export HGRCPATH=
$ export PATH=$TESTDIR/../repo-scripts/:${PATH}
$ mkdir references
$ hg init test-repo
$ hg -R test-repo debugbuilddag --new-file .+2000
$ cd references
$ make-reference ../test-repo smoke-test-A
hg cloning repository from: ../test-repo
building a reference tarball
result available at: smoke-test-A-reference.tar
$ make-reference ../test-repo smoke-test-B
hg cloning repository from: ../test-repo
building a reference tarball
result available at: smoke-test-B-reference.tar
$ make-partial-repos $TESTDIR/../partial-sets.yaml *.benchrepo
Cloning smoke-test-A (smoke-test-A-reference stripped of last(all(), 100)) into ./smoke-test-A-partial-last-hundred (for exchange benchmarks)
Cloning smoke-test-A (smoke-test-A-reference stripped of last(all(), 10)) into ./smoke-test-A-partial-last-ten (for exchange benchmarks)
Cloning smoke-test-A (smoke-test-A-reference stripped of last(all(), 1000)) into ./smoke-test-A-partial-last-thousand (for exchange benchmarks)
Cloning smoke-test-A (smoke-test-A-reference stripped of None) into ./smoke-test-A-partial-same (for exchange benchmarks)
Cloning smoke-test-B (smoke-test-B-reference stripped of last(all(), 100)) into ./smoke-test-B-partial-last-hundred (for exchange benchmarks)
Cloning smoke-test-B (smoke-test-B-reference stripped of last(all(), 10)) into ./smoke-test-B-partial-last-ten (for exchange benchmarks)
Cloning smoke-test-B (smoke-test-B-reference stripped of last(all(), 1000)) into ./smoke-test-B-partial-last-thousand (for exchange benchmarks)
Cloning smoke-test-B (smoke-test-B-reference stripped of None) into ./smoke-test-B-partial-same (for exchange benchmarks)
$ cd ..
setup a copy of the asv repo
$ rsync --exclude 'repos' --exclude 'mercurial' -ra $TESTDIR/.. test-asv
$ cd test-asv
$ cat << EOF > repos.make
> SOURCE="$TESTTMP/references"
> all: repos/smoke-test-A.benchrepo repos/smoke-test-B.benchrepo
>
> repos/smoke-test-A.benchrepo:
> (cd repos; tar xf \$(SOURCE)/smoke-test-A-reference.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-A-partial-last-hundred.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-A-partial-last-ten.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-A-partial-last-thousand.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-A-partial-same.tar --warning=no-timestamp )
> repos/smoke-test-B.benchrepo:
> (cd repos; tar xf \$(SOURCE)/smoke-test-B-reference.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-B-partial-last-hundred.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-B-partial-last-ten.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-B-partial-last-thousand.tar --warning=no-timestamp )
> (cd repos/.cache/; tar xf \$(SOURCE)/smoke-test-B-partial-same.tar --warning=no-timestamp )
> EOF
setup appropriate virtual env
$ virtualenv $TESTTMP/venv 1>/dev/null 2>&1
$ export PATH=$TESTTMP/venv/bin:${PATH}
$ pip install --requirement requirements.txt --quiet
run
$ ASV_SKIP_SYSTEM_CHECK=1 ./launch.sh fe3ca1e6 -b track_commit
+ asv machine --yes
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
* (glob)
+ echo check that system is ready for benchmark
check that system is ready for benchmark
+ echo '(use ASV_SKIP_SYSTEM_CHECK=1 to skip it)'
(use ASV_SKIP_SYSTEM_CHECK=1 to skip it)
+ '[' -z 1 ']'
+ '[' '!' -d mercurial ']'
+ hg clone https://www.mercurial-scm.org/repo/hg mercurial
requesting all changes
adding changesets
adding manifests
adding file changes
added * changesets with * changes to * files (glob)
new changesets * (glob)
updating to bookmark @* (glob)
* files updated, 0 files merged, 0 files removed, 0 files unresolved (glob)
+ mkdir -p repos/
+ mkdir -p repos/.cache
+ make MAKE_VERBOSE=1 -f repos.make
(cd repos; tar xf "$TESTTMP/references"/smoke-test-A-reference.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-A-partial-last-hundred.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-A-partial-last-ten.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-A-partial-last-thousand.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-A-partial-same.tar --warning=no-timestamp )
(cd repos; tar xf "$TESTTMP/references"/smoke-test-B-reference.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-B-partial-last-hundred.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-B-partial-last-ten.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-B-partial-last-thousand.tar --warning=no-timestamp )
(cd repos/.cache/; tar xf "$TESTTMP/references"/smoke-test-B-partial-same.tar --warning=no-timestamp )
+ ./repo-scripts/rename-reference-directories repos
Renaming repos/smoke-test-A-reference into repos/smoke-test-A
Renaming repos/smoke-test-B-reference into repos/smoke-test-B
+ python create_skip_file.py
+ target=fe3ca1e6
+ shift
+ taskset -c 2,3 asv run --show-stderr --skip-existing-successful fe3ca1e6 -b track_commit
- Fetching recent changes
- Creating environments
- Discovering benchmarks
-- Uninstalling from virtualenv-py2.7
-- Building fe3ca1e6 for virtualenv-py2.7
-- Installing into virtualenv-py2.7
- Running 1 total benchmarks (1 commits * 1 environments * 1 benchmarks)
[ 0.00%] - For mercurial commit hash fe3ca1e6:
[ 0.00%] -- Building for virtualenv-py2.7
[ 0.00%] -- Benchmarking virtualenv-py2.7
[100.00%] --- Running basic_commands.TestSuite.track_commit ok
[100.00%] ----
============== =====================
repo
-------------- ---------------------
smoke-test-A * (glob)
smoke-test-B * (glob)
============== =====================
real * (glob)
user * (glob)
sys * (glob)
+ asv publish --no-pull
[ 11.11%] - Loading machine info
[ 22.22%] - Getting params, commits, tags and branches
[ 33.33%] - Loading results
[ 44.44%] - Detecting steps
[ 55.56%] - Generating graphs
[ 66.67%] - Generating output for SummaryGrid
[ 77.78%] - Generating output for SummaryList
[ 88.89%] - Generating output for Regressions
[100.00%] - Writing index
+ echo DONE
DONE
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment