Skip to content
Snippets Groups Projects
Commit 7f848cfc4286 authored by Pierre-Yves David's avatar Pierre-Yves David :octopus:
Browse files

stream-clone-v2: use dedicated threads to write the data on disk

This could provide a significant performance boost, but various implementation
details means it is currently slower. More update will help make this boost
real, but here is the basic idea. The implementation in this patch is unbounded
in memory which could be a problem in some situation. We will deal with that
soon.

There is the benchmark result showing a slower run

### benchmark.name                         = hg.perf.exchange.stream.consume
  # bin-env-vars.hg.flavor                 = default
  # bin-env-vars.hg.py-re2-module          = default
  # benchmark.variants.parallel-processing = yes
  # benchmark.variants.progress            = no
  # benchmark.variants.read-from-memory    = yes
  # benchmark.variants.version             = v2
 ## data-env-vars.name                     = mercurial-public-2024-03-22-zstd-sparse-revlog
before: 0.249693  ~~~~~
after:  0.275081  (+10.17%, +0.03)
 ## data-env-vars.name                     = netbsd-xsrc-all-2024-09-19-zstd-sparse-revlog
before: 5.317709  ~~~~~
after:  6.783031  (+27.56%, +1.47)
 ## data-env-vars.name                     = netbsd-xsrc-draft-2024-09-19-zstd-sparse-revlog
before: 5.398368  ~~~~~
after:  6.737864  (+24.81%, +1.34)
 ## data-env-vars.name                     = pypy-2024-03-22-zstd-sparse-revlog
before: 3.acbb55  ~~~~~
after:  3.758324  (+22.48%, +0.69)
 ## data-env-vars.name                     = heptapod-public-2024-03-25-zstd-sparse-revlog
before: 7.244015  ~~~~~
after:  9.128669  (+26.02%, +1.88)
 ## data-env-vars.name                     = netbeans-2019-11-07-zstd-sparse-revlog
before: 13.136674  ~~~~~
after:  16.374306  (+24.65%, +3.24)
 ## data-env-vars.name                     = mozilla-unified-2024-03-22-zstd-sparse-revlog
before: 52.253858  ~~~~~
after:  66.955037 (+28.13%, +14.70)
 ## data-env-vars.name                     = mozilla-central-2024-03-22-zstd-sparse-revlog
before: 51.934795  ~~~~~
after:  66.561340 (+28.16%, +14.63)
 ## data-env-vars.name                     = mozilla-try-2024-03-26-zstd-sparse-revlog
  # benchmark.variants.read-from-memory    = no
before: 130.584329  ~~~~~
after:  138.770454   (+6.27%, +8.19)
parent 307c4a0b91a0
2 merge requests!1292Draft: 7.0rc preparation,!1217add the ability to use multiple thread to apply stream bundles
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment