Skip to content
Snippets Groups Projects
  1. Jun 19, 2023
  2. Jun 17, 2023
  3. Apr 20, 2023
  4. Apr 17, 2023
  5. Jun 12, 2023
    • Raphaël Gomès's avatar
      match: add `filepath:` pattern to match an exact filepath relative to the root · 1c31b343
      Raphaël Gomès authored
      It's useful in certain automated workflows to make sure we recurse in
      directories whose name conflicts with files in other revisions.
      
      In addition it makes it possible to avoid building a potentially costly regex,
      improving performance when the set of files to match explicitly is large.
      
      The benchmark below are run in the following configuration :
       # data-env-vars.name             = mozilla-central-2018-08-01-zstd-sparse-revlog
       # benchmark.name                 = files
       # benchmark.variants.rev         = tip
       # benchmark.variants.files       = all-list-filepath-sorted
       # bin-env-vars.hg.flavor         = no-rust
      
      It also includes timings using the re2 engine (through the `google-re2` module)
      to show how much can be saved by just using a better regexp engine.
      
      
      Pattern                time (seconds)        time using re2
      -----------------------------------------------------------
      just "."                0.4                    0.4
      list of "filepath:…"    1.3                    1.3
      list of "path:…"       25.7                    3.9
      list of patterns       29.7                   10.4
      
      As you can see, Without re2, using "filepath:" instead of "path:" is a huge
      win. With re2, it is still about three times faster to not have to build the
      regex.
      1c31b343
  6. Jun 17, 2023
  7. Jun 20, 2023
    • Pierre-Yves David's avatar
      backout: remove db9b6ce803a9 · b9a35576
      Pierre-Yves David authored
      That changeset lost its topic for unknown reason and ended up being picked by
      some automation. So I backout it previous incarnation until it can get properly
      reviewed.
      b9a35576
  8. Jun 19, 2023
  9. Jun 13, 2023
    • Raphaël Gomès's avatar
      rust-dependencies: switch from `users` to `whoami` · d39ac346
      Raphaël Gomès authored
      `users` is unmaintained, with the maintainer apparently MIA. `whoami` is a
      popular crate that does simple wrapping of platform-specific calls.
      
      This makes the overhead from using `blackbox` lower. It used to be up 10ms
      on slower hardware for both calls, now down to <1ms which is always good.
      d39ac346
  10. Jun 12, 2023
    • Raphaël Gomès's avatar
      rust-hg-core: move from `ouroboros` to `self_cell` · 2cc5de26
      Raphaël Gomès authored
      `ouroboros` has a fundamental soundness problem that, while not applicable
      today, could become applicable given new compiler optimizations.¹
      
      `self_cell` is a crate that accomplishes a lot of the same things that
      `ouroboros` did while remaining sound (that is, unless a new soundness issue
      is discovered) by not assuming as much about the memory layout of the program.
      `self_cell` has been scrutinized heavily in the past few months by very
      competent people, some from the compiler team and has shown no weaknesses
      for a while, with a 1.0 stable release coming out a couple months ago.
      
      Our internal API is exactly the same, this is just an implementation detail.
      To reiterate, no actual soundness issue was found with our use of `ouroboros`,
      but there might be evolutions of `rustc` (or even a future separate compiler)
      that could generate unsound code.
      
      [1] https://github.com/joshua-maros/ouroboros/issues/88
      2cc5de26
  11. Jun 02, 2023
  12. Jun 12, 2023
  13. Jun 06, 2023
  14. Jun 12, 2023
  15. Jun 05, 2023
  16. May 18, 2023
  17. May 31, 2023
    • Arseniy Alekseyev's avatar
      stream-clone: avoid opening a revlog in case we do not need it · 3b563954
      Arseniy Alekseyev authored
      Opening an revlog has a cost, especially if it is inline as we have to scan the
      file and construct an index.
      
      To prevent the associated slowdown, we just do a minimal scan to check that an
      inline file is still inline, and simply stream the file without creating a
      revlog when we can.
      
      This provides a big boost compared to the previous changeset, even if the full
      generation is still penalized by the initial gathering of information.
      
      All benchmarks are run on linux with Python 3.10.7.
      # benchmark.name             = hg.exchange.stream.generate
      # benchmark.variants.version = v2
      
      ### Compared to the previous changesets
      
      We get a large win all across the board!
      
      # mercurial-2018-08-01-zstd-sparse-revlog
      before:  0.250694 seconds
      after:   0.105986 seconds  (-57.72%)
      # pypy-2018-08-01-zstd-sparse-revlog
      before:  3.885657 seconds
      after:   1.709748 seconds  (-56.00%)
      # netbeans-2018-08-01-zstd-sparse-revlog
      before: 16.679371 seconds
      after:   7.687469 seconds  (-53.91%)
      # mozilla-central-2018-08-01-zstd-sparse-revlog
      before: 38.575482 seconds
      after:  17.520316 seconds  (-54.58%)
      # mozilla-try-2019-02-18-zstd-sparse-revlog
      before: 81.160994 seconds
      after:  37.073753 seconds  (-54.32%)
      
      
      
      ### Compared to 6.4.3
      
      We are still significantly slower than 6.4.3, the extra time is usually twice
      slower than the extra time we observe on the locked section, which is a quite
      interesting information.
      
      Except for mercurial-central that is much faster. That discrepancy is not really
      explained yet.
      
      # mercurial-2018-08-01-zstd-sparse-revlog
      6.4.3:  0.072560 seconds
      after:  0.105986 seconds  (+46.07%) (- 0.03 seconds)
      # pypy-2018-08-01-zstd-sparse-revlog
      6.4.3:  1.211193 seconds
      after:  1.709748 seconds  (+41.16%) (-0.45 seconds)
      # netbeans-2018-08-01-zstd-sparse-revlog
      6.4.3:  4.932843 seconds
      after:  7.687469 seconds  (+55.84%) (-2.75 seconds)
      # mozilla-central-2018-08-01-zstd-sparse-revlog
      6.4.3: 34.012226 seconds
      after: 17.520316 seconds  (-48.49%) (-16.49 seconds)
      # mozilla-try-2019-02-18-zstd-sparse-revlog
      6.4.3: 23.850555 seconds
      after: 37.073753 seconds  (+55.44%) (+13.22 seconds)
      3b563954
  18. May 30, 2023
    • Pierre-Yves David's avatar
      store: stop relying on a `revlog_type` property · e06d1a77
      Pierre-Yves David authored
      We want to know if a file is related to a revlog, but the rest is dealt with
      differently already, so we simplify things further.
      
      as a bonus, this cleanup This provides a small but noticeable  speedup.
      
      The number below use `hg perf::stream-locked-section` to measure the time spend
      in the locked section of the streaming clone. Number are run on various
      repository and compare different steps.:
      
      1) the effect of this patchs,
      2) the effect of the cleanup series,
      2) current state compared to because large refactoring.
      
      All benchmarks are run on linux with Python 3.10.7.
      
      
      ### Effect of this patch
      # mercurial-2018-08-01-zstd-sparse-revlog
        # benchmark.name         = perf-stream-locked-section
      before:   0.030246 seconds
      after:    0.029274 seconds   (-3.21%)
      # pypy-2018-08-01-zstd-sparse-revlog
      before:   0.545012 seconds
      after:    0.520872 seconds   (-4.43%)
      # netbeans-2018-08-01-zstd-sparse-revlog
      before:   2.719939 seconds
      after:    2.626791 seconds   (-3.42%)
      # mozilla-central-2018-08-01-zstd-sparse-revlog
      before:   6.304179 seconds
      after:    6.096700 seconds   (-3.29%)
      # mozilla-try-2019-02-18-zstd-sparse-revlog
      before:  14.142687 seconds
      after:   13.640779 seconds   (-3.55%)
      
      ### Effect of this series
      
      A small but sizeable speedup
      
      # mercurial-2018-08-01-zstd-sparse-revlog
      before:  0.031122 seconds
      after:   0.029274 seconds   (-5.94%)
      # pypy-2018-08-01-zstd-sparse-revlog
      before:  0.589970 seconds
      after:   0.520872 seconds  (-11.71%)
      # netbeans-2018-08-01-zstd-sparse-revlog
      before:  2.980300 seconds
      after:   2.626791 seconds  (-11.86%)
      # mozilla-central-2018-08-01-zstd-sparse-revlog
      before:  6.863204 seconds
      after:   6.096700 seconds  (-11.17%)
      # mozilla-try-2019-02-18-zstd-sparse-revlog
      before: 14.921393 seconds
      after:  13.640779 seconds   (-8.58%)
      
      
      ### Current state compared to the pre-refactoring state
      
      The refactoring introduced multiple string manipulation and dictionary creation
      that seems to induce a signifiant slowdown
      Slowdown
      
      # mercurial-2018-08-01-zstd-sparse-revlog
      6.4.3:  0.019459 seconds
      after:  0.029274 seconds  (+50.44%)
      ## pypy-2018-08-01-zstd-sparse-revlog
      6.4.3:  0.290715 seconds
      after:  0.520872 seconds  (+79.17%)
      # netbeans-2018-08-01-zstd-sparse-revlog
      6.4.3:  1.403447 seconds
      after:  2.626791 seconds  (+87.17%)
      # mozilla-central-2018-08-01-zstd-sparse-revlog
      6.4.3:  3.163549 seconds
      after:  6.096700 seconds  (+92.72%)
      # mozilla-try-2019-02-18-zstd-sparse-revlog
      6.4.3:  6.702184 seconds
      after: 13.640779 seconds (+103.53%)
      e06d1a77
    • Pierre-Yves David's avatar
      store: directly pass the filesize in the `details` of revlog · 11562d72
      Pierre-Yves David authored
      The dictionary only contains 1 (or 0) entries, we can directly store that
      information (or None).
      
      Moving to a simpler argument passing result in a noticable speedup (because
      Python)
      
      The number below use `hg perf::stream-locked-section` to measure the time spend
      in the locked section of the streaming clone. Number are run on various
      repository.
      
      ### mercurial-2018-08-01-zstd-sparse-revlog
      before: 0.031247 seconds
      after:  0.030246 seconds   (-3.20%)
      ### mozilla-central-2018-08-01-zstd-sparse-revlog
      before: 6.718968 seconds
      after:  6.304179 seconds   (-6.17%)
      ### mozilla-try-2019-02-18-zstd-sparse-revlog
      before: 14.631343 seconds
      after:  14.142687 seconds   (-3.34%)
      ### netbeans-2018-08-01-zstd-sparse-revlog
      before: 2.895584 seconds
      after:  2.719939 seconds   (-6.07%)
      ### pypy-2018-08-01-zstd-sparse-revlog
      before: 0.561843 seconds
      after:  0.543034 seconds   (-3.35%)
      11562d72
    • Pierre-Yves David's avatar
      store: explicitly pass file_size when creating StoreFile · cdb471c8
      Pierre-Yves David authored
      A small cleanup before large cleanup in the next patch.
      cdb471c8
    • Pierre-Yves David's avatar
      store: have the revlog determine which files are volatile itself · 6b522a9e
      Pierre-Yves David authored
      This is a first step toward simplifying the walk step.
      6b522a9e
  19. Jun 12, 2023
  20. Jun 08, 2023
  21. Jun 05, 2023
  22. Jun 08, 2023
Loading