Skip to content
Snippets Groups Projects
  1. Feb 22, 2025
  2. Jul 29, 2024
    • Raphaël Gomès's avatar
      revlog: add glue to use a pure-Rust VFS · 72bc29f01570
      Raphaël Gomès authored
      This will save us a lot of calling back into Python, which is always
      horribly expensive.
      
      We are now faster in all benchmarked cases except for `log --patch`
      specifically on mozilla-try. Fixing this will happen in a later patch.
      
      ```
      ### data-env-vars.name            = mercurial-devel-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 1.760765  ~~~~~
      5559d7e63ec3: 1.555513  (-11.66%, -0.21)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 62.848869  ~~~~~
      5559d7e63ec3: 58.113051   (-7.54%, -4.74)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 3.173532  ~~~~~
      5559d7e63ec3: 3.543591  (+11.66%, +0.37)
      ### data-env-vars.name            = mozilla-try-2024-03-26-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 1.214698  ~~~~~
      5559d7e63ec3: 1.192478   (-1.83%, -0.02)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 56.205474  ~~~~~
      5559d7e63ec3: 51.520074   (-8.34%, -4.69)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 2.105419  ~~~~~
      5559d7e63ec3: 2.051849   (-2.54%, -0.05)
      ### data-env-vars.name            = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.309960  ~~~~~
      5559d7e63ec3: 0.299035   (-3.52%, -0.01)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.cat
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.files      = all-root
        # benchmark.variants.output     = plain
        # benchmark.variants.rev        = tip
      e679697a6ca4: 1.849832  ~~~~~
      5559d7e63ec3: 1.805076   (-2.42%, -0.04)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 10
        # benchmark.variants.patch      = yes
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.289521  ~~~~~
      5559d7e63ec3: 0.279889   (-3.33%, -0.01)
      ### data-env-vars.name            = tryton-public-2024-03-22-ds2-pnm
        # benchmark.name                = hg.command.log
        # bin-env-vars.hg.flavor        = rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.limit-rev  = 1000
        # benchmark.variants.patch      = no
        # benchmark.variants.rev        = none
      e679697a6ca4: 0.332270  ~~~~~
      5559d7e63ec3: 0.323324   (-2.69%, -0.01)
      ```
      72bc29f01570
  3. Jun 19, 2024
    • Raphaël Gomès's avatar
      revlog: add the glue to use the Rust `InnerRevlog` from Python · 7346f93be7a4
      Raphaël Gomès authored
      The performance of this has been looked at for quite some time, and some
      workflows are actually quite a bit faster than with the Python + C code.
      
      However, we are still (up to 20%) slower in some crucial places like cloning
      certain repos, log, cat, which makes this an incomplete rewrite. This is
      mostly due to the high amount of overhead in Python <-> Rust FFI, especially
      around the VFS code. A future patch series will rewrite the VFS code in
      pure Rust, which should hopefully get us up to par with current perfomance,
      if not better in all important cases.
      
      This is a "save state" of sorts, as this is a ton of code, and I don't want
      to pile up even more things in a single review.
      
      Continuing to try to match the current performance will take an extremely
      long time, if it's not impossible, without the aforementioned VFS work.
      7346f93be7a4
  4. Mar 10, 2023
  5. May 02, 2022
  6. Feb 02, 2022
  7. Feb 01, 2022
  8. Dec 13, 2021
  9. Nov 09, 2021
    • Simon Sapin's avatar
      rhg: Enable `rhg status` in most tests · b7fde9237c92
      Simon Sapin authored
      This subcommand is disabled by default because of bugs that make some test fail.
      Enable it in the rest of the tests in order to avoid regressing them.
      
      As with `RHG_ON_UNSUPPORTED`, an environment variable is used instead of
      a configuration file and `HGRCPATH` because some tests override `HGRCPATH`.
      
      Running `unset RHG_STATUS` at the start of a test restores the default of
      `rhg status` being disabled. Hopefully it can be increasingly removed
      from test files as bugs are fixed.
      
      Differential Revision: https://phab.mercurial-scm.org/D11756
      b7fde9237c92
  10. Oct 19, 2021
    • Pierre-Yves David's avatar
      dirstate-v2: freeze the on-disk format · bf11ff22a9af
      Pierre-Yves David authored
      It seems the format as reached a good balance. With a core of new capabilities
      that motivated it initially and enough new feature and room for future
      improvement to be a clear progress we can set a milestone for.
      
      Having the format frozen will help the feature to get real life testing, outside
      of the test suite.
      
      The feature itself stay experimental but the config gains a new name to avoid
      people enable non-frozen version by default.
      
      If too many bugs are reported during the RC we might move the format back to
      experimental and drop its support in future version (in favor of a new one)
      
      Differential Revision: https://phab.mercurial-scm.org/D11709
      bf11ff22a9af
  11. Oct 14, 2021
  12. Jul 16, 2021
    • Simon Sapin's avatar
      dirstate-v2: Add support when Rust is not enabled · b4f83c9e7905
      Simon Sapin authored
      This wires into `dirstatemap` the parser and serializer added in previous
      changesets. The memory representation is still the same, with a flat `dict`
      for `DirstateItem`s and another one for copy sources. Serialization always
      creates a new dirstate-v2 data file and does not support (when Rust is not
      enabled) appending to an existing one, since we don’t keep track of which
      tree nodes are new or modified. Instead the tree is reconstructed during
      serialization.
      
      Differential Revision: https://phab.mercurial-scm.org/D11520
      b4f83c9e7905
  13. Sep 27, 2021
    • Simon Sapin's avatar
      dirstate: Remove the flat Rust DirstateMap implementation · bf8837e3d7ce
      Simon Sapin authored
      Before this changeset we had two Rust implementations of `DirstateMap`.
      This removes the "flat" DirstateMap so that the "tree" DirstateMap is always
      used when Rust enabled. This simplifies the code a lot, and will enable
      (in the next changeset) further removal of a trait abstraction.
      
      This is a performance regression when:
      
      * Rust is enabled, and
      * The repository uses the legacy dirstate-v1 file format, and
      * For `hg status`, unknown files are not listed (such as with `-mard`)
      
      The regression is about 100 milliseconds for `hg status -mard` on a
      semi-large repository (mozilla-central), from ~320ms to ~420ms.
      We deem this to be small enough to be worth it.
      
      The new dirstate-v2 is still experimental at this point, but we aim to
      stabilize it (though not yet enable it by default for new repositories)
      in Mercurial 6.0. Eventually, upgrating repositories to dirsate-v2 will
      eliminate this regression (and enable other performance improvements).
      
      
      # Background
      
      The flat DirstateMap was introduced with the first Rust implementation of the
      status algorithm. It works similarly to the previous Python + C one, with a
      single `HashMap` that associates file paths to a `DirstateEntry` (where Python
      has a dict).
      
      We later added the tree DirstateMap where the root of the tree contains nodes
      for files and directories that are directly at the root of the repository,
      and nodes for directories can contain child nodes representing the files and
      directly that *they* contain directly. The shape of this tree mirrors that of
      the working directory in the filesystem. This enables the status algorithm to
      traverse this tree in tandem with traversing the filesystem tree, which in
      turns enables a more efficient algorithm.
      
      Furthermore, the new dirstate-v2 file format is also based on a tree of the
      same shape. The tree DirstateMap can access a dirstate-v2 file without parsing
      it: binary data in a single large (possibly memory-mapped) bytes buffer is
      traversed on demand. This allows `DirstateMap` creation to take `O(1)` time.
      (Mutation works by creating new in-memory nodes with copy-on-write semantics,
      and serialization is append-mostly.)
      
      The tradeoff is that for "legacy" repositories that use the dirstate-v1 file
      format, parsing that file into a tree DirstateMap takes more time. Profiling
      shows that this time is dominated by `HashMap`. For a dirstate containing `F`
      files with an average `D` directory depth, the flat DirstateMap does parsing
      in `O(F)` number of HashMap operations but the tree DirstateMap in `O(F × D)`
      operations, since each node has its own HashMap containing its child nodes.
      This slower costs ~140ms on an old snapshot of mozilla-central, and ~80ms
      on an old snapshot of the Netbeans repository.
      
      The status algorithm is faster, but with `-mard` (when not listing unknown
      files) it is typically not faster *enough* to compensate the slower parsing.
      
      Both Rust implementations are always faster than the Python + C implementation
      
      
      # Benchmark results
      
      All benchmarks are run on changeset 98c0408324e6, with repositories that use
      the dirstate-v1 file format, on a server with 4 CPU cores and 4 CPU threads
      (no HyperThreading).
      
      `hg status` benchmarks show wall clock times of the entire command as the
      average and standard deviation of serveral runs, collected by
      https://github.com/sharkdp/hyperfine and reformated.
      
      Parsing benchmarks are wall clock time of the Rust function that converts a
      bytes buffer of the dirstate file into the `DirstateMap` data structure as
      used by the status algorithm. A single run each, collected by running
      `hg status` this environment variable:
      
          RUST_LOG=hg::dirstate::dirstate_map=trace,hg::dirstate_tree::dirstate_map=trace
      
      
      Benchmark 1: Rust flat DirstateMap → Rust tree DirstateMap
      
          hg status
      
          mozilla-clean    	562.3 ms ± 2.0 ms	→	462.5 ms ± 0.6 ms	 1.22 ± 0.00 times faster
          mozilla-dirty    	859.6 ms ± 2.2 ms	→	719.5 ms ± 3.2 ms	 1.19 ± 0.01 times faster
          mozilla-ignored  	558.2 ms ± 3.0 ms	→	457.9 ms ± 2.9 ms	 1.22 ± 0.01 times faster
          mozilla-unknowns 	859.4 ms ± 5.7 ms	→	716.0 ms ± 4.7 ms	 1.20 ± 0.01 times faster
          netbeans-clean   	336.5 ms ± 0.9 ms	→	339.5 ms ± 0.4 ms	 0.99 ± 0.00 times faster
          netbeans-dirty   	491.4 ms ± 1.6 ms	→	475.1 ms ± 1.2 ms	 1.03 ± 0.00 times faster
          netbeans-ignored 	343.7 ms ± 1.0 ms	→	347.8 ms ± 0.4 ms	 0.99 ± 0.00 times faster
          netbeans-unknowns	484.3 ms ± 1.0 ms	→	466.0 ms ± 1.2 ms	 1.04 ± 0.00 times faster
      
          hg status -mard
      
          mozilla-clean    	317.3 ms ± 0.6 ms	→	422.5 ms ± 1.2 ms	 0.75 ± 0.00 times faster
          mozilla-dirty    	315.4 ms ± 0.6 ms	→	417.7 ms ± 1.1 ms	 0.76 ± 0.00 times faster
          mozilla-ignored  	314.6 ms ± 0.6 ms	→	417.4 ms ± 1.0 ms	 0.75 ± 0.00 times faster
          mozilla-unknowns 	312.9 ms ± 0.9 ms	→	417.3 ms ± 1.6 ms	 0.75 ± 0.00 times faster
          netbeans-clean   	212.0 ms ± 0.6 ms	→	283.6 ms ± 0.8 ms	 0.75 ± 0.00 times faster
          netbeans-dirty   	211.4 ms ± 1.0 ms	→	283.4 ms ± 1.6 ms	 0.75 ± 0.01 times faster
          netbeans-ignored 	211.4 ms ± 0.9 ms	→	283.9 ms ± 0.8 ms	 0.74 ± 0.01 times faster
          netbeans-unknowns	211.1 ms ± 0.6 ms	→	283.4 ms ± 1.0 ms	 0.74 ± 0.00 times faster
      
          Parsing
      
          mozilla-clean       38.4ms → 177.6ms
          mozilla-dirty       38.8ms → 177.0ms
          mozilla-ignored     38.8ms → 178.0ms
          mozilla-unknowns    38.7ms → 176.9ms
          netbeans-clean      16.5ms → 97.3ms
          netbeans-dirty      16.5ms → 98.4ms
          netbeans-ignored    16.9ms → 97.4ms
          netbeans-unknowns   16.9ms → 96.3ms
      
      
      Benchmark 2: Python + C dirstatemap → Rust tree DirstateMap
      
          hg status
      
          mozilla-clean    	1261.0 ms ± 3.6 ms	→	461.1 ms ± 0.5 ms	 2.73 ± 0.00 times faster
          mozilla-dirty    	2293.4 ms ± 9.1 ms	→	719.6 ms ± 3.6 ms	 3.19 ± 0.01 times faster
          mozilla-ignored  	1240.4 ms ± 2.3 ms	→	457.7 ms ± 1.9 ms	 2.71 ± 0.00 times faster
          mozilla-unknowns 	2283.3 ms ± 9.0 ms	→	719.7 ms ± 3.8 ms	 3.17 ± 0.01 times faster
          netbeans-clean   	879.7 ms ± 3.5 ms	→	339.9 ms ± 0.5 ms	 2.59 ± 0.00 times faster
          netbeans-dirty   	1257.3 ms ± 4.7 ms	→	474.6 ms ± 1.6 ms	 2.65 ± 0.01 times faster
          netbeans-ignored 	943.9 ms ± 1.9 ms	→	347.3 ms ± 1.1 ms	 2.72 ± 0.00 times faster
          netbeans-unknowns	1188.1 ms ± 5.0 ms	→	465.2 ms ± 2.3 ms	 2.55 ± 0.01 times faster
      
          hg status -mard
      
          mozilla-clean    	903.2 ms ± 3.6 ms	→	423.4 ms ± 2.2 ms	 2.13 ± 0.01 times faster
          mozilla-dirty    	884.6 ms ± 4.5 ms	→	417.3 ms ± 1.4 ms	 2.12 ± 0.01 times faster
          mozilla-ignored  	881.9 ms ± 1.3 ms	→	417.3 ms ± 0.8 ms	 2.11 ± 0.00 times faster
          mozilla-unknowns 	878.5 ms ± 1.9 ms	→	416.4 ms ± 0.9 ms	 2.11 ± 0.00 times faster
          netbeans-clean   	434.9 ms ± 1.8 ms	→	284.0 ms ± 0.8 ms	 1.53 ± 0.01 times faster
          netbeans-dirty   	434.1 ms ± 0.8 ms	→	283.1 ms ± 0.8 ms	 1.53 ± 0.00 times faster
          netbeans-ignored 	431.7 ms ± 1.1 ms	→	283.6 ms ± 1.8 ms	 1.52 ± 0.01 times faster
          netbeans-unknowns	433.0 ms ± 1.3 ms	→	283.5 ms ± 0.7 ms	 1.53 ± 0.00 times faster
      
      Differential Revision: https://phab.mercurial-scm.org/D11516
      bf8837e3d7ce
  14. May 19, 2021
  15. May 03, 2021
  16. Jan 26, 2019
  17. Sep 04, 2018
  18. Apr 04, 2018
  19. Jul 12, 2017
  20. Aug 06, 2014
  21. Apr 13, 2014
  22. Nov 15, 2013
  23. Jun 10, 2012
    • kiilerix's avatar
      tests: add missing trailing 'cd ..' · f2719b387380
      kiilerix authored
      Many tests didn't change back from subdirectories at the end of the tests ...
      and they don't have to. The missing 'cd ..' could always be added when another
      test case is added to the test file.
      
      This change do that tests (99.5%) consistently end up in $TESTDIR where they
      started, thus making it simpler to extend them or move them around.
      f2719b387380
  24. Nov 07, 2011
  25. Oct 09, 2010
  26. Sep 22, 2010
    • Brodie Rao's avatar
      tests: add glob matching for unified tests · 97ffc68f71d3
      Brodie Rao authored
      This adds a " (glob)" marker that works like a simpler version of
      (re): "*" is converted to ".*", and "?" is converted to ".".
      
      Both special characters can be escaped using "\", and the backslash
      itself can be escaped as well.
      
      Other glob-style syntax, like "**", "[chars]", or "[!chars]", isn't
      supported.
      97ffc68f71d3
    • Brodie Rao's avatar
      tests: require regexes in unified tests to be marked with " (re)" · 02990e22150b
      Brodie Rao authored
      Consider this test:
      
        $ hg glog --template '{rev}:{node|short} "{desc}"\n'
        @  2:20c4f79fd7ac "3"
        |
        | o  1:38f24201dcab "2"
        |/
        o  0:2a18120dc1c9 "1"
      
      Because each line beginning with "|" can be compiled as a regular
      expression (equivalent to ".*|"), they will match any output.
      
      Similarly:
      
        $ echo foo
      
      
      The blank output line can be compiled as a regular expression and will
      also match any output.
      
      With this patch, none of the above output lines will be matched as
      regular expressions. A line must end in " (re)" in order to be matched
      as one.
      
      Lines are still matched literally first, so the following will pass:
      
        $ echo 'foo (re)'
        foo (re)
      02990e22150b
  27. Sep 21, 2010
  28. Sep 14, 2010
  29. Sep 02, 2010
    • Martin Geisler's avatar
      tests: remove unneeded -d flags · 4c94b6d0fb1c
      Martin Geisler authored
      Many tests fixed the commit date of their changesets at '1000000 0' or
      similar. However testing with "Mon Jan 12 13:46:40 1970 +0000" is not
      better than testing with "Thu Jan 01 00:00:00 1970 +0000", which is
      the default run-tests.py installs.
      
      Removing the unnecessary flag removes some clutter and will hopefully
      make it clearer what the tests are really trying to test. Some tests
      did not even change their output when the dates were changed, in which
      case the -d flag was truly irrelevant.
      
      Dates used in sequence (such as '0 0', '1 0', etc...) were left alone
      since they may make the test easier to understand.
      4c94b6d0fb1c
  30. Jul 22, 2010
  31. Oct 18, 2008
  32. Oct 15, 2008
  33. Nov 09, 2007
  34. Aug 23, 2007
  35. Dec 01, 2006
  36. May 08, 2006
  37. Mar 13, 2006
  38. Nov 03, 2005
Loading