Skip to content
Snippets Groups Projects
  1. Oct 08, 2024
  2. Oct 07, 2024
    • Matt Harbison's avatar
      tests: stabilize `test-largefiles-cache.t` on Windows · b99f0b76
      Matt Harbison authored
      The `undo.backup.dirstate.bck` was created by `hg commit -m 'add largefile'` at
      line 18, and deleted in the conditional block by `hg push -q http://localhost:$HGPORT/`
      at line 138.  That's... surprising, but probably not harmful, and can be debugged
      from Linux if it is a problem.
      
      This was showing up in `find src/.hg/largefiles/* | grep -E "(dirstate|$hash)"`.
      b99f0b76
    • Matt Harbison's avatar
      tests: stabilize `test-journal.t` on Windows · df887ca0
      Matt Harbison authored
      The file in these lines are double quoted, but single quoted on other platforms.
      Not sure why, other than `cmd.exe` doesn't recognize single quotes.  But it's a
      cosmetic difference, so glob over it and move on.
      df887ca0
    • Matt Harbison's avatar
      tests: stabilize `test-clonebundles.t` on Windows · d8b98196
      Matt Harbison authored
      The `remote: {foo,bar}` lines were different because `echo` in MSYS uses `\r\n`.
      I couldn't make it work with the previous echoing of individual lines, changing
      the internal `echo` to `printf "foo\n"`, because that output as "foon".  This
      works on Linux and Windows, so I'm not thinking too hard about it.
      d8b98196
    • Matt Harbison's avatar
      tests: cleanup some special casing of `seq` emitting '\r\n' · 92a708af
      Matt Harbison authored
      My guess is these predate the commit referenced in the previous commit.
      92a708af
    • Matt Harbison's avatar
      tests: force `seq` to print with '\n' EOL · 2924676d
      Matt Harbison authored
      It looks like consistent EOL is the reason for 0605726179a0, but now on py3,
      `print()` uses the platform EOL without regard to binary mode.  The tests mostly
      use this to loop over a sequence of number in the shell, but there are a handful
      that redirect output to a file.  Specifically, this fixes Windows runs of
      `test-bundle2-multiple-changegroups.t`, but there may be other tests this fixes.
      
      Some other `tests/*.py` files also set binary mode on stdout, but they also
      write bytes directly to `sys.stdout.buffer`.  I'm not doing that here because
      PyCharm flags these write calls for passing bytes instead of str (PyCharm is
      likely wrong, but possibly confused because the code falls back to `sys.stdout`
      if there is no `.buffer` attribute), and it's annoying.
      2924676d
    • Matt Harbison's avatar
      tests: stabilize `test-bundle-phase-internal.t` on Windows · a878d260
      Matt Harbison authored
      This is a silly difference, and we control the Windows side of things from
      mercurial/windows.py:195.  I'll swap and quote the values to be like `no-windows`
      at some point, but I suspect this exception output would appear elsewhere, and
      don't feel like waiting for a 2h+ test run to find all of them.
      a878d260
    • Matt Harbison's avatar
      tests: skip `test-wsgicgi.t` on MSYS · 46afce95
      Matt Harbison authored
      The test is attempting to set `PATH_INFO="/rev/\xe2\x80\x94"` into the
      environment, which it does.  The problem is that when MSYS sees a leading '/' in
      an environment variable, it thinks it's a unix filesystem path, so it "helpfully"
      prepends the Windows path to the MSYS root directory before running a non-MSYS
      process.  hgweb would then split this value on '/', so it would get 'C:' instead
      of 'rev', and return a 400 since that isn't a valid web command.
      
      I tried generating a *.bat file, but had trouble running that via `cmd.exe`
      inside the test.  I also tried generating an equivalent *.py launcher that would
      set the environment variables itself.  But there is no `os.environb` on Windows,
      and the value was getting mangled when put into the script.  So, I give up.  If
      it's encoding stuff on Windows, it's probably broken.
      46afce95
  3. Jun 27, 2024
  4. Oct 03, 2024
  5. Oct 09, 2024
  6. Sep 27, 2024
    • Pierre-Yves David's avatar
      head-revs: add a native implementation of the `stop_rev` parameter · 609700e5
      Pierre-Yves David authored
      This does not add too much complexity to the native code and help with
      branchmap v3 performance.
      
      Note that the final conversion of the heads from native-code to Python is still
      too costly, especially in Rust. In addition the current caching around headrevs
      is too simple and fragile. However these are an unrelated problem.
      
      
      ### benchmark.name                                 = hg.command.unbundle
        # bin-env-vars.hg.py-re2-module                  = default
        # benchmark.variants.issue6528                   = disabled
        # benchmark.variants.resource-usage              = default
        # benchmark.variants.reuse-external-delta-parent = yes
        # benchmark.variants.revs                        = any-1-extra-rev
        # benchmark.variants.source                      = unbundle
        # benchmark.variants.validate                    = default
        # benchmark.variants.verbosity                   = quiet
      
       ## data-env-vars.name     = netbeans-2018-08-01-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.233711  ~~~~~
      branch-v3 before: 0.239857   (+2.63%, +0.01)
      branch-v3 after:  0.239558   (+2.50%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.235230  ~~~~~
      branch-v3 before: 0.240972   (+2.44%, +0.01)
      branch-v3 after:  0.239917   (+1.99%, +0.00)
       ## data-env-vars.name     = netbeans-2018-08-01-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.255586  ~~~~~
      branch-v3 before: 0.268560   (+5.08%, +0.01)
      branch-v3 after:  0.262261   (+2.61%, +0.01)
      
       ## data-env-vars.name     = mozilla-central-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.339010  ~~~~~
      branch-v3 before: 0.349389   (+3.06%, +0.01)
      branch-v3 after:  0.348247   (+2.72%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.346525  ~~~~~
      branch-v3 before: 0.355661   (+2.64%, +0.01)
      branch-v3 after:  0.350906   (+1.26%, +0.00)
       ## data-env-vars.name     = mozilla-central-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.380202  ~~~~~
      branch-v3 before: 0.408851   (+7.54%, +0.03)
      branch-v3 after:  0.406511   (+6.92%, +0.03)
      
      
       ## data-env-vars.name     = mozilla-unified-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.412165  ~~~~~
      branch-v3 before: 0.427782   (+3.79%, +0.02)
      branch-v3 after:  0.422595   (+2.53%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.412397  ~~~~~
      branch-v3 before: 0.422354   (+2.41%, +0.01)
      branch-v3 after:  0.421079   (+2.11%, +0.01)
       ## data-env-vars.name     = mozilla-unified-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.429501  ~~~~~
      branch-v3 before: 0.443197   (+3.19%, +0.01)
      branch-v3 after:  0.449432   (+4.64%, +0.02)
      
       ## data-env-vars.name     = mozilla-try-2024-03-26-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        3.403171  ~~~~~
      branch-v3 before: 3.819477  (+12.23%, +0.42)
      branch-v3 after:  3.658482   (+7.50%, +0.26)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.454876  ~~~~~
      branch-v3 before: 3.590284   (+3.92%, +0.14)
      branch-v3 after:  3.545843   (+2.63%, +0.09)
       ## data-env-vars.name     = mozilla-try-2024-03-26-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.465435  ~~~~~
      branch-v3 before: 3.633278   (+4.84%, +0.17)
      branch-v3 after:  3.556074   (+2.62%, +0.09)
      609700e5
  7. Sep 25, 2024
  8. Sep 03, 2024
    • Pierre-Yves David's avatar
      branchmap-v3: introduce a "stop_rev" argument to `headsrevs` · 42a116f1
      Pierre-Yves David authored
      The `headsrevs` method of the revlog already have a `revs` argument to compute
      the headrevs of a limited set of heads. However, it disable the use of the
      native compiled code to compute the heads, which slows down the branchmap v3
      code a lot.
      
      The branchmap v3 usage is actually quite constrained as we will always only
      ignores a part at the top of the graph. So we could be significantly faster.
      
      We start by making small change to the python side to improve the situation and
      introduce the new API. More collaboration with the native code are coming later.
      
      This massively speedup operation and close most of the remaining gaps between
      branchmap-v3 and branchmap-v2. especially on repository with many revs like
      mozilla-try. A small overhead remains mostly because the `headrevs` logic
      currently has some inefficiently. We will look into them from there.
      
      ### benchmark.name                                 = hg.command.unbundle
        # bin-env-vars.hg.py-re2-module                  = default
        # benchmark.variants.issue6528                   = disabled
        # benchmark.variants.resource-usage              = default
        # benchmark.variants.reuse-external-delta-parent = yes
        # benchmark.variants.revs                        = any-1-extra-rev
        # benchmark.variants.source                      = unbundle
        # benchmark.variants.validate                    = default
        # benchmark.variants.verbosity                   = quiet
      
       ## data-env-vars.name                             = netbeans-2018-08-01-zstd-sparse-revlog
        # bin-env-vars.hg.flavor                         = default
      branch-v2:        0.233711  ~~~~~
      branch-v3 before: 0.368769  (+57.79%, +0.14)
      branch-v3 after:  0.239857   (+2.63%, +0.01)
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.235230  ~~~~~
      branch-v3 before: 0.372460  (+58.34%, +0.14)
      branch-v3 after:  0.240972   (+2.44%, +0.01)
       ## data-env-vars.name                             = netbeans-2018-08-01-ds2-pnm
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.255586  ~~~~~
      branch-v3 before: 0.318907  (+24.78%, +0.06)
      branch-v3 after:  0.268560   (+5.08%, +0.01)
      
       ## data-env-vars.name                             = mozilla-central-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor                         = default
      branch-v2:        0.339010  ~~~~~
      branch-v3 before: 0.349752   (+3.17%, +0.01)
      branch-v3 after:  0.349389   (+3.06%, +0.01)
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.346525  ~~~~~
      branch-v3 before: 0.354300   (+2.24%, +0.01)
      branch-v3 after:  0.355661   (+2.64%, +0.01)
       ## data-env-vars.name                             = mozilla-central-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.380202  ~~~~~
      branch-v3 before: 0.396293   (+4.23%, +0.02)
      branch-v3 after:  0.408851   (+7.54%, +0.03)
      
       ## data-env-vars.name                             = mozilla-unified-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor                         = default
      branch-v2:        0.412165  ~~~~~
      branch-v3 before: 0.424769   (+3.06%, +0.01)
      branch-v3 after:  0.427782   (+3.79%, +0.02)
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.412397  ~~~~~
      branch-v3 before: 0.421796   (+2.28%, +0.01)
      branch-v3 after:  0.422354   (+2.41%, +0.01)
       ## data-env-vars.name                             = mozilla-unified-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        0.429501  ~~~~~
      branch-v3 before: 0.443849   (+3.34%, +0.01)
      branch-v3 after:  0.443197   (+3.19%, +0.01)
      
       ## data-env-vars.name                             = mozilla-try-2024-03-26-zstd-sparse-revlog
        # bin-env-vars.hg.flavor                         = default
      branch-v2:        3.403171  ~~~~~
      branch-v3 before: 6.234055  (+83.18%, +2.83)
      branch-v3 after:  3.819477  (+12.23%, +0.42)
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        3.454876  ~~~~~
      branch-v3 before: 6.307813  (+82.58%, +2.85)
      branch-v3 after:  3.590284   (+3.92%, +0.14)
       ## data-env-vars.name                             = mozilla-try-2024-03-26-ds2-pnm
        # bin-env-vars.hg.flavor                         = rust
      branch-v2:        3.465435  ~~~~~
      branch-v3 before: 5.176076  (+49.36%, +1.71)
      branch-v3 after:  3.633278   (+4.84%, +0.17)
      42a116f1
    • Pierre-Yves David's avatar
      branchmap-v3: filter topo heads using node for performance reason · 41b8892a
      Pierre-Yves David authored
      The branchmap currently contains heads as nodeid. If we build a set of revnum
      with the topological heads, we need to turn the nodeid in the branchmap to revnum to
      be able to check if they are topo-heads. That nodeid → revnum lookup is
      "expensive" and adds up to something noticeable if you do it hundreds of thousand
      of time.
      
      Instead we turn all the topo-heads revnums into nodes and build a set. So we can
      directly test membership of the nodeids stored in the branchmap. That is much
      faster.
      
      Ideally we would have revnum in the branchmap and could directly test revnum
      against a revnum set and that would be even faster. However that's an adventure
      for another time.
      
      Without this change, the branchmap format "v3" was significantly slower than the
      "v2" format. With this changes, some of that gap is recovered
      
      With rust + persistent nodemap, this overhead was smaller because the extra
      lookup did not had to to build the nodemap from scratch.
      
      In addition the mozilla-unified repository is able to use the "pure_top" mode of
      branchmap v3, so it was not really affected by this.
      
      Future changeset will work of the remaining of the performance gap.
      
      
      ### benchmark.name                                 = hg.command.unbundle
        # bin-env-vars.hg.py-re2-module                  = default
        # benchmark.variants.issue6528                   = disabled
        # benchmark.variants.resource-usage              = default
        # benchmark.variants.reuse-external-delta-parent = yes
        # benchmark.variants.revs                        = any-1-extra-rev
        # benchmark.variants.source                      = unbundle
        # benchmark.variants.validate                    = default
        # benchmark.variants.verbosity                   = quiet
      
       ## data-env-vars.name     = netbeans-2018-08-01-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.233711  ~~~~~
      branch-v3 before: 0.380994  (+63.02%, +0.15)
      branch-v3 after:  0.368769  (+57.79%, +0.14)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.235230  ~~~~~
      branch-v3 before: 0.385060  (+63.70%, +0.15)
      branch-v3 after:  0.372460  (+58.34%, +0.14)
       ## data-env-vars.name     = netbeans-2018-08-01-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.255586  ~~~~~
      branch-v3 before: 0.317524  (+24.23%, +0.06)
      branch-v3 after:  0.318907  (+24.78%, +0.06)
      
       ## data-env-vars.name     = mozilla-central-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.339010  ~~~~~
      branch-v3 before: 0.410007  (+20.94%, +0.07)
      branch-v3 after:  0.349752   (+3.17%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.346525  ~~~~~
      branch-v3 before: 0.410428  (+18.44%, +0.06)
      branch-v3 after:  0.354300   (+2.24%, +0.01)
       ## data-env-vars.name     = mozilla-central-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.380202  ~~~~~
      branch-v3 before: 0.393871   (+3.60%, +0.01)
      branch-v3 after:  0.396293   (+4.23%, +0.02)
      
       ## data-env-vars.name     = mozilla-unified-2024-03-22-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        0.412165  ~~~~~
      branch-v3 before: 0.438105   (+6.29%, +0.03)
      branch-v3 after:  0.424769   (+3.06%, +0.01)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.412397  ~~~~~
      branch-v3 before: 0.438405   (+6.31%, +0.03)
      branch-v3 after:  0.421796   (+2.28%, +0.01)
       ## data-env-vars.name     = mozilla-unified-2024-03-22-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        0.429501  ~~~~~
      branch-v3 before: 0.452692   (+5.40%, +0.02)
      branch-v3 after:  0.443849   (+3.34%, +0.01)
      
       ## data-env-vars.name     = mozilla-try-2024-03-26-zstd-sparse-revlog
        # bin-env-vars.hg.flavor = default
      branch-v2:        3.403171  ~~~~~
      branch-v3 before: 6.562345  (+92.83%, +3.16)
      branch-v3 after:  6.234055  (+83.18%, +2.83)
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.454876  ~~~~~
      branch-v3 before: 6.160248  (+78.31%, +2.71)
      branch-v3 after:  6.307813  (+82.58%, +2.85)
       ## data-env-vars.name     = mozilla-try-2024-03-26-ds2-pnm
        # bin-env-vars.hg.flavor = rust
      branch-v2:        3.465435  ~~~~~
      branch-v3 before: 5.381648  (+55.30%, +1.92)
      branch-v3 after:  5.176076  (+49.36%, +1.71)
      41b8892a
  9. Sep 27, 2024
  10. Sep 26, 2024
  11. Sep 25, 2024
  12. Oct 05, 2024
    • Matt Harbison's avatar
      interfaces: introduce and use a protocol class for the `mpatch` module · d7f17819
      Matt Harbison authored
      See f2832de2a46c for details when this was done for the `bdiff` module.
      
      Two things worth pointing out-
      
        1) The `cffi` module "inherits" the `pure` implementation of `patchedsize()`
           because of its wildcard import.
        2) It's odd that the `mpatchError` lives in both `pure` and `cext` modules.
      
      I initially thought to move the exception into the new class, and make the
      existing class name an alias to the class in the new location, but the exception
      is created in C code by the `cext` module, so that won't work.  I don't think a
      protocol class is approriate, because there's nothing special about the class to
      distinguish from any other `Exception`.  Fortunately, nobody is catching this
      exception in core, so we can kick the can down the road.
      d7f17819
  13. Oct 08, 2024
  14. Oct 04, 2024
    • Matt Harbison's avatar
    • Matt Harbison's avatar
      tests: disable `worker.backgroundclose` to stabilize a test on Windows · b8528c8b
      Matt Harbison authored
      TIL that `worker.enabled=0` doesn't prevent these workers from spinning up.  At
      any rate, there's already a whole lot of conditionalized output following
      `cat client.log`, the placement of the "starting 4 threads for background file
      closing" message seems unstable, and we don't care about those worker threads
      here.  Preventing the message is better for test maintenance.
      b8528c8b
    • Matt Harbison's avatar
      tests: fix lock file path mangling in `test-racy-mutations.t` on Windows · b6ce567d
      Matt Harbison authored
      I guess `$TESTTMP_FORWARD_SLASH` gets translated by MSYS.  This was in the
      `.foo_commit_out` file:
      
          sh: C;C:\\MinGW\\msys\\1.0\\Users\\Matt\\AppData\\Local\\Temp\\hgtests.1qc8jmdl\\child2\\test-racy-mutations.t-skip-detection\\waitlock_editor.sh: $ENOENT
      b6ce567d
    • Matt Harbison's avatar
      tests: stabilize `test-status-eacces.t` on Windows · fdb1971b
      Matt Harbison authored
      As noted earlier, `chmod` doesn't complain in MSYS, but also doesn't alter the
      file permissions such that they are unreadable.
      
      I'm guessing the other lines of output in this area that are gated on `rhg` (or
      not) will also need this, but I don't want to dig too deeply into something that
      is apparently working well enough.
      fdb1971b
    • Matt Harbison's avatar
      run-tests: bump the default timeout on Windows to 4x the normal value · f4c03808
      Matt Harbison authored
      There are a ridiculous number of tests that timeout on Windows with the 360 sec
      default (~60).  And because of the bug where timed out tests still run to
      completion before the results are thrown away[1], the timeout does nothing but
      waste time, so there's no reason to try to find a lower value that still works.
      For reference on my system:
      
          # Ran 909 tests, 116 skipped, 119 failed.
          python hash seed: 2052473208
      
          real    151m44.322s
          user    0m0.077s
          sys     0m0.046s
      
      [1] I thought that I wrote a bug for this, but search isn't finding it.
      f4c03808
Loading