Skip to content
Snippets Groups Projects
  1. May 02, 2022
  2. Dec 19, 2022
  3. Dec 07, 2022
    • Pierre-Yves David's avatar
      bundle: emit full snapshot as is, without doing a redelta · e1953a34
      Pierre-Yves David authored
      With the new `forced` delta-reused policy, it become important to be able to
      send full snapshot where full snapshot are needed. Otherwise, the fallback delta
      will simply be used on the client side… creating monstrous delta chain, since
      revision that are meant as a reset of delta-chain chain becoming too complex are
      simply adding a new full delta-tree on the leaf of another one.
      
      In the `non-forced` cases, client process full snapshot from the bundle
      differently from deltas, so client will still try to convert the full snapshot
      into a delta if possible. So this will no lead to pathological storage
      explosion.
      
      I have considered making this configurable, but the impact seems limited enough
      that it does not seems to be worth it. Especially with the current
      sparse-revlog format that use "delta-tree" with multiple level snapshots, full
      snapshot are much less frequent and not that different from other intermediate
      snapshot that we are already sending over the wire anyway.
      
      CPU wise, this will help the bundling side a little as it will not need to
      reconstruct revisions and compute deltas. The unbundling side might save a tiny
      amount of CPU as it won't need to reconstruct the delta-base to reconstruct the
      revision full text. This only slightly visible in some of the benchmarks. And
      have no real impact on most of them.
      
      ### data-env-vars.name      = pypy-2018-08-01-zstd-sparse-revlog
        # benchmark.name          = perf-bundle
        # benchmark.variants.revs = last-40000
      before:          11.467186 seconds
      just-emit-full:  11.190576 seconds   (-2.41%)
      with-pull-force: 11.041091 seconds   (-3.72%)
        # benchmark.name          = perf-unbundle
        # benchmark.variants.revs = last-40000
      before:     16.744862
      just-emit-full:: 16.561036 seconds   (-1.10%)
      with-pull-force: 16.389344 seconds   (-2.12%)
        # benchmark.name          = pull
        # benchmark.variants.revs = last-40000
      before:     26.870569
      just-emit-full:  26.391188 seconds   (-1.78%)
      with-pull-force: 25.633184 seconds   (-4.60%)
      
      
      Space wise (so network-wise) the impact is fairly small. When taking compression into
      account.
      
      Below are tests the size of `hg bundle --all` for a handful of benchmark repositories
      (with bzip, zstd compression and without it)
      
      This show a small increase in the bundle size, but nothing really significant
      except maybe for mozilla-try (+12%) that nobody really pulls large chunk of anyway.
      Mozilla-try is also the repository that benefit the most for not having to
      recompute deltas client size.
      
      ### mercurial:
      bzip-before:     26 406 342 bytes
      bzip-after:      26 691 543 bytes  +1.08%
      
      zstd-before:     27 918 645 bytes
      zstd-after:      28 075 896 bytes  +0.56%
      
      none-before:     98 675 601 bytes
      none-after:     100 411 237 bytes  +1.76%
      
      
      ### pypy
      bzip-before:    201 295 752 bytes
      bzip-after:     209 780 282 bytes  +4.21%
      
      zstd-before:    202 974 795 bytes
      zstd-after:     205 165 780 bytes  +1.08%
      
      none-before:    871 070 261 bytes
      none-after:     993 595 057 bytes +14.07%
      
      
      ### netbeans
      bzip-before:    601 314 330 bytes
      bzip-after:     614 246 241 bytes  +2.15%
      
      zstd-before:    604 745 136 bytes
      zstd-after:     615 497 705 bytes  +1.78%
      
      none-before:  3 338 238 571 bytes
      none-after:   3 439 422 535 bytes  +3.03%
      
      
      ### mozilla-central
      bzip-before:  1 493 006 921 bytes
      bzip-after:   1 549 650 570 bytes  +3.79%
      
      zstd-before:  1 481 910 102 bytes
      zstd-after:   1 513 052 415 bytes  +2.10%
      
      none-before:  6 535 929 910 bytes
      none-after:   7 010 191 342 bytes  +7.26%
      
      
      ### mozilla-try
      bzip-before:  6 583 425 999 bytes
      bzip-after:   7 423 536 928 bytes +12.76%
      
      zstd-before:  6 021 009 212 bytes
      zstd-after:   6 674 922 420 bytes +10.86%
      
      none-before: 22 954 739 558 bytes
      none-after:  26 013 854 771 bytes +13.32%
      e1953a34
    • Pierre-Yves David's avatar
      delta-find: properly report full snapshot used from cache as such · d57b966c
      Pierre-Yves David authored
      The number of tries and the delta base is reported differently so we missed
      there detection initially.
      d57b966c
    • Pierre-Yves David's avatar
      test-acl: glob the payload size again · 5d972518
      Pierre-Yves David authored
      This size of bundle-2 payload are irrelevant for this test and only appears in
      its output because other pieces of the debug output are important.
      
      We glob it these number before they get in our way again.
      5d972518
  4. Nov 29, 2022
  5. Nov 08, 2022
  6. Dec 03, 2022
  7. Dec 05, 2022
    • Pierre-Yves David's avatar
      delta-find: add a test checking various simple behavior · e300f445
      Pierre-Yves David authored
      There are enough work happening in this area that it is worth having a dedicated
      test for it. So far we add to small test checking that the "best" parent is
      picked as the delta base and that this behavior can be controlled during commit
      and unbundle.
      e300f445
  8. Dec 02, 2022
  9. Nov 06, 2022
  10. Nov 07, 2022
  11. Nov 21, 2022
  12. Nov 07, 2022
    • Pierre-Yves David's avatar
      debug-revlog: details about non-ancestors delta-bases · 511106bc
      Pierre-Yves David authored
      Deltas against a base that is not an ancestor of the revision that owns this
      delta are notable.
      
      For example, they introduce complexity during the bundling process as the base
      might not exist on the unbundling side.
      
      We detect them in `hg debugrevlog` and print information about them.
      511106bc
  13. Nov 24, 2022
    • Pierre-Yves David's avatar
      test: adjust test-push-race.t timeout's to overall test timeout · c6f0bcb7
      Pierre-Yves David authored
      The generic `tests/testlib/wait-on-file` mechanism scale its timeout with the
      value of `HGTEST_TIMEOUT`, the `delaypush.py` in `test-push-race.t` is not doing
      this, and we have been seeing more and more timeout from loaded CI worker
      lately.
      
      Adding this timeout scaling should help with that.
      c6f0bcb7
  14. Nov 21, 2022
    • Matt Harbison's avatar
      tests: update test-util.py for modern attrs package · 7e6f3c69
      Matt Harbison authored
      When updating to 22.1.0, this test started failing:
      
          Traceback (most recent call last):
            File "/tmp/mercurial-ci/tests/test-util.py", line 53, in <module>
              _start_default = (util.timedcmstats.start.default, 'factory')
          AttributeError: type object 'timedcmstats' has no attribute 'start'
      
      Poking around in `hg debugshell`, the attribute is indeed missing, but looks to
      be attached to `__attrs_attrs__` in both the currently vendored and the modern
      version of attrs.  The old attrs packages will print the same for both accesses,
      so fingers crossed...
      
          >>> print((util.timedcmstats.start.default, 'factory'))
          (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory')
      
          >>> print((util.timedcmstats.__attrs_attrs__.start.default, 'factory'))
          (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory')
      7e6f3c69
  15. Nov 14, 2022
    • Raphaël Gomès's avatar
      rhg: upgrade `clap` dependency · 37bc3ede
      Raphaël Gomès authored
      This one is the worst one to upgrade since v2 -> v4 broke a ton of API,
      which thankfully seems saner now.
      
      Contrary to what was done in the `hg-core/src/examples/nodemap` rewrite,
      we're not switching from the "builder" pattern to the "derive" pattern,
      since that would imply a much larger diff. It can be done incrementally.
      37bc3ede
  16. Nov 19, 2022
    • Pierre-Yves David's avatar
      tests: fix test-sparse-revlog · da636e7a
      Pierre-Yves David authored
      This one is not covered by the CIbecause I requires an expensive artifact to be
      cached. So it goes out of think on regular basis (we should fix that…)
      
      The test ouput was affected by e706bb41fdb3 as we filtering now happens sooner,
      removing for the output.
      da636e7a
  17. Nov 18, 2022
  18. Nov 17, 2022
  19. Nov 06, 2022
  20. Oct 17, 2022
  21. Nov 15, 2022
  22. Nov 16, 2022
    • Pierre-Yves David's avatar
      matcher: do not prepend '.*' to pattern using ^ after flags · b3480822
      Pierre-Yves David authored
      Since the previous commit (fixing wider issue), the code generated strange
      regex. This is now fixed and tested.
      b3480822
    • Pierre-Yves David's avatar
      matcher: fix the issue with regex inline-flag in rust oo · 086b0c4f
      Pierre-Yves David authored
      Same problem same solution.
      086b0c4f
    • Pierre-Yves David's avatar
      matcher: fix issues regex flag contained in pattern (issue6759) · 3eda36e9
      Pierre-Yves David authored
      Python 3.11 is now enforcing that flag must be at the beginning of the regex
      
      This creates a serious regression for people using Python 3.11 with an hgignore
      using flag in a "relre" pattern.
      
      We now detect any flags in such pattern and "prepend" our ".*" pattern after them.
      
      In addition, we now insert the flag in the regexp to only affect the pattern we
      are rewriting. Otherwise, the regex built from the combined pattern would these
      flags in the middle of it anyway.
      
      As a side effect of this last change, we fix a bug… before this change regex
      flag in a pattern would affect all combined patterns. That was bad and is not
      longer the case.
      
      The Rust code needs to be updated to fix that very bug, but we will do it in
      another changeset.
      3eda36e9
  23. Nov 14, 2022
  24. Jun 20, 2022
  25. Nov 14, 2022
  26. Nov 04, 2022
  27. Nov 02, 2022
Loading