Skip to content
Snippets Groups Projects
  1. Jan 19, 2024
  2. Feb 23, 2024
  3. Jan 17, 2024
    • Georges Racinet's avatar
      chainsaw-update: log actual locks breaking · fe68a2dc
      Georges Racinet authored
      Previously, the command would simply state that it was about
      to break locks, not if there was actually some to break.
      
      This version is race-free. It would be also possible to display
      the content of the lock before hand (not race-free but informative
      in almost all cases).
      fe68a2dc
    • Georges Racinet's avatar
      vfs: have tryunlink tell what it did · 187c5769
      Georges Racinet authored
      It is useful in certain circumstances to know whether vfs.tryunlink()
      actually removed something or not, be it for logging purposes.
      187c5769
  4. Nov 26, 2022
    • Georges Racinet's avatar
      chainsaw: new extension for dangerous operations · bc88aa74
      Georges Racinet authored
      The first provided command is `chainsaw-update`, whose one and single job is
      to make sure that it will pull, update and purge the target repository,
      no matter what may be in the way (locks, notably), see docstring for rationale.
      bc88aa74
  5. Feb 23, 2024
  6. Feb 22, 2024
    • Raphaël Gomès's avatar
      rust-index: remove one collect when converting back · 3099f1c6
      Raphaël Gomès authored
      Turns out this is slightly faster. Sending the results back to Python is still
      the most costly (like 75% of the time) of the whole method, but it's about
      as fast as it can be now.
      
      hg perf::phases on mozilla-try-2023-03-22
      before: 0.267114
      after:  0.247101
      3099f1c6
    • Raphaël Gomès's avatar
      rust-index: improve phase computation speed · 7c6d0b9d
      Raphaël Gomès authored
      While less memory efficient, using an array is *much* faster than using a
      HashMap, especially with the default hasher. It even makes the code simpler,
      so I'm not really sure what I was thinking in the first place, maybe it's more
      obvious now.
      
      This fix a significant performance regression when using the rust version of the
      code. (however, the C code still outperform rust on this operation)
      
      hg perf::phases on mozilla-try-2023-03-22
      - 6.6.3:  0.451239 seconds
      - before: 0.982495 seconds
      - after:  0.265347 seconds
      - C code: 0.183241 second
      7c6d0b9d
  7. Feb 23, 2024
    • Pierre-Yves David's avatar
      phases: directly update the phase sets in advanceboundary · 3cee8706
      Pierre-Yves David authored
      This is similar to what we do in retractboundary. There is no need to invalidate
      the cache if we have everything at hand to update it.
      3cee8706
    • Pierre-Yves David's avatar
      phases: large rework of advance boundary · 23950e39
      Pierre-Yves David authored
      In a similar spirit as the rework of retractboundary, the new algorithm is doing
      an amount of work in the order of magnitude of the amount of changeset that
      changes phases. (except to find new roots in impacted higher phases if any may
      exists).
      
      This result in a very significant speedup for repository with many old draft
      like mozilla try.
      
      runtime of perf:unbundle for a bundle constaining a single changeset (C code):
      
      before 6.7 phase work: 14.497 seconds
      before this change:     6.311 seconds (-55%)
      with this change:       2.240 seconds (-85%)
      
      Combined with the other patches that fixes the phases computation in the Rust
      index, the rust code with a persistent nodemap get back to quite interresting
      performances with 2.026 seconds for the same operation, about 10% faster than
      the C code.
      23950e39
  8. Feb 22, 2024
  9. Feb 21, 2024
    • Pierre-Yves David's avatar
      phases: filter revision that are already in the right phase · 2eb93812
      Pierre-Yves David authored
      No need to compute new roots if everything is already in order.
      2eb93812
    • Pierre-Yves David's avatar
      phases: invalidate the phases set less often on retract boundary · ac1c7518
      Pierre-Yves David authored
      We already have the information to update the phase set, so we do so directly
      instead of invalidating the cache.
      
      This show a sizeable speedup in our `perf::unbundle` benchmark on the
      many-draft mozilla-try repository.
      
      ### data-env-vars.name            = mozilla-try-2023-03-22-zstd-sparse-revlog
        # benchmark.name                = hg.perf.perf-unbundle
        # bin-env-vars.hg.flavor        = no-rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.issue6528  = disabled
      
        # benchmark.variants.revs = last-10
      before: 2.055259 seconds
      after:  1.887064 seconds (-8.18%)
        # benchmark.variants.revs = last-100
      before: 2.409239 seconds
      after:  2.222429 seconds (-7.75%)
        # benchmark.variants.revs = last-1000
      before: 3.945648 seconds
      after:  3.762480 seconds (-4.64%)
      ac1c7518
    • Pierre-Yves David's avatar
      phases: incrementally update the phase sets when reasonable · e57d4b86
      Pierre-Yves David authored
      When the amount of manual walking is small, we update the phases set manually
      instead of computing them from scratch. This should help small update. The next
      changesets will make this used more often by reducing the amount of full
      invalidation we do on roots upgrade.
      
      The criteria for using an incremental upgrade are arbitrary, however, it "should
      never hurt".
      e57d4b86
  10. Feb 22, 2024
  11. Feb 21, 2024
  12. Feb 22, 2024
  13. Feb 21, 2024
  14. Feb 20, 2024
    • Pierre-Yves David's avatar
      phases: leverage the collected information to record phase update · eababb7b
      Pierre-Yves David authored
      Since the lower level function already gather this information, we can directly
      use it.
      
      This comes with a small change to the test that are actually fixing them. The
      previous version over-reported some phase change that did not exists. In both
      case, we are force revision `1` to be secret and `0` remains draft`, the
      previous code wrongly reported `0` as moving to secret while it properly
      remained draft in the repository.
      eababb7b
  15. Feb 21, 2024
    • Pierre-Yves David's avatar
      phases: large rewrite on retract boundary · 2f39c7ae
      Pierre-Yves David authored
      The new code is still pure Python, so we still have room to going significantly
      faster. However its complexity of the complex part is `O(|[min_new_draft, tip]|)` instead of
      `O(|[min_draft, tip]|` which should help tremendously one repository with old
      draft (like mercurial-devel or mozilla-try).
      
      This is especially useful as the most common "retract boundary" operation
      happens when we commit/rewrite new drafts or when we push new draft to a
      non-publishing server. In this case, the smallest new_revs is very close to the
      tip and there is very few work to do.
      
      A few smaller optimisation could be done for these cases and will be introduced in
      later changesets.
      
      We still have iterate over large sets of roots, but this is already a great
      improvement for a very small amount of work. We gather information on the
      affected changeset as we go as we can put it to use in the next changesets.
      This extra data collection might slowdown the `register_new` case a bit, however
      for register_new, it should not really matters. The set of new nodes is either
      small, so the impact is negligible, or the set of new nodes is large, and the
      amount of work to do to had them will dominate the overhead the collecting
      information in `changed_revs`.
      
      As this new code compute the changes on the fly, it unlock other interesting
      improvement to be done in later changeset.
      2f39c7ae
  16. Feb 22, 2024
  17. Feb 21, 2024
  18. Feb 20, 2024
    • Pierre-Yves David's avatar
      phases: keep internal state as rev-num instead of node-id · f8bf1a8e
      Pierre-Yves David authored
      Node-id are expensive to work with, dealing with revision is much simple and
      faster.
      
      The fact we still used node-id here shows how few effort have been put into
      making the phase logic fast. We tend to no longer use node-id internally for
      about ten years.
      
      This has a large impact of repository with many draft roots. For example this
      Mozilla-try copy have ½ Million draft roots and `perf::unbundle` see a
      significant improvement.
      
      ### data-env-vars.name            = mozilla-try-2023-03-22-zstd-sparse-revlog
        # benchmark.name                = hg.perf.perf-unbundle
        # bin-env-vars.hg.flavor        = no-rust
        # bin-env-vars.hg.py-re2-module = default
        # benchmark.variants.issue6528  = disabled
      
        # benchmark.variants.revs = last-1
      before:: 1.746791 seconds
      after::  1.278379 seconds  (-26.82%)
      
        # benchmark.variants.revs = last-10
      before:: 3.145774 seconds
      after::  2.103735 seconds  (-33.13%)
      
        # benchmark.variants.revs = last-100
      before:: 3.487635 seconds
      after::  2.446749 seconds  (-29.85%)
      
        # benchmark.variants.revs = last-1000
      before:: 5.007568 seconds
      after::  3.989923 seconds  (-20.32%)
      f8bf1a8e
    • Pierre-Yves David's avatar
      phases: do filtering at read time · 12881244
      Pierre-Yves David authored
      This remove the need for the `filterunknown` method at all.
      12881244
    • Pierre-Yves David's avatar
      phases: always write with a repo · 04111ef0
      Pierre-Yves David authored
      In the future change that move the internal representation of phase-roots from
      node-id to rev-num, we will use a repository to translate revision numbers back
      to node at write time.
      
      Since that future change is quite complicated already, we do this small API
      change beforehand.
      04111ef0
    • Pierre-Yves David's avatar
      phases: mark `phasecache.phaseroots` private · 68289ed1
      Pierre-Yves David authored
      We are about to change its content from nodeid to revnum. So anyone directly
      using the content might be in unexpected troubles. We start by making it private
      to explicitly break any such user (and discourage them to do so).
      68289ed1
    • Pierre-Yves David's avatar
      phases: check secret presence the right way during discovery · 2e10ddbb
      Pierre-Yves David authored
      There is an official function for this, lets use it.
      
      This will prevent the code to break in the future while we refactor the phase
      code.
      2e10ddbb
    • Pierre-Yves David's avatar
      phases: explicitly filter stripped revision at strip time · 8f2ea3fa
      Pierre-Yves David authored
      Explicit is better than implicit. The current logic is bit subtle and fragile.
      
      It also get in the way of using something else than node-id as internal storage.
      We replace it with a more explicit filtering while striping.
      8f2ea3fa
  19. Feb 23, 2024
    • Pierre-Yves David's avatar
      debug: add a debug::unbundle command that simulate the unbundle from a push · e2dfa403
      Pierre-Yves David authored
      The code have different behavior when the unbundle comes from a push, so we
      introduce a command that can simulate such unbundle.
      
      For our copy of mozilla-try-2023-03-22, this make the unbundle jump from 2.5
      seconds (with `hg unbundle`) to 15 seconds (with `hg debug::unbundle`).
      
      That 15 seconds timings is consistent with the issue seen in production.
      e2dfa403
  20. Feb 22, 2024
Loading