Skip to content
Snippets Groups Projects
  1. Nov 28, 2024
  2. Oct 01, 2024
    • Raphaël Gomès's avatar
      update: add a Rust fast-path when updating from null (and clean) · 8b7123c8947b
      Raphaël Gomès authored
      This case is easy to detect and we have all we need to generate a valid
      working copy and dirstate entirely in Rust, which speeds things up
      considerably:
      
      On my machine updating a repo of ~300k files goes from 10.00s down to 4.2s,
      all while consuming 50% less system time, with all caches hot.
      Something to note is that further improvements will probably happen
      with the upcoming `InnerRevlog` series that does smarter
      mmap hanlding, especially for filelogs.
      
      Here are benchmark numbers on a machine with only 4 cores (and no SMT enabled)
      
      ```
      ### data-env-vars.name               = heptapod-public-2024-03-25-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 5.328762  ~~~~~
      rust: 1.308654  (-75.44%, -4.02)
      ### data-env-vars.name               = mercurial-devel-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 1.693271  ~~~~~
      rust: 1.151053  (-32.02%, -0.54)
      ### data-env-vars.name               = mozilla-unified-2024-03-22-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 38.901613  ~~~~~
      rust: 11.637880 (-70.08%, -27.26)
      ### data-env-vars.name               = netbsd-xsrc-public-2024-09-19-ds2-pnm
        # benchmark.name                   = hg.command.update
        # bin-env-vars.hg.py-re2-module    = default
        # bin-env-vars.hg.changeset.node   = <this change>
        # benchmark.variants.atomic-update = no
        # benchmark.variants.scenario      = null-to-tip
        # benchmark.variants.worker        = default
      default: 4.793727  ~~~~~
      rust: 1.505905  (-68.59%, -3.29)
      ```
      8b7123c8947b
  3. Sep 24, 2024
  4. Apr 09, 2024
  5. Dec 11, 2023
    • Pierre-Yves David's avatar
      changelog: never inline changelog · dcaa2df1f688
      Pierre-Yves David authored
      The test suite mostly use small repositories, that implies that most changelog in the
      tests are inlined. As a result, non-inlined changelog are quite poorly tested.
      Since non-inline changelog are most common case for serious repositories, this
      lack of testing is a significant problem that results in high profile issue like
      the one recently fixed by 66417f55ea33 and 849745d7da89.
      
      Inlining the changelog does not bring much to the table, the number of total
      file saved is negligible, and the changelog will be read by most operation
      anyway.
      
      So this changeset is make it so we never inline the changelog, and de-inline the
      one that are still inlined whenever we touch them.
      
      By doing that, we remove the "dual code path" situation for writing new entry to
      the changelog and move to a "single code path" situation. Having a single
      code path simplify the code and make sure it is covered by test (if test cover
      that situation obviously)
      
      
      This impact all tests that care about the number of file and the exchange size,
      but there is nothing too complicated in them just a lot of churn.
      
      The churn is made "worse" by the fact rust will use the persistent nodemap on
      any changelog now. Which is overall a win as it means testing the persistent
      nodemap more and having less special cases.
      
      In short, having inline changelog is mostly useless and an endless source of
      pain. We get rid of it.
      dcaa2df1f688
  6. May 23, 2023
  7. Jun 01, 2023
    • Arseniy Alekseyev's avatar
      stream-clone: add a v3 version of the protocol · 0452af304808
      Arseniy Alekseyev authored
      This new version is less rigid regarding the extract number of files and number
      of bytes to be actually transfered, it also lays the groundwork for other
      improvements.
      
      The format stays experimental, but this is an interesting base to build upon.
      0452af304808
  8. Jun 19, 2023
  9. May 22, 2023
  10. May 21, 2023
  11. Jan 27, 2022
    • Pierre-Yves David's avatar
      stream-clone: allow to change persistent-nodemap format during stream clone · de3ac3d2c60b
      Pierre-Yves David authored
      Persistent nodemap affect the store format. However it is fairly isolated and
      fast to generate locally. So not making it a fixed part of the stream clone is
      useful.
      
      This allow clients without persistent-nodemap support (default for client
      without Rust enabled, or simply older client). So it make it possible to enable
      persistent nodemap on client, where it can provide a massive boost. without too
      much consequence.
      
      To do so, we stop using it in the advertisement requirements for streaming and
      let the client add/remove the necessary file depending of its configuration.
      
      We still send the files as it seems like a small save to not regenerate them.
      In addition, the way we match them will overlap with the changelog-v2/revlog-v2
      so we can't simply skip the associated patterns.
      
      Differential Revision: https://phab.mercurial-scm.org/D12096
      de3ac3d2c60b
  12. Feb 02, 2022
  13. Feb 01, 2022
  14. Jan 17, 2022
  15. Oct 19, 2021
    • Pierre-Yves David's avatar
      dirstate-v2: freeze the on-disk format · bf11ff22a9af
      Pierre-Yves David authored
      It seems the format as reached a good balance. With a core of new capabilities
      that motivated it initially and enough new feature and room for future
      improvement to be a clear progress we can set a milestone for.
      
      Having the format frozen will help the feature to get real life testing, outside
      of the test suite.
      
      The feature itself stay experimental but the config gains a new name to avoid
      people enable non-frozen version by default.
      
      If too many bugs are reported during the RC we might move the format back to
      experimental and drop its support in future version (in favor of a new one)
      
      Differential Revision: https://phab.mercurial-scm.org/D11709
      bf11ff22a9af
  16. Aug 03, 2021
    • sliquister's avatar
      tests: rely on dummyssh being the default · 9c4204b7f3e4
      sliquister authored
      This commit is exactly the result of running this command:
      
      sed -i -e 's! *\(-e \|--ssh \|--config ui.ssh=\)[ \"]*$PYTHON[ \"]*$\(RUN\|\)TESTDIR/dummyssh[\"]* *! !g' -e '/^[ >]*ssh *=[ "]*$PYTHON[ "]*$\(RUN\|\)TESTDIR\/dummyssh[ "]*$/d' -e 's/^\(  [$] .*[^ ]\) *$/\1/' *.t *.sh
      
      Sometimes the tests can be simplified further, but I think it's
      preferable to do the simplification separately.
      
      Differential Revision: https://phab.mercurial-scm.org/D11245
      9c4204b7f3e4
  17. Aug 25, 2021
  18. May 19, 2021
  19. Jan 14, 2021
  20. Apr 06, 2021
  21. Jan 15, 2021
    • Pierre-Yves David's avatar
      clone: make sure we warm the cache after a clone · 95a615dd77bf
      Pierre-Yves David authored
      This work around any deviciency/limitation of the clone process. In our case
      this ensure the persistent nodemap exist with valid content.
      
      Ideally, the cloning process would also do "the right thing". However since
      older server will never be able to do "the right thing". The local workaround
      will be necessary anyway.
      
      I am not worried by the performance impact of this as `hg clone` is non-instant
      on large repositories where is could matters. Warming the cache if they are
      already correct is very fast. And if they are not already warm, this seems like
      a good time to do so.
      
      This impact various test as more cache are now warmed sooner, all the change
      should be harmless.
      
      Differential Revision: https://phab.mercurial-scm.org/D9789
      95a615dd77bf
  22. Sep 11, 2019
  23. Nov 12, 2018
  24. Oct 01, 2018
    • Gregory Szorc's avatar
      httppeer: report http statistics · 393e44324037
      Gregory Szorc authored
      Now that keepalive.py records HTTP request count and the
      number of bytes sent and received as part of performing those
      requests, we can easily print a report on the activity when
      closing a peer instance!
      
      Exact byte counts are globbed in tests because they are influenced
      by non-deterministic things, such as hostnames and port numbers.
      Plus, the exact byte count isn't too important anyway.
      
      I feel obliged to note that printing the byte count could have
      security implications. e.g. if sending a password via HTTP basic
      auth, the length of that password will influence the byte count
      and the reporting of the byte count could be a side-channel leak
      of the password length. I /think/ this is beyond our threshold
      for concern. But if we think it poses a problem, we can teach the
      byte count logging code to e.g. ignore sensitive HTTP request
      headers. We could also consider not reporting the byte count of
      request headers altogether. But since the wire protocol uses HTTP
      headers for sending command arguments, it is kind of important to
      report their size.
      
      Differential Revision: https://phab.mercurial-scm.org/D4858
      393e44324037
  25. Aug 25, 2018
    • Yuya Nishihara's avatar
      transaction: remember original len(repo) instead of tracking added revs (API) · 5763216ba311
      Yuya Nishihara authored
      It's silly to keep updating xrange(len(changelog), len(changelog) + 1) for
      each added revision. Instead, let's simply remember the first revision to
      be added.
      
      The test output slightly changed as the branch cache is also warmed up by
      stream clone, which seems more consistent.
      
      .. api::
      
         ``tr.changes['revs']`` is replaced by ``tr.changes['origrepolen']`` which
         is the first revision number to be added.
      5763216ba311
  26. May 08, 2018
  27. Apr 03, 2018
  28. Mar 31, 2018
  29. Jan 31, 2018
Loading