Skip to content
Snippets Groups Projects
  1. Feb 22, 2025
  2. Nov 28, 2024
  3. Jun 19, 2024
    • Raphaël Gomès's avatar
      revlog: add the glue to use the Rust `InnerRevlog` from Python · 7346f93be7a4
      Raphaël Gomès authored
      The performance of this has been looked at for quite some time, and some
      workflows are actually quite a bit faster than with the Python + C code.
      
      However, we are still (up to 20%) slower in some crucial places like cloning
      certain repos, log, cat, which makes this an incomplete rewrite. This is
      mostly due to the high amount of overhead in Python <-> Rust FFI, especially
      around the VFS code. A future patch series will rewrite the VFS code in
      pure Rust, which should hopefully get us up to par with current perfomance,
      if not better in all important cases.
      
      This is a "save state" of sorts, as this is a ton of code, and I don't want
      to pile up even more things in a single review.
      
      Continuing to try to match the current performance will take an extremely
      long time, if it's not impossible, without the aforementioned VFS work.
      7346f93be7a4
  4. Mar 10, 2024
    • Pierre-Yves David's avatar
      branchcache: stop writing more branchcache file on disk than needed · 2e8a88e5809f
      Pierre-Yves David authored
      Before this change, we were unconditionally writing a branchmap file for the
      filter level passed to `update_disk`. This is actually counter productive if no
      update were needed for this filter level. In many case, the branch cache for a
      filter level is identical to its parent "subset" and it is better to simply
      keep the subset update and reuse it every time instead of having to do identical
      work for similar subset.
      
      So we change the `update_disk` method to only write a file when that filter
      level differ from its parent. This removes many cases where identical files were
      written, requiring multiple boring update in the test suite.
      
      The only notable changes is the change to `test-strip-branch-cache.t`, this
      case was checking a scenario that no longer reproduce the bug as writing less
      branchmap file result in less stalled cache on disk.
      
      Strictly speaking, we could create a more convoluted scenario that create a
      similar issue. However the next changeset would also cover that scenario so we
      directly updated that test case to a "no longer buggy" state.
      2e8a88e5809f
  5. Dec 11, 2023
    • Pierre-Yves David's avatar
      changelog: never inline changelog · dcaa2df1f688
      Pierre-Yves David authored
      The test suite mostly use small repositories, that implies that most changelog in the
      tests are inlined. As a result, non-inlined changelog are quite poorly tested.
      Since non-inline changelog are most common case for serious repositories, this
      lack of testing is a significant problem that results in high profile issue like
      the one recently fixed by 66417f55ea33 and 849745d7da89.
      
      Inlining the changelog does not bring much to the table, the number of total
      file saved is negligible, and the changelog will be read by most operation
      anyway.
      
      So this changeset is make it so we never inline the changelog, and de-inline the
      one that are still inlined whenever we touch them.
      
      By doing that, we remove the "dual code path" situation for writing new entry to
      the changelog and move to a "single code path" situation. Having a single
      code path simplify the code and make sure it is covered by test (if test cover
      that situation obviously)
      
      
      This impact all tests that care about the number of file and the exchange size,
      but there is nothing too complicated in them just a lot of churn.
      
      The churn is made "worse" by the fact rust will use the persistent nodemap on
      any changelog now. Which is overall a win as it means testing the persistent
      nodemap more and having less special cases.
      
      In short, having inline changelog is mostly useless and an endless source of
      pain. We get rid of it.
      dcaa2df1f688
  6. May 18, 2023
  7. May 31, 2023
  8. May 29, 2023
  9. May 02, 2022
  10. May 31, 2022
  11. Jan 18, 2022
  12. Dec 08, 2021
  13. Aug 03, 2021
  14. May 07, 2021
  15. Jan 14, 2021
  16. Apr 06, 2021
  17. Jan 15, 2021
    • Pierre-Yves David's avatar
      clone: make sure we warm the cache after a clone · 95a615dd77bf
      Pierre-Yves David authored
      This work around any deviciency/limitation of the clone process. In our case
      this ensure the persistent nodemap exist with valid content.
      
      Ideally, the cloning process would also do "the right thing". However since
      older server will never be able to do "the right thing". The local workaround
      will be necessary anyway.
      
      I am not worried by the performance impact of this as `hg clone` is non-instant
      on large repositories where is could matters. Warming the cache if they are
      already correct is very fast. And if they are not already warm, this seems like
      a good time to do so.
      
      This impact various test as more cache are now warmed sooner, all the change
      should be harmless.
      
      Differential Revision: https://phab.mercurial-scm.org/D9789
      95a615dd77bf
  18. Jun 28, 2019
  19. Nov 12, 2018
  20. Sep 17, 2018
    • Anton Shestakov's avatar
      bundle2: make server.bundle2.stream default to True · 4bd6e444c76f
      Anton Shestakov authored
      Support for bundle2 streaming clones has been shipped in Mercurial 4.5
      (7eedbd5d4880), but was never activated by default. It's time to have more
      people use it. The new format allows streaming clones to transport cache
      (hooray for speed) and phaseroots (fixes phase-related issues).
      
      Changes in tests:
      
      bundle2 capabilities now have "stream=v2" (plus a '\n' as a separator) and
      therefore take 14 bytes more: "%0Astream%3Dv2". Tip for tests that have data
      encoded with CBOR: 0xd3 - 0xc5 = 14.
      
      $USUAL_BUNDLE2_CAPS$ replaces $USUAL_BUNDLE2_CAPS_SERVER$, which is the same
      thing, but without "stream=v2".
      
      Since streaming clones now also transfer caches, the reported byte and file
      counts are higher (e.g. 816 bytes in 9 files instead of 613 bytes in 4 files,
      a bit of --debug and manual math confirms that the caches take these extra 203
      bytes in 5 files).
      
      Differential Revision: https://phab.mercurial-scm.org/D4680
      4bd6e444c76f
  21. Sep 19, 2018
  22. Sep 04, 2018
  23. Aug 29, 2018
    • Boris Feld's avatar
      phase: report number of non-public changeset alongside the new range · 89630d0b3e23
      Boris Feld authored
      When interacting with non-publishing repository or bundle, it is useful to
      have some information about the phase of the changeset we just pulled.
      
      This changeset updates the "new changesets MIN:MAX" output to also includes
      phases information for non-public changesets. Displaying extra data about
      non-public changesets means the output for exchange with publishing repository
      (the default) is unaffected.
      89630d0b3e23
  24. May 01, 2018
  25. Apr 05, 2018
  26. Apr 04, 2018
    • Gregory Szorc's avatar
      simplestore: use a custom store for the simple store repo · c2c8962a9465
      Gregory Szorc authored
      Before, we used the default store, which was based on fncache
      and dotencode. After attempting to port tests to work with the
      simple store, I realized that fncache was more trouble than it is
      worth.
      
      This commit implements a proper store type for the simple repo -
      one that isn't based off fncache.
      
      This causes a number of new test failures because of tests
      expecting the full fncache store filename encoding. I may
      extend the store format in a subsequent commit to take the
      filename encoding parts of fncache that we can take
      (basically everything except hash encoding, since that isn't
      reversible). But for now, let's use encoded store.
      
      As part of this, we implement proper requirements support for
      repos created with the simple store. This should have been
      done from the beginning, as a requirement is needed to lock
      out clients that don't understand a storage format.
      
      A new hghave feature advertising the presence of fncache in repos
      has been added. Most tests touching the fncache are now conditional
      on that feature.
      
      Other tests have added the optional repo requirement to output.
      
      Differential Revision: https://phab.mercurial-scm.org/D3095
      c2c8962a9465
    • Gregory Szorc's avatar
      verify: drop "revlog" from warning message · a6651f5e2c78
      Gregory Szorc authored
      Not all stores may be backed by revlogs. Switch to a more generic
      error message.
      
      Differential Revision: https://phab.mercurial-scm.org/D3094
      a6651f5e2c78
    • Gregory Szorc's avatar
      tests: disallow using simple store repo with bundlerepo · 45a4799174a1
      Gregory Szorc authored
      bundlerepo is... going to be difficult to port to an alternate
      store because it assumes revlogs for storage and essentially
      overlays the contents of a bundle onto a fake revlog-like
      primitive. It will be a good test case for our eventual new
      storage interface.
      
      Refactoring bundlerepo to make it work with non-revlog storage is
      going to be a bit of work. So for now, let's refuse to use the
      simple store repo when a bundlerepo is in play.
      
      A new test requirement advertising support for treating bundle
      files as repo instances has been added. Some tests have been
      made conditional on this feature. Additional tests will be
      annotated in subsequent commits.
      
      Having positive opt-in to repo features will be simpler in the
      long run because it will allow multiple storage backends to
      declare feature support and we won't have to annotate each test
      with the set of repo backends that are supported. Again, we'll
      probably want better integration between repo features and
      tests. But this is the easiest we can do at the moment.
      
      Differential Revision: https://phab.mercurial-scm.org/D3060
      45a4799174a1
  27. Apr 03, 2018
    • Gregory Szorc's avatar
      tests: use `hg unbundle` instead of `hg pull` in some tests · 5d10f41ddcc4
      Gregory Szorc authored
      `hg pull <bundle>` uses the special "bundlerepo" repository. The
      bundlerepo code makes many assumptions about the storage of
      repositories. It will be difficult to teach bundlerepo to use
      non-revlog storage before a better storage interface is established.
      
      Many test failures using our "simple store" are related to
      bundlerepo: the simple store just isn't compatible with bundlerepo
      because of storage assumptions in bundlerepo.
      
      In order to mitigate the impact of bundlerepo on our code base,
      this commit changes various tests to use `hg unbundle` instead
      of `hg pull`. This bypasses the bundlerepo code.
      
      Tests exercising exchange functionality have not been altered, as
      they should be using `hg pull` and going through the bundlerepo
      code paths.
      
      Differential Revision: https://phab.mercurial-scm.org/D3059
      5d10f41ddcc4
  28. Apr 04, 2018
  29. Apr 02, 2018
    • Gregory Szorc's avatar
      debugcommands: drop offset and length from debugindex by default · d4e62df1c73d
      Gregory Szorc authored
      These fields are an implementation detail of revlog storage. As
      such, they are not part of the generic storage "index" interface
      and shouldn't be displayed by default.
      
      Because we don't have another way to display these fields, we've
      retained support for printing these fields via --verbose.
      
      Yes, I know we should probably be doing all this formatting using
      modern formatting/templater APIs. I didn't feel like scope
      bloating this patch.
      
      Differential Revision: https://phab.mercurial-scm.org/D3028
      d4e62df1c73d
    • Gregory Szorc's avatar
      debugcommands: drop base revision from debugindex · 009d0283de5f
      Gregory Szorc authored
      Revlog index data consists of generic index metadata that will
      likely be implemented across all storage engines and revlog-specifc
      metadata.
      
      Most tests printing index data only care about the generic fields.
      
      This commit drops the printing of the base revision from
      `hg debugindex`. This value is an implementation detail of
      revlogs / delta chains. If tests are interested in verifying this
      implementation detail, `hg debugdeltachain` is a better command.
      
      Most tests were skipping over this field anyway. Tests that weren't
      looked like they were newer. So my guess is we forgot to make them
      skip the field to match the style of the older tests. This reinforces
      my belief that the base revision is not worth having in
      `hg debugindex`.
      
      Differential Revision: https://phab.mercurial-scm.org/D3027
      009d0283de5f
  30. Dec 11, 2017
    • Matt Harbison's avatar
      tests: remove (glob) annotations that were only for '\' matches · 4441705b7111
      Matt Harbison authored
      # skip-blame because this was mechanically rewritten the following script.  I
      ran it on both *.t and *.py, but none of the *.py changes were proper.  All *.t
      ones appear to be, and they run without addition failures on both Windows and
      Linux.
      
        import argparse
        import os
        import re
      
        ap = argparse.ArgumentParser()
        ap.add_argument('path', nargs='+')
        opts = ap.parse_args()
      
        globre = re.compile(r'^(.*) \(glob\)(.*)$')
      
        for p in opts.path:
            tmp = p + '.tmp'
            with open(p, 'rb') as src, open(tmp, 'wb') as dst:
                for line in src:
                    m = globre.match(line)
                    if not m or '$LOCALIP' in line or '*' in line:
                        dst.write(line)
                        continue
                    if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'):
                        dst.write(line)
                        continue
                    dst.write(m.group(1) + m.group(2) + '\n')
            os.unlink(p)
            os.rename(tmp, p)
      4441705b7111
Loading