Skip to content
Snippets Groups Projects
  1. Feb 22, 2025
  2. Sep 24, 2024
  3. Dec 11, 2023
    • Pierre-Yves David's avatar
      changelog: never inline changelog · dcaa2df1f688
      Pierre-Yves David authored
      The test suite mostly use small repositories, that implies that most changelog in the
      tests are inlined. As a result, non-inlined changelog are quite poorly tested.
      Since non-inline changelog are most common case for serious repositories, this
      lack of testing is a significant problem that results in high profile issue like
      the one recently fixed by 66417f55ea33 and 849745d7da89.
      
      Inlining the changelog does not bring much to the table, the number of total
      file saved is negligible, and the changelog will be read by most operation
      anyway.
      
      So this changeset is make it so we never inline the changelog, and de-inline the
      one that are still inlined whenever we touch them.
      
      By doing that, we remove the "dual code path" situation for writing new entry to
      the changelog and move to a "single code path" situation. Having a single
      code path simplify the code and make sure it is covered by test (if test cover
      that situation obviously)
      
      
      This impact all tests that care about the number of file and the exchange size,
      but there is nothing too complicated in them just a lot of churn.
      
      The churn is made "worse" by the fact rust will use the persistent nodemap on
      any changelog now. Which is overall a win as it means testing the persistent
      nodemap more and having less special cases.
      
      In short, having inline changelog is mostly useless and an endless source of
      pain. We get rid of it.
      dcaa2df1f688
  4. May 15, 2023
    • Pierre-Yves David's avatar
      transaction: use a ".bck" extension for all backup file · 63dc24be635d
      Pierre-Yves David authored
      There are too much code in Mercurial that assume file function from extension.
      Keeping the original extension when doing backup is just creating confusion
      (for exemple, backup of revlog's confused with new version revlogs).
      
      So we change the backup name from now on to be explicilty backup file and
      prevent this kind of problem.
      
      (note that repository using fncache is less affected (but still affected) by
      this kind of problem as the backup are not listed in the fncache).
      63dc24be635d
  5. Feb 23, 2023
    • Pierre-Yves David's avatar
      transaction: use the standard transaction mechanism to backup branch · 11e6eee4b063
      Pierre-Yves David authored
      Branch is a bit special :
      - It currently does not collaborate with the transaction (or any scoping) for
        writing (this is bad)
      - It can change without the lock being taken (it is protected by `wlock`)
      
      So we rely on the same mechanism as for the backup of the other dirstate file:
      - we only do a backup if we hold the wlock
      - we force a backup though the transaction
      
      
      Since "branch" write does not collaborate with the transaction, we cannot back
      it up "at the last minute" as we do for the dirstate. We have to back it up
      "upfront". Since we have a backup, the transaction is no longer doing its
      "quick_abort" and get noisy. Which is quite annoying. To work around this, and
      to avoid jumping in yet-another-rabbit-hole of "getting branch written
      properly", I am doing horrible things to the transaction in the meantime.
      We should be able to get this code go away during the next cycle.
      
      In the meantime, I prefer to take this small stop so that we stop abusing the
      "journal" and "undo" mechanism instead of the proper backup mechanism of the
      transaction.
      
      
      Also note that this change regress the warning message for the legacy fallback
      introduced in 2008 when issue902 got fixed in dd5a501cb97f (Mercurial 1.0).
      
      I feel like this is fine as issue 902 remains fixed, and this would only affect
      people deploying a mix of 15 year old Mercurial and modern mercurial, and using
      branch and rollback extensively.
      11e6eee4b063
    • Pierre-Yves David's avatar
      transaction: no longer explicitly cache bookmarks · 8fb391363aad
      Pierre-Yves David authored
      The transaction file generation is already dealing with the backup for this.
      So, no need to duplicate such backup.
      8fb391363aad
  6. Feb 22, 2023
  7. Feb 15, 2023
    • Pierre-Yves David's avatar
      commit: use `dirstate.change_files` to scope the associated `addremove` · 28dfb2df4ab9
      Pierre-Yves David authored
      This was significantly more complicated than I expected, because multiple
      extensions get in the way.
      
      I introduced a context that lazily open the transaction and associated context
      to work around these complication. See the inline documentation for details.
      
      Introducing the wrapping transaction remove the need for dirstate-guard (one of
      the ultimate goal of all this), and slightly affect the result of a `hg
      rollback` after a `hg commit --addremove`. That last part is deemed fine. It
      aligns the behavior with what happens after a failed `hg commit --addremove` and
      nobody should be using `hg rollback` anyway.
      
      The small output change in the test come from the different transaction timing
      and fact the transaction now backup the dirstate before the addremove, which
      might mean "no file to backup" when the repository starts from an empty state.
      28dfb2df4ab9
  8. Feb 14, 2023
    • Pierre-Yves David's avatar
      dirstate: do not write an empty dirstate just for backup · e358f6e0e50e
      Pierre-Yves David authored
      This will get in the way when we get more strict about holding the lock when
      writing the dirstate.
      
      Instead, we simply don't copy dirstate files around if there are None at backup
      time.
      
      A couple of tests are impacted they no longer need to backup such "empty"
      dirstate.
      e358f6e0e50e
  9. Feb 20, 2022
  10. Jan 11, 2022
  11. Jan 15, 2021
    • Jörg Sonnenberger's avatar
      branchmap: update rev-branch-cache incrementally · 3e91d9978bec
      Jörg Sonnenberger authored
      Historically, the revision to branch mapping cache was updated on demand
      and shared via bundle2 to avoid the cost of rebuilding on first use.
      
      Use the new `register_changeset` callback and update rbc directly on
      every change. Make the transfer of the bundle part redundant, but keep
      it for the moment to avoid the test churn.
      
      Over all, "hg unbundle" for large bundles is less than 1.8% slower for
      different larger repositories and that seems to a reasonable trade off.
      
      Differential Revision: https://phab.mercurial-scm.org/D9781
      3e91d9978bec
  12. Mar 14, 2019
    • Pierre-Yves David's avatar
      manifestcache: use `wcache` directory for manifest cache · e4ac7e63c213
      Pierre-Yves David authored
      The manifest full text cache is tightly related to the working copy. We should
      use the `wcache` directory for it, instead of the `cache`. Otherwise, multiple
      shares would keep overwriting each other cache entry and we loose its benefit.
      
      This is also more consistent with the fact this cache file is protected by
      `wlock`.
      e4ac7e63c213
  13. Jan 18, 2019
  14. Dec 04, 2018
  15. Nov 15, 2018
    • Boris Feld's avatar
      cache: create `wcache` directory at init time · d5622dfe4ba3
      Boris Feld authored
      The cache directory will be needed very quickly, so it seems simpler to create
      it early to make sure it has the same owner and permission than the other
      directory in the repository.
      d5622dfe4ba3
    • Boris Feld's avatar
      cache: create `cache` directory at init time · 64cdfcc73706
      Boris Feld authored
      The cache directory will be needed very quickly, so it seems simpler to create
      it early to make sure it has the same owner and permission than the other
      directory in the repository.
      64cdfcc73706
    • Boris Feld's avatar
      check-exec: write file in 'wcache' instead of 'cache' · 47e3f554df35
      Boris Feld authored
      Some cache are relevant or affected by the working copy used. So the `.hg/cache`
      directory is not the best place for them because multiple shared repository can
      end up fighting over them.
      
      To address this issue, we introduce a new 'wcache' directory to host this kind
      of cache.
      
      The first user are the `checkisexec` type file. These files describe property of
      the working copy and fit the use-case well.
      47e3f554df35
  16. Nov 29, 2018
  17. Nov 15, 2018
    • Boris Feld's avatar
      checkexec: create destination directory if necessary · bd0874977a5e
      Boris Feld authored
      Since 460733327640, a "share" use the cache of the source repository. A side
      effect is that no `.hg/cache` directory exists in the "share" anymore. As a
      result, the checkexec logic can't use it to create its temporary file and have
      to use the working copy for that.
      
      This is suboptimal, it pollutes the working copy and prevents them to keep the
      file around in cache. We do not want to use the cache directory for the share
      target, it might be on a different file system.
      
      So instead, we (try to) create the directory if it is missing. This is a
      simple change that fixes the current behavior regression on stable.
      
      On default, we should probably ensure the proper directories are created when
      initializing the repository. We should also introduce a 'wcache' directory to
      hold cache file related to the working copy. This would clarify the cache
      situation regarding shares.
      
      The tests catch a couple of other affected cases.
      bd0874977a5e
  18. Sep 19, 2018
  19. Aug 15, 2018
  20. Jul 31, 2018
    • Martijn Pieters's avatar
      manifest: persist the manifestfulltext cache · 0a57945aaf7f
      Martijn Pieters authored
      Reconstructing the manifest from the revlog takes time, so much so that there
      already is a LRU cache to avoid having to load a manifest multiple times.
      
      This patch persists that LRU cache in the .hg/cache directory, so we can re-use
      this cache across hg commands. Commit benchmark (run on Macos 10.13 on a
      2017-model Macbook Pro with Core i7 2.9GHz and flash drive), testing without
      and with patch run 5 times, baseline is r2a227782e754:
      
      * committing to an existing file, against the mozilla-central repository.
        Baseline real time average 1.9692, with patch 1.3786.
      
      A new debugcommand "hg debugmanifestfulltextcache" lets you inspect the cache,
      clear it, or add specific manifest nodeids to it. When calling
      repo.updatecaches(), the manifest(s) for the working copy parents are added to
      the cache.
      
      The hg perfmanifest command has an additional --clear-disk switch to clear this
      cache when testing manifest loading performance.
      
      Using this command to test performance on the firefox repository for revision
      f947d902ed91, whose manifest has a delta chain length of 60540, we see:
      
      $ hg perfmanifest f947d902ed91 --clear-disk
      ! wall 0.972253 comb 0.970000 user 0.850000 sys 0.120000 (best of 10)
      $ hg debugmanifestfulltextcache -a `hg log --debug -r f947d902ed91 | grep manifest | cut -d: -f3`
      Cache contains 1 manifest entries, in order of most to least recent:
      id: 0294517df4aad07c70701db43bc7ff24c3ce7dbc, size 25.6 MB
      Total cache data size 25.6 MB, on-disk 0 bytes
      $ hg perfmanifest f947d902ed91
      ! wall 0.036748 comb 0.040000 user 0.020000 sys 0.020000 (best of 100)
      
      Worst-case scenario: a manifest text loaded from a single delta; in the firefox
      repository manifest node 9a1246ff762e is the chain base for the manifest
      attached to revision f947d902ed91. Loading this from a full cache file is just
      as fast as without the cache; the extra node ids ensure a big full cache:
      
      $ for node in 9a1246ff762e 1a1922c14a3e 54a31d11a36a 0294517df4aa; do
      >   hgd debugmanifestfulltextcache -a $node > /dev/null
      > done
      $ hgd perfmanifest -m 9a1246ff762e
      ! wall 0.077513 comb 0.080000 user 0.030000 sys 0.050000 (best of 100)
      $ hgd perfmanifest -m 9a1246ff762e --clear-disk
      ! wall 0.078547 comb 0.080000 user 0.070000 sys 0.010000 (best of 100)
      0a57945aaf7f
  21. Apr 04, 2018
    • Gregory Szorc's avatar
      tests: conditionalize tests for various repo features · 7542e97c7867
      Gregory Szorc authored
      Working my down the long tail of test failures due to the simple
      store.
      
      We're now down to 38 failures with the simple store.
      
      Differential Revision: https://phab.mercurial-scm.org/D3096
      7542e97c7867
    • Gregory Szorc's avatar
      simplestore: use a custom store for the simple store repo · c2c8962a9465
      Gregory Szorc authored
      Before, we used the default store, which was based on fncache
      and dotencode. After attempting to port tests to work with the
      simple store, I realized that fncache was more trouble than it is
      worth.
      
      This commit implements a proper store type for the simple repo -
      one that isn't based off fncache.
      
      This causes a number of new test failures because of tests
      expecting the full fncache store filename encoding. I may
      extend the store format in a subsequent commit to take the
      filename encoding parts of fncache that we can take
      (basically everything except hash encoding, since that isn't
      reversible). But for now, let's use encoded store.
      
      As part of this, we implement proper requirements support for
      repos created with the simple store. This should have been
      done from the beginning, as a requirement is needed to lock
      out clients that don't understand a storage format.
      
      A new hghave feature advertising the presence of fncache in repos
      has been added. Most tests touching the fncache are now conditional
      on that feature.
      
      Other tests have added the optional repo requirement to output.
      
      Differential Revision: https://phab.mercurial-scm.org/D3095
      c2c8962a9465
  22. Aug 22, 2017
  23. Jun 15, 2017
  24. Jul 12, 2017
  25. Jun 20, 2017
  26. May 02, 2017
    • Pierre-Yves David's avatar
      caches: stop warming the cache after changegroup application · 24f55686a63d
      Pierre-Yves David authored
      Now that we garantee that branchmap cache is updated at the end of the
      transaction we can drop this update. This removes a problematic case with
      nested transaction where the new cache could be written on disk before the
      transaction is finished (and even roll-backed)
      
      Such premature cache write was visible in the following test:
      
      * tests/test-acl.t
      * tests/test-rebase-conflicts.t
      
      In addition, running the cache update later means having more date about the
      state of the repository (in particular: phases). So we can generate caches with
      more information. This creates harmless changes to the following tests:
      
      * tests/test-hardlinks-whitelisted.t
      * tests/test-hardlinks.t
      * tests/test-phases.t
      * tests/test-tags.t
      * tests/test-inherit-mode.t
      24f55686a63d
  27. May 11, 2016
  28. Nov 18, 2015
    • Laurent Charignon's avatar
      localrepo: put bookmark move following commit in one transaction · 4414d500604f
      Laurent Charignon authored
      Before this patch, making a commit on a local repo could move a bookmark and
      both operations would not be grouped as one transaction. This patch makes both
      operations part of one transaction. This is necessary to switch to the new api
      to save bookmarks repo._bookmarks.recordchange if we don't want to change the
      current behavior of rollback.
      
      Dirstate change happening after the commit is done is now part of the
      transaction mentioned above. This leads to a change in the expected output of
      several tests.
      
      The change to test-fncache happens because both lock are now released in the
      same finally clause. The lock release is made explicitly buggy in this test.
      Previously releasing lock would crash triggering release of wlock that crashes
      too. Now lock release crash does not directly result in the release of wlock.
      Instead wlock is released at garbage collection time and the error raised at
      that time "confuses" python.
      4414d500604f
  29. Jan 17, 2015
    • Pierre-Yves David's avatar
      transaction: include backup file in the "undo" transaction · d251da5e0e84
      Pierre-Yves David authored
      Once the transaction is closed, we now write transaction related data for
      possible future undo. For now, we only do it for full file "backup" because
      their were not handle at all in that case. In the future, we could move all the
      current logic to set undo up (that currently exists in localrepository) inside
      transaction itself, but it is not strictly requires to solve the current
      situation.
      d251da5e0e84
  30. Jan 07, 2015
    • Mads Kiilerich's avatar
      branchmap: use revbranchcache when updating branch map · 7d63398fbfd1
      Mads Kiilerich authored
      The revbranchcache is read on demand before it will be used for updating the
      branch map. It is written back when the branchmap is written and it will thus
      use the same locking as branchmap. The revbranchcache instance is short-lived;
      it is only stored in the branchmap from .update() is invoked and until .write()
      is invoked. Branchmap already assume that the repo is locked in that case.
      
      The use of revbranchcache for branch map updates will make sure that the
      revbranchcache "always" is kept up-to-date.
      
      The perfbranchmap benchmark is somewhat bogus, especially when we can see that
      the caching makes a significant difference between the realistic case of a
      first run and the rare case of rerunning it with a full cache. Here are some
      'base' numbers on mozilla-central:
      Before:
      ! wall 6.912745 comb 6.910000 user 6.840000 sys 0.070000 (best of 3)
      After - initial, cache is empty:
      ! wall 7.792569 comb 7.790000 user 7.720000 sys 0.070000 (best of 3)
      After - cache is full:
      ! wall 0.879688 comb 0.880000 user 0.870000 sys 0.010000 (best of 4)
      
      The overhead when running with empty cache comes from checking, missing and
      updating it every time.
      
      Most of the performance improvement comes from not having to extract the branch
      info from the changelog. The last doubling of performance comes from no longer
      having to convert all branch names to local encoding but reuse the few already
      converted branch names.
      
      On the hg repo:
      Before:
      ! wall 0.715703 comb 0.710000 user 0.710000 sys 0.000000 (best of 14)
      After:
      ! wall 0.105489 comb 0.110000 user 0.110000 sys 0.000000 (best of 87)
      7d63398fbfd1
  31. Aug 07, 2014
    • Pierre-Yves David's avatar
      phase: attach phase to the transaction instead of the lock · 37f46575d9c2
      Pierre-Yves David authored
      The phase cache file is no longer written on lock release, it is now handled by
      the transaction (as changesets and obsolescence markers are).
      
      (Hooray)
      
      As we stop relying on the lock to write phase, repos with no existing phase
      information will need to wait for a phase move or a strip to happen in order to
      get the first write in the `phaseroots` file. This explain the change in
      test-inherit-mode.t.
      
      This should not have any side effects but in very obscure cases where
      people interact with pre-2.1 and post-2.1 versions of Mercurial on the
      same repo while having MQ patches applied but the MQ extension
      disabled from time to time. A case unlikely enough to not be worth
      preserving the old behavior with awful hacks.
      37f46575d9c2
  32. Aug 06, 2014
  33. Sep 16, 2013
  34. Jan 13, 2013
    • Kevin Bullock's avatar
      filtering: rename filters to their antonyms · f3b21beb9802
      Kevin Bullock authored
      Now that changelog filtering is in place, it's become evident that
      naming the filters according to the set of revs _not_ included in the
      filtered changelog is confusing. This is especially evident in the
      collaborative branch cache scheme.
      
      This changes the names of the filters to reflect the revs that _are_
      included:
      
        hidden -> visible
        unserved -> served
        mutable -> immutable
        impactable -> base
      
      repoview.filteredrevs is renamed to filterrevs, so that callers read a
      bit more sensibly, e.g.:
      
        filterrevs('visible') # filter revs according to what's visible
      f3b21beb9802
Loading