Skip to content
Snippets Groups Projects
  1. Jan 04, 2023
  2. Jan 03, 2023
  3. Dec 06, 2022
    • Matt Harbison's avatar
      packaging: add dependencies to the PyOxidizer build on macOS · 3d7bf111
      Matt Harbison authored
      Otherwise, we get a bunch of test failures for missing things like pygments, or
      tests skipped entirely.  The input file is a copy/paste from the equivalent
      Windows file, but with dulwich, pygit2, and pytest-vcr commented out because
      the build process errors out with them, flagging them as incompatible with
      loading from memory.  I have no idea if that's actually true or not, because
      I've noticed that if I don't `make clean` after every build, the next build
      flags the watchman stuff as incompatible with loading from memory.
      
      The remaining failures are:
      
          Failed test-alias.t: output changed
          Failed test-basic.t: output changed
          Failed test-check-help.t: output changed
          Failed test-commit-interactive.t: output changed
          Failed test-extension.t: output changed
          Failed test-help.t: output changed
          Failed test-i18n.t: output changed
          Failed test-log.t: output changed
          Failed test-qrecord.t: output changed
          Failed test-share-safe.t: output changed
      
      Most of the issues seem related to loading help for disabled extensions from
      `hgext.__index__`, namely the full extension help being unavailable, not being
      able to resolve what commands are provided by what extension, and not having the
      command level help available.
      
      test-log.t, test-commit-interactive.t, and test-i18n.t look like i18n (or lack
      thereof) issues.
      
      test-basic.t is just odd:
          @@ -55,7 +55,7 @@
           On Python 3, stdio may be None:
      
             $ hg debuguiprompt --config ui.interactive=true 0<&-
          -   abort: Bad file descriptor (no-rhg !)
          +   abort: response expected
              abort: response expected (rhg !)
             [255]
             $ hg version -q 0<&-
      3d7bf111
    • Matt Harbison's avatar
    • Matt Harbison's avatar
      tests: conditionalize test output for in-filesystem pyoxidizer resources · 6a3549a0
      Matt Harbison authored
      The in-memory pyoxidizer builds apparently behave as expected.
      6a3549a0
    • Matt Harbison's avatar
      hghave: add predicates for embedded and filesystem pyoxidizer resources · a2356e15
      Matt Harbison authored
      There are a handful of tests with different output between the two flavors of
      pyoxidizer builds (like the location of the modules and templates), and a few
      others that avoid `known-bad-output` cases with the embedded resources that
      shouldn't cause the tests to fail.
      a2356e15
  4. Dec 05, 2022
  5. Jan 02, 2023
  6. Dec 12, 2022
  7. Nov 11, 2022
    • Pierre-Yves David's avatar
      emitrevision: consider ancestors revision to emit as available base · f463eb67
      Pierre-Yves David authored
      This should make more delta base valid. This notably affects:
      
      * case where we skipped some parent with empty delta to directly delta against
        an ancestors
      * case where an intermediate snapshots is stored.
      
      This change means we could sent largish intermediate snapshots over the wire.
      However this is actually a sub goal here. Sending snapshots over the wire means
      the client have a high odd of simply storing the pre-computed delta instead of
      doing a lengthy process that will… end up doing the same intermediate snapshot.
      
      In addition the overall size of snapshot (or any level) is "only" some or the
      overall delta size. (0.17% for my mercurial clone, 20% for my clone of Mozilla
      try). So Sending them other the wire is unlikely to change large impact on the
      bandwidth used.
      
      If we decide that minimising the bandwidth is an explicit goal, we should
      introduce new logic to filter-out snapshot as delta. The current code has no
      notion explicite of snapshot so far, they just tended to fall into the wobbly
      filtering options.
      
      In some cases, this patch can yield large improvement to the bundling time:
      
          ### data-env-vars.name      = mozilla-try-2019-02-18-zstd-sparse-revlog
            # benchmark.name          = perf-bundle
            # benchmark.variants.revs = last-100000
          before: 68.787066 seconds
          after:  47.552677 seconds (-30.87%)
      
      That translate to large improvement to the pull time :
      
          ### data-env-vars.name           = mozilla-try-2019-02-18-zstd-sparse-revlog
            # benchmark.name               = pull
            # benchmark.variants.issue6528 = disabled
            # benchmark.variants.revs      = last-100000
          before: 142.186625 seconds
          after:   75.897745 seconds  (-46.62%)
      
      No significant negative impact have been observed.
      f463eb67
  8. Nov 09, 2022
  9. Nov 24, 2022
    • Pierre-Yves David's avatar
      emitrevision: if we need to compute a delta on the fly, try p1 or p2 first · 31b4675c
      Pierre-Yves David authored
      Falling back to `prev` does not yield any real value on modern storage and
      result in pathological changes to be created on the other side. Doing a delta
      against a parent will likely be smaller (helping the network) and will be safer
      to apply on the client (helping future pulls by Triggering intermediate
      snapshop where they will be needed by later deltas).
      31b4675c
  10. Nov 28, 2022
  11. Dec 08, 2022
  12. Dec 11, 2022
  13. Dec 05, 2022
    • Matt Harbison's avatar
      extensions: load help from hgext.__index__ as a fallback this time · f4a363b2
      Matt Harbison authored
      Prior to 843418dc0b1b, `hgext.__index__` was consulted first if present, which
      caused the longer help from the extension modules to be ignored, even when
      available.  But that change causes a bunch of test failures when the pyoxidized
      binary bundles *.pyc in the binary, saying the there's no help topic for
      `hg help $disabled_extension` and suggesting the use of `--keyword`, rather than
      showing a summary and indicating that it is disabled.  Current failures were in
      test-check-help.t, test-extension.t, test-help.t, and test-qrecord.t.
      
      Ideally, we would read the various *.pyc files from memory and slurp in the
      docstring, but I know that they used to not be readable as resources, and I
      can't figure out how to make it work now.  So maybe 3.9 and/or the current
      PyOxidizer doesn't support it yet.  I got closer in py2exe with
      `importlib.resources.open_binary("hgext", "rebase.pyc")`, but `open_binary()` on
      *.pyc fails in pyoxidizer.[1]  Either way, the *.pyc can't be passed to
      `ast.parse()` as `extensions._disabledcmdtable()` is doing, so I'm setting that
      aside for now.
      
      [1] https://github.com/indygreg/PyOxidizer/issues/649
      f4a363b2
  14. Dec 07, 2022
    • Matt Harbison's avatar
      extensions: process disabled external paths when `hgext` package is in-memory · 9d8757dd
      Matt Harbison authored
      This fixes `hg help -e ambiguous` in test-helpt.t:2055 with the
      `ambiguous = !./ambiguous.py` configuration, when `hgext` is not in the
      filesystem (e.g. pyoxidizer builds with in-memory resources, or TortoiseHg with
      py2exe), but the disabled external extension is.  Now instead of aborting with a
      suggestion to try `--keyword`, the help command prints text for the extension.
      9d8757dd
  15. Dec 05, 2022
    • Matt Harbison's avatar
      hg: show the correct message when cloning an LFS repo with extension disabled · 7d6c8943
      Matt Harbison authored
      The `extensions._disabledpaths()` doesn't handle fetching help from `__index__`,
      so it returns an empty dictionary of paths.  That means None is always returned
      from `extensions.disabled_help()` when embedding resources inside the pyoxidizer
      or py2exe binary, regardless of the arg or if is an external extension stored in
      the filesystem.  And that means wrongly telling the user with an explicitly
      disabled LFS extension that it will be enabled locally upon cloning from an LFS
      remote.  That causes test-lfs-serve.t:295 to fail.
      
      This effectively reverts most of the rest of 843418dc0b1b, while keeping the
      help text change in place (which was specifically identified as a problem).
      7d6c8943
  16. Dec 09, 2022
    • Matt Harbison's avatar
      demandimport: fix a crash in LazyFinder.__delattr__ · 48e38b17
      Matt Harbison authored
      I was tinkering with `with hgdemandimport.deactivated()` wrapped around loading
      the keyring module, and got spew that seemed to be confirmed by PyCharm.  But I
      can't believe we haven't seen this before (and phabricator uses the same
      pattern):
      
          ** Unknown exception encountered with possibly-broken third-party extension "mercurial_keyring" 1.4.3 (keyring 23.11.0, backend unknown)
          ** which supports versions unknown of Mercurial.
          ** Please disable "mercurial_keyring" and try your action again.
          ** If that fixes the bug please report it to https://foss.heptapod.net/mercurial/mercurial_keyring/issues
          ** Python 3.9.15 (main, Oct 13 2022, 04:28:25) [GCC 7.5.0]
          ** Mercurial Distributed SCM (version 6.3.1)
          ** Extensions loaded: absorb, attorc 20220315, blackbox, eol, extdiff, fastannotate, lfs, mercurial_keyring 1.4.3 (keyring 23.11.0, backend unknown), phabblocker 20220315, phabricator 20220315, purge, rebase, schemes, share, show, strip, uncommit
          Traceback (most recent call last):
            File "/usr/local/bin/hg", line 59, in <module>
              dispatch.run()
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 143, in run
              status = dispatch(req)
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 232, in dispatch
              status = _rundispatch(req)
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 276, in _rundispatch
              ret = _runcatch(req) or 0
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 451, in _runcatch
              return _callcatch(ui, _runcatchfunc)
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 461, in _callcatch
              return scmutil.callcatch(ui, func)
            File "/usr/local/lib/python3.9/site-packages/mercurial/scmutil.py", line 153, in callcatch
              return func()
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 441, in _runcatchfunc
              return _dispatch(req)
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1265, in _dispatch
              return runcommand(
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 899, in runcommand
              ret = _runcommand(ui, options, cmd, d)
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1277, in _runcommand
              return cmdfunc()
            File "/usr/local/lib/python3.9/site-packages/mercurial/dispatch.py", line 1263, in <lambda>
              d = lambda: util.checksignature(func)(ui, *args, **strcmdopt)
            File "/usr/local/lib/python3.9/site-packages/mercurial/util.py", line 1880, in check
              return func(*args, **kwargs)
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 962, in cmd_keyring_check
              user, pwd, source, final_url = handler.get_credentials(
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 497, in get_credentials
              keyring_pwd = password_store.get_http_password(keyring_url, actual_user)
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 287, in get_http_password
              return self._read_password_from_keyring(
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 335, in _read_password_from_keyring
              keyring = import_keyring()
      >> `with hgdemandimport.deactivated()` inserted here
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 120, in import_keyring
              return _import_keyring()
            File "/root/mercurial_keyring/mercurial_keyring/mercurial_keyring.py", line 133, in _import_keyring
              mod, was_imported_now = meu.direct_import_ext(
            File "/usr/lib/python3.9/site-packages/mercurial_extension_utils.py", line 1381, in direct_import_ext
              __import__(module_name)
            File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
            File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
            File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
              self.loader.exec_module(module)
            File "/usr/lib/python3.9/site-packages/keyring/__init__.py", line 1, in <module>
              from .core import (
            File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
            File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
            File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
              self.loader.exec_module(module)
            File "/usr/lib/python3.9/site-packages/keyring/core.py", line 11, in <module>
              from . import backend, credentials
            File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
            File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
            File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
              self.loader.exec_module(module)
            File "/usr/lib/python3.9/site-packages/keyring/backend.py", line 13, in <module>
              from .py312compat import metadata
            File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
            File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
            File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
              self.loader.exec_module(module)
            File "/usr/lib/python3.9/site-packages/keyring/py312compat.py", line 10, in <module>
              import importlib_metadata as metadata  # type: ignore
            File "<frozen importlib._bootstrap>", line 1007, in _find_and_load
            File "<frozen importlib._bootstrap>", line 986, in _find_and_load_unlocked
            File "<frozen importlib._bootstrap>", line 680, in _load_unlocked
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 46, in exec_module
              self.loader.exec_module(module)
            File "/usr/lib/python3.9/site-packages/importlib_metadata/__init__.py", line 715, in <module>
              class MetadataPathFinder(NullFinder, DistributionFinder):
            File "/usr/lib/python3.9/site-packages/importlib_metadata/_compat.py", line 24, in install
              disable_stdlib_finder()
            File "/usr/lib/python3.9/site-packages/importlib_metadata/_compat.py", line 43, in disable_stdlib_finder
              del finder.find_distributions
            File "/usr/local/lib/python3.9/site-packages/hgdemandimport/demandimportpy3.py", line 88, in __delattr__
              return delattr(object.__getattribute__(self, "_finder"))
          TypeError: delattr expected 2 arguments, got 1
      48e38b17
  17. Dec 05, 2022
  18. Dec 01, 2022
  19. Nov 28, 2022
  20. Nov 24, 2022
    • Pierre-Yves David's avatar
      test: adjust test-push-race.t timeout's to overall test timeout · c6f0bcb7
      Pierre-Yves David authored
      The generic `tests/testlib/wait-on-file` mechanism scale its timeout with the
      value of `HGTEST_TIMEOUT`, the `delaypush.py` in `test-push-race.t` is not doing
      this, and we have been seeing more and more timeout from loaded CI worker
      lately.
      
      Adding this timeout scaling should help with that.
      c6f0bcb7
  21. Nov 23, 2022
  22. Nov 22, 2022
  23. Nov 20, 2022
    • Matt Harbison's avatar
      help: fix a py3 error interpolating Set into b'%s' · f09bc2ed
      Matt Harbison authored
      I can't reproduce it, but a coworker hit this with `hg help -v` with 6.2.3:
      
          ...
            File "mercurial\help.pyc", line 865, in helplist
          TypeError: %b requires a bytes-like object, or an object that implements __bytes__, not 'set'
      
      I can confirm that the original expression fails in `hg debugshell`, and the new
      one works.  The second instance was found by searching for "%s", but PyCharm
      detects a lot of variables as Any type, so I have no idea if there are other
      lurking problems.
      f09bc2ed
  24. Nov 19, 2022
    • Raphaël Gomès's avatar
      3324f394
    • Raphaël Gomès's avatar
      010d9e36
    • Raphaël Gomès's avatar
    • Pierre-Yves David's avatar
      memory-usage: fix `hg log --follow --rev R F` space complexity · dcb2581e
      Pierre-Yves David authored
      When running `hg log --follow --rev REVS FILES`, the log code will walk the
      history of all FILES starting from the file revisions that exists in each REVS.
      
      Before doing so, it looks if the files actually exists in the target revisions.
      To do so, it opens the manifest of each revision in REVS to look up if we find
      the associated items in FILES.
      
      Before this changeset this was done in a way that created a changectx for
      each target revision, keeping them in memory while we look into each file.
      
      If the set of REVS is large, this means keeping the manifest for each entry in
      REVS in memory. That can be large… if REV is in the form `::X`, this can quickly
      become huge and saturate the memory. We have seen usage allocating 2GB per
      second until memory runs out.
      
      So this changeset invert the two loop so that only one revision is kept in
      memory during the operation. This solve the memory explosion issue.
      dcb2581e
  25. Nov 18, 2022
Loading