Skip to content
Snippets Groups Projects
  1. Apr 08, 2021
    • Simon Sapin's avatar
      rust: Remove the compile-time 'dirstate-tree' feature flag · 441024b279a6
      Simon Sapin authored
      This code has compiler errors since it is not built on CI and nobody has been
      working on it for some time.
      
      We (Octobus) are still pursuing status optimizations based on a tree data
      structure for the dirstate, but upcoming patches will use a run-time opt-in
      instead of compile-time, so that at least corresponding Rust code keeps
      compiling when other changes are made.
      
      Differential Revision: https://phab.mercurial-scm.org/D10329
      441024b279a6
  2. Sep 14, 2020
    • sliquister's avatar
      procutil: avoid using os.fork() to implement runbgcommand · 8759e22f1649
      sliquister authored
      We ran into the following deadlock:
      
      - some command creates an ssh peer, then raises without explicitly
        closing the peer (hg id + extension in our case)
      
      - dispatch catches the exception, calls ui.log('commandfinish', ..)
        (the sshpeer is still not closed), which calls logtoprocess, which
        calls procutil.runbgcommand.
      
      - in the child of runbgcommand's fork(), between the fork and the
        exec, the opening of file descriptors triggers a gc which runs the
        destructor for sshpeer, which waits on ssh's stderr being closed,
        which never happens since ssh's stderr is held open by the parent of
        the fork where said destructor hasn't run
      
      Remotefilelog appears to have a hack around this deadlock as well.
      
      I don't know if there's more subtlety to it, because even though the
      problem is determistic, it is very fragile, so I didn't manage to
      reduce it.
      
      I can imagine three ways of tackling this problem:
      1. don't run any python between fork and exec in runbgcommand
      2. make the finalizer harmless after the fork
      3. close the peer without relying on gc behavior
      
      This commit goes with 1, as forking without exec'ing is tricky in
      general in a language with gc finalizers. And maybe it's better in the
      presence of rust threads. A future commit will try 2 or 3.
      
      Performance wise: at low memory usage, it's an improvement. At higher
      memory usage, it's about 2x faster than before when ensurestart=True,
      but 2x slower when ensurestart=False. Not sure if that matters. The
      reason for that last bit is that the subprocess.Popen always waits for
      the execve to finish, and at high memory usage, execve is slow because
      it deallocates the large page table. Numbers and script:
      
      				 before after
          mem=1.0GB, ensurestart=True  52.1ms 26.0ms
          mem=1.0GB, ensurestart=False 14.7ms 26.0ms
          mem=0.5GB, ensurestart=True  23.2ms 11.2ms
          mem=0.5GB, ensurestart=False  6.2ms 11.3ms
          mem=0.2GB, ensurestart=True  15.7ms 7.4ms
          mem=0.2GB, ensurestart=False  4.3ms 8.1ms
          mem=0.0GB, ensurestart=True   2.3ms 0.7ms
          mem=0.0GB, ensurestart=False  0.8ms 0.8ms
      
          import time
          for memsize in [1_000_000_000, 500_000_000, 250_000_000, 0]:
              mem = 'a' * memsize
              for ensurestart in [True, False]:
                  now = time.time()
                  n = 100
                  for i in range(n):
                      procutil.runbgcommand([b'true'], {}, ensurestart=ensurestart)
                  after = time.time()
                  ms = (after - now) / float(n) * 1000
                  print(f'mem={memsize / 1e9:.1f}GB, ensurestart={ensurestart} -> {ms:.1f}ms')
      
      Differential Revision: https://phab.mercurial-scm.org/D9019
      8759e22f1649
  3. Apr 08, 2021
    • Matt Harbison's avatar
      share: store relative share paths with '/' separators · 218a26df7813
      Matt Harbison authored
      I created a relative share in Windows and tried to use it in WSL, and it failed:
      
          abort: .hg/sharedpath points to nonexistent directory
              /mnt/c/Users/Matt/hg-review/.hg/..\..\hg\.hg
      
      Use `normpath` on the read side so that the code has the usual Windows style
      paths it always had (I don't think that matters much), but it also eliminates
      the directory escaping path components in the case where the path is printed.
      
      This will not fix repositories that have already been created, but it's trivial
      enough to hand edit the file to correct it.
      
      Differential Revision: https://phab.mercurial-scm.org/D10330
      218a26df7813
  4. Apr 09, 2021
  5. Jan 14, 2021
  6. Apr 09, 2021
  7. Apr 08, 2021
  8. Apr 06, 2021
  9. Apr 07, 2021
  10. Mar 31, 2021
    • Kyle Lippincott's avatar
      match: convert O(n) to O(log n) in exactmatcher.visitchildrenset · 8bca353b1ebc
      Kyle Lippincott authored
      When using narrow, during rebase this is called (at least) once per directory in
      the set of files in the commit being rebased. Every time it's called, we did the
      set arithmetic (now extracted and cached), which was probably pretty cheap but
      not necessary to repeat each time, looped over every item in the matcher and
      kept things that started with the directory we were querying.
      
      With very large narrowspecs, and a commit that touched a file in a large number
      of directories, this was slow. In a pathological repo, the rebase of a single
      commit (that touched over 17k files, I believe in approximately as many
      directories) with a narrowspec that had >32k entries took 8,246s of profiled
      time, with 5,007s of that spent in visitchildrenset (transitively). With this
      change, the time spent in visitchildrenset is less than 34s (which is where my
      profile cut off). Most of the remaining time was network access due to our
      custom remotefilelog-based setup not properly prefetching.
      
      Differential Revision: https://phab.mercurial-scm.org/D10294
      8bca353b1ebc
  11. Mar 30, 2021
  12. Apr 05, 2021
    • Matt Harbison's avatar
      tests: restore the ability to run `black` on Windows · 41d43d12c2c4
      Matt Harbison authored
      The hghave test for black silently stopped working with the change in
      08fd76a553c9.  This was the output of what it was hitting when run in the test
      environment:
      
          Traceback (most recent call last):\r (esc)
             File "c:\\users\\matt\\appdata\\local\\programs\\python\\python39\\lib\\runpy.py", line 197, in _run_module_as_main\r (esc)
               return _run_code(code, main_globals, None,\r (esc)
             File "c:\\users\\matt\\appdata\\local\\programs\\python\\python39\\lib\\runpy.py", line 87, in _run_code\r (esc)
               exec(code, run_globals)\r (esc)
             File "c:\\Users\\Matt\\AppData\\Roaming\\Python\\Python39\\Scripts\\black.exe\\__main__.py", line 4, in <module>\r (esc)
             File "C:\\Users\\Matt\\AppData\\Roaming\\Python\\Python39\\site-packages\\black\\__init__.py", line 70, in <module>\r (esc)
               CACHE_DIR = Path(user_cache_dir("black", version=__version__))\r (esc)
             File "C:\\Users\\Matt\\AppData\\Roaming\\Python\\Python39\\site-packages\\appdirs.py", line 293, in user_cache_dir\r (esc)
               path = os.path.normpath(_get_win_folder("CSIDL_LOCAL_APPDATA"))\r (esc)
             File "C:\\Users\\Matt\\AppData\\Roaming\\Python\\Python39\\site-packages\\appdirs.py", line 481, in _get_win_folder_with_pywin32\r (esc)
               dir = shell.SHGetFolderPath(0, getattr(shellcon, csidl_name), 0, 0)\r (esc)
           pywintypes.com_error: (-2147024893, '$ENOTDIR$.', None, None)\r (esc)
           [1]
      
      Differential Revision: https://phab.mercurial-scm.org/D10310
      41d43d12c2c4
  13. Apr 04, 2021
  14. Apr 03, 2021
  15. Mar 25, 2021
  16. Apr 05, 2021
  17. Apr 06, 2021
  18. Apr 07, 2021
Loading