This instance will be upgraded to Heptapod 0.27.1 on 2021-12-08 at 22:00 UTC+1 (a few minutes of down time)

  1. 08 Dec, 2020 1 commit
  2. 10 Dec, 2020 1 commit
  3. 02 Dec, 2020 1 commit
    • Pulkit Goyal's avatar
      procutils: don't try to get `.buffer` if sys.stdin is None · 81c1f5d1801f
      Pulkit Goyal authored
      While hunting down following test failure of test-chg.t on Python 3, I stumbled
      the case when `.buffer` is not available as sys.stdin is None.
      
      	--- /home/pulkit/repo/hg-committed/tests/test-chg.t
      	+++ /home/pulkit/repo/hg-committed/tests/test-chg.t.err
      	@@ -203,7 +203,31 @@
      	   $ CHGDEBUG=1 chg version -q 0<&-
      		 chg: debug: * stdio fds are missing (glob)
      	     chg: debug: * execute original hg (glob)
      		 -  Mercurial Distributed SCM * (glob)
      		 +  Traceback (most recent call last):
      		 +    File "/tmp/hgtests.avspvsq4/install/bin/hg", line 43, in <module>
      		 +      dispatch.run()
      		 +    File "/usr/lib/python3.6/importlib/util.py", line 233, in
      		 __getattribute__
      		 +      self.__spec__.loader.exec_module(self)
      		 +    File "<frozen importlib._bootstrap_external>", line 678, in
      		 exec_module
      		 +    File "<frozen importlib._bootstrap>", line 219, in
      		 _call_with_frames_removed
      		 +    File
      		 "/tmp/hgtests.avspvsq4/install/lib/python/mercurial/dispatch.py", line
      		 726, in <module>
      		 +      class lazyaliasentry(object):
      		 +    File
      		 "/tmp/hgtests.avspvsq4/install/lib/python/mercurial/dispatch.py", line
      		 737, in lazyaliasentry
      		 +      @util.propertycache
      		 +    File "/usr/lib/python3.6/importlib/util.py", line 233, in
      		 __getattribute__
      		 +      self.__spec__.loader.exec_module(self)
      		 +    File "<frozen importlib._bootstrap_external>", line 678, in
      		 exec_module
      		 +    File "<frozen importlib._bootstrap>", line 219, in
      		 _call_with_frames_removed
      		 +    File "/tmp/hgtests.avspvsq4/install/lib/python/mercurial/util.py",
      		 line 3473, in <module>
      		 +      f=procutil.stderr,
      		 +    File "/usr/lib/python3.6/importlib/util.py", line 233, in
      		 __getattribute__
      		 +      self.__spec__.loader.exec_module(self)
      		 +    File "<frozen importlib._bootstrap_external>", line 678, in
      		 exec_module
      		 +    File "<frozen importlib._bootstrap>", line 219, in
      		 _call_with_frames_removed
      		 +    File
      		 "/tmp/hgtests.avspvsq4/install/lib/python/mercurial/utils/procutil.py",
      		 line 127, in <module>
      		 +      stdin = sys.stdin.buffer
      		 +  AttributeError: 'NoneType' object has no attribute 'buffer'
      		 +  [1]
      
      		  server lifecycle
      		   ----------------
      
      Differential Revision: https://phab.mercurial-scm.org/D9500
      81c1f5d1801f
  4. 01 Dec, 2020 1 commit
  5. 25 Jun, 2020 1 commit
  6. 05 Jun, 2020 1 commit
  7. 07 May, 2020 2 commits
    • Raphaël Gomès's avatar
      formatting: add missing newline · 79f6f856c53f
      Raphaël Gomès authored
      Differential Revision: https://phab.mercurial-scm.org/D8509
      
      --HG--
      branch : stable
      79f6f856c53f
    • Rodrigo Damazio Bovendorp's avatar
      procutil: always waiting on child processes to prevent zombies with 'hg serve' · ed684a82e29b
      Rodrigo Damazio Bovendorp authored
      When runbgcommand is invoked by an extension with ensurestart=False, we never
      called waitpid - which is fine in most cases, except if that's happening on a
      command server (e.g. chg), in which case the child defunct process will just
      sit there for as long as the server is running.
      
      The actual semantics of SIGCHLD signal handling is a lot more complex than
      it seems, and the POSIX standard *seems* to read that it's ignored by default
      and everything would just work without the waitpid if we're not listening for
      it, but the truth is that it's only ignored if we *explicitly* set it to
      SIG_IGN. We further cannot set it to SIG_IGN or to a catch-all handler across
      all of 'hg serve', because Python's suprocess.Popen relies on that signal,
      and a few specific parts of hg also set custom handlers, so instead we wait
      for specific PIDs in dedicated threads.
      
      I did a poor-man's benchmark of the thread creation and it seems to take
      about 1ms, which is way better than the 20+ms from ensurestart=True.
      
      Differential Revision: https://phab.mercurial-scm.org/D8497
      
      --HG--
      branch : stable
      ed684a82e29b
  8. 17 Jul, 2020 1 commit
    • Manuel Jacob's avatar
      windows: always work around EINVAL in case of broken pipe for stdout / stderr · a37f290a7124
      Manuel Jacob authored
      In 29a905fe23ae, I missed the fact that the `winstdout` class works around two
      unrelated bugs (size limit when writing to consoles and EINVAL in case of
      broken pipe) and that the latter bug happens even when no console is involved.
      When writing a test for this, I realized that the same problem applies to
      stderr, so I applied the workaround for EINVAL to both stdout and stderr.
      
      The size limit is worked around in the same case as before (consoles on Windows
      on Python 2). For that, I changed the `winstdout` class.
      a37f290a7124
  9. 05 Dec, 2019 1 commit
    • Augie Fackler's avatar
      procutil: try and avoid angering CoreFoundation on macOS · a89381e04c58
      Augie Fackler authored
      We've seen failures like this:
      
          objc[57662]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called.
          objc[57662]: +[__NSCFConstantString initialize] may have been in progress in another thread when fork() was called. We cannot safely call it or ignore it in the fork() child process. Crashing instead. Set a breakpoint on objc_initializeAfterForkError to debug.
      
      I think this is due to forking off some background processes during
      `hg update` or similar. I don't have any conclusive proof this is the
      fork() call that's to blame, but it's the most likely one since the
      regular `hg update` codepath uses the other fork() invocation (via
      workers) and we don't get this report from non-Google macOS users.
      
      Ugh.
      
      Differential Revision: https://phab.mercurial-scm.org/D7615
      a89381e04c58
  10. 09 Dec, 2019 1 commit
  11. 01 Dec, 2019 1 commit
  12. 14 Nov, 2019 4 commits
  13. 27 Nov, 2020 1 commit
    • durin42's avatar
      formating: upgrade to black 20.8b1 · 89a2afe31e82
      durin42 authored
      This required a couple of small tweaks to un-confuse black, but now it
      works. Big formatting changes come from:
      
       * Dramatically improved collection-splitting logic upstream
       * Black having a strong (correct IMO) opinion that """ is better than '''
      
      Differential Revision: https://phab.mercurial-scm.org/D9430
      89a2afe31e82
  14. 17 Nov, 2020 1 commit
    • Yuya Nishihara's avatar
      chgserver: backport py3 buffered I/O workarounds from procutil · b56feaa9b520
      Yuya Nishihara authored
      I've recently switched to new machine and I found chg's stdout is fully
      buffered.
      
      Even though chg server is a daemon process, it inherits the environment
      where the chg client originally forked the server. This means the server's
      stdout might have been wrapped by LineBufferedWrapper. That's why we need
      to do wrap/unwrap in both ways.
      
      The "if" condition in _restoreio() looks weird, but I'm not willing to
      clean things up because stdio behavior is fundamentally different between
      py2 and py3, and py2 support will be dropped anyway.
      b56feaa9b520
  15. 14 Oct, 2020 1 commit
  16. 15 Jul, 2020 1 commit
  17. 10 Jul, 2020 1 commit
    • Manuel Jacob's avatar
      procutil: ensure that procutil.std{out,err}.write() writes all bytes · 8e04607023e5
      Manuel Jacob authored
      Python 3 offers different kind of streams and it’s not guaranteed for all of
      them that calling write() writes all bytes.
      
      When Python is started in unbuffered mode, sys.std{out,err}.buffer are
      instances of io.FileIO, whose write() can write less bytes for
      platform-specific reasons (e.g. Linux has a 0x7ffff000 bytes maximum and could
      write less if interrupted by a signal; when writing to Windows consoles, it’s
      limited to 32767 bytes to avoid the "not enough space" error). This can lead to
      silent loss of data, both when using sys.std{out,err}.buffer (which may in fact
      not be a buffered stream) and when using the text streams sys.std{out,err}
      (I’ve created a CPython bug report for that:
      https://bugs.python.org/issue41221).
      
      Python may fix the problem at some point. For now, we implement our own wrapper
      for procutil.std{out,err} that calls the raw stream’s write() method until all
      bytes have been written. We don’t use sys.std{out,err} for larger writes, so I
      think it’s not worth the effort to patch them.
      8e04607023e5
  18. 11 Jul, 2020 1 commit
    • Manuel Jacob's avatar
      procutil: move assignments · b4c35e439ea5
      Manuel Jacob authored
      This should probably be part of the previous patch, but folding it results in a
      less useful word diff, so I decided to keep it separate for review.
      b4c35e439ea5
  19. 10 Jul, 2020 5 commits
  20. 11 Jul, 2020 1 commit
    • Manuel Jacob's avatar
      procutil: back out 8403cc54bc83 (make ....procutil.stderr unbuffered) · fa270dcbdb55
      Manuel Jacob authored
      Changeset 8403cc54bc83 introduced code that opens a second file object
      referring to the stderr file descriptor. This broke tests on Windows. The
      reason is that on Windows, sys.stderr is buffered and procutil.stderr closed
      the file descriptor when it got garbage collected before sys.stderr had the
      chance to flush buffered data.
      
      `procutil.stdout` had the same problem for a long time, but we didn’t realize,
      as in CI test runs, stdout is not a TTY and in this case no second file object
      is opened.
      fa270dcbdb55
  21. 06 Jul, 2020 1 commit
  22. 05 Jul, 2020 1 commit
    • Manuel Jacob's avatar
      procutil: make mercurial.utils.procutil.stderr unbuffered · 8403cc54bc83
      Manuel Jacob authored
      For most Mercurial code, it doesn’t make a difference, as the ui object flushes
      stderr explicitly (after the change, we could get rid of the explicit flush).
      One example where it makes a observable difference is mercurial.util.timed().
      Without the patch, the time is not immediately shown on Python 3. With the
      patch, it’s shown immediately on all Python versions and platforms.
      8403cc54bc83
  23. 04 Jul, 2020 4 commits
  24. 02 Jul, 2020 1 commit
  25. 10 Jun, 2020 1 commit
    • Manuel Jacob's avatar
      py3: make stdout line-buffered if connected to a TTY · f9734b2d59cc
      Manuel Jacob authored
      Status messages that are to be shown on the terminal should be written to the
      file descriptor before anything further is done, to keep the user updated.
      
      One common way to achieve this is to make stdout line-buffered if it is
      connected to a TTY. This is done on Python 2 (except on Windows, where libc,
      which the CPython 2 streams depend on, does not properly support this).
      
      Python 3 rolls it own I/O streams. On Python 3, buffered binary streams can't be
      set line-buffered. The previous code (added in 227ba1afcb65) incorrectly
      assumed that on Python 3, pycompat.stdout (sys.stdout.buffer) is already
      line-buffered. However the interpreter initializes it with a block-buffered
      stream or an unbuffered stream (when the -u option or the PYTHONUNBUFFERED
      environment variable is set), never with a line-buffered stream.
      
      One example where the current behavior is unacceptable is when running
      `hg pull https://www.mercurial-scm.org/repo/hg` on Python 3, where the line
      "pulling from https://www.mercurial-scm.org/repo/hg" does not appear on the
      terminal before the hg process blocks while waiting for the server.
      
      Various approaches to fix this problem are possible, including:
      
      1. Weaken the contract of procutil.stdout to not give any guarantees about
         buffering behavior. In this case, users of procutil.stdout need to be
         changed to do enough flushes. In particular,
        1. either ui must insert enough flushes for ui.write() and friends, or
        2. ui.write() and friends get split into flushing and fully buffered
           methods, or
        3. users of ui.write() and friends must flush explicitly.
      2. Make stdout unbuffered.
      3. Make stdout line-buffered. Since Python 3 does not natively support that for
         binary streams, we must implement it ourselves.
      
      (2.) is problematic because using unbuffered I/O changes the performance
      characteristics significantly compared to line-buffered (which is used on
      Python 2) and this would be a regression.
      
      (1.2.) and (1.3) are a substantial amount of work. It’s unclear whether the
      added complexity would be justified, given that raw performance doesn’t matter
      that much when writing to a terminal much faster than the user could read it.
      
      (1.1.) pushes complexity into the ui class instead of separating the concern of
      how stdout is buffered. Other users of procutil.stdout would still need to take
      care of the flushes.
      
      This patch implements (3.). The general performance considerations are very
      similar to (1.1.). The extra method invocation and method forwarding add a
      little more overhead if the class is used. In exchange, it doesn’t add overhead
      if not used.
      
      
      For the benchmarks, I compared the previous implementation (incorrect on Python
      3), (1.1.), (3.) and (2.). The command was chosen so that the streams were
      configured as if they were writing to a TTY, but actually write to a pager,
      which is also the default:
      HGRCPATH=/dev/null python3 ./hg --cwd ~/vcs/mozilla-central --time --pager yes --config pager.pager='cat > /dev/null' status --all
      
      previous:
      time: real 7.880 secs (user 7.290+0.050 sys 0.580+0.170)
      time: real 7.830 secs (user 7.220+0.070 sys 0.590+0.140)
      time: real 7.800 secs (user 7.210+0.050 sys 0.570+0.170)
      
      (1.1.) using Yuya Nishihara’s patch:
      time: real 9.860 secs (user 8.670+0.350 sys 1.160+0.830)
      time: real 9.540 secs (user 8.430+0.370 sys 1.100+0.770)
      time: real 9.830 secs (user 8.630+0.370 sys 1.180+0.840)
      
      (3.) using this patch:
      time: real 9.580 secs (user 8.480+0.350 sys 1.090+0.770)
      time: real 9.670 secs (user 8.480+0.330 sys 1.170+0.860)
      time: real 9.640 secs (user 8.500+0.350 sys 1.130+0.810)
      
      (2.) using a previous patch by me:
      time: real 10.480 secs (user 8.850+0.720 sys 1.590+1.500)
      time: real 10.490 secs (user 8.750+0.750 sys 1.710+1.470)
      time: real 10.240 secs (user 8.600+0.700 sys 1.590+1.510)
      
      
      As expected, there’s no difference on Python 2, as exactly the same code paths
      are used:
      
      previous:
      time: real 6.950 secs (user 5.870+0.330 sys 1.070+0.770)
      time: real 7.040 secs (user 6.040+0.360 sys 0.980+0.750)
      time: real 7.070 secs (user 5.950+0.360 sys 1.100+0.760)
      
      this patch:
      time: real 7.010 secs (user 5.900+0.390 sys 1.070+0.730)
      time: real 7.000 secs (user 5.850+0.350 sys 1.120+0.760)
      time: real 7.000 secs (user 5.790+0.380 sys 1.170+0.710)
      f9734b2d59cc
  26. 29 May, 2020 1 commit
  27. 08 Nov, 2019 1 commit
    • Augie Fackler's avatar
      cleanup: remove pointless r-prefixes on single-quoted strings · 9f70512ae2cf
      Augie Fackler authored
      This is the promised second step on single-quoted strings. These had
      existed because our source transformer didn't turn r'' into b'', so we
      had tagged some strings as r-strings to get "native" strings on both
      Pythons. Now that the transformer is gone, we can dispense with this
      nonsense.
      
      Methodology:
      
      I ran
      
          hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$'  | xargs egrep --color=never -n  -- \[\^b\]\[\^a-z\]r\'\[\^\'\\\\\]\*\'\[\^\'\
      
      in an emacs grep-mode buffer, and then used a keyboard macro to
      iterate over the results and remove the r prefix as needed.
      
      # skip-blame removing unneeded r prefixes left over from Python 3 migration.
      
      Differential Revision: https://phab.mercurial-scm.org/D7306
      9f70512ae2cf
  28. 07 Nov, 2019 1 commit
    • Augie Fackler's avatar
      cleanup: remove pointless r-prefixes on double-quoted strings · 313e3a279828
      Augie Fackler authored
      This is only double-quoted strings. I'll do single-quoted strings as a
      second step. These had existed because our source transformer didn't
      turn r"" into b"", so we had tagged some strings as r-strings to get
      "native" strings on both Pythons. Now that the transformer is gone, we
      can dispense with this nonsense.
      
      Methodology:
      
      I ran
      
          hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$'  | xargs egrep --color=never -n  -- \[\^a-z\]r\"\[\^\"\\\\\]\*\"\[\^\"\]
      
      in an emacs grep-mode buffer, and then used a keyboard macro to
      iterate over the results and remove the r prefix as needed.
      
      # skip-blame removing unneeded r prefixes left over from Python 3 migration.
      
      Differential Revision: https://phab.mercurial-scm.org/D7305
      313e3a279828
  29. 10 Oct, 2019 1 commit