Skip to content
Snippets Groups Projects
  1. Jul 19, 2016
    • Simon Farnsworth's avatar
      rebase: turn rebase revs into set before filtering obsolete · 754f63671229
      Simon Farnsworth authored
      When the inhibit extension from mutable-history is enabled, it attempts to
      iterate over the rebaseset to prevent the nodes being rebased from being
      marked obsolete. This happens at the same time as rebase's
      _filterobsoleterevs function trying to iterate over the rebaseset to figure
      out which ones are obsolete. The two of these iterating over the same
      revset generatorset cause a 'generator already executing' exception. This is
      probably a flaw in the revset implementation, since iterating over the same
      set twice should be supported.
      
      This regression was introduced in 5d16ebe7b14f, since it changed
      _filterobsoleterevs to be called before the rebaseset was turned into a
      set(). For now let’s just make the rebaseset an actual set again before
      calling that function. This was caught by the inhibit tests.
      
      The relevant call stack from test-inhibit.t:
      
         File "/tmp/hgtests.jgjrN5/install/lib/python/hgext/rebase.py", line 285, in _preparenewrebase
           obsrevs = _filterobsoleterevs(self.repo, rebaseset)
         File "/data/hgbuild/facebook-hg-rpms/mutable-history/hgext/inhibit.py", line 197, in _filterobsoleterevswrap
           r = orig(repo, rebasesetrevs, *args, **kwargs)
         File "/tmp/hgtests.jgjrN5/install/lib/python/hgext/rebase.py", line 1380, in _filterobsoleterevs
           return set(r for r in revs if repo[r].obsolete())
         File "/tmp/hgtests.jgjrN5/install/lib/python/hgext/rebase.py", line 1380, in <genexpr>
           return set(r for r in revs if repo[r].obsolete())
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3079, in _iterordered
           val2 = next(iter2)
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3417, in gen
           yield nextrev()
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3424, in _consumegen
           for item in self._gen:
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 71, in iterate
           cl = repo.changelog
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/repoview.py", line 319, in changelog
           revs = filterrevs(unfi, self.filtername)
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/repoview.py", line 261, in filterrevs
           repo.filteredrevcache[filtername] = func(repo.unfiltered())
         File "/data/hgbuild/facebook-hg-rpms/mutable-history/hgext/directaccess.py", line 65, in _computehidden
           hidden = repoview.filterrevs(repo, 'visible')
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/repoview.py", line 261, in filterrevs
           repo.filteredrevcache[filtername] = func(repo.unfiltered())
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/repoview.py", line 175, in computehidden
           hideable = hideablerevs(repo)
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/repoview.py", line 33, in hideablerevs
           return obsolete.getrevs(repo, 'obsolete')
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/obsolete.py", line 1097, in getrevs
           repo.obsstore.caches[name] = cachefuncs[name](repo)
         File "/data/hgbuild/facebook-hg-rpms/mutable-history/hgext/inhibit.py", line 255, in _computeobsoleteset
           if getrev(n) not in blacklist:
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3264, in __contains__
           return x in self._r1 or x in self._r2
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3348, in __contains__
           for l in self._consumegen():
         File "/tmp/hgtests.jgjrN5/install/lib/python/mercurial/revset.py", line 3424, in _consumegen
           for item in self._gen:
       ValueError: generator already executing
      754f63671229
  2. Jul 18, 2016
    • Jun Wu's avatar
      commandserver: update comment about setpgid · 591c3badff2e
      Jun Wu authored
      Now setpgid has 2 main purposes: better handling for terminal-generated
      SIGTSTP, SIGINT, and process-exit-generated SIGHUP. Update the comment to
      explain things more clearly.
      591c3badff2e
  3. Jul 17, 2016
    • Jun Wu's avatar
      chg: forward SIGINT, SIGHUP to process group · 681fe090d82e
      Jun Wu authored
      These signals are meant to send to a process group, instead of a single
      process: SIGINT is usually emitted by the terminal and sent to the process
      group. SIGHUP usually happens to a process group if termination of a process
      causes that process group to become orphaned.
      
      Before this patch, chg will only forward these signals to the single server
      process. This patch changes it to the server process group.
      
      This will allow us to properly kill processes started by the forked server
      process, like a ssh process. The behavior difference can be observed by
      setting SSH_ASKPASS to a dummy script doing "sleep 100" and then run
      "chg push ssh://dest-need-password-auth". Before this patch, the first Ctrl+C
      will kill the hg process while ssh-askpass and ssh will remain alive. This
      patch will make sure they are killed properly.
      681fe090d82e
  4. Jul 19, 2016
  5. Jul 18, 2016
    • Mads Kiilerich's avatar
      rbc: fix invalid rbc-revs entries caused by missing cache growth · db0095c83344
      Mads Kiilerich authored
      It was in some cases possible to end up writing to the cache file without
      growing it first. The range assignment in _setcachedata would append instead of
      writing at the requested position and thus write the new record in the wrong
      place.
      
      To fix this, we avoid looking up in too small caches, and when growing the
      cache, do it right before writing the new record to it so we know it has been
      done correctly.
      db0095c83344
    • Mads Kiilerich's avatar
    • Jun Wu's avatar
      chg: handle EOF reading data block · 4fc4b8cc9957
      Jun Wu authored
      We recently discovered a case in production that chg uses 100% CPU and is
      trying to read data forever:
      
        recvfrom(4, "", 1814012019, 0, NULL, NULL) = 0
      
      Using gdb, apparently readchannel() got wrong data. It was reading in an
      infinite loop because rsize == 0 does not exit the loop, while the server
      process had ended.
      
        (gdb) bt
        #0 ... in recv () at /lib64/libc.so.6
        #1 ... in readchannel (...) at /usr/include/bits/socket2.h:45
        #2 ... in readchannel (hgc=...) at hgclient.c:129
        #3 ... in handleresponse (hgc=...) at hgclient.c:255
        #4 ... in hgc_runcommand (hgc=..., args=<optimized>, argsize=<optimized>)
        #5 ... in main (argc=...486922636, argv=..., envp=...) at chg.c:661
        (gdb) frame 2
        (gdb) p *hgc
        $1 = {sockfd = 4, pid = 381152, ctx = {ch = 108 'l',
              data = 0x7fb05164f010 "st):\nTraceback (most recent call last):\n"
              "Traceback (most recent call last):\ne", maxdatasize = 1814065152,"
              " datasize = 1814064225}, capflags = 16131}
      
      This patch addresses the infinite loop issue by detecting continuously empty
      responses and abort in that case.
      
      Note that datasize can be translated to ['l', ' ', 'l', 'a']. Concatenate
      datasize and data, it forms part of "Traceback (most recent call last):".
      
      This may indicate a server-side channeledoutput issue. If it is a race
      condition, we may want to use flock to protect the channels.
      4fc4b8cc9957
    • Gregory Szorc's avatar
      sslutil: more robustly detect protocol support · 6cff2ac0ccb9
      Gregory Szorc authored
      The Python ssl module conditionally sets the TLS 1.1 and TLS 1.2
      constants depending on whether HAVE_TLSv1_2 is defined. Yes, these
      are both tied to the same constant (I would think there would be
      separate constants for each version). Perhaps support for TLS 1.1
      and 1.2 were added at the same time and the assumption is that
      OpenSSL either has neither or both. I don't know.
      
      As part of developing this patch, it was discovered that Apple's
      /usr/bin/python2.7 does not support TLS 1.1 and 1.2 (only TLS 1.0)!
      On OS X 10.11, Apple Python has the modern ssl module including
      SSLContext, but it doesn't appear to negotiate TLS 1.1+ nor does
      it expose the constants related to TLS 1.1+. Since this code is
      doing more robust feature detection (and not assuming modern ssl
      implies TLS 1.1+ support), we now get TLS 1.0 warnings when running
      on Apple Python. Hence the test changes.
      
      I'm not super thrilled about shipping a Mercurial that always
      whines about TLS 1.0 on OS X. We may want a follow-up patch to
      suppress this warning.
      6cff2ac0ccb9
  6. Jul 11, 2016
  7. Jun 05, 2016
    • Yuya Nishihara's avatar
      url: drop support for proxying HTTP (not HTTPS) over CONNECT tunneling · e3dc96834126
      Yuya Nishihara authored
      It's been broken since cca59ef27e60, which made ui argument mandatory. I've
      tried several combinations of HTTP/HTTPS proxying on old/new Python versions,
      but I couldn't figure out how to reach this code path. Also, wrapping HTTP
      connection by SSLSocket seems wrong. My understanding is that self.realhostport
      is set by _generic_start_transaction() if HTTPS connection is tunneled.
      
      This patch removes proxy tunneling from httpconnection.connect() assuming
      that it was dead code from the beginning. Note that HTTPS over tunneling
      should be handled by httpsconnection class.
      e3dc96834126
  8. May 21, 2016
  9. May 22, 2016
  10. May 21, 2016
  11. Jun 05, 2016
  12. Jul 17, 2016
    • Gregory Szorc's avatar
      bundle2: store changeset count when creating file bundles · 953839de96ab
      Gregory Szorc authored
      The bundle2 changegroup part has an advisory param saying how many
      changesets are in the part. Before this patch, we were setting
      this part when generating bundle2 parts via the wire protocol but
      not when generating local bundle2 files.
      
      A side effect of not setting the changeset count part is that progress
      bars don't work when applying changesets. As the tests show, this
      impacted clone bundles, shelve, backup bundles, `hg unbundle`, and
      anything touching bundle2 files.
      
      This patch adds a backdoor to allow us to pass state from
      changegroup generation into the unbundler. We store the number
      of changesets in the changegroup in this state and use it to
      populate the aforementioned advisory part parameter when generating
      the bundle2 bundle.
      
      I concede that I'm not thrilled by how state is being passed in
      changegroup.py (it feels a bit hacky). I would love to overhaul the
      rather confusing set of functions in changegroup.py with something that
      passes rich objects around instead of e.g. low-level generators.
      However, given the code freeze for 3.9 is imminent, I'd rather not
      undertake this endeavor right now. This feels like the easiest way
      to get the parameter added to the changegroup part.
      953839de96ab
    • Gregory Szorc's avatar
      util: implement a deterministic __repr__ on sortdict · 37cccad55410
      Gregory Szorc authored
      `hg debugbundle` is calling repr() on bundle2 part params, which are
      now util.sortdict instances. Unfortunately, repr() doesn't appear
      to be deterministic for util.sortdict. So, we implement one.
      
      We include the type name because that's the common convention for
      __repr__ implementations. Having the type name in `hg debugbundle`
      is a bit ugly. But it's a debug command and I don't care enough to
      fix it.
      37cccad55410
    • Gregory Szorc's avatar
      bundle2: use a sorted dict for holding parameters · 6215b5537ba5
      Gregory Szorc authored
      An upcoming change that introduces a 2nd part parameter to a part
      reveals that `hg debugbundle` isn't deterministic because parameters
      are stored on n plain, unsorted dict.
      
      While we could change that command to sort before output, I think
      the more important underlying issue is that bundle2 reading is taking
      an ordered data structure and converting it to an unordered one.
      Plugging in util.sortdict() fixes that problem while preserving API
      compatibility.
      
      This patch also appears to shine light on the fact that we don't
      have tests verifying parts with multiple parameters roundtrip
      correctly. That would be a good thing to test (and fuzz)... someday.
      6215b5537ba5
  13. Jul 15, 2016
    • Gregory Szorc's avatar
      wireproto: extract repo filtering to standalone function · 84c1a5942f1d
      Gregory Szorc authored
      As part of teaching Mozilla's replication extension to better handle
      repositories with obsolescence data, I encountered a few scenarios
      where I wanted built-in wire protocol commands from replication clients
      to operate on unfiltered repositories so they could have access to
      obsolete changesets.
      
      While the undocumented "web.view" config option provides a mechanism
      to choose what filter/view hgweb operates on, this doesn't apply
      to wire protocol commands because wireproto.dispatch() is always
      operating on the "served" repo.
      
      This patch extracts the line for obtaining the repo that
      wireproto commands operate on to its own function so extensions
      can monkeypatch it to e.g. return an unfiltered repo.
      
      I stopped short of exposing a config option because I view the use
      case for changing this as a niche feature, best left to the domain
      of extensions.
      84c1a5942f1d
    • Gregory Szorc's avatar
      url: add distribution and version to user-agent request header (BC) · 486de14eb394
      Gregory Szorc authored
      As a server operator, I've always wanted to know what Mercurial
      version clients are running so I can track version adoption and
      make informed decisions about which versions of Mercurial to
      support in extensions. Unfortunately, there is no easy way to discern
      this today: the best you can do is look for high-level feature usage
      (e.g. bundle2) or sniff capabilities from bundle2 commands. And these
      things aren't changed frequently enough to tell you anything that
      interesting.
      
      Nearly every piece of software talking HTTP sends its version in
      the user agent. This includes web browsers, curl, and even Git.
      
      This patch adds the distribution name and version to the user-agent
      HTTP request header. We choose "Mercurial" for the distribution
      name because that seems appropriate. The version string comes
      from __version__.
      
      The value is inside parenthesis for a few reasons:
      
      * The version *may* contain spaces
      * Alternate forms like "Mercurial/<version>" imply structure and
        since the user agent should not be used by servers for protocol
        or feature negotiation/detection, we don't want to even give the
        illusion that the value should be parsed. A free form field is
        the most hostile to parsing.
      
      Flagging the patch as BC so it shows up in release notes. This
      change should be backwards compatible. But I wouldn't be surprised if
      a server somewhere is filtering on the exact old user agent string. So
      I want to make noise about this change.
      486de14eb394
  14. Jul 16, 2016
  15. May 22, 2016
  16. May 21, 2016
  17. Jul 17, 2016
    • Pulkit Goyal's avatar
      pycompat: make pycompat demandimport friendly · 06587edd1233
      Pulkit Goyal authored
      pycompat.py includes hack to import modules whose names are changed in Python 3.
      We use try-except to load module according to the version of python. But this
      method forces us to import the modules to raise an ImportError and hence making
      it demandimport unfriendly.
      
      This patch changes the try-except blocks to a single if-else block. To avoid
      test-check-pyflakes.t complain about unused imports, pycompat.py is excluded
      from the test.
      06587edd1233
  18. Jul 18, 2016
  19. Jul 17, 2016
    • Jun Wu's avatar
      chg: add pgid to hgclient struct · c66bc06f1bf6
      Jun Wu authored
      The previous patch makes the server tell the client its pgid. This patch
      stores it in hgclient_t and adds a function to get it.
      c66bc06f1bf6
    • Jun Wu's avatar
      commandserver: send pgid in hello message · ee8186457516
      Jun Wu authored
      See the next patches for why we need it.
      ee8186457516
    • Gregory Szorc's avatar
      tests: update test certificate generation instructions · 43f3c0df2fab
      Gregory Szorc authored
      Suggestions from Anton Shestakov and Julien Cristau to use
      -subj and faketime, respectively.
      43f3c0df2fab
    • Gregory Szorc's avatar
      sslutil: move comment about protocol constants · 4a4b8d3b4e43
      Gregory Szorc authored
      protocolsettings() is the appropriate place for this comment.
      4a4b8d3b4e43
    • Gregory Szorc's avatar
      sslutil: support defining cipher list · 9654ef41f7cc
      Gregory Szorc authored
      Python 2.7 supports specifying a custom cipher list to TLS sockets.
      Advanced users may wish to specify a custom cipher list to increase
      security. Or in some cases they may wish to prefer weaker ciphers
      in order to increase performance (e.g. when doing stream clones
      of very large repositories).
      
      This patch introduces a [hostsecurity] config option for defining
      the cipher list. The help documentation states that it is for
      advanced users only.
      
      Honestly, I'm a bit on the fence about providing this because
      it is a footgun and can be used to decrease security. However,
      there are legitimate use cases for it, so I think support should
      be provided.
      9654ef41f7cc
    • Gregory Szorc's avatar
      hghave: add test for Python 2.7+ · d5067913f97b
      Gregory Szorc authored
      Setting ciphers in the ssl module requires Python 2.7. Surprisingly,
      we didn't have a test for running on Python 2.7.
      d5067913f97b
  20. Jul 16, 2016
  21. May 20, 2016
    • Katsunori FUJIWARA's avatar
      tests: check importing modules in perf.py for historical portability · d1a7d9c279bb
      Katsunori FUJIWARA authored
      To check importing modules in perf.py for historical portability, this
      patch lists up files by "hg files" both for "1.2" and tip, and builds
      up "module whitelist" check from those files.
      
      This patch uses "1.2" as earlier side version of "module whitelist",
      because "mercurial.error" module is a blocker for loading perf.py with
      Mercurial earlier than 1.2, and just importing "mercurial.error"
      separately isn't enough.
      d1a7d9c279bb
Loading