Skip to content
Snippets Groups Projects
  1. Sep 29, 2018
  2. Sep 28, 2018
  3. Oct 04, 2018
  4. Oct 05, 2018
  5. Oct 02, 2018
  6. Sep 28, 2018
    • Pulkit Goyal's avatar
      narrow: the first version of narrow_widen wireprotocol command · 8feae5b9
      Pulkit Goyal authored
      This patch introduces a wireprotocol command narrow_widen() which will be used
      to widen a narrow copy using `hg tracked` command provided by narrow extension.
      
      The wireprotocol command takes the old and new includes and excludes, common
      heads, changegroup version, known revs, and a boolean ellipses and generates a
      bundle2 of the required data and send it. The clients receives the bundle2
      and applies that.
      
      A bundle2 instead of changegroup because in future we might want to add more
      things to send while widening. Thanks for martinvonz for the suggestion.
      
      I am not sure whether we need changegroup version as an argument to the command
      as I *think* narrow needs changegroup3 already.
      
      The tests shows that we don't exchange phase data now while widening which is
      nice. Also we don't check for pushkeys, rbc-cache, bookmarks etc.
      
      This does not support ellipses cases for now but will be supported in future
      patches. Since we send bundle2, it won't be hard to plug the ellipses logic in
      here.
      
      The existing code for widening a non-ellipses case is also dropped in this
      patch.
      
      Differential Revision: https://phab.mercurial-scm.org/D4813
      8feae5b9
  7. Oct 05, 2018
  8. Oct 04, 2018
  9. Oct 05, 2018
  10. Jul 11, 2018
    • Pulkit Goyal's avatar
      remotenames: add names argument to remotenames revset · fda1df3d
      Pulkit Goyal authored
      This patch adds names argument to the revsets provided by the remotenames
      extension. The revsets are remotenames(), remotebranches() and
      remotebookmarks(). names can be a single names, list of names or can be empty too
      which means it's an optional argument.
      
      If names is/are passed, changesets which have those remotenames will be
      returned.
      If names are not passed, changesets from all the remotenames are shown.
      
      Passing an invalid remotename does not throw error.
      
      The name argument also supports pattern matching.
      
      Tests are added for the argument in tests/test-logexchange.t
      
      Differential Revision: https://phab.mercurial-scm.org/D3639
      fda1df3d
  11. Sep 07, 2018
  12. Oct 03, 2018
    • Gregory Szorc's avatar
      revlog: rewrite censoring logic · 324b4b10
      Gregory Szorc authored
      I was able to corrupt a revlog relatively easily with the existing
      censoring code. The underlying problem is that the existing code
      doesn't fully take delta chains into account. When copying revisions
      that occur after the censored revision, the delta base can refer
      to a censored revision. Then at read time, things blow up due to the
      revision data not being a compressed delta.
      
      This commit rewrites the revlog censoring code to take a higher-level
      approach. We now create a new revlog instance pointing at temp files.
      We iterate through each revision in the source revlog and insert
      those revisions into the new revlog, replacing the censored revision's
      data along the way.
      
      The new implementation isn't as efficient as the old one. This is
      because it will fully engage delta computation on insertion. But I
      don't think it matters.
      
      The new implementation is a bit hacky because it attempts to reload
      the revlog instance with a new revlog index/data file. This is fragile.
      But this is needed because the index (which could be backed by C) would
      have a cached copy of the old, possibly changed data and that could
      lead to problems accessing index or revision data later.
      
      One benefit of the new approach is that we integrate with the
      transaction. The old revlog is backed up and if the transaction is
      rolled back, the original revlog is restored.
      
      As part of this, we had to teach the transaction about the store
      vfs. I'm not super keen about this. But this was the easiest way
      to hook things up to the transaction. We /could/ just ignore the
      transaction like we were doing before. But any file mutation should
      be governed by transaction semantics, including undo during rollback.
      
      Differential Revision: https://phab.mercurial-scm.org/D4869
      324b4b10
    • Gregory Szorc's avatar
      revlog: move loading of index data into own method · 0a4625ff
      Gregory Szorc authored
      This will allow us to "reload" a revlog instance from a rewritten
      index file, which will be used in a subsequent commit.
      
      Differential Revision: https://phab.mercurial-scm.org/D4868
      0a4625ff
    • Gregory Szorc's avatar
      revlog: clear revision cache on hash verification failure · 801ccd8e
      Gregory Szorc authored
      The revision cache is populated after raw revision fulltext is
      retrieved but before hash verification. If hash verification
      fails, the revision cache will be populated and subsequent
      operations to retrieve the invalid fulltext may return the cached
      fulltext instead of raising.
      
      This commit changes hash verification so it will invalidate the
      revision cache if the cached node fails hash verification. The
      side-effect is that subsequent operations to request the revision
      text - even the raw revision text - will always fail.
      
      The new behavior is consistent and is definitely less wrong. There
      is an open question of whether revision(raw=True) should validate
      hashes. But I'm going to punt on this problem. We can always change
      behavior later. And to be honest, I'm not sure we should expose
      raw=True on the storage interface at all. Another day...
      
      Differential Revision: https://phab.mercurial-scm.org/D4867
      801ccd8e
  13. Sep 06, 2018
    • Augie Fackler's avatar
      fuzz: new fuzzer for cext/manifest.c · 8c692a6b
      Augie Fackler authored
      This is a bit messy, because lazymanifest is tightly coupled to the
      cpython API for performance reasons. As a result, we have to build a
      whole Python without pymalloc (so ASAN can help us out) and link
      against that. Then we have to use an embedded Python interpreter. We
      could manually drive the lazymanifest in C from that point, but
      experimentally just using PyEval_EvalCode isn't really any slower so
      we may as well do that and write the innermost guts of the fuzzer in
      Python.
      
      Leak detection is currently disabled for this fuzzer because there are
      a few global-lifetime things in our extensions that we more or less
      intentionally leak and I didn't want to take the detour to work around
      that for now.
      
      This should not be pushed to our repo until
      https://github.com/google/oss-fuzz/pull/1853 is merged, as this
      depends on having the Python tarball around.
      
      Differential Revision: https://phab.mercurial-scm.org/D4879
      8c692a6b
  14. Oct 03, 2018
    • Gregory Szorc's avatar
      revlog: rename _cache to _revisioncache · 55db747a
      Gregory Szorc authored
      "cache" is generic and revlog instances have multiple caches. Let's
      be descriptive about what this is a cache for.
      
      Differential Revision: https://phab.mercurial-scm.org/D4866
      55db747a
    • Gregory Szorc's avatar
      testing: add file storage integration for bad hashes and censoring · cdf61ab1
      Gregory Szorc authored
      In order to implement these tests, we need a backdoor to write data
      into storage backends while bypassing normal checks. We invent a
      callable to do that.
      
      As part of writing the tests, I found a bug with censorrevision()
      pretty quickly! After calling censorrevision(), attempting to
      access revision data for an affected node raises a cryptic error
      related to malformed compression. This appears to be due to the
      revlog not adjusting delta chains as part of censoring.
      
      I also found a bug with regards to hash verification and revision
      fulltext caching. Essentially, we cache the fulltext before hash
      verification. If we look up the fulltext after a failed hash
      verification, we don't get a hash verification exception. Furthermore,
      the behavior of revision(raw=True) can be inconsistent depending on
      the order of operations.
      
      I'll be fixing both these bugs in subsequent commits.
      
      Differential Revision: https://phab.mercurial-scm.org/D4865
      cdf61ab1
    • Gregory Szorc's avatar
    • Gregory Szorc's avatar
      wireprotov2: always advertise raw repo requirements · 39074a35
      Gregory Szorc authored
      I'm pretty sure my original thinking behind making it conditional
      on stream clone support was that the behavior mirrored wire protocol
      version 1.
      
      I don't see a compelling reason for us to not advertise the server's
      storage requirements. The proper way to advertise stream clone support
      in wireprotov2 would be to not advertise the command(s) required to
      perform stream clone or to advertise a separate capability denoting
      stream clone support.
      
      Stream clone isn't yet implemented on wireprotov2, so we can cross
      this bridge later.
      
      Differential Revision: https://phab.mercurial-scm.org/D4863
      39074a35
    • Gregory Szorc's avatar
      tests: don't be as verbose in wireprotov2 tests · a732d702
      Gregory Szorc authored
      I don't think that printing low-level I/O and frames is beneficial to
      testing command-level functionality. Protocol-level testing, yes. But
      command-level functionality shouldn't care about low-level details in
      most cases. This output makes tests more verbose and harder to read.
      It also makes them harder to maintain, as you need to glob over various
      dynamic width fields.
      
      Let's remove these low-level details from many of the wireprotov2
      tests.
      
      Differential Revision: https://phab.mercurial-scm.org/D4861
      a732d702
    • Gregory Szorc's avatar
      repository: define and use revision flag constants · 8e398628
      Gregory Szorc authored
      Revlogs have a per-revision 2 byte field holding integer flags that
      define how revision data should be interpreted. For historical reasons,
      these integer values are sent verbatim on the wire protocol as part of
      changegroup data.
      
      From a semantic standpoint, the flags that go out over the wire are
      different from the flags stored internally by revlogs. Failure to
      establish this semantic distinction creates unwanted strong coupling
      between revlog's internals and the wire protocol.
      
      This commit establishes new constants on the repository module that
      define the revision flags used by the wire protocol (and by some
      internal storage APIs, sadly). The changegroups internals documentation
      has been updated to document them explicitly. Various references
      throughout the repo now use the repository constants instead of the
      revlog constants. This is done to make it clear that we're operating
      on generic revision data and this isn't tied to revlogs.
      
      Differential Revision: https://phab.mercurial-scm.org/D4860
      8e398628
    • Boris Feld's avatar
      context: reverse conditional branch order in introrev · 50700a02
      Boris Feld authored
      Positive logic will be simpler to follow. It will help to clarify coming
      refactoring.
      50700a02
  15. Oct 04, 2018
    • Boris Feld's avatar
      context: drop a redundant fast path in introrev · 6ed53b19
      Boris Feld authored
      Now that _adjustlinkrev fast path this case itself, we no longer need an extra
      conditional.
      
      A nice side effect is that we are no longer calling `self.rev()`. In case
      where `_descendantrev` is set, calling `self.rev` will trigger a potentially
      expensive `_adjustlinkrev` call. So blindly calling `self.rev()` to avoid
      another `_adjustlinkrev` call can be counterproductive.
      
      Note that `_descendantrev` is currently never taken into account in `introrev`
      so far which is wrong. We'll fix that in changeset later in this series.
      6ed53b19
Loading