Skip to content
Snippets Groups Projects
  1. Aug 21, 2018
  2. Sep 14, 2018
    • Kyle Lippincott's avatar
      narrow: intersect provided matcher with narrowmatcher in `hg diff` · ee7ee0c5
      Kyle Lippincott authored
      This provides significant speedups when running diff, and no change in behavior
      that I'm aware of (or that the tests found). I tested with a repo that I started
      using narrow in after it was created and attempted to run `hg diff -c .` and
      similar commands in it on a commit that had files not in the narrowspec.
      
      Timing numbers below, using a similar setup as my previous commits.
      before=9db856446298, m-u is mozilla-unified at eb39298e432d (flatmanifest) and
      0553b7f29eaf (treemanifest). l-d-r is a repo simulating a situation I've
      encountered where there's one directory with 30k+ subdirectories. N means
      narrow, T means treemanifest. The narrowspec is pretty small when in use, and
      importantly the narrowspec is applied *after* doing the initial checkout
      (without narrowing), so all of these files exist in the filesystem, which is not
      normally the case if someone has been using narrow for the entire life of the
      clone.
      
      Anything less than a 5% difference in performance is most likely noise.
      
          diff --git:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 1.292 s +-  0.009 s    | 1.295 s +-  0.010 s   | 100.2%
          m-u   |   | x | 1.296 s +-  0.042 s    | 1.299 s +-  0.026 s   | 100.2%
          m-u   | x |   | 1.292 s +-  0.010 s    | 1.297 s +-  0.021 s   | 100.4%
          m-u   | x | x | 84.2 ms +-   1.2 ms    | 83.6 ms +-   0.2 ms   |  99.3%
          l-d-r |   |   | 188.7 ms +-   2.7 ms   | 188.8 ms +-   2.0 ms  | 100.1%
          l-d-r |   | x | 189.9 ms +-   1.5 ms   | 189.4 ms +-   1.2 ms  |  99.7%
          l-d-r | x |   | 97.1 ms +-   1.0 ms    | 87.1 ms +-   1.0 ms   |  89.7% <--
          l-d-r | x | x | 96.9 ms +-   0.8 ms    | 87.2 ms +-   0.7 ms   |  90.0% <--
      
          diff -c . --git:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 231.6 ms +-   3.1 ms   | 228.9 ms +-   1.6 ms  |  98.8%
          m-u   |   | x | 150.5 ms +-   1.7 ms   | 150.7 ms +-   1.4 ms  | 100.1%
          m-u   | x |   | 233.7 ms +-   2.4 ms   | 232.2 ms +-   1.9 ms  |  99.4%
          m-u   | x | x | 126.1 ms +-   1.2 ms   | 126.8 ms +-   1.2 ms  | 100.6%
          l-d-r |   |   | 82.1 ms +-   2.0 ms    | 81.8 ms +-   1.4 ms   |  99.6%
          l-d-r |   | x | 3.732 s +-  0.020 s    | 3.746 s +-  0.027 s   | 100.4%
          l-d-r | x |   | 83.1 ms +-   0.8 ms    | 107.6 ms +-   2.4 ms  | 129.5% <--
          l-d-r | x | x | 758.2 ms +-  38.8 ms   | 188.5 ms +-   1.8 ms  |  24.9% <--
      
          rebase -r . --keep -d .^^:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 5.532 s +-  0.087 s    | 5.496 s +-  0.016 s   |  99.3%
          m-u   |   | x | 5.554 s +-  0.061 s    | 5.532 s +-  0.013 s   |  99.6%
          m-u   | x |   | 5.602 s +-  0.134 s    | 5.508 s +-  0.035 s   |  98.3%
          m-u   | x | x | 582.2 ms +-  15.2 ms   | 572.9 ms +-  12.0 ms  |  98.4%
          l-d-r |   |   | 629.5 ms +-  12.3 ms   | 622.5 ms +-   7.3 ms  |  98.9%
          l-d-r |   | x | 6.173 s +-  0.062 s    | 6.185 s +-  0.076 s   | 100.2%
          l-d-r | x |   | 274.5 ms +-  10.0 ms   | 272.1 ms +-   6.2 ms  |  99.1%
          l-d-r | x | x | 4.835 s +-  0.056 s    | 4.826 s +-  0.034 s   |  99.8%
      
          status --change . --copies:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 214.4 ms +-   1.4 ms   | 212.2 ms +-   1.7 ms  |  99.0%
          m-u   |   | x | 130.9 ms +-   1.2 ms   | 131.7 ms +-   1.1 ms  | 100.6%
          m-u   | x |   | 215.0 ms +-   2.1 ms   | 214.9 ms +-   2.7 ms  | 100.0%
          m-u   | x | x | 109.5 ms +-   2.3 ms   | 107.8 ms +-   0.9 ms  |  98.4%
          l-d-r |   |   | 79.6 ms +-   0.9 ms    | 79.8 ms +-   1.6 ms   | 100.3%
          l-d-r |   | x | 3.799 s +-  0.037 s    | 3.928 s +-  0.021 s   | 103.4% <--?
          l-d-r | x |   | 82.7 ms +-   0.7 ms    | 83.2 ms +-   1.0 ms   | 100.6%
          l-d-r | x | x | 746.8 ms +-   6.1 ms   | 739.0 ms +-   4.2 ms  |  99.0%
      
          status --copies:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 1.884 s +-  0.012 s    | 1.885 s +-  0.013 s   | 100.1%
          m-u   |   | x | 1.897 s +-  0.027 s    | 1.909 s +-  0.077 s   | 100.6%
          m-u   | x |   | 1.886 s +-  0.021 s    | 1.891 s +-  0.030 s   | 100.3%
          m-u   | x | x | 92.0 ms +-   0.7 ms    | 92.4 ms +-   0.4 ms   | 100.4%
          l-d-r |   |   | 570.3 ms +-  18.7 ms   | 552.2 ms +-   4.5 ms  |  96.8%
          l-d-r |   | x | 568.9 ms +-  16.1 ms   | 567.2 ms +-  11.9 ms  |  99.7%
          l-d-r | x |   | 171.1 ms +-   2.5 ms   | 170.4 ms +-   1.2 ms  |  99.6%
          l-d-r | x | x | 171.6 ms +-   3.4 ms   | 171.5 ms +-   1.7 ms  |  99.9%
      
          update $rev^; ~/src/hg/hg{hg}/hg update $rev:
          repo  | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before
          ------+---+---+------------------------+-----------------------+------------
          m-u   |   |   | 3.107 s +-  0.017 s    | 3.116 s +-  0.012 s   | 100.3%
          m-u   |   | x | 2.943 s +-  0.010 s    | 2.945 s +-  0.019 s   | 100.1%
          m-u   | x |   | 3.116 s +-  0.033 s    | 3.118 s +-  0.027 s   | 100.1%
          m-u   | x | x | 318.5 ms +-   2.7 ms   | 320.8 ms +-   4.8 ms  | 100.7%
          l-d-r |   |   | 428.9 ms +-   4.4 ms   | 429.5 ms +-   4.0 ms  | 100.1%
          l-d-r |   | x | 9.593 s +-  0.081 s    | 9.869 s +-  0.043 s   | 102.9%
          l-d-r | x |   | 253.2 ms +-   3.6 ms   | 254.0 ms +-   2.8 ms  | 100.3%
          l-d-r | x | x | 1.613 s +-  0.009 s    | 1.630 s +-  0.017 s   | 101.1%
      
      Differential Revision: https://phab.mercurial-scm.org/D4587
      ee7ee0c5
  3. Sep 01, 2018
  4. Jun 07, 2018
    • Yuya Nishihara's avatar
      formatter: populate ctx from repo and node value · 34ecc0a0
      Yuya Nishihara authored
      This will basically replace the fm.contexthint() API. I originally thought
      this would be too complicated, and I wrote 8399438bc7ef "formatter: provide
      hint of context keys required by template" because of that. However, I had
      to add a similar mechanism for fctx templates, and the overall machinery
      became way simpler than my original patch.
      
      The test output slightly changed as {author} is no longer available in
      the {manifest} context, which isn't the point this test targeted on.
      34ecc0a0
  5. Sep 14, 2018
  6. Sep 13, 2018
  7. Sep 14, 2018
  8. Sep 13, 2018
    • Augie Fackler's avatar
      py3: whitelist another passing test · 053aa673
      Augie Fackler authored
      Differential Revision: https://phab.mercurial-scm.org/D4562
      053aa673
    • Matt Harbison's avatar
      py3: prevent the win32 ctype _fields_ from being transformed to bytes · f3900f4c
      Matt Harbison authored
      Otherwise, any hg invocation dies with
      
          TypeError: '_fields_' must be a sequence of (name, C type) pairs
      
      # skip-blame just a r prefix
      f3900f4c
    • Matt Harbison's avatar
      cext: fix warnings when building for py3 on Windows · 11ed2ead
      Matt Harbison authored
      MSVC++ 14 now has standard int types that don't need to be redefined (I didn't
      go back to see when they came along since the build system wants either 2008 or
      2015), but doesn't have ssize_t.  The FILE pointer in posixfile is only used on
      python2.
      11ed2ead
    • Matt Harbison's avatar
      cext: stop preprocessing a partial function call · c06c585f
      Matt Harbison authored
      MSVC++ 14 yelled:
      
          mercurial/cext/revlog.c(1913): fatal error C1057: unexpected end of file in
                                         macro expansion
      
      At this point, the C extensions build (with warnings), and it dies in win32.py
      because the `_fields_` strings in the ctypes classes are being converted to
      bytes by the source translator.
      c06c585f
    • Matt Harbison's avatar
      py3: add b'' to some setup.py strings for Windows · ec68135a
      Matt Harbison authored
      These were things found trying to do `make PYTHON="py -3" local`.  The following
      is dumped out, before dying while compiling the C extensions:
      
      C:\Program Files\Python37\lib\site-packages\setuptools\dist.py:406: UserWarning:
        The version specified (b'4.7.1') is an invalid version, this may not work as
        expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440
        for more details.
        "details." % self.metadata.version
      running build_py
      byte-compiling .\mercurial\thirdparty\concurrent\futures\_base.py to _base.cpython-37.pyc
        File "mercurial\thirdparty\concurrent\futures\_base.py", line 416
          raise exception_type, self._exception, self._traceback
                              ^
      SyntaxError: invalid syntax
      
      # skip-blame since these are just converting to bytes literals
      ec68135a
    • Augie Fackler's avatar
    • Gregory Szorc's avatar
      hg: don't reuse repo instance after unshare() · c5e6c1ba
      Gregory Szorc authored
      Unsharing a repository is a pretty invasive procedure and fundamentally
      changes the behavior of the repository.
      
      Currently, hg.unshare() calls into localrepository.__init__ to
      re-initialize the repository instance. This is a bit hacky. And
      future commits that refactor how localrepository instances are
      constructed will make this difficult to support.
      
      This commit changes unshare() so it constructs a new repo instance
      once the unshare I/O has completed. It then poisons the old repo
      instance so any further use will result in error.
      
      Surprisingly, nothing in core appears to access a repo instance
      after it has been unshared!
      
      .. api::
      
         ``hg.unshare()`` now poisons the repo instance so it can't be used.
      
         It also returns a new repo instance suitable for interacting with
         the unshared repository.
      
      Differential Revision: https://phab.mercurial-scm.org/D4557
      c5e6c1ba
  9. Sep 12, 2018
    • Gregory Szorc's avatar
      unionrepo: dynamically create repository type from base repository · 23f2299e
      Gregory Szorc authored
      This is basically the same thing we just did for bundlerepo except
      for union repositories.
      
      .. api::
      
         ``unionrepo.unionrepository()`` is no longer usable on its own.
      
         To instantiate an instance, call ``unionrepo.instance()`` or
         ``unionrepo.makeunionrepository()``.
      
      Differential Revision: https://phab.mercurial-scm.org/D4556
      23f2299e
    • Gregory Szorc's avatar
      bundlerepo: dynamically create repository type from base repository · 335ae4d0
      Gregory Szorc authored
      Previously, bundlerepository inherited from localrepo.localrepository.
      You simply instantiated a bundlerepository and its __init__ called
      localrepo.localrepository.__init__. Things were simple.
      
      Unfortunately, this strategy is limiting because it assumes that
      the base repository is a localrepository instance. And it assumes
      various properties of localrepository, such as the arguments its
      __init__ takes. And it prevents us from changing behavior of
      localrepository.__init__ without also having to change derived classes.
      
      Previous and ongoing work to abstract storage revealed these
      limitations.
      
      This commit changes the initialization strategy of bundle repositories
      to dynamically create a type to represent the repository. Instead of
      a static type, we instantiate a new local repo instance via
      localrepo.instance(). We then combine its __class__ with
      bundlerepository to produce a new type. This ensures that no matter
      how localrepo.instance() decides to create a repository object, we
      can derive a bundle repo object from it. i.e. localrepo.instance()
      could return a type that isn't a localrepository and it would "just
      work."
      
      Well, it would "just work" if bundlerepository's custom implementations
      only accessed attributes in the documented repository interface. I'm
      pretty sure it violates the interface contract in a handful of
      places. But we can worry about that another day. This change gets us
      closer to doing more clever things around instantiating repository
      instances without having to worry about teaching bundlerepository about
      them.
      
      .. api::
      
         ``bundlerepo.bundlerepository`` is no longer usable on its own.
      
         The class is combined with the class of the base repository it is
         associated with at run-time.
      
         New bundlerepository instances can be obtained by calling
         ``bundlerepo.instance()`` or ``bundlerepo.makebundlerepository()``.
      
      Differential Revision: https://phab.mercurial-scm.org/D4555
      335ae4d0
    • Gregory Szorc's avatar
      bundlerepo: factor out code for instantiating a bundle repository · a8d2faec
      Gregory Szorc authored
      This code will soon become a bit more complicated. So extract to its
      own function.
      
      And change both instantiators of bundlerepository to use it.
      
      Differential Revision: https://phab.mercurial-scm.org/D4554
      a8d2faec
    • Gregory Szorc's avatar
      bundlerepo: pass create=True · 2d2bbf3f
      Gregory Szorc authored
      I don't want to know how this came to be. Maybe a holdover from the
      days before Python had a bool type?
      
      Differential Revision: https://phab.mercurial-scm.org/D4553
      2d2bbf3f
    • Gregory Szorc's avatar
      shelve: use bundlerepo.instance() to construct a repo object · 84d6e9a2
      Gregory Szorc authored
      The instance() functions are preferred over cls.__init__ for
      creating repo instances. It doesn't really matter now. But future
      commits will refactor the bundlerepository class in ways that will
      cause the old way to break.
      
      Differential Revision: https://phab.mercurial-scm.org/D4552
      84d6e9a2
  10. Jul 29, 2018
  11. Sep 13, 2018
  12. Sep 10, 2018
    • Boris Feld's avatar
      revlog: reuse cached delta for identical base revision (issue5975) · a911932d
      Boris Feld authored
      Since 8f83a953dddf, we skip over empty deltas when choosing a delta base. Such
      delta happens when two distinct revisions have the same content.
      
      The remote might be sending a delta against such revision within the bundle.
      In that case, the delta base is no longer considered, but the cached one could
      still, be used with the equivalent revision.
      
      Not reusing the delta from the bundle can have a significant performance
      impact, so we now make sure with doing so when possible.
      a911932d
    • Boris Feld's avatar
      snapshot: fix line order when skipping over empty deltas · bdb41eaa
      Boris Feld authored
      The code movement in 37957e07138c introduced an error.
      
      Since 8f83a953dddf, we discarded some revisions because they are identical to
      their delta base (and use that delta base instead). That logic is good,
      however, in 37957e07138c we mixed up the order of two line, adding the "new"
      revision to the set of already tested one, instead of the discarded one. So in
      practice, we were never investigating any revisions in a chain starting with
      an empty delta. Creating significantly worst delta chain (eg: Mercurial's
      manifest move goes from about 60MB up to about 80MB).
      bdb41eaa
  13. Sep 13, 2018
  14. Aug 21, 2018
  15. Sep 12, 2018
  16. Sep 13, 2018
Loading