- Sep 12, 2018
-
-
Gregory Szorc authored
This commit introduces the "changesetdata" wire protocol command. The role of the command is to expose data associated with changelog revisions, including the raw revision data itself. This command is the first piece of a new clone/pull strategy that is built on top of domain-specific commands for data retrieval. Instead of a monolithic "getbundle" command that transfers all of the things, we'll be introducing commands for fetching specific pieces of data. Since the changeset is the fundamental unit from which we derive pointers to other data (manifests, file nodes, etc), it makes sense to start reimplementing pull with this data. The command accepts as arguments a set of root and head revisions defining the changesets that should be fetched as well as an explicit list of nodes. By default, the command returns only the node values: the client must explicitly request additional fields be added to the response. Current supported fields are the list of parent nodes and the revision fulltext. My plan is to eventually add support for transferring other data associated with changesets, including phases, bookmarks, obsolescence markers, etc. Since the response format is CBOR, we'll be able to add this data into the response object relatively easily (it should be as simple as adding a key in a map). The documentation captures a number of TODO items. Some of these may require BC breaking changes. That's fine: wire protocol v2 is still highly experimental. Differential Revision: https://phab.mercurial-scm.org/D4481
-
Gregory Szorc authored
Wire protocol version 2 will take a substantially different approach to exchange than version 1 (at least as far as pulling is concerned). This commit establishes a new exchangev2 module for holding code related to exchange using wire protocol v2. I could have added things to the existing exchange module. But it is already quite big. And doing things inline isn't in question because the existing code is already littered with conditional code for various states of support for the existing wire protocol as it evolved over 10+ years. A new module gives us a chance to make a clean break. This approach does mean we'll end up writing some duplicate code. And there's a significant chance we'll miss functionality as code is ported. The plan is to eventually add #testcase's to existing tests so the new wire protocol is tested side-by-side with the existing one. This will hopefully tease out any features that weren't ported properly. But before we get there, we need to build up support for the new exchange methods. Our journey towards implementing a new exchange begins with pulling. And pulling begins with discovery. The discovery code added to exchangev2 is heavily drawn from the following functions: * exchange._pulldiscoverychangegroup * discovery.findcommonincoming For now, we build on top of existing discovery mechanisms. The new wire protocol should be capable of doing things more efficiently. But I'd rather defer on this problem. To foster the transition, we invent a fake capability on the HTTPv2 peer and have the main pull code in exchange.py call into exchangev2 when the new wire protocol is being used. Differential Revision: https://phab.mercurial-scm.org/D4480
-
- Aug 21, 2018
-
-
Gregory Szorc authored
This will help code using peers to sniff out exactly what servers support. Differential Revision: https://phab.mercurial-scm.org/D4436
-
- Sep 14, 2018
-
-
Kyle Lippincott authored
This provides significant speedups when running diff, and no change in behavior that I'm aware of (or that the tests found). I tested with a repo that I started using narrow in after it was created and attempted to run `hg diff -c .` and similar commands in it on a commit that had files not in the narrowspec. Timing numbers below, using a similar setup as my previous commits. before=9db856446298, m-u is mozilla-unified at eb39298e432d (flatmanifest) and 0553b7f29eaf (treemanifest). l-d-r is a repo simulating a situation I've encountered where there's one directory with 30k+ subdirectories. N means narrow, T means treemanifest. The narrowspec is pretty small when in use, and importantly the narrowspec is applied *after* doing the initial checkout (without narrowing), so all of these files exist in the filesystem, which is not normally the case if someone has been using narrow for the entire life of the clone. Anything less than a 5% difference in performance is most likely noise. diff --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.292 s +- 0.009 s | 1.295 s +- 0.010 s | 100.2% m-u | | x | 1.296 s +- 0.042 s | 1.299 s +- 0.026 s | 100.2% m-u | x | | 1.292 s +- 0.010 s | 1.297 s +- 0.021 s | 100.4% m-u | x | x | 84.2 ms +- 1.2 ms | 83.6 ms +- 0.2 ms | 99.3% l-d-r | | | 188.7 ms +- 2.7 ms | 188.8 ms +- 2.0 ms | 100.1% l-d-r | | x | 189.9 ms +- 1.5 ms | 189.4 ms +- 1.2 ms | 99.7% l-d-r | x | | 97.1 ms +- 1.0 ms | 87.1 ms +- 1.0 ms | 89.7% <-- l-d-r | x | x | 96.9 ms +- 0.8 ms | 87.2 ms +- 0.7 ms | 90.0% <-- diff -c . --git: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 231.6 ms +- 3.1 ms | 228.9 ms +- 1.6 ms | 98.8% m-u | | x | 150.5 ms +- 1.7 ms | 150.7 ms +- 1.4 ms | 100.1% m-u | x | | 233.7 ms +- 2.4 ms | 232.2 ms +- 1.9 ms | 99.4% m-u | x | x | 126.1 ms +- 1.2 ms | 126.8 ms +- 1.2 ms | 100.6% l-d-r | | | 82.1 ms +- 2.0 ms | 81.8 ms +- 1.4 ms | 99.6% l-d-r | | x | 3.732 s +- 0.020 s | 3.746 s +- 0.027 s | 100.4% l-d-r | x | | 83.1 ms +- 0.8 ms | 107.6 ms +- 2.4 ms | 129.5% <-- l-d-r | x | x | 758.2 ms +- 38.8 ms | 188.5 ms +- 1.8 ms | 24.9% <-- rebase -r . --keep -d .^^: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 5.532 s +- 0.087 s | 5.496 s +- 0.016 s | 99.3% m-u | | x | 5.554 s +- 0.061 s | 5.532 s +- 0.013 s | 99.6% m-u | x | | 5.602 s +- 0.134 s | 5.508 s +- 0.035 s | 98.3% m-u | x | x | 582.2 ms +- 15.2 ms | 572.9 ms +- 12.0 ms | 98.4% l-d-r | | | 629.5 ms +- 12.3 ms | 622.5 ms +- 7.3 ms | 98.9% l-d-r | | x | 6.173 s +- 0.062 s | 6.185 s +- 0.076 s | 100.2% l-d-r | x | | 274.5 ms +- 10.0 ms | 272.1 ms +- 6.2 ms | 99.1% l-d-r | x | x | 4.835 s +- 0.056 s | 4.826 s +- 0.034 s | 99.8% status --change . --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 214.4 ms +- 1.4 ms | 212.2 ms +- 1.7 ms | 99.0% m-u | | x | 130.9 ms +- 1.2 ms | 131.7 ms +- 1.1 ms | 100.6% m-u | x | | 215.0 ms +- 2.1 ms | 214.9 ms +- 2.7 ms | 100.0% m-u | x | x | 109.5 ms +- 2.3 ms | 107.8 ms +- 0.9 ms | 98.4% l-d-r | | | 79.6 ms +- 0.9 ms | 79.8 ms +- 1.6 ms | 100.3% l-d-r | | x | 3.799 s +- 0.037 s | 3.928 s +- 0.021 s | 103.4% <--? l-d-r | x | | 82.7 ms +- 0.7 ms | 83.2 ms +- 1.0 ms | 100.6% l-d-r | x | x | 746.8 ms +- 6.1 ms | 739.0 ms +- 4.2 ms | 99.0% status --copies: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 1.884 s +- 0.012 s | 1.885 s +- 0.013 s | 100.1% m-u | | x | 1.897 s +- 0.027 s | 1.909 s +- 0.077 s | 100.6% m-u | x | | 1.886 s +- 0.021 s | 1.891 s +- 0.030 s | 100.3% m-u | x | x | 92.0 ms +- 0.7 ms | 92.4 ms +- 0.4 ms | 100.4% l-d-r | | | 570.3 ms +- 18.7 ms | 552.2 ms +- 4.5 ms | 96.8% l-d-r | | x | 568.9 ms +- 16.1 ms | 567.2 ms +- 11.9 ms | 99.7% l-d-r | x | | 171.1 ms +- 2.5 ms | 170.4 ms +- 1.2 ms | 99.6% l-d-r | x | x | 171.6 ms +- 3.4 ms | 171.5 ms +- 1.7 ms | 99.9% update $rev^; ~/src/hg/hg{hg}/hg update $rev: repo | N | T | before (mean +- stdev) | after (mean +- stdev) | % of before ------+---+---+------------------------+-----------------------+------------ m-u | | | 3.107 s +- 0.017 s | 3.116 s +- 0.012 s | 100.3% m-u | | x | 2.943 s +- 0.010 s | 2.945 s +- 0.019 s | 100.1% m-u | x | | 3.116 s +- 0.033 s | 3.118 s +- 0.027 s | 100.1% m-u | x | x | 318.5 ms +- 2.7 ms | 320.8 ms +- 4.8 ms | 100.7% l-d-r | | | 428.9 ms +- 4.4 ms | 429.5 ms +- 4.0 ms | 100.1% l-d-r | | x | 9.593 s +- 0.081 s | 9.869 s +- 0.043 s | 102.9% l-d-r | x | | 253.2 ms +- 3.6 ms | 254.0 ms +- 2.8 ms | 100.3% l-d-r | x | x | 1.613 s +- 0.009 s | 1.630 s +- 0.017 s | 101.1% Differential Revision: https://phab.mercurial-scm.org/D4587
-
- Sep 01, 2018
-
-
Yuya Nishihara authored
This is a part of the name unification. {parents} is a list of nodes in "hg log -Tjson" output. Since {rev} can be computed from (repo, node) pair, we no longer need to put it to provide {rev} to user templates. https://www.mercurial-scm.org/wiki/GenericTemplatingPlan#Dictionary
-
Yuya Nishihara authored
This fixes the length of {id} in JSON and template outputs.
-
Yuya Nishihara authored
And pass in repo instead to resolve ctx from (repo, node) pair.
-
- Jun 07, 2018
-
-
Yuya Nishihara authored
This will basically replace the fm.contexthint() API. I originally thought this would be too complicated, and I wrote 8399438bc7ef "formatter: provide hint of context keys required by template" because of that. However, I had to add a similar mechanism for fctx templates, and the overall machinery became way simpler than my original patch. The test output slightly changed as {author} is no longer available in the {manifest} context, which isn't the point this test targeted on.
-
- Sep 14, 2018
-
-
Augie Fackler authored
-
Pulkit Goyal authored
# skip-blame because just b'' prefixes I believe this should fix some tests. Differential Revision: https://phab.mercurial-scm.org/D4594
-
Pulkit Goyal authored
Differential Revision: https://phab.mercurial-scm.org/D4593
-
Pulkit Goyal authored
Differential Revision: https://phab.mercurial-scm.org/D4592
-
Pulkit Goyal authored
Differential Revision: https://phab.mercurial-scm.org/D4591
-
Pulkit Goyal authored
# skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D4590
-
Pulkit Goyal authored
I still don't know why python-dev thought it was a nice idea to do this. Differential Revision: https://phab.mercurial-scm.org/D4589
-
- Sep 13, 2018
-
-
Valentin Gatien-Baron authored
Before this change, trying to censor some random revision uses an ever increasing amount of memory (I stopped at 20GB, but it was by no means finished), presumably because these contexts have a lot of information that is kept alive. After this change, the memory usage plateaus quickly. Differential Revision: https://phab.mercurial-scm.org/D4582
-
- Sep 14, 2018
-
-
Yuya Nishihara authored
-
Yuya Nishihara authored
The vendored future can't live on Python 3.
-
- Sep 13, 2018
-
-
Augie Fackler authored
Differential Revision: https://phab.mercurial-scm.org/D4562
-
Matt Harbison authored
Otherwise, any hg invocation dies with TypeError: '_fields_' must be a sequence of (name, C type) pairs # skip-blame just a r prefix
-
Matt Harbison authored
MSVC++ 14 now has standard int types that don't need to be redefined (I didn't go back to see when they came along since the build system wants either 2008 or 2015), but doesn't have ssize_t. The FILE pointer in posixfile is only used on python2.
-
Matt Harbison authored
MSVC++ 14 yelled: mercurial/cext/revlog.c(1913): fatal error C1057: unexpected end of file in macro expansion At this point, the C extensions build (with warnings), and it dies in win32.py because the `_fields_` strings in the ctypes classes are being converted to bytes by the source translator.
-
Matt Harbison authored
These were things found trying to do `make PYTHON="py -3" local`. The following is dumped out, before dying while compiling the C extensions: C:\Program Files\Python37\lib\site-packages\setuptools\dist.py:406: UserWarning: The version specified (b'4.7.1') is an invalid version, this may not work as expected with newer versions of setuptools, pip, and PyPI. Please see PEP 440 for more details. "details." % self.metadata.version running build_py byte-compiling .\mercurial\thirdparty\concurrent\futures\_base.py to _base.cpython-37.pyc File "mercurial\thirdparty\concurrent\futures\_base.py", line 416 raise exception_type, self._exception, self._traceback ^ SyntaxError: invalid syntax # skip-blame since these are just converting to bytes literals
-
Augie Fackler authored
Differential Revision: https://phab.mercurial-scm.org/D4584
-
Gregory Szorc authored
Unsharing a repository is a pretty invasive procedure and fundamentally changes the behavior of the repository. Currently, hg.unshare() calls into localrepository.__init__ to re-initialize the repository instance. This is a bit hacky. And future commits that refactor how localrepository instances are constructed will make this difficult to support. This commit changes unshare() so it constructs a new repo instance once the unshare I/O has completed. It then poisons the old repo instance so any further use will result in error. Surprisingly, nothing in core appears to access a repo instance after it has been unshared! .. api:: ``hg.unshare()`` now poisons the repo instance so it can't be used. It also returns a new repo instance suitable for interacting with the unshared repository. Differential Revision: https://phab.mercurial-scm.org/D4557
-
- Sep 12, 2018
-
-
Gregory Szorc authored
This is basically the same thing we just did for bundlerepo except for union repositories. .. api:: ``unionrepo.unionrepository()`` is no longer usable on its own. To instantiate an instance, call ``unionrepo.instance()`` or ``unionrepo.makeunionrepository()``. Differential Revision: https://phab.mercurial-scm.org/D4556
-
Gregory Szorc authored
Previously, bundlerepository inherited from localrepo.localrepository. You simply instantiated a bundlerepository and its __init__ called localrepo.localrepository.__init__. Things were simple. Unfortunately, this strategy is limiting because it assumes that the base repository is a localrepository instance. And it assumes various properties of localrepository, such as the arguments its __init__ takes. And it prevents us from changing behavior of localrepository.__init__ without also having to change derived classes. Previous and ongoing work to abstract storage revealed these limitations. This commit changes the initialization strategy of bundle repositories to dynamically create a type to represent the repository. Instead of a static type, we instantiate a new local repo instance via localrepo.instance(). We then combine its __class__ with bundlerepository to produce a new type. This ensures that no matter how localrepo.instance() decides to create a repository object, we can derive a bundle repo object from it. i.e. localrepo.instance() could return a type that isn't a localrepository and it would "just work." Well, it would "just work" if bundlerepository's custom implementations only accessed attributes in the documented repository interface. I'm pretty sure it violates the interface contract in a handful of places. But we can worry about that another day. This change gets us closer to doing more clever things around instantiating repository instances without having to worry about teaching bundlerepository about them. .. api:: ``bundlerepo.bundlerepository`` is no longer usable on its own. The class is combined with the class of the base repository it is associated with at run-time. New bundlerepository instances can be obtained by calling ``bundlerepo.instance()`` or ``bundlerepo.makebundlerepository()``. Differential Revision: https://phab.mercurial-scm.org/D4555
-
Gregory Szorc authored
This code will soon become a bit more complicated. So extract to its own function. And change both instantiators of bundlerepository to use it. Differential Revision: https://phab.mercurial-scm.org/D4554
-
Gregory Szorc authored
I don't want to know how this came to be. Maybe a holdover from the days before Python had a bool type? Differential Revision: https://phab.mercurial-scm.org/D4553
-
Gregory Szorc authored
The instance() functions are preferred over cls.__init__ for creating repo instances. It doesn't really matter now. But future commits will refactor the bundlerepository class in ways that will cause the old way to break. Differential Revision: https://phab.mercurial-scm.org/D4552
-
- Jul 29, 2018
-
-
Yuya Nishihara authored
This is another example of fctx-based keywords. I think this is somewhat useful in log templates.
-
Yuya Nishihara authored
They will be necessary to provide {status} of files.
-
Yuya Nishihara authored
There's no point to drop tail elements, which are mostly empty lists.
-
Yuya Nishihara authored
-
- Sep 13, 2018
-
-
Yuya Nishihara authored
msg.decode('utf8') may fail if msg isn't an ASCII string, and that's possible as we sometimes embed a filename in the error message for example.
-
- Sep 10, 2018
-
-
Boris Feld authored
Since 8f83a953dddf, we skip over empty deltas when choosing a delta base. Such delta happens when two distinct revisions have the same content. The remote might be sending a delta against such revision within the bundle. In that case, the delta base is no longer considered, but the cached one could still, be used with the equivalent revision. Not reusing the delta from the bundle can have a significant performance impact, so we now make sure with doing so when possible.
-
Boris Feld authored
The code movement in 37957e07138c introduced an error. Since 8f83a953dddf, we discarded some revisions because they are identical to their delta base (and use that delta base instead). That logic is good, however, in 37957e07138c we mixed up the order of two line, adding the "new" revision to the set of already tested one, instead of the discarded one. So in practice, we were never investigating any revisions in a chain starting with an empty delta. Creating significantly worst delta chain (eg: Mercurial's manifest move goes from about 60MB up to about 80MB).
-
- Sep 13, 2018
-
-
Matt Harbison authored
The change originated in cb1329738d64. I suspect the problem is with the combination of (re) and the '\' to '/' retry on Windows. I've no idea if py3 on Windows needs the quoting, since it can't even run `hg` with no arguments. (It's dying somewhere on the ctype declarations when win32.py is imported.)
-
- Aug 21, 2018
-
-
Augie Fackler authored
This should help us have a better idea of what "interpreter startup costs" look like. This does omit the HGUNICODEPEDANTRY block and the LIBDIR dancing to set up sys.path, but the former is usually off and the latter is unavoidable and should be very fast. If we get worried about those cases we can consider open-coding the tracing logic here. Differential Revision: https://phab.mercurial-scm.org/D4346
-
- Sep 12, 2018
-
-
Martin von Zweigbergk authored
It looks like this was lost in 7ce9dea3a14a (localrepo: move repo creation logic out of localrepository.__init__ (API), 2018-09-11). I don't know when it makes a difference (maybe on Windows, since urllocalpath() mentions something about drive letters). Differential Revision: https://phab.mercurial-scm.org/D4550
-