- Nov 12, 2018
-
-
Boris Feld authored
Now that sparse-revlog is enabled by default, we no longer needs it. Differential Revision: https://phab.mercurial-scm.org/D5346
-
Boris Feld authored
We are about to enable sparse-revlog globally. To help with reviewing the tests change, we isolate them in individual changesets. Differential Revision: https://phab.mercurial-scm.org/D5335
-
- Oct 13, 2018
-
-
Matt Harbison authored
These tests are broken under py3 on Windows to the point where the `cd ..` was actually escaping into the system wide $TEMP. The subsequent `hg init` created a repo there, and then added a local extension to the hgrc. This breaks every single subsequent test when it tries to `hg init` in its $TESTTMP, and can't load the localwrite.py extension. And since I botched this the first time and replaced the wrong `cd ..`, this just replaces all of them. I've noticed test garbage in $TEMP recently, and maybe this will help. Perhaps `hg init` shouldn't load the config for the local repo, but this is an easy enough workaround for now.
-
- Oct 09, 2018
-
-
Gregory Szorc authored
Right now, we have a handful of arguments for specifying the revisions whose data should be returned. Defining how all these arguments interact when various combinations are present is difficult. This commit establishes a new, generic mechanism for specifying revisions. Instead of a hodgepodge of arguments defining things, we have a list of dicts that specify revision selectors. The final set of revisions is a union of all these selectors. We implement support for specifying revisions based on: * An explicit list of changeset revisions * An explicit list of changeset revisions plus ancestry depth * A DAG range between changeset roots and heads If you squint hard enough, this problem has already been solved by revsets. But I'm reluctant to expose revsets to the wire protocol because that would require servers to implement a revset parser. Plus there are security and performance implications: the set of revision selectors needs to be narrowly and specifically tailored for what is appropriate to be executing on a server. Perhaps there would be a way for us to express the "parse tree" of a revset query, for example. I'm not sure. We can explore this space another time. For now, the new mechanism should bring sufficient flexibility while remaining relatively simple. The selector "types" are prefixed with "changeset" because I plan to add manifest and file-flavored selectors as well. This will enable us to e.g. select file revisions based on a range of changeset revisions. Differential Revision: https://phab.mercurial-scm.org/D4979
-
- Sep 20, 2018
-
-
Gregory Szorc authored
The *data wire protocol commands emit a series of CBOR values. Because revision/delta data may be large, their data is emitted outside the map as a top-level bytestring value. Before this commit, we'd emit a single optional bytestring value after the revision descriptor map. This got the job done. But it was limiting in that we could only send a single field. And, it required the consumer to know that the presence of a key in the map implied the existence of a following bytestring value. This commit changes the encoding strategy so top-level bytestring values in the stream are explicitly denoted in a "fieldsfollowing" key. This key contains an array defining what fields that follow and the expected size of each field. By defining things this way, we can easily send N bytestring values without any ambiguity about their order. In addition, clients only need to know how to parse ``fieldsfollowing`` to know if extra values are present. Because this breaks backwards compatibility, we've bumped the version number of the wire protocol version 2 API endpoint. Differential Revision: https://phab.mercurial-scm.org/D4620
-
- Sep 17, 2018
-
-
Anton Shestakov authored
Support for bundle2 streaming clones has been shipped in Mercurial 4.5 (7eedbd5d4880), but was never activated by default. It's time to have more people use it. The new format allows streaming clones to transport cache (hooray for speed) and phaseroots (fixes phase-related issues). Changes in tests: bundle2 capabilities now have "stream=v2" (plus a '\n' as a separator) and therefore take 14 bytes more: "%0Astream%3Dv2". Tip for tests that have data encoded with CBOR: 0xd3 - 0xc5 = 14. $USUAL_BUNDLE2_CAPS$ replaces $USUAL_BUNDLE2_CAPS_SERVER$, which is the same thing, but without "stream=v2". Since streaming clones now also transfer caches, the reported byte and file counts are higher (e.g. 816 bytes in 9 files instead of 613 bytes in 4 files, a bit of --debug and manual math confirms that the caches take these extra 203 bytes in 5 files). Differential Revision: https://phab.mercurial-scm.org/D4680
-
- Sep 19, 2018
-
-
Matt Harbison authored
Python3 defaults to installing under "Program Files".
-
- Sep 04, 2018
-
-
Meirambek Omyrzak authored
output before: "500 files, 2035 changesets, 2622 total revisions" output after: "checked 2035 changesets with 2622 changes to 500 files" new one was suggested in the comments inside the issue. Differential Revision: https://phab.mercurial-scm.org/D4476
-
- Aug 18, 2018
-
-
Matt Harbison authored
This was exposed by ac0a87160012. The paths for convert and ssh respectively are being turned into: $TESTTMP\bzr+ssh:\foobar@selenic.com\baz\.hg 'C:\\C:\\Users\\Matt\\AppData\\Local\\Temp\\hgtests.8awa9x\\child1\\test-ssh-bundle1.t-sshv2\\nonexistent\\.hg The errno value raised is 123, and the message is The filename, directory name, or volume label syntax is incorrect
-
- May 20, 2018
-
-
Pulkit Goyal authored
# skip-blame because just b'' prefixes Differential Revision: https://phab.mercurial-scm.org/D3630
-
- Apr 06, 2018
-
-
Jörg Sonnenberger authored
Differential Revision: https://phab.mercurial-scm.org/D3171
-
- Mar 24, 2018
-
-
Jörg Sonnenberger authored
For HTTP, this refactors the existing logic, including the parsing of the compression engine capability. For SSH, this adds a ssh-only capability "protocaps" and a command for informing the server on what the client supports. Since SSH is stateful, keep track of the capabilities in the server instance. Differential Revision: https://phab.mercurial-scm.org/D1944
-
- Apr 03, 2018
-
-
Gregory Szorc authored
There are a handful of test failures in the simple store with regards to stream clones. Fixing them will require a lot of conditional output. Disabling the tests wholesale is easier at this juncture. Differential Revision: https://phab.mercurial-scm.org/D3064
-
- Apr 04, 2018
-
-
Gregory Szorc authored
format.usegeneraldelta defaults to true. Differential Revision: https://phab.mercurial-scm.org/D3057
-
- Mar 14, 2018
-
-
Gregory Szorc authored
The unified frame-based protocol will (eventually) support multiple requests per client transmission. This means that the [very hacky] "batch" command has no purpose existing in this protocol. This commit marks the command as applying to v1 transports only. But because SSHv2 == SSHv1 currently, we had to hack it back in for the SSHv2 transport. Bleh. Tests changed because the capabilities string changed. The order of tokens in the string is not important. Differential Revision: https://phab.mercurial-scm.org/D2856
-
- Jan 18, 2018
-
-
Boris Feld authored
The feature is now advertised and use. Updating the branchmap cache can be very expensive (up to minutes on large repository) and fetching revision branch data is about 80% of that. Exchanging the rev branch cache over the wire really help to recover from branch map invalidation. (There is a good chance other in flight chance would conflict on test-http-bad-server.t and other. So here is a small note to help update that test again: capabilities=19bytes, part-107bytes)
-
- Feb 27, 2018
-
-
Gregory Szorc authored
We just marked the changegroupsubset command as only available to version 1 of the wire transports. There is a capability of the same name of the command that indicates if the command is supported. This commit teaches the capabilities code to conditionally emit that capability depending on whether the command is available for the current transport. Most test output is reordering of capabilities. But the limited tests for version 2 of the SSH protocol do show the capability disappearing. Differential Revision: https://phab.mercurial-scm.org/D2486
-
- Feb 13, 2018
-
-
Gregory Szorc authored
Now that the version 2 of the SSH protocol is usable in core, we can start actively testing it more widely outside of low-level protocol tests. We add #testcases variants to a handful of tests so we exercise both version 1 and version 2 of the SSH protocol when testing. This will allow us to more easily find regressions and variances as protocol 2 is developed. It will also make it easier to continue testing with protocol version 1 once version 2 is enabled by default. There are a handful of tests using ssh:// that should also gain test variances. One - test-push-race.t - already has a #testcases. This would require combinatorial cases. I didn't want to go down that rabbit hole, so that test is unchanged. Thinking aloud, there is probably an opportunity to automatically run tests with multiple server/protocol implementations. Ideally any test that performed server interaction would run with all supported server implementations and protocols so we could find variances between servers and protocols. But this has been a long-standing issue with our test harness. I don't think it is an easily solved problem. But it would be nice... Differential Revision: https://phab.mercurial-scm.org/D2206
-
- Jan 22, 2018
-
-
Gregory Szorc authored
Previously, enabling of stream clone over bundle2 was a server-side only change. And clients would only advertise bundle2 support for stream clones if an experimental config option was set. This situation wasn't forward compatible because in the future (when the feature is enabled on servers by default), an old client would send a request to the server but it wouldn't send its own bundle2 capability support for stream parts. Servers would have to infer that clients not sending this capability were old Mercurial clients that only sent the capability if the feature was explicitly enabled. Implicit and inferred behavior makes implementing servers hard. It's much better to be explicit about client features. We should either make the config option for bundle2 stream clones disable the feature client-side as well (so a server doesn't see a request from a client not advertising stream support). Or we should always advertise stream support if a client is willing to accept stream parts. Since I anticipating stabilizing stream clone support in bundle2 quickly, I think it's safe to always advertise client support in the bundle2 capabilities. So this commit changes things to do that. Because capabilities now vary between client and server, we had to create variations of the variable substitutions for those strings. Differential Revision: https://phab.mercurial-scm.org/D1931
-
- Dec 11, 2017
-
-
Matt Harbison authored
# skip-blame because this was mechanically rewritten the following script. I ran it on both *.t and *.py, but none of the *.py changes were proper. All *.t ones appear to be, and they run without addition failures on both Windows and Linux. import argparse import os import re ap = argparse.ArgumentParser() ap.add_argument('path', nargs='+') opts = ap.parse_args() globre = re.compile(r'^(.*) \(glob\)(.*)$') for p in opts.path: tmp = p + '.tmp' with open(p, 'rb') as src, open(tmp, 'wb') as dst: for line in src: m = globre.match(line) if not m or '$LOCALIP' in line or '*' in line: dst.write(line) continue if '?' in line[:-3] or ('?' in line[:-3] and line[-3:] != '(?)'): dst.write(line) continue dst.write(m.group(1) + m.group(2) + '\n') os.unlink(p) os.rename(tmp, p)
-
- Nov 13, 2017
-
-
Boris Feld authored
Before updating the actual bookmark update, we can start with updating the way we check for push race. Checking bookmarks state earlier is useful even if we still use pushkey. Aborting before the changegroup is added can save a lot of time.
-
- Nov 05, 2017
-
-
Boris Feld authored
We mass update the tests now. This will help the next soul touching the http protocol.
-
- Oct 12, 2017
-
-
Denis Laxalde authored
Upon pull or unbundle, we display a message with the range of new revisions fetched. This revision range could readily be used after a pull to look out what's new with 'hg log'. The algorithm takes care of filtering "obsolete" revisions that might be present in transaction's "changes" but should not be displayed to the end user.
-
- Oct 01, 2017
-
-
Gregory Szorc authored
--uncompressed isn't a very good name and its description in the help documentation isn't very useful. We refer to this concept as "stream clones" in a number of places. I think it makes sense to change the user-facing argument to use the mode --stream. So this commit does that. We keep --uncompressed around for backwards compatibility. While we're here, we overhaul the help docs for streaming clones to be somewhat useful. All tests have been updated to reflect the new preferred --stream argument. A test for backwards compatibility of --uncompressed has been added. .. bc:: `hg clone --stream` should now be used instead of --uncompressed. --uncompressed is marked as deprecated and is an alias for --stream. There is no schedule for elimination of --uncompressed. Differential Revision: https://phab.mercurial-scm.org/D864
-
- Sep 24, 2017
-
-
Boris Feld authored
A new bundle2 capability 'phases' has been added. If 'heads' is part of the supported value for 'phases', the server supports reading and sending 'phase- heads' bundle2 part. Server is now able to process a 'phases' boolean parameter to 'getbundle'. If 'True', a 'phase-heads' bundle2 part will be included in the bundle with phase information relevant to the whole pulled set. If this method is available the phases listkey namespace will no longer be listed. Beside the more efficient encoding of the data, this new method will greatly improve the phase exchange efficiency for repositories with non-served changesets (obsolete, secret) since we'll no longer send data about the filtered heads. Add a new 'devel.legacy.exchange' config item to allow fallback to the old 'listkey in bundle2' method. Reminder: the pulled set is not just the changesets bundled by the pull. It also contains changeset selected by the "pull specification" on the client side (eg: everything for bare pull). One of the reason why the 'pulled set' is important is to make sure we can move -common- nodes to public.
-
- Aug 05, 2017
-
-
Jun Wu authored
This patch uses shellquote to quote ssh parameters more strictly to avoid shell injection.
-
- Jul 07, 2017
-
-
Matt Harbison authored
When unquoted, MSYS sees the colon between the drive letter and path as a Unix path separator and unhelpfully splits on it, feeding only the drive letter as the command. Much chaos ensues. I vaguely remember trying to get the test runner to use /letter/path/to/exe syntax the last time this happened, without success. I doubt a check-code rule would work, since sometimes it is quoted, and sometimes the quotes are escaped.
-
- Jun 20, 2017
-
-
Augie Fackler authored
This time ones that are prefixed with =, ", ', or `. This appears to be the last of them. Differential Revision: https://phab.mercurial-scm.org/D14
-
- Jul 05, 2017
-
-
Augie Fackler authored
These are some simple cases. More to come in a future change. Reviewers: krbullock Reviewed By: krbullock Differential Revision: https://phab.mercurial-scm.org/D4
-
- Mar 31, 2017
-
-
Pierre-Yves David authored
While we are here, we can also add the hook name information to external hook.
-
Pierre-Yves David authored
The python hooks have access to the hook type information. There is not reason for external hook to not be aware of it too. For the record my use case is to make sure a hook script is configured for the right type.
-
- Mar 30, 2017
-
-
Pierre-Yves David authored
Hooks related to the transaction are aware of the transaction id. By definition this txn-id is unique and different for each transaction. As a result it can never be predicted in test and always needs matching. As a result, touching any like with this data is annoying. We solve the problem once and for all by installing an automatic replacement. In test, this will now show as: TXNID=TXN:$ID$
-
- Feb 16, 2017
-
-
Jun Wu authored
This patch replaces hardcoded 127.0.0.1 with $LOCALIP in all tests. Till now, the IPv6 series should make tests pass on common IPv6 systems where the local device has the address "::1" and the hostname "localhost" resolves to "::1".
-
- Feb 10, 2017
-
-
Pierre-Yves David authored
Previously Abort raised during 'getbundle' call poorly reported (HTTP-500 for http, some scary messages for ssh). Abort error have been properly reported for "push" for a long time, there is not reason to be different for 'getbundle'. We properly catch such error and report them back the best way available. For bundle, we issue a valid bundle2 reply (as expected by the client) with an 'error:abort' part. With bundle1 we do as best as we can depending of http or ssh.
-
- Nov 29, 2016
-
-
Gregory Szorc authored
Previously, the capabilities list was protocol agnostic and we advertised the same capabilities list to all clients, regardless of transport protocol. A few capabilities are specific to HTTP. I see no good reason why we should advertise them to SSH clients. So this patch limits their advertisement to HTTP clients. This patch is BC, but SSH clients shouldn't be using the removed capabilities so there should be no impact.
-
- Oct 28, 2016
-
-
Katsunori FUJIWARA authored
On Windows platform, invoking printenv.py directly via hook is problematic, because: - unless binding between *.py suffix and python runtime, application selector dialog is displayed, and running test is blocked at each printenv.py invocations - it isn't safe to assume binding between *.py suffix and python runtime, because application binding is easily broken For example, installing IDE (VisualStudio with Python Tools, or so) often requires binding between source files and IDE itself. This patch invokes printenv.py via sh -c for test portability. This is a kind of follow up for d19787db6fe0, which eliminated explicit "python" for printenv.py. There are already other 'sh -c "printenv.py"' in *.t files, and this fix should be reasonable. This changes were confirmed in cases below: - without any application binding for *.py suffix - with binding between *.py suffix and VisualStudio This patch also replaces "echo + redirection" style with "heredoc" style, because: - hook command line is parsed by cmd.exe as shell at first, and - single quotation can't quote arguments on cmd.exe, therefore, - "printenv.py foobar" should be quoted by double quotation, but - nested quoting (or tricky escaping) isn't readable
-
- Oct 13, 2016
-
-
Gregory Szorc authored
This patch disables delta chains on changelogs. After this patch, new entries on changelogs - including existing changelogs - will be stored as the fulltext of that data (likely compressed). No delta computation will be performed. An overview of delta chains and data justifying this change follows. Revlogs try to store entries as a delta against a previous entry (either a parent revision in the case of generaldelta or the previous physical revision when not using generaldelta). Most of the time this is the correct thing to do: it frequently results in less CPU usage and smaller storage. Delta chains are most effective when the base revision being deltad against is similar to the current data. This tends to occur naturally for manifests and file data, since only small parts of each tend to change with each revision. Changelogs, however, are a different story. Changelog entries represent changesets/commits. And unless commits in a repository are homogonous (same author, changing same files, similar commit messages, etc), a delta from one entry to the next tends to be relatively large compared to the size of the entry. This means that delta chains tend to be short. How short? Here is the full vs delta revision breakdown on some real world repos: Repo % Full % Delta Max Length hg 45.8 54.2 6 mozilla-central 42.4 57.6 8 mozilla-unified 42.5 57.5 17 pypy 46.1 53.9 6 python-zstandard 46.1 53.9 3 (I threw in python-zstandard as an example of a repo that is homogonous. It contains a small Python project with changes all from the same author.) Contrast this with the manifest revlog for these repos, where 99+% of revisions are deltas and delta chains run into the thousands. So delta chains aren't as useful on changelogs. But even a short delta chain may provide benefits. Let's measure that. Delta chains may require less CPU to read revisions if the CPU time spent reading smaller deltas is less than the CPU time used to decompress larger individual entries. We can measure this via `hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's fulltext. Here are the results of that command on a repo using delta chains in its changelog and on a repo without delta chains: hg (forward) ! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25) ! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26) hg (reverse) ! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19) ! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25) mozilla-central (forward) ! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3) ! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3) mozilla-central (reverse) ! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3) ! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3) mozilla-unified (forward) ! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3) ! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3) mozilla-unified (reverse) ! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3) ! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3) pypy (forward) ! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8) ! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9) pypy (reverse) ! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7) ! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9) The data clearly shows that it takes less wall and CPU time to resolve revisions when there are no delta chains in the changelogs, regardless of the direction of traversal. Furthermore, not using a delta chain means that fulltext resolution in reverse is as fast as iterating forward. So not using delta chains on the changelog is a clear CPU win for reading operations. An example of a user-visible operation showing this speed-up is revset evaluation. Here are results for `hg perfrevset 'author(gps) or author(mpm)'`: hg ! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6) ! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7) mozilla-central ! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3) ! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3) What about 00changelog.i size? Repo Delta Chains No Delta Chains hg 7,033,250 6,976,771 mozilla-central 82,978,748 81,574,623 mozilla-unified 88,112,349 86,702,162 pypy 20,740,699 20,659,741 The data shows that removing delta chains from the changelog makes the changelog smaller. Delta chains are also used during changegroup generation. This operation essentially converts a series of revisions to one large delta chain. And changegroup generation is smart: if the delta in the revlog matches what the changegroup is emitting, it will reuse the delta instead of recalculating it. We can measure the impact removing changelog delta chains has on changegroup generation via `hg perfchangegroupchangelog`: hg ! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7) ! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6) mozilla-central ! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3) ! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3) mozilla-unified ! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3) ! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3) pypy ! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3) ! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3) The data shows eliminating delta chains makes the changelog part of changegroup generation slower. This is expected since we now have to compute deltas for revisions where we could recycle the delta before. It is worth putting this regression into context of overall changegroup times. Here is the rough total CPU time spent in changegroup generation for various repos while using delta chains on the changelog: Repo CPU Time (s) CPU Time w/ compression hg 4.50 7.05 mozilla-central 111.1 222.0 pypy 28.68 75.5 Before compression, removing delta chains from the changegroup adds ~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central, and 2.0% to pypy. When you factor in zlib compression, these percentages are roughly divided by 2. While the increased CPU usage for changegroup generation is unfortunate, I think it is acceptable because the percentage is small, server operators (those likely impacted most by this) have other mechanisms to mitigate CPU consumption (namely reducing zlib compression level and pre-generated clone bundles), and because there is room to optimize this in the future. For example, we could use the nullid as the base revision, effectively encoding the full revision for each entry in the changegroup. When doing this, `hg perfchangegroupchangelog` nearly halves: mozilla-unified ! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3) ! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3) This looks very promising as a future optimization opportunity. It's worth that the changes in test-acl.t to the changegroup part size. This is because revision 6 in the changegroup had a delta chain of length 2 before and after this patch the base revision is nullrev. When the base revision is nullrev, cg2packer.deltaparent() hardcodes the *previous* revision from the changegroup as the delta parent. This caused the delta in the changegroup to switch base revisions, the delta to change, and the size to change accordingly. While the size increased in this case, I think sizes will remain the same on average, as the delta base for changelog revisions doesn't matter too much (as this patch shows). So, I don't consider this a regression.
-
- Aug 03, 2016
-
-
Pierre-Yves David authored
The new option will stay around. The experimental option was only meant to be temporary.
-
- Mar 15, 2016
-
-
Jun Wu authored
chg currently does not support hg serve -d. It has a quick path testing if the command is hg serve -d and fallbacks to hg if so. But the test only works if "serve" is the first argument since the test wants to avoid false positives (for example, "-r serve" is different). This patch reorders "hg server" commands in tests, making them chg friendly.
-
- Jan 13, 2016
-
-
Martin von Zweigbergk authored
In order to give us the freedom to change the changegroup3 format, let's hide it behind an experimental config. Since it is required by treemanifests, that will override the cg3 config.
-