- Jun 20, 2017
-
-
Martin von Zweigbergk authored
-
- May 02, 2017
-
-
Pierre-Yves David authored
Now that we garantee that branchmap cache is updated at the end of the transaction we can drop this update. This removes a problematic case with nested transaction where the new cache could be written on disk before the transaction is finished (and even roll-backed) Such premature cache write was visible in the following test: * tests/test-acl.t * tests/test-rebase-conflicts.t In addition, running the cache update later means having more date about the state of the repository (in particular: phases). So we can generate caches with more information. This creates harmless changes to the following tests: * tests/test-hardlinks-whitelisted.t * tests/test-hardlinks.t * tests/test-phases.t * tests/test-tags.t * tests/test-inherit-mode.t
-
Pierre-Yves David authored
We are about to remove the branchmap cache update in changegroup application. There is a debug message alongside this update that we do not want to loose. We move the message beforehand to simplify the test update in the next changeset. The message move is quite noisy and isolating that noise is useful. Most tests update are just line reordering since the message is issued at a later point during the transaction. After this changes, the message is displayed in more case since local commit creation also issue it.
-
- Oct 13, 2016
-
-
Pierre-Yves David authored
Revlog can now be configured to store full snapshot only. This is used on the changelog. However, the changegroup packing was still recomputing deltas to be sent over the wire. We now just reuse the full snapshot directly in this case, skipping delta computation. This provides use with a large speed up(-30%): # perfchangegroupchangelog on mercurial ! wall 2.010326 comb 2.020000 user 2.000000 sys 0.020000 (best of 5) ! wall 1.382039 comb 1.380000 user 1.370000 sys 0.010000 (best of 8) # perfchangegroupchangelog on pypy ! wall 5.792589 comb 5.780000 user 5.780000 sys 0.000000 (best of 3) ! wall 3.911158 comb 3.920000 user 3.900000 sys 0.020000 (best of 3) # perfchangegroupchangelog on mozilla central ! wall 20.683727 comb 20.680000 user 20.630000 sys 0.050000 (best of 3) ! wall 14.190204 comb 14.190000 user 14.150000 sys 0.040000 (best of 3) Many tests have to be updated because of the change in bundle content. All theses update have been verified. Because diffing changelog was not very valuable, the resulting bundle have similar size (often a bit smaller): # full bundle of mozilla central with delta: 1142740533B without delta: 1142173300B So this is a win all over the board.
-
Gregory Szorc authored
This patch disables delta chains on changelogs. After this patch, new entries on changelogs - including existing changelogs - will be stored as the fulltext of that data (likely compressed). No delta computation will be performed. An overview of delta chains and data justifying this change follows. Revlogs try to store entries as a delta against a previous entry (either a parent revision in the case of generaldelta or the previous physical revision when not using generaldelta). Most of the time this is the correct thing to do: it frequently results in less CPU usage and smaller storage. Delta chains are most effective when the base revision being deltad against is similar to the current data. This tends to occur naturally for manifests and file data, since only small parts of each tend to change with each revision. Changelogs, however, are a different story. Changelog entries represent changesets/commits. And unless commits in a repository are homogonous (same author, changing same files, similar commit messages, etc), a delta from one entry to the next tends to be relatively large compared to the size of the entry. This means that delta chains tend to be short. How short? Here is the full vs delta revision breakdown on some real world repos: Repo % Full % Delta Max Length hg 45.8 54.2 6 mozilla-central 42.4 57.6 8 mozilla-unified 42.5 57.5 17 pypy 46.1 53.9 6 python-zstandard 46.1 53.9 3 (I threw in python-zstandard as an example of a repo that is homogonous. It contains a small Python project with changes all from the same author.) Contrast this with the manifest revlog for these repos, where 99+% of revisions are deltas and delta chains run into the thousands. So delta chains aren't as useful on changelogs. But even a short delta chain may provide benefits. Let's measure that. Delta chains may require less CPU to read revisions if the CPU time spent reading smaller deltas is less than the CPU time used to decompress larger individual entries. We can measure this via `hg perfrevlog -c -d 1` to iterate a revlog to resolve each revision's fulltext. Here are the results of that command on a repo using delta chains in its changelog and on a repo without delta chains: hg (forward) ! wall 0.407008 comb 0.410000 user 0.410000 sys 0.000000 (best of 25) ! wall 0.390061 comb 0.390000 user 0.390000 sys 0.000000 (best of 26) hg (reverse) ! wall 0.515221 comb 0.520000 user 0.520000 sys 0.000000 (best of 19) ! wall 0.400018 comb 0.400000 user 0.390000 sys 0.010000 (best of 25) mozilla-central (forward) ! wall 4.508296 comb 4.490000 user 4.490000 sys 0.000000 (best of 3) ! wall 4.370222 comb 4.370000 user 4.350000 sys 0.020000 (best of 3) mozilla-central (reverse) ! wall 5.758995 comb 5.760000 user 5.720000 sys 0.040000 (best of 3) ! wall 4.346503 comb 4.340000 user 4.320000 sys 0.020000 (best of 3) mozilla-unified (forward) ! wall 4.957088 comb 4.950000 user 4.940000 sys 0.010000 (best of 3) ! wall 4.660528 comb 4.650000 user 4.630000 sys 0.020000 (best of 3) mozilla-unified (reverse) ! wall 6.119827 comb 6.110000 user 6.090000 sys 0.020000 (best of 3) ! wall 4.675136 comb 4.670000 user 4.670000 sys 0.000000 (best of 3) pypy (forward) ! wall 1.231122 comb 1.240000 user 1.230000 sys 0.010000 (best of 8) ! wall 1.164896 comb 1.160000 user 1.160000 sys 0.000000 (best of 9) pypy (reverse) ! wall 1.467049 comb 1.460000 user 1.460000 sys 0.000000 (best of 7) ! wall 1.160200 comb 1.170000 user 1.160000 sys 0.010000 (best of 9) The data clearly shows that it takes less wall and CPU time to resolve revisions when there are no delta chains in the changelogs, regardless of the direction of traversal. Furthermore, not using a delta chain means that fulltext resolution in reverse is as fast as iterating forward. So not using delta chains on the changelog is a clear CPU win for reading operations. An example of a user-visible operation showing this speed-up is revset evaluation. Here are results for `hg perfrevset 'author(gps) or author(mpm)'`: hg ! wall 1.655506 comb 1.660000 user 1.650000 sys 0.010000 (best of 6) ! wall 1.612723 comb 1.610000 user 1.600000 sys 0.010000 (best of 7) mozilla-central ! wall 17.629826 comb 17.640000 user 17.600000 sys 0.040000 (best of 3) ! wall 17.311033 comb 17.300000 user 17.260000 sys 0.040000 (best of 3) What about 00changelog.i size? Repo Delta Chains No Delta Chains hg 7,033,250 6,976,771 mozilla-central 82,978,748 81,574,623 mozilla-unified 88,112,349 86,702,162 pypy 20,740,699 20,659,741 The data shows that removing delta chains from the changelog makes the changelog smaller. Delta chains are also used during changegroup generation. This operation essentially converts a series of revisions to one large delta chain. And changegroup generation is smart: if the delta in the revlog matches what the changegroup is emitting, it will reuse the delta instead of recalculating it. We can measure the impact removing changelog delta chains has on changegroup generation via `hg perfchangegroupchangelog`: hg ! wall 1.589245 comb 1.590000 user 1.590000 sys 0.000000 (best of 7) ! wall 1.788060 comb 1.790000 user 1.790000 sys 0.000000 (best of 6) mozilla-central ! wall 17.382585 comb 17.380000 user 17.340000 sys 0.040000 (best of 3) ! wall 20.161357 comb 20.160000 user 20.120000 sys 0.040000 (best of 3) mozilla-unified ! wall 18.722839 comb 18.720000 user 18.680000 sys 0.040000 (best of 3) ! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3) pypy ! wall 4.828317 comb 4.830000 user 4.820000 sys 0.010000 (best of 3) ! wall 5.415455 comb 5.420000 user 5.410000 sys 0.010000 (best of 3) The data shows eliminating delta chains makes the changelog part of changegroup generation slower. This is expected since we now have to compute deltas for revisions where we could recycle the delta before. It is worth putting this regression into context of overall changegroup times. Here is the rough total CPU time spent in changegroup generation for various repos while using delta chains on the changelog: Repo CPU Time (s) CPU Time w/ compression hg 4.50 7.05 mozilla-central 111.1 222.0 pypy 28.68 75.5 Before compression, removing delta chains from the changegroup adds ~4.4% overhead to hg changegroup generation, 1.3% to mozilla-central, and 2.0% to pypy. When you factor in zlib compression, these percentages are roughly divided by 2. While the increased CPU usage for changegroup generation is unfortunate, I think it is acceptable because the percentage is small, server operators (those likely impacted most by this) have other mechanisms to mitigate CPU consumption (namely reducing zlib compression level and pre-generated clone bundles), and because there is room to optimize this in the future. For example, we could use the nullid as the base revision, effectively encoding the full revision for each entry in the changegroup. When doing this, `hg perfchangegroupchangelog` nearly halves: mozilla-unified ! wall 21.168075 comb 21.170000 user 21.130000 sys 0.040000 (best of 3) ! wall 11.196461 comb 11.200000 user 11.190000 sys 0.010000 (best of 3) This looks very promising as a future optimization opportunity. It's worth that the changes in test-acl.t to the changegroup part size. This is because revision 6 in the changegroup had a delta chain of length 2 before and after this patch the base revision is nullrev. When the base revision is nullrev, cg2packer.deltaparent() hardcodes the *previous* revision from the changegroup as the delta parent. This caused the delta in the changegroup to switch base revisions, the delta to change, and the size to change accordingly. While the size increased in this case, I think sizes will remain the same on average, as the delta base for changelog revisions doesn't matter too much (as this patch shows). So, I don't consider this a regression.
-
- Aug 09, 2016
-
-
Gregory Szorc authored
Before, we logged about performing a branch cache update when we weren't actually doing it. Fix that.
-
- Aug 02, 2016
-
-
Pierre-Yves David authored
This is an old config that predate bundle2 on by default. This should have been remove after Mercurail 3.6 got released.
-
- Jan 13, 2016
-
-
Martin von Zweigbergk authored
In order to give us the freedom to change the changegroup3 format, let's hide it behind an experimental config. Since it is required by treemanifests, that will override the cg3 config.
-
- Dec 14, 2015
-
-
Mike Edgar authored
This lets revlog flags be transmitted over the wire. Right now this is useful for censored nodes and for narrowhg's ellipsis nodes.
-
- Dec 11, 2015
-
-
Augie Fackler authored
I'm not entirely happy with using a trailing / on a "file" entry for transferring a treemanifest. We've discussed putting some flags on each file header[0], but I'm unconvinced that's actually any better: if we were going to add another feature to the cg format we'd still be doing a version bump anyway to cg4, so I'm inclined to not spend time coming up with a more sophisticated format until we actually know what the next feature we want to stuff in a changegroup will be. Test changes outside test-treemanifest.t are only due to the new CG3 bundlecap showing up in the wire protocol. Many thanks to adgar@google.com and martinvonz@google.com for helping me with various odd corners of the changegroup and treemanifest API. 0: It's not hard refactoring, nor is it a lot of work. I'm just disinclined to do speculative work when it's not clear what the customer would actually be.
-
- Jun 11, 2015
-
-
Pierre-Yves David authored
The message was issued unconditionally. Move it inside the `if` that actually tries to push obsmarkers.
-
- May 28, 2015
-
-
Pierre-Yves David authored
When using bundle2, the phase pushkey parts are now made mandatory. As a result, failure to update the bookmark server side will result in the transaction being aborted.
-
- Jun 10, 2015
-
-
Pierre-Yves David authored
We add a way to convey the precise exception. This will allow better error message on the server.
-
- Jun 06, 2015
-
-
Pierre-Yves David authored
This capability will be extended as new error type is introduced.
-
- May 27, 2015
-
-
Pierre-Yves David authored
The current behavior (with bundle1) is to let the rest of the push succeed if the pushkey call (phases, bookmarks) failed (this comes from the fact that each item is sent in its own command). We kept this behavior with bundle2, which is highly debatable, but let us keep thing as they are now as a start. We are about to enforce 'mandatory' pushkey part as 'mandatory' successful, so we need to marks parts as advisory to preserve the current (debatable) behavior.
-
- Jun 02, 2015
-
-
Gregory Szorc authored
.hgtags fnodes cache entries can be expensive to compute, especially if there are hundreds of even thousands of them. This patch implements support for receiving a bundle2 part that contains a mapping of changeset to .hgtags fnodes. An upcoming patch will teach the server to send this part, allowing clients to bypass having to redundantly compute these values. A number of tests changed due to the client advertising the "hgtagsfnodes" capability.
-
- May 27, 2015
-
-
Pierre-Yves David authored
This test makes extensive use of --debug so moving to bundle2 based exchange has a massive impact. We do it early to reduce the noise create by a future usage of bundle2 as the default protocol.
-
- May 10, 2015
-
-
Pierre-Yves David authored
When the progress extension is not enabled, each call to 'ui.progress' used to issue a debug message. This results is a very verbose output and often redundant in tests. Dropping it makes tests less volatile to factor they do not meant to test. We had to alter the sed trick in 'test-rename-merge2.t'. Sed is used to drop all output from a certain point and hidding the progress output remove its anchor. So we anchor on something else.
-
- Sep 09, 2014
-
-
durin42 authored
A future patch is going to add some extra commented-out boilerplate to the top of .hg/hgrc during clone. In order to make this test not require regular updates, switch to searching for [hooks] or [acl] and print file from the first match to that pattern.
-
- Aug 16, 2014
-
-
Pierre-Yves David authored
The discovery of necessary bookmark updates is now done within the "discovery phase". This opens the door to the inclusion of bookmarks in a unified bundle2 push.
-
- Jul 31, 2014
-
-
Pierre-Yves David authored
This will allow including phase information in the same bundle2 as the changesets.
-
- Apr 15, 2014
-
-
Pierre-Yves David authored
This input will have to travel over the wire anyway, so we feed the peer method with a simple binary stream and rely on the server side to use `readbundle` to create the python object. The test output changes because the bundle is created marginally sooner and the debug output interleaves in a different way.
-
- Apr 05, 2014
-
-
Pierre-Yves David authored
Localrepo now supports the unbundle method of pushing changegroups. We plan to use the unbundle call for bundle2 so it is important that all peers supports it. The `peer.unbundle` and `peer.addchangegroup` code path have small difference so cause some test output changes. None of those changes seems problematic.
-
- Jan 15, 2013
-
-
Pierre-Yves David authored
The strip code used a trick to lower the cost of branchcache update after a strip. However is less necessary since we have branchcache collaboration. Invalid branchcache are likely to be cheaply rebuilt again a near subset of the repo. Moreover, this trick would need update to be relevant in the now filtered repository world. It currently update the unfiltered branchcache that few people cares about. Make it smarter on that aspect would need complexes update of the calling logic So this mechanism is: - Arguably needed, - Currently irrelevant, - Hard to update and I'm dropping it. We now update the branchcache in all case by courtesy of the read only reader. This changeset have a few expected impact on the testsuite are different cache are updated.
-
- Jan 13, 2013
-
-
Kevin Bullock authored
Now that changelog filtering is in place, it's become evident that naming the filters according to the set of revs _not_ included in the filtered changelog is confusing. This is especially evident in the collaborative branch cache scheme. This changes the names of the filters to reflect the revs that _are_ included: hidden -> visible unserved -> served mutable -> immutable impactable -> base repoview.filteredrevs is renamed to filterrevs, so that callers read a bit more sensibly, e.g.: filterrevs('visible') # filter revs according to what's visible
-
- Jan 02, 2013
-
-
Pierre-Yves David authored
It filters all mutable changesets, leaving only public changeset unfiltered. This filtering set is expected to be much more stable that the previous one as public changeset are unlikely to disapear. The only official use of this filter is for branchcache.
-
- Jan 07, 2013
-
-
Pierre-Yves David authored
Filtered repository are *subset* of unfiltered repository. This means that a filtered branchmap could be use to compute the unfiltered version. And filtered version happen to be subset of each other: - "all() - unserved()" is a subset of "all() - hidden()" - "all() - hidden()" is a subset of "all()" This means that branchmap with "unfiltered" filter can be used as a base for "hidden" branchmap that itself could be used as a base for unfiltered branchmap. unserved < hidden < None This changeset implements this mechanism. If the on disk branchcache is not valid we use the branchcache of the nearest subset as base instead of computing it from scratch. Such fallback can be cascaded multiple time is necessary. Note that both "hidden" and "unserved" set are a bit volatile. We will add more stable filtering in next changesets. This changeset enables collaboration between no filtering and "unserved" filtering. Fixing performance regression introduced by 47f00b0de337
-
- Jan 02, 2013
-
-
Pierre-Yves David authored
Disables this simple optimisation to allow coming more powerfull approach: cache collaboration. Our goal is to have branchcache collaborate. This means that unfiltered branchcache will fallback to some filtered branchcache if invalid. We can't have the filtered branchcache to use the unfiltered one. That would loop.
-
- Dec 27, 2012
-
-
Pierre-Yves David authored
This factors out the generation of the message. This helps future error reporting when reading cache for filtered repository.
-
- Dec 04, 2012
-
-
Julien Cristau authored
Hurd seems to set ENOENT to 2 + 2**30, unlike everyone else.
-
- Jul 26, 2012
-
-
Pierre-Yves David authored
-
- Jul 28, 2012
-
-
Pierre-Yves David authored
Very few data are displayed now, making it hard to debug phases and obsolete related issues.
-
- May 28, 2012
-
-
Elifarley Callado Coelho Cruz authored
The "!" prefix allows you to prevent anyone except a given user or group to push changesets in a given branch or path. This patch enables a use case suggested by a user (Julien Bonnet): There's a branch that only a given user (or group) should be able to push to, and you don't want to restrict access to any other branch that may be created. With the "!" prefix, you simply deny access to "!givenuser" (or "!@givengroup").
-
- Jun 01, 2012
-
-
Elifarley Callado Coelho Cruz authored
Declaring 'acl' in the [extensions] section doesn't change anything.
-
- Dec 08, 2011
-
-
Matt Mackall authored
-
Matt Mackall authored
-
- Nov 16, 2011
-
-
kiilerix authored
Windows will use a different and localized message.
-
- Oct 06, 2011
-
-
Elifarley Callado Coelho Cruz authored
-
- Sep 18, 2011
-
-
Greg Ward authored
If the working dir parent was destroyed by rollback, then the old behaviour is perfectly reasonable: restore dirstate, branch, and bookmarks. That way the working dir moves back to an existing changeset rather than becoming an orphan. But if the working dir parent was unaffected -- say, you updated to an older changeset and then did rollback -- then it's silly to restore dirstate and branch. So don't do that. Leave the status of the working dir alone. (But always restore bookmarks, because that file refers to changeset IDs that may have been destroyed.)
-
- Jun 03, 2011
-
-
Sune Foldager authored
Progress for files was off by one, and the code was rather hackish.
-