- Nov 21, 2022
-
-
Pierre-Yves David authored
This is an interesting statistis, so let's display it.
-
- Nov 07, 2022
-
-
Pierre-Yves David authored
Deltas against a base that is not an ancestor of the revision that owns this delta are notable. For example, they introduce complexity during the bundling process as the base might not exist on the unbundling side. We detect them in `hg debugrevlog` and print information about them.
-
Pierre-Yves David authored
We have a module dedicated to debug code, let us use it.
-
Pierre-Yves David authored
We have a module dedicated to debug code, let us use it.
-
- Nov 23, 2022
-
-
Pierre-Yves David authored
I ran performance and storage tests on repositories of various sizes and shapes for the following values of the config : 5, 10, 20, 50, 100, no-chunking The performance tests do not show any statistical impact on computation times for large pushes and pulls. For searching for an individual delta, this can provide a significant performance improvement with a minor degradation of space-quality on the result. (see data at the end of the commit). For overall store size, the change : - does not have any impact on many small repositories, - has an observable, but very negligible impact on most larger repositories. - One private repository we use for testing sees a small increase in size (1%) in the narrower version. We will try to get more numbers on a larger version of that repository to make sure nothing pathological happens. We pick "10" as the limit as "5" seems a bit more risky. There are room to improve the current code, by using more aggressive filtering and better (i.e any) sorting of the candidates. However this is already a large improvement for pathological cases, with little impact in the common situations. The initial motivation for this change is to fix performance of delta computation for a file where the previous code ended up testing 20 000 possible candidate-bases in one go, which is… slow. This affected about ½ of the file revisions leading to atrocious performance, especially during some push/pull operations. Details about individual delta finding timing: ---------------------------------------------- The vast majority of benchmark cases are unchanged but the three below. The first two do not see any impact on the final delta. The last one sees a change in delta-size that is negligible compared to the full text size. ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-a (revision 756096) ∞: 5.844783 5: 4.473523 (-23.46%) 10: 4.970053 (-14.97%) 20: 5.770386 (-1.27%) 50 5.821358 100: 5.834887 MANIFESTLOG: rev = 756096: (no-limit) delta-base = 301840 search-rounds = 6 try-count = 60 delta-type = snapshot snap-depth = 7 delta-size = 179 MANIFESTLOG: rev=756096: (limit = 10) delta-base=301840 search-rounds=9 try-count=51 delta-type=snapshot snap-depth=7 delta-size=179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # benchmark.variants.rev = manifest-snapshot-many-tries-d (revision 754060) ∞: 5.017663 5: 3.655931 (-27.14%) 10: 4.095436 (-18.38%) 20: 4.828949 (-3.76%) 50 4.987574 100: 4.994889 MANIFESTLOG: rev=754060: (no limit) delta-base=301840 search-rounds=5 try-count=53 delta-type=snapshot snap-depth=7 delta-size = 179 MANIFESTLOG: rev=754060: (limite = 10) delta-base=301840 search-rounds=8 try-count=45 delta-type=snapshot snap-depth=7 delta-size = 179 ### data-env-vars.name = mozilla-try-2019-02-18-zstd-sparse-revlog # benchmark.name = perf-delta-find # bin-env-vars.hg.flavor = rust # benchmark.variants.rev = manifest-snapshot-many-tries-e (revision 693368) ∞: 4.869282 5: 2.039732 (-58.11%) 10: 2.413537 (-50.43%) 20: 4.449639 (-8.62%) 50 4.865863 100: 4.882649 MANIFESTLOG: rev=693368: delta-base=693336 search-rounds=6 try-count=53 delta-type=snapshot snap-depth=6 full-test-size=131065 delta-size=199 MANIFESTLOG: rev=693368: delta-base=278023 search-rounds=5 try-count=21 delta-type=snapshot snap-depth=4 full-test-size=131065 delta-size=278 Raw data for store size (in bytes) for various chunk size value below: ---------------------------------------------------------------------- 440 134 384 5 pypy/.hg/store/ 440 134 384 10 pypy/.hg/store/ 440 134 384 20 pypy/.hg/store/ 440 134 384 50 pypy/.hg/store/ 440 134 384 100 pypy/.hg/store/ 440 134 384 ... pypy/.hg/store/ 666 987 471 5 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 10 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 20 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 50 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 100 netbsd-xsrc-2022-11-15/.hg/store/ 666 987 471 ... netbsd-xsrc-2022-11-15/.hg/store/ 852 844 884 5 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 10 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 20 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 50 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 100 netbsd-pkgsrc-2022-11-15/.hg/store/ 852 844 884 ... netbsd-pkgsrc-2022-11-15/.hg/store/ 1 504 227 981 5 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 871 10 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 20 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 50 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 100 netbeans-2018-08-01-sparse-zstd/.hg/store/ 1 504 227 813 ... netbeans-2018-08-01-sparse-zstd/.hg/store/ 3 875 801 068 5 netbsd-src-2022-11-15/.hg/store/ 3 875 696 767 10 netbsd-src-2022-11-15/.hg/store/ 3 875 696 757 20 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 50 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 100 netbsd-src-2022-11-15/.hg/store/ 3 875 696 653 ... netbsd-src-2022-11-15/.hg/store/ 4 531 441 314 5 mozilla-central/.hg/store/ 4 531 435 157 10 mozilla-central/.hg/store/ 4 531 432 045 20 mozilla-central/.hg/store/ 4 531 429 119 50 mozilla-central/.hg/store/ 4 531 429 119 100 mozilla-central/.hg/store/ 4 531 429 119 ... mozilla-central/.hg/store/ 4 875 861 390 5 mozilla-unified/.hg/store/ 4 875 855 155 10 mozilla-unified/.hg/store/ 4 875 852 027 20 mozilla-unified/.hg/store/ 4 875 848 851 50 mozilla-unified/.hg/store/ 4 875 848 851 100 mozilla-unified/.hg/store/ 4 875 848 851 ... mozilla-unified/.hg/store/ 11 498 764 601 5 mozilla-try/.hg/store/ 11 497 968 858 10 mozilla-try/.hg/store/ 11 497 958 730 20 mozilla-try/.hg/store/ 11 497 927 156 50 mozilla-try/.hg/store/ 11 497 925 963 100 mozilla-try/.hg/store/ 11 497 923 428 ... mozilla-try/.hg/store/ 10 047 914 031 5 private-repo 9 969 132 101 10 private-repo 9 944 745 015 20 private-repo 9 939 756 703 50 private-repo 9 939 833 016 100 private-repo 9 939 822 035 ... private-repo
-
- Nov 06, 2022
-
-
Pierre-Yves David authored
See inline comment for details. The feature is currently disabled, but should be enabled by default to mitigate some existing pathological cases. Also see the next changeset for details.
-
- Nov 24, 2022
-
-
Matt Harbison authored
While pytype may not support PEP 561, PyCharm does, so having the stubs available means it can determine `foo = attr.ib(type=int)` means `foo` is an int. This only applies when using Mercurial as a library, like with TortoiseHg development- PyCharm is already smart enough to use the *.pyi files in the Mercurial source tree when hacking on Mercurial itself. I left the mercurial.cext stubs out because it seems very low level, that 3rd parties shouldn't be using directly.
-
Matt Harbison authored
No particular reason, other than the current build is fairly old. It flagged a few more things (that weren't errors based on the logic around them), but OTOH, some of the pyi stubs it generates are less specific.
-
Matt Harbison authored
Since CI runs from docker images, it doesn't matter that this mucks with the typeshed bundled with pytype.
-
Matt Harbison authored
I really hate this, but pytype doesn't support PEP 561 and doesn't seem to have the equivalent of `MYPYPATH` to point to custom stubs. Ignoring the vendored stubs isn't necessarily harmful, but pytype has been choking on the vendored attr package after pytype 2022.03.29 with errors like this: File "/mnt/c/Users/Matt/hg/mercurial/linelog.py", line 52, in __iter__: Built-in function iter was called with the wrong arguments [wrong-arg-types] Expected: (collection: bytearray) Actually passed: (collection: mercurial.thirdparty.attr._make._CountingAttr) File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 143, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 144, in pack: No attribute 'rfind' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 146, in pack: Built-in function len was called with the wrong arguments [wrong-arg-types] Expected: (obj: Sized) Actually passed: (obj: mercurial.thirdparty.attr._make._CountingAttr) Attributes of protocol Sized are not implemented on mercurial.thirdparty.attr._make._CountingAttr: __len__ File "/mnt/c/Users/Matt/hg/mercurial/dirstateutils/v2.py", line 152, in pack: No attribute 'v2_data' on mercurial.thirdparty.attr._make._CountingAttr [attribute-error] File "/mnt/c/Users/Matt/hg/mercurial/util.py", line 2817, in go: unsupported operand type(s) for /: 'count: mercurial.thirdparty.attr._make._CountingAttr' and 'float: float' [unsupported-operands] No attribute '__truediv__' on 'count: mercurial.thirdparty.attr._make._CountingAttr' or '__rtruediv__' on 'float: float' Called from (traceback): line 2981, in __bytes__ This is essentially the same hack we've been using in TortoiseHg to add the vendored PyQt5 stubs. What I don't understand is pytype *still* generates *.pyi files under .pytype/pyi/mercurial/thirdparty/attr, even when the package is explicitly ignored in the pytype command line args. But it avoids the errors, which means we aren't stuck on pytype==2022.03.29. https://github.com/google/pytype/issues/151
-
Matt Harbison authored
The same logic is in the TortoiseHg tests for running pytype, and it's useful to know if a new version of pytype is better or worse.
-
- Nov 23, 2022
-
-
Matt Harbison authored
Unfortunately, pytype doesn't support this yet. But it was included with the attr package, so we might as well do it here for consistency. Unlike the attr package, these type hints are only partial, so they are marked as such[1] (but who knows if it matters, given these are C extensions, so no local source code to scan). [1] https://peps.python.org/pep-0561/#partial-stub-packages
-
Matt Harbison authored
Flagged by pytype 2022.11.18 when the cext stubs are made visible to it. There are very likely other signatures that are missing, but this is enough to keep it happy for now.
-
Matt Harbison authored
-
- Nov 21, 2022
-
-
Matt Harbison authored
Was hoping to find more issues like f09bc2ed9100, but it may be that nothing checks the args to that operation. In any event, the work is done and pytype doesn't do a very good job inferring the types. A few of th emore complicated things like the command table are left untyped, because they come from modules that aren't typed yet.
-
Matt Harbison authored
When trying to figure out how `hg help -v` took the Set interpolation path in f09bc2ed9100, I turned on devel warnings and noticed this (unrelated) warning: devel-warn: specifying a mismatched default value for a registered config item: 'merge-tools.beyondcompare4.regappend' '' at: c:\Users\Matt\hg\mercurial\filemerge.py:46 (_toolstr) The previous default value for this config was `None`, but that slightly complicates the code at the only site it is used, referenced above.
-
Matt Harbison authored
The previous version was 5 years old, and pytype 2022.06.30 started complaining about various uses (e.g. seeing `mercurial.thirdparty.attr._make._CountingAttr` instead of `bytearray`). Hopefully this helps. Additionally, this has official python 3.11 support. The `attrs` package is left out, because it is simply a bunch of *.pyi stubs and `from attr.X import *`, and that's not how they've been used up to this point. We'd probably need to customize those anyway to `from mercurial.thirdparty.attr import *`.
-
Matt Harbison authored
When updating to 22.1.0, this test started failing: Traceback (most recent call last): File "/tmp/mercurial-ci/tests/test-util.py", line 53, in <module> _start_default = (util.timedcmstats.start.default, 'factory') AttributeError: type object 'timedcmstats' has no attribute 'start' Poking around in `hg debugshell`, the attribute is indeed missing, but looks to be attached to `__attrs_attrs__` in both the currently vendored and the modern version of attrs. The old attrs packages will print the same for both accesses, so fingers crossed... >>> print((util.timedcmstats.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory') >>> print((util.timedcmstats.__attrs_attrs__.start.default, 'factory')) (Factory(factory=<function timedcmstats.<lambda> at 0x000001EFDF0F21F0>, takes_self=False), 'factory')
-
- Nov 15, 2022
-
-
Raphaël Gomès authored
These are painless, so they are all grouped in this changeset.
-
- Nov 14, 2022
-
-
Raphaël Gomès authored
This one is the worst one to upgrade since v2 -> v4 broke a ton of API, which thankfully seems saner now. Contrary to what was done in the `hg-core/src/examples/nodemap` rewrite, we're not switching from the "builder" pattern to the "derive" pattern, since that would imply a much larger diff. It can be done incrementally.
-
Raphaël Gomès authored
`hg-cpython` has no BC breaking dependencies, we can group them all in this changeset.
-
Raphaël Gomès authored
Finally, these dependencies do not require any code changes, so they are all grouped in the same changeset.
-
Raphaël Gomès authored
Upgrading is a finally possible now that we're supporting a more recent version of Rust. This required changes in the `nodemap` example.
-
Raphaël Gomès authored
Now that we support a newer version of Rust, we can update this dependency to get all the latest bugfixes and improvements. A slight API adjustment was needed.
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
-
Raphaël Gomès authored
We've changed our minimum Rust version to 1.61.0 in the previous patch, and edition 2021 predates that version.
-
- Oct 20, 2022
-
-
Raphaël Gomès authored
A few months ago¹, a decision was made to move the Rust toolchain target to whatever Debian Testing was tracking. I didn't have the bandwidth to act on it until now. This is starting to be even more problematic than before, now that edition 2021 is out. The CI has been updated to track the current Debian testing version, 1.61.0. [1] https://lists.mercurial-scm.org/pipermail/mercurial-packaging/2022-April/000338.html
-
- Nov 19, 2022
-
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
Raphaël Gomès authored
-
Pierre-Yves David authored
This one is not covered by the CIbecause I requires an expensive artifact to be cached. So it goes out of think on regular basis (we should fix that…) The test ouput was affected by e706bb41fdb3 as we filtering now happens sooner, removing for the output.
-
Pierre-Yves David authored
When running `hg log --follow --rev REVS FILES`, the log code will walk the history of all FILES starting from the file revisions that exists in each REVS. Before doing so, it looks if the files actually exists in the target revisions. To do so, it opens the manifest of each revision in REVS to look up if we find the associated items in FILES. Before this changeset this was done in a way that created a changectx for each target revision, keeping them in memory while we look into each file. If the set of REVS is large, this means keeping the manifest for each entry in REVS in memory. That can be large… if REV is in the form `::X`, this can quickly become huge and saturate the memory. We have seen usage allocating 2GB per second until memory runs out. So this changeset invert the two loop so that only one revision is kept in memory during the operation. This solve the memory explosion issue.
-
- Nov 18, 2022
-
-
Arseniy Alekseyev authored
This is useful so we can store other files in $TESTTMP (in particular tests that use docket files (nodemap, dirstate-v2) keep file uids in $TESTTMP/UID)
-
Arseniy Alekseyev authored
-
- Nov 17, 2022
-
-
Arseniy Alekseyev authored
since the transaction reverts the store, while the dirstate is stored separately
-