- Feb 21, 2022
-
-
Gregory Szorc authored
The inheritance from object is implied in Python 3. So this should be equivalent. This change was generated via an automated search and replace. So there may have been some accidental changes. Differential Revision: https://phab.mercurial-scm.org/D12352
-
Gregory Szorc authored
pycompat.unicode is an alias to str. Differential Revision: https://phab.mercurial-scm.org/D12340
-
Gregory Szorc authored
Differential Revision: https://phab.mercurial-scm.org/D12322
-
- Mar 04, 2022
-
-
Gregory Szorc authored
pycompat.iteritems() just calls .items(). This commit applies a regular expression search and replace to convert simple instances of pycompat.iteritems() with .items(). There are still a handful of calls to pycompat.iteritems() remaining. But these all have more complicated expressions that I wasn't comfortable performing an automated replace on. In addition, some simple replacements were withheld because they broke pytype. These will be handled by their own changesets. Differential Revision: https://phab.mercurial-scm.org/D12318
-
- Mar 03, 2022
-
-
Gregory Szorc authored
These were needed for Python 2 support. Now that our linter no longer mandates these, we can start deleting them. Differential Revision: https://phab.mercurial-scm.org/D12254
-
- Jan 07, 2022
-
-
Pierre-Yves David authored
With `share-safe`, we will also write file in the store, so we take the store lock before writing requirements. Differential Revision: https://phab.mercurial-scm.org/D11991
-
- Sep 28, 2021
-
-
Martin von Zweigbergk authored
Most callers of `scmutil.revrange()` pass in a revset provided by the user. If there are problems resolving that, it should result in an `InputError` and exit code 10 (when using detailed exit codes). However, there are also some callers that pass in revsets not provided by the user. `InputError` is not appropriate in those cases. This patch therefore introduces a wrapper around `scmutil.revrange()` that simply converts the exception type. I put it in `logcmdutil.py` since that seems to be the lowest-level module in the (poorly defined) UI layer. Differential Revision: https://phab.mercurial-scm.org/D11560
-
- Mar 28, 2021
-
-
Jörg Sonnenberger authored
The introduction of 256bit hashes require changes to nullid and other constant magic values. Start pushing them down from repository and revlog where sensible. Differential Revision: https://phab.mercurial-scm.org/D9465
-
- Apr 12, 2021
-
-
Pierre-Yves David authored
The new module is well fitting for this new code. And this will be useful to make the gathered code collaborate more later. Differential Revision: https://phab.mercurial-scm.org/D10374
-
- Jan 18, 2021
-
-
Raphaël Gomès authored
The current (experimental) sidedata system uses flagprocessors to signify the presence and store/retrieve sidedata from the raw revlog data. This proved to be quite fragile from an exchange perspective and a lot more complex than simply having a dedicated space in the new revlog format. This change does not handle exchange (ironically), so the test for amend - that uses a bundle - is broken. This functionality is split into the next patches. Differential Revision: https://phab.mercurial-scm.org/D9993
-
- Dec 31, 2020
-
-
Pulkit Goyal authored
Well there are no private public functions in Python, but we generally treat functions and variables starting with `_` as private ones. A function which needs to be overrided in extension should be a public one. Differential Revision: https://phab.mercurial-scm.org/D9672
-
- Dec 01, 2020
-
-
Jörg Sonnenberger authored
There is no point in lazy importing mercurial.node, it is used all over the place anyway. So consistently import the used symbols directly. Fix one file using symbols indirectly via mercurial.revlog. Differential Revision: https://phab.mercurial-scm.org/D9480
-
Pierre-Yves David authored
This is a second step to clarify and clean up this code. The code responsible for definition which action exist, are possible and their compatibility if moved into a sub module. This clarify the main code and prepare further cleanup. Differential Revision: https://phab.mercurial-scm.org/D9477
-
Pierre-Yves David authored
The main module is getting big and hard to follow. So we are splitting all the logic to actually run an upgrade in a sub module. It nicely highlight that there are very few actual call point to the code we just moved. Differential Revision: https://phab.mercurial-scm.org/D9476
-
- Nov 27, 2020
-
-
durin42 authored
This required a couple of small tweaks to un-confuse black, but now it works. Big formatting changes come from: * Dramatically improved collection-splitting logic upstream * Black having a strong (correct IMO) opinion that """ is better than ''' Differential Revision: https://phab.mercurial-scm.org/D9430
-
- Oct 15, 2020
-
-
Pierre-Yves David authored
In the process on general clone bundle automatically, we need to make some function available more widely. This is a good opportunity to extract a significant amount of code from `mercurial.exchange` into a new `mercurial.bundlecaches`. This make `mercurial.exchange` move under the 3K line range (hooray…). The module is called `bundlecaches` because I expect it to be eventually useful for more than just clone bundle (like pull bunbles). Differential Revision: https://phab.mercurial-scm.org/D9208
-
- Apr 14, 2020
-
-
Pulkit Goyal authored
In upcoming patches, to implement Share Safe plan we will be introducing requires file in store. We need to route all callers to a single function to check for a share-safe requirement and if present, write requirements to .hg/store/requires instead. After this patch, callers directly calling scmutil.writerequires() are only those where we don't have the repo object, for example when initializing the repository object itself. Differential Revision: https://phab.mercurial-scm.org/D8631
-
- Jul 10, 2020
-
-
Rodrigo Damazio Bovendorp authored
The old API takes a list of revision separate from the file matcher, and thus provides no way to fetch different sets of files from each revision. In preparation for adding one such usage, I'm changing the API to take a list of (revision, file matcher) tuples instead. Differential Revision: https://phab.mercurial-scm.org/D8721
-
- Feb 09, 2020
-
-
Matt Harbison authored
The other fix for this was to leave the mode as bytes, and import `pycompat.open()` like a bunch of other modules do. But I think it's confusing to still use bytes at the python boundary, and obviously error prone. Grepping for ` open\(.+, ['"][a-z]+['"]\)` and ` open\(.+, b['"][a-z]+['"]\)` outside of `tests`, there are 51 and 87 uses respectively, so it's not like this is a rare direct usage. Differential Revision: https://phab.mercurial-scm.org/D8099
-
- Jan 21, 2020
-
-
Matt Harbison authored
With the stall issue seemingly fixed, there's no reason not to use workers. The setting is left for now to keep the test output deterministic, and in case other issues come up. If none do, this can be converted to a developer setting for usage with the tests. Differential Revision: https://phab.mercurial-scm.org/D7963
-
Matt Harbison authored
We've avoided the issue up to this point by gating worker usage with an experimental config. See 10e62d5efa73, and the thread linked there for some of the initial diagnosis, but essentially some data was being read from the blob before an error occurred and `keepalive` retried, but didn't rewind the file pointer. So the leading data was lost from the blob on the server, and the connection stalled, trying to send more data than available. In trying to recreate this, I was unable to do so uploading from Windows to CentOS 7. But it reproduced every time going from CentOS 7 to another CentOS 7 over https. I found recent fixes in the FaceBook repo to address this[1][2]. The commit message for the first is: The KeepAlive HTTP implementation is bugged in it's retry logic, it supports reading from a file pointer, but doesn't support rewinding of the seek cursor when it performs a retry. So it can happen that an upload fails for whatever reason and will then 'hang' on the retry event. The sequence of events that get triggered are: - Upload file A, goes OK. Keep-Alive caches connection. - Upload file B, fails due to (for example) failing Keep-Alive, but LFS file pointer has been consumed for the upload and fd has been closed. - Retry for file B starts, sets the Content-Length properly to the expected file size, but since file pointer has been consumed no data will be uploaded, causing the server to wait for the uploaded data until either client or server reaches a timeout, making it seem as our mercurial process hangs. This is just a stop-gap measure to prevent this behavior from blocking Mercurial (LFS has retry logic). A proper solutions need to be build on top of this stop-gap measure: for upload from file pointers, we should support fseek() on the interface. Since we expect to consume the whole file always anyways, this should be safe. This way we can seek back to the beginning on a retry. I ported those two patches, and it works. But I see that `url._sendfile()` does a rewind on `httpsendfile` objects[3], so maybe it's better to keep this all in one place and avoid a second seek. We may still want the first FaceBook patch as extra protection for this problem in general. The other two uses of `httpsendfile` are in the wire protocol to upload bundles, and to upload largefiles. Neither of these appear to use a worker, and I'm not sure why workers seem to trigger this, or if this could have happened without a worker. Since `httpsendfile` already has a `close()` method, that is dropped. That class also explicitly says there's no `__len__` attribute, so that is removed too. The override for `read()` is necessary to avoid the progressbar usage per file. [1] https://github.com/facebookexperimental/eden/commit/c350d6536d90c044c837abdd3675185644481469 [2] https://github.com/facebookexperimental/eden/commit/77f0d3fd0415e81b63e317e457af9c55c46103ee [3] https://www.mercurial-scm.org/repo/hg/file/5.2.2/mercurial/url.py#l176 Differential Revision: https://phab.mercurial-scm.org/D7962
-
Matt Harbison authored
This is less ugly than passing an open callback to the `httpsendfile` constuctor. Differential Revision: https://phab.mercurial-scm.org/D7961
-
Matt Harbison authored
This (almost) guarantees that the file is closed in the case of an exception. The one hole is if the `seek(SEEK_END)`/`tell()`/`seek(0)` sequence fails. But that's going to go away when subclassing `httpconnection.httpsendfile` to fix the worker problem, so I'm not going to worry too much. (And that class appears to have the same problem.) Differential Revision: https://phab.mercurial-scm.org/D7959
-
Matt Harbison authored
No longer necessary since the source transformer was removed. # skip-blame for changing string prefixes Differential Revision: https://phab.mercurial-scm.org/D7958
-
Matt Harbison authored
The previous code relied on reading the blob fully to close it. The obvious problem is if an error occurs before that point. But there is also a problem when using workers where the data may need to be re-read, which can't happen once it is closed. This eliminates the surprising behavior before attempting to fix the worker problem. Differential Revision: https://phab.mercurial-scm.org/D7957
-
Matt Harbison authored
This has been unused since f98fac24b757, which added worker based transfers for concurrency, shifting the progressbar maintenance to the single thread waiting on the worker to complete. Since the name no longer fits, rename the class. Differential Revision: https://phab.mercurial-scm.org/D7956
-
- Jan 15, 2020
-
-
Matt Harbison authored
This is also adapted from the Facebook repo[1]. Unlike there, we were already reading the download stream in chunks and immediately writing it to disk, so we basically avoided the problem on download. There shouldn't be a lot of data to read on upload, but it's better to get rid of this pattern. [1] https://github.com/facebookexperimental/eden/commit/82df66ffe97e21f3ee73dfec093c87500fc1f6a7 Differential Revision: https://phab.mercurial-scm.org/D7882
-
Matt Harbison authored
Adapted from the Facebook repo[1]. The intent is to distinguish between the connection dying and getting served a corrupt blob. The original message: HTTP makes no provision to tell your client that you failed halfway through producing your response and won't have the answer they're looking for. So, if a LFS server fails while producing a response, then we'll report an OID mismatch. We can do a little better and disambiguate between "the server sent us the wrong blob" (very scary) and "the server crashed" (merely annoying) by looking at the content length of the response we got back. If it's not what was advertised, we can reasonably safely assume the server crashed. [1] https://github.com/facebookexperimental/eden/commit/2a4a6fab4e882ed89b948bfc1e7d56d7c3c99dd2 Differential Revision: https://phab.mercurial-scm.org/D7881
-
- Jan 14, 2020
-
-
Matt Harbison authored
This is the response object, not a request. Differential Revision: https://phab.mercurial-scm.org/D7880
-
Matt Harbison authored
Facebook also passes `util.bytecount()` as a pretty formatter here, but our progress bar doesn't support that. Differential Revision: https://phab.mercurial-scm.org/D7872
-
- Dec 23, 2019
-
-
Matt Harbison authored
This applies the new marker in the lfs handler to show it in action, and adds the test mentioned at the beginning of the series to show that fulltext isn't necessary in the LFS case. The existing `skipread` isn't enough, because it is also set if an error occurs reading the revlog data, or the data is censored. It could probably be cleared, but then it technically violates the interface contract. That wouldn't matter for the existing verify algorithm, but it isn't clear how that will change as alternate storage support is added. The flag is probably pretty revlog specific, given the comments in verify.py. But there's already filelog specific stuff in there and I'm not sure what future storage will bring, so I don't want to over-engineer this. Likewise, I'm not sure that we want the verify method for each storage type to completely drive the bus when it comes to detecting renames, so I don't want to go down the rabbithole of having verifyintegrity() return metadata hints at this point. Differential Revision: https://phab.mercurial-scm.org/D7713
-
Matt Harbison authored
The `skipflags` config was introduced in a2ab9ebcd85b, which specifically calls out downloading and storing all blobs as potentially too expensive. But I don't see any reason to skip blobs that are already available locally. Hashing the blob is the only way to indirectly verify the rawdata content stored in the revlog. (The note in that commit about skipping renamed is still correct, but the reason given about needing fulltext isn't.) Differential Revision: https://phab.mercurial-scm.org/D7712
-
- Dec 20, 2019
-
-
Matt Harbison authored
Trying to validate the fulltext of an external revision causes missing blobs to be downloaded and cached. Since the downloads aren't batch prefetched[1] and aren't compressed, this can be expensive both in terms of time and space. I made this a tri-state instead of a simple bool because there's an existing (undocumented) config to handle this, and it would be weird if `hg verify` were to suddenly start ignoring that config but an `hg recover` initiated verify honors it. Since this uses the same config setting, it too will skip rename verification (which requires fulltext, but not for LFS). [1] https://www.mercurial-scm.org/pipermail/mercurial-devel/2018-April/116118.html Differential Revision: https://phab.mercurial-scm.org/D7708
-
- Dec 23, 2019
-
-
Matt Harbison authored
This conceptually broke in 1541e1a8e87d when the filelog isa revlog relationship was changed to containment of the revlog. It was made more obvious in 62a532045e71 and related API simplification. It's resolved in favor of passing a revlog because the revlog verification code doesn't have a reference to a filelog. Differential Revision: https://phab.mercurial-scm.org/D7711
-
- Nov 08, 2019
-
-
Augie Fackler authored
This is the promised second step on single-quoted strings. These had existed because our source transformer didn't turn r'' into b'', so we had tagged some strings as r-strings to get "native" strings on both Pythons. Now that the transformer is gone, we can dispense with this nonsense. Methodology: I ran hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$' | xargs egrep --color=never -n -- \[\^b\]\[\^a-z\]r\'\[\^\'\\\\\]\*\'\[\^\'\ in an emacs grep-mode buffer, and then used a keyboard macro to iterate over the results and remove the r prefix as needed. # skip-blame removing unneeded r prefixes left over from Python 3 migration. Differential Revision: https://phab.mercurial-scm.org/D7306
-
- Nov 07, 2019
-
-
Augie Fackler authored
This is only double-quoted strings. I'll do single-quoted strings as a second step. These had existed because our source transformer didn't turn r"" into b"", so we had tagged some strings as r-strings to get "native" strings on both Pythons. Now that the transformer is gone, we can dispense with this nonsense. Methodology: I ran hg locate 'set:added() or modified() or clean()' | egrep '.*\.py$' | xargs egrep --color=never -n -- \[\^a-z\]r\"\[\^\"\\\\\]\*\"\[\^\"\] in an emacs grep-mode buffer, and then used a keyboard macro to iterate over the results and remove the r prefix as needed. # skip-blame removing unneeded r prefixes left over from Python 3 migration. Differential Revision: https://phab.mercurial-scm.org/D7305
-
- Nov 02, 2019
-
-
Gregory Szorc authored
Python 3.5's json.loads() requires a str. Only Python 3.6+ supports passing a bytes or bytearray. This commit implements a json.loads() polyfill on Python 3.5 so that we can use bytes. The added function to detect encodings comes verbatim from Python 3.7.
-
- Oct 21, 2019
-
-
Denis Laxalde authored
The leading ".hglfs::" is interpreted as a macro in generated man page and, as it is unknown, the whole section does not render. Also, having the section marked as preformatted (::) makes it render verbatim in HTML, which is not desired as the text contains formatting markers. So we just dedent the section and remove the ".hglfs::" line. The example file is still indented and rendered preformatted.
-
- Oct 07, 2019
-
-
Martin von Zweigbergk authored
Differential Revision: https://phab.mercurial-scm.org/D7029
-
- Oct 06, 2019
-
-
Gregory Szorc authored
.iteritems() -> .items() is the last source transform being performed. But it is also the most widely used. This commit adds a pycompat.iteritems symbol and imports it in place of .iteritems() for usage in hgext/. I chose to stop at just hgext/ because the patch will be large and it is an easy boundary to stop at since we can disable source transformation on a per-package basis. There are places where the type does implement items() and we could call items() directly. However, this would require critical thought and I thought it would be easier to just blindly change the code. We know which call sites need to be audited in the future because they have "pycompat.iteritems." With this change, we no longer perform source transformation on hgext! Differential Revision: https://phab.mercurial-scm.org/D7014
-