- Apr 12, 2023
-
-
Georges Racinet authored
Part of #80. As noticed by the functional tests at heptapod-tests, this is mandatory for Wikis using regular gRPC calls, which are rolled out in GitLab 15.5
-
- Mar 31, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
-
Georges Racinet authored
The only change we had to make is a workaround for what appears to be a minor bug in Mercurial or, if it is intended that some cache has to be updated explicitely, a missing compatibility in `mercurial-testhelper`.
-
Georges Racinet authored
This is in advance of !140 (introduction of RHGitaly), but is harmless if the directory does not exist and removes an irritating pain when switching topics.
-
Georges Racinet authored
py-heptapod 4.0 requires hg-evolve<11 and hence also Mercurial<6.4
-
- Mar 23, 2023
-
-
Georges Racinet authored
It turns out that `CommitService` does not return the `INTERNAL` gRPC error any more in Gitaly 15.5 when storage is incorrect. Now that is `INVALID_ARGUMENT`, same as with other gRPC methods.
-
Georges Racinet authored
The tests for `TreeEntry` and `GetTreeEntries` predate the introduction of the `RpcHelper` class by a few months, and were probably important in defining what we wanted to mutualize. Switching these tests to `RpcHelper` makes them directly applicable to `RHGitaly` when the time comes. The Blob/Tree methods are natural candidates for oxidation, as it is obvious that they are too slow in the Python implementation. We had to overcome two challenges for this: - Gitaly chunking being a bit non-deterministic in `TreeEntry`, we needed a way to compare streaming requests without comparing response by response. This now done by `RpcHelper.assert_compare_aggregated`, which takes care of the small data fields (`oid` and `mode` in the case of `TreeEntry`) with the common Gitaly pattern of providing them in the first response only. Two new hooks are to be registered in `RpcHelper` for this: one for the concatenation, one to remove the main data so that the small fields can be compared. - To take pagination into account, we needed a way to expose the first response of each backend as they were before normalization removed the cursor information (which is backend-specific). Since the pagination subprotocol is meant to be used in several streaming gRPC methods, we'll probably provide this in `RpcHelper` itself in a second move. In the process we're doing a more thorough job of comparing the `oid` response fields, thanks to a normalizer that knows the mapping between HGitaly and Gitaly OIDs.
-
- Mar 22, 2023
-
-
Georges Racinet authored
We had a Gitaly Comparison test, but it was actually comparing HGitaly with itself (!), see the change in `assert_compare_tree_entry()`. Then of course, whether it was initially correct or not, we had discrepancies to fix: - a gRPC `NOT_FOUND` is expected for unknown paths and revisions - `.` is not a valid path *for this method* - chunking in Gitaly happens in chunks of 16 kiB, but it's not consistent: sometimes the last chunk can be up to 16 kiB + 1023B long, sometimes, we get exact chunks. It's not clear in Gitaly code base exactly what happens: added logs in `streamio.go` showed us that this is what gets passed to it (not recut, then, since it's meant to cut at 128kiB). For these reasons, we gave up on comparing chunk by chunk and test the metadata only.
-
Georges Racinet authored
Since we don't currently run tests for the `stable` and `default` Mercurial/Evolve revisions (see 1c187a5a5a0c), we are at risk of losing coverage without the Gitaly Comparison tests. Hence we introduce the `tests-hg-current` job. Because the `gitaly-comparison` job (new name) will become heavier with the introduction of RHGitaly, we're pushing it to the compat stage. We could also just run `tests_with_gitaly` and don't bother with coverage (since it is done in a harder way by `tests-hg-current`).
-
Georges Racinet authored
-
Georges Racinet authored
Turns out Gitaly issues an `INTERNAL` error in that case. Probably not intended, but we're ready for further changes down the road.
-
Georges Racinet authored
It passes right away in that case.
-
Georges Racinet authored
As it was uncovered by experimental implementation in RHGitaly of `RepositoryExists`, it's possible that the `repository` argument is entirely missing. In that case, we are now raising `ValueError` internally, and convert it to the gRPC code `HGitalyServicer.STATUS_CODE_STORAGE_NOT_FOUND`.
-
Georges Racinet authored
Up to know, we were thinking that raising `KeyError` and setting code and details on the context was enough, but apparently, it is at least problematic with streaming gRPC methods. We're instead using the two-layer organisation that was suggested as comment in `service.repository`. As it turns out, the error code to set in case of unknown storage depends at least on the service. This was already visible in the error treatment of the full path methods of `RepositoryService`, which does not need to be specific any more. The new `STATUS_CODE_*`class attributes make it easy to tweak the error code in service implementations. Specific methods could override them with instance attributes if needed. In the process, we're basically giving up on providing the same `details` as Gitaly: these are highly dependent on implementation (it's just stderr if the implementation relies on a `git` subprocess) and chances that it may matter are low, given that GitLab these days pushes towards "structured errors", i.e. with programmatically usable details provided in metadata. As a result, the normalization of error details in `assert_compare_errors` is not used any more. We're keeping it in the code base, as it may turn useful for forthcoming tests were the details would not be a simple stderr echo.
-
- Mar 15, 2023
-
-
Georges Racinet authored
This is the only difference the tests could find with Gitaly 15.5.9
-
Georges Racinet authored
The changes do not warrant immediate implementation: - refine the IP address passing to Git commands to avoid rebind attacks - structured errors for Git-specific methods - more parameters to `RepositoryService.SearchFiles` method, that are not implemented in HGitaly, see #80, but perhaps should.
-
Georges Racinet authored
Next release will be at least about Gitaly 15.5
-
Georges Racinet authored
-
Georges Racinet authored
-
- Mar 13, 2023
-
-
Georges Racinet authored
-
- Mar 12, 2023
-
-
Georges Racinet authored
This finally leverages the new revset predicates in a straightforward manner. Closes #105
-
Georges Racinet authored
This will be required for the revsets, but it makes sense to do it inconditionally: this is after all closer to how the software is actually running, so it makes sense for integration and comparison tests (and all tests pass).
-
Georges Racinet authored
`logger.warn` is deprecated but I have to force myself not to use it.
-
Georges Racinet authored
We had the usual minor discrepancy: returning an empty response instead of an empty stream (the new Comparision Tests assertion does see the difference)
-
Georges Racinet authored
-
Georges Racinet authored
Mercurial core `grep` revset is also matching user (committer) and files, which can lead to surprises depending on how this is to be used by the Rails app (or perhaps API users if it gets exposed). Also, there was no case-insensitive regular expression. After some hesitation, we decided to make a separate predicate, rather than using the `(?i)` prefix, to avoid interfering with other in-pattern flags. Predicate names are meant to avoid colliding with other extensions, but this could go submitted UPSTREAM.
-
- Mar 10, 2023
-
-
Georges Racinet authored
We will soon have to enrich the revset with predicates using bytes strings.
-
- Mar 08, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
-
Georges Racinet authored
Unfortunaltely, e559b74680c4, which is already published on PyPI contained local changes in proto files. There were no change in stubs, so that functionality was not impaired. Still better clean up right away and republish something consistent.
-
Georges Racinet authored
-
- Feb 20, 2023
-
-
Georges Racinet authored
The end goal is to allow configuring logging in one shot, both for sporadic logs from service methods and the systematic request logging done in interceptors. Such loags require to tie a format string using `correlation_id` to the relevant loggers, and can now be done for the `hgitaly.service` only. Of course it is lacking in consistency, as interceptors are not services. Hence the docstring proposing to introduce an intermediate subpackage. That would be a chore for another day, though.
-
Georges Racinet authored
This will now be our primary source of informatiopn, with the supported `correlation_id`, provided the format string uses is. Converserly, logging the loading operation on repos themselves is generally speaking redundant, being useful only to investigate problems before the start of a request and the related load, hence we demote it to `DEBUG`
-
Georges Racinet authored
More generally, we make use of the logger adapter that leverages information from the gRPC context in the whole `hgitaly.service` package, and we won't from other packages. The intention and the constraints that lead to it are explained in the new REAdME section.
-
Georges Racinet authored
This provide san adapter to extract `correlation_id` from the gRPC context and add it to the logging `extra`, so that it may be used in a format, e.g.: `[%(asctime)s] [%(correlation_id)s] [%(levelname)s] [%(name)s] %(message)s` with the drawback that all loggers using this format will have to use the adapter. Minimal usage is added (`RequestLoggerInterceptor`) and support in `Service` fixture to get coverage with actual `ServicerContext` objects. Closes #122
-
Georges Racinet authored
-
Georges Racinet authored
These were remnant of times when we didn't have the systematic request logging interceptor.
-