- Apr 24, 2023
-
-
Georges Racinet authored
This wraps things together by passing down the proper root of working directories for the given repo (as prescribed in the Gitaly `Repository` message. The test is a simplified version of a service method creating changesets.
-
Georges Racinet authored
It has already been the case in Heptapod that relative path in Gitaly Repository messages were to be passed as if the repository was a Git repository (hence ending in `.git`) and then to be normalized as `.hg` for Mercurial operation. But this is due to the legacy hg-git based integration, and could not be necessary in some future. Meanwhile, we'll need to reuse the logic for working directories.
-
Georges Racinet authored
This is to be called as part of ordinary housekeeping. Another subject to be tackled on would be release of working directories held by defunct processes, which would introduce a more satisfactory breaking logic, also applicable to the roster lock.
-
- Apr 09, 2023
-
-
Georges Racinet authored
This creates the pool, recorded as a "roster" file in the main repository. To obtain an exclusive working directory, callers should use the `working_directory` context manager only. The roster file is protected by a lock, with input/output occurring only at acquire and release times of working directories, allowing to acquire working directories freely when one is already held. The lock is for now provided by Mercurial standard utilities, but we will probably implement a more suitable one later on. A cleanup system for old working directories shall be implemented in a following changeset. It will be based on the recorded timestamp.
-
- Apr 28, 2023
-
-
Georges Racinet authored
-
- Apr 27, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
Worth of note is that the log formatting will not be applied unless the level if fixed low enough. That's why the tests did not detect the problem in CreateBundleFromRefList, But it is visible in the production log
-
- Apr 26, 2023
-
-
Georges Racinet authored
We've been worried to see so many "defaulting to slow direct analysis" in the logs (especially in developer setups), as we are on the verge to rely on the GitLab state files only in some cases (RHGitaly). It turns out that all such logs emitted by a local functional tests run were actually about empty repos (`heptapod-tests-run --heptapod-remote --heptapod-hg-native=full`). It will be interesting to check foss.heptapod.net logs. In theory, we should not see any such warning. The early returns are of course much simpler, not sure if that is a performance boost in general, though, since the condition will be checked for all calls and most of them are not on empty repositories.
-
Georges Racinet authored
Logging all invocation metadata is verbose and contains secrets (authentication header). Of these, only the `correlation_id` is interesting to us at the moment. To achieve this, we split the service method with an inner method taking the inner request and the correlation id. Only the inner method is now instrumented. This is heavy for the implementation. We tried not to split, using the `fields` modifier like this: ``` #[instrument(skip(self, request), fields(request = request.get_ref()))] ``` and it ended up giving "`RepositoryRequest` does not implement tracing::Value`" The splitting will be good enough for now.
-
Georges Racinet authored
Basically, Gitaly chunk lengths are partly unpredictable. We tried to approximate them, but we can't really depend on them in assertions. The irritating thing is that this test failed often in CI (clear majority of runs, yet not all of them) and passed almost always on my development setup. The CI flakiness is now too much of a pain to keep it. Passing the `compare_first_chunks=False` does not change the fact that the entire content is compared and that metadata (aka small fields or unchunked fiels) are compared for all chunks (added a clarification in docstring about that).
-
- Apr 25, 2023
-
-
Georges Racinet authored
We are now on hg-evolve 11. This is the promised backout of 1c187a5a5a0c, with conflict resolution.
-
- Apr 24, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
The pipeline has much grown since the beginning of the HGitaly project. With the recent addition of RHGitaly, it has crossed a threshold. Besides, in recent versions of GitLab, branch pipelines for a commit heading a MR are not displayed in the pipelines tag of the MR, which we fear would be surprising contributors.
-
Georges Racinet authored
-
Georges Racinet authored
This freezes in particular the Mercurial sources revision, expressed in the new `rust/mercurial.rev` file. There is no longer need for the the `ci/prepare-rhgitaly` script and its associated revision variable. Run `make -C rust help` for more details.
-
Georges Racinet authored
Instead of requiring a clone of rust-protobuf, we now include them directly, which is much easier for development and packaging. These versions are copied over from rust-protobuf, because it was on hand, but it is obviously not the ultimate source of truth.
-
Georges Racinet authored
-
- Apr 13, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
Closes #118
-
Georges Racinet authored
This parallels the Python implementation: returning a `GitCommit`, this time from a `ChangelogEntry` instead of Python `changectx` will be a common need. The user line parsing is implemented to match the Python implementation (module `stringutil`). We considered using a regular expression, but it is not so obvious to obtain the exact same result (actually first experiments where splitting the brackets with a regexp for quick validation of the first gRPC method that will use this). User line and timestamp parsing should find eventually their way into `hg-core`.
-
Georges Racinet authored
All future implementations of streaming methods with repositories will follow the same pattern, hence we provide a generic helper right away, taking care notably of repetitive error treatment (sending errors etc.). Detailed explanations of the constraints and how it will work are provided in the doc-comment of the `load_repo_and_stream` function. Concrete gRPC method implementations will consist mostly of calling it with the closure doing the actual work on the loaded repository.
-
Georges Racinet authored
As the Gitaly Comparison tests showed us in the Python case, the errors to return on wrong `Repository` specifications depend at least on the service, hence we need an added flexibility. For that, we need internally to make a difference between client-side errors (e.g., repo does not exist) and internal errors (failed to load repo, e.g., due to a filesystem issue).
-
- Apr 12, 2023
-
-
Georges Racinet authored
That no response at all should be sent is confirmed by the added assertions in the Gitaly Comparison test
-
- Mar 24, 2023
-
-
Georges Racinet authored
Basic, yet will provide the baseline for RHGitaly development.
-
Georges Racinet authored
not immediately needed, but was overlooked
-
Georges Racinet authored
Reflection with `isinstance` is good enough, but for the sake of symmetry with responses we should perhaps add brackets to the specification, as in `request_sha_attrs='oids[]`
-
- Apr 02, 2023
-
-
Georges Racinet authored
As we don't want to resort ot unstable features of rustfmt, we wrote a simple checker in Python. Also we have to run after the tests, because we need the protocol source files (e.g., `gitaly.rs` to be present). Without them, `cargo fmt` fails at the `use` statement.
-
Georges Racinet authored
Some warnings have been fixed lower in the stack. However, the Camel Case rule for acronyms hurts my eyes in the case of TCP (and leads to plain wrong output when it ends up in startup logs through `fmt::Debug`). We had to change the way the generated code is included. This also has the nice effect of ignoring `gitaly.rs` in `rustfmt`, thus making `cargo fmt --check` work for us. The missing `Eq` impl when deriving `PartialEq` is new in Rust 1.63
-
- Mar 26, 2023
-
-
Georges Racinet authored
Nicer when starting from scratch: we avoid a timeout error. It's still worthwile to have CI do it before hand, because the build log ends up in the job log.
-
- Mar 23, 2023
-
-
Georges Racinet authored
We're defining a new fixture and `RpcHelper` for such comparisons. As expected, this is much simpler than the one for Gitaly Comparison tests. The included test is actually useful: we are sure that we don't have version skew between RHGitaly and HGitaly. Given how it is implemented, this is not a surprise, but it'll have our back in case the implementation has to change.
-
Georges Racinet authored
This new base class does not carry the specificities of the Gitaly comparison, but still has feature flags, default args, and calling primitives. It will be useful to introduce Comparison tests between HGitaly and RHGitaly with room for growth.
-
Georges Racinet authored
It was nice to use the purely stateless `ServerInfo` gRPC method (at least with the info that HGitaly provides) as a first example of an implementation using Tonic. Now it's time to provide the actual information, consistent with the Python implementation. We're using the `build_const` crate to inject the value at build time. This is actually importing from the `constants.rs` file in the relevant build `out/` directory. Also the `..Default::default()` syntax (taught us by clippy) allows to avoid starting from the defaults to mutate a few fields.
-
Georges Racinet authored
This artifacts built by `tests-rust` will serve as precompilation for gitaly-comparison. For the cache, we follow the instructions of https://doc.rust-lang.org/nightly/cargo/guide/cargo-home.html#caching-the-cargo-home-in-ci but still exclude installed binaries information Hopefully, this will speed up the build a bit.
-
Georges Racinet authored
A simple `cargo test`. Because of the dependencies, we had to provide a new base image for this (we could have used heptapod-gitaly, but it is unsatisfactory to rely on a huge image when much less is needed).
-
- Mar 19, 2023
-
-
Georges Racinet authored
We chose to compare RHGitaly with Gitaly rather than with HGitaly because it's not impossible that we stop at some point maintaining some methods in the Python code (time will tell). It is likely that we'll end up implementing some of the HGitaly-specific methods in RHGitaly (e.g. reading phases, topics and whatnot). In that case we'll need comparison between HGitaly and RHGitaly, but that will be some much simpler (no correspondence of Git content) that we will provide it directly. This fully hooked in CI/CD. We had to anyway: otherwise the new test would not pass. Implementation details ---------------------- We add the capability to start RHGitaly, with a simple `cargo run`. This is validated independently by `test_rhgitaly_server` (homologous to `test_gitaly_server`). The `GitalyComparison` class (of which the `gitaly_comparison` fixture is an instance) now gains an optional `rhgitaly_channel`, which is set and exposed in the new `gitaly_rhgitaly_comparison` fixture. Finally, using `rhgitaly_channel` rather than `hgitaly_channel` is left to the `RPCHelper` class, with its `hg_server` argument. Since it is instantiated from inside the tests, it is easy to pass `hg_server` down from a parametrization, reaching the end goal: the very same test is run for HGitaly and RHGitaly, compared in both cases with Gitaly. This is illustrated in the test for `RepositoryExists`. Two annoyances had to be taken care of: - we need to ignore the `rust/` directory in `run-all-tests` because it makes pytest collection crazy (hangs forever) due at least in part to Mercurial full source being present by default in `rust/dependencies` - Tonic seems to add trailing metadata that Gitaly does not. We had to remove them in `assert_compare_errors()`. CI/CD assumptions ----------------- The base image `heptapod-gitaly` is now expected to provide - the Rust toolchain - a clone of Mercurial sources. It can be daily or weekly, we make a share and update it - `rust-protobuf`: could be trimmed, as we only need the protocol files and don't need to update them for each build
-
- Mar 18, 2023
-
-
Georges Racinet authored
We keep on configuring through environment variables, in order to focus on the task at hand: stop hardoding a TCP address and support Unix Domain sockets. The latter are the natural choice for integration tests, such as the Gitaly Comparison tests, as they come with strong guarantees not to be already in use.
-
- Mar 17, 2023
-
-
Georges Racinet authored
Similarly to the Python implementation, this introduces a top level module to take care of logic that will be useful for several methods. These top-level modules don't know about actual method or services, but they handle protobuf messages, `Status` errors etc. Like the Python implementation, we support only one storage (named `default`) and configuration does its repositories root.
-
- Mar 28, 2023
-
-
Georges Racinet authored
This removes the (blocking) `println!` in favour of the standard `tracing` crate, which is precisely meant for asynchronous code. For now, we're exclusively logging to `stdout`, at the `INFO` level. Configurability will appear later. The `DEBUG` level is very verbose, as it includes lots of events from the tonic/h2 etc lower layers. These will certainly be interesting, yet not at this stage of developpement. Quoting from current documentation: The #[instrument] attribute provides an easy way to add tracing spans to functions. A function annotated with #[instrument] will create and enter a span with that function’s name every time the function is called, with arguments to that function will be recorded as fields using `fmt::Debug`. Hence we don't need to include the passed `request` in the call to `info!`. This also raises the question of the laziness of formatting, something that we will have to check later. Finally, Tonic's `Request` wrapper does include all invocation metadata, which is great to have `correlation_id` but won't be acceptable when we implement authentication, as it will also probably have the related secrets. It could be a simple matter of using an appropriate subscriber. We'll see.
-