- May 14, 2023
-
-
Georges Racinet authored
In case the `gitaly` executable is to be found in a subdirectory of `GITALY_INSTALL_DIR`, then that same subdirectory has to be provided as the `bin_dir` configuration item, so that Gitaly can find all its non-ruby bundled executables, notably the bundled Git. It was working until now because Gitaly falls back to a system-wide Git if the specified one is not found, whose version is acceptable in CI because the `heptapod-gitaly` base image is itself based on the `heptapod-base` image, which provides the Git version generally accepted by the target GitLab version. Since there is no doubt as to where the `gitaly` executable is supposed to sit, we're only testing one possibility (minimal change to keep the diff understable, to be refactored).
-
- May 03, 2023
-
-
Georges Racinet authored
-
- May 01, 2023
-
-
Georges Racinet authored
Hoping this will reuse intermediate object files from the artifact uploaded by the `tests-rust` job
-
- Apr 30, 2023
-
-
Georges Racinet authored
Again, we don't want the application to be locked for the wrong reason, e.g. lock held by a former process, but hostname has changed in the mean time. We also take the opportunity to use shorter timeout and delay betweeni locking attempts. The customatbility of the breaking logic and of the timeouts could be submitted UPSTREAM as options to Mercurial core, although it would be of limited interest to Heptapod right now: we cannot readily apply the same logic to other Mercurial locks, because HGitaly does not have exclusivity on them. Querying the system service manager (systemd or runit) about the Heptapod services would be a promising idea to do better, but we'd still have the issue that pushes over SSH would create standalone processes.
-
- Apr 26, 2023
-
-
Georges Racinet authored
The rationale is that if a working directory is being held by a process (existing or not) that is not considered part of the HGitaly system service, it means that the process was killed before it could release the working directory and the pid is either not in use or has been reused for an unrelated process. The underlying assumption is that only HGitaly is supposed to make such working directories. This could be further expanded to the roster lock. Had to put `is_current_service_process` in a separate top level module instead of, e.g., `server.mono` to avoid import loop: server.mono imports services import workdir import server.mono We need to use pytest-cov's `cleanup_on_signal` again. This time, it's easier as the subprocess startup function is defined by the test itself.
-
- Apr 25, 2023
-
-
Georges Racinet authored
The trick is that we fall precisely in the case where pytest-cov needs a hand: termination by signal (SIGTERM), hence we need the `cleanup_on_signal` method. Unfortunately, this does not work with pytest-7.3.1 and pytest-cov-4.0.0 as the previously lax `test-requirements.txt` would install (detected on the `heptapod-gitaly` base image, which does not preinstall these), so we fix versions as they are in py-heptapod (hence also as they already were in HDK settups, since `py-heptapod` is installed first). Then, we need the server to actually be running to be sure that the handling has been done before we terminate. Still not covering the case of `SIGINT` while doing some Mercurial processing. We'd probably need to launch a long enough gRPC method for that.
-
Georges Racinet authored
This makes it importable from the whole code base, and is already somewhat cleaner in the `tests_with_gitaly.rhgitaly` module.
-
- Apr 24, 2023
-
-
Georges Racinet authored
This wraps things together by passing down the proper root of working directories for the given repo (as prescribed in the Gitaly `Repository` message. The test is a simplified version of a service method creating changesets.
-
Georges Racinet authored
It has already been the case in Heptapod that relative path in Gitaly Repository messages were to be passed as if the repository was a Git repository (hence ending in `.git`) and then to be normalized as `.hg` for Mercurial operation. But this is due to the legacy hg-git based integration, and could not be necessary in some future. Meanwhile, we'll need to reuse the logic for working directories.
-
Georges Racinet authored
This is to be called as part of ordinary housekeeping. Another subject to be tackled on would be release of working directories held by defunct processes, which would introduce a more satisfactory breaking logic, also applicable to the roster lock.
-
- Apr 09, 2023
-
-
Georges Racinet authored
This creates the pool, recorded as a "roster" file in the main repository. To obtain an exclusive working directory, callers should use the `working_directory` context manager only. The roster file is protected by a lock, with input/output occurring only at acquire and release times of working directories, allowing to acquire working directories freely when one is already held. The lock is for now provided by Mercurial standard utilities, but we will probably implement a more suitable one later on. A cleanup system for old working directories shall be implemented in a following changeset. It will be based on the recorded timestamp.
-
- Apr 28, 2023
-
-
Georges Racinet authored
-
- Apr 27, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
Worth of note is that the log formatting will not be applied unless the level if fixed low enough. That's why the tests did not detect the problem in CreateBundleFromRefList, But it is visible in the production log
-
- Apr 26, 2023
-
-
Georges Racinet authored
We've been worried to see so many "defaulting to slow direct analysis" in the logs (especially in developer setups), as we are on the verge to rely on the GitLab state files only in some cases (RHGitaly). It turns out that all such logs emitted by a local functional tests run were actually about empty repos (`heptapod-tests-run --heptapod-remote --heptapod-hg-native=full`). It will be interesting to check foss.heptapod.net logs. In theory, we should not see any such warning. The early returns are of course much simpler, not sure if that is a performance boost in general, though, since the condition will be checked for all calls and most of them are not on empty repositories.
-
Georges Racinet authored
Logging all invocation metadata is verbose and contains secrets (authentication header). Of these, only the `correlation_id` is interesting to us at the moment. To achieve this, we split the service method with an inner method taking the inner request and the correlation id. Only the inner method is now instrumented. This is heavy for the implementation. We tried not to split, using the `fields` modifier like this: ``` #[instrument(skip(self, request), fields(request = request.get_ref()))] ``` and it ended up giving "`RepositoryRequest` does not implement tracing::Value`" The splitting will be good enough for now.
-
Georges Racinet authored
Basically, Gitaly chunk lengths are partly unpredictable. We tried to approximate them, but we can't really depend on them in assertions. The irritating thing is that this test failed often in CI (clear majority of runs, yet not all of them) and passed almost always on my development setup. The CI flakiness is now too much of a pain to keep it. Passing the `compare_first_chunks=False` does not change the fact that the entire content is compared and that metadata (aka small fields or unchunked fiels) are compared for all chunks (added a clarification in docstring about that).
-
- Apr 25, 2023
-
-
Georges Racinet authored
We are now on hg-evolve 11. This is the promised backout of 1c187a5a5a0c, with conflict resolution.
-
- Apr 24, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
The pipeline has much grown since the beginning of the HGitaly project. With the recent addition of RHGitaly, it has crossed a threshold. Besides, in recent versions of GitLab, branch pipelines for a commit heading a MR are not displayed in the pipelines tag of the MR, which we fear would be surprising contributors.
-
Georges Racinet authored
-
Georges Racinet authored
This freezes in particular the Mercurial sources revision, expressed in the new `rust/mercurial.rev` file. There is no longer need for the the `ci/prepare-rhgitaly` script and its associated revision variable. Run `make -C rust help` for more details.
-
Georges Racinet authored
Instead of requiring a clone of rust-protobuf, we now include them directly, which is much easier for development and packaging. These versions are copied over from rust-protobuf, because it was on hand, but it is obviously not the ultimate source of truth.
-
Georges Racinet authored
-
- Apr 13, 2023
-
-
Georges Racinet authored
-
Georges Racinet authored
Closes #118
-
Georges Racinet authored
This parallels the Python implementation: returning a `GitCommit`, this time from a `ChangelogEntry` instead of Python `changectx` will be a common need. The user line parsing is implemented to match the Python implementation (module `stringutil`). We considered using a regular expression, but it is not so obvious to obtain the exact same result (actually first experiments where splitting the brackets with a regexp for quick validation of the first gRPC method that will use this). User line and timestamp parsing should find eventually their way into `hg-core`.
-
Georges Racinet authored
All future implementations of streaming methods with repositories will follow the same pattern, hence we provide a generic helper right away, taking care notably of repetitive error treatment (sending errors etc.). Detailed explanations of the constraints and how it will work are provided in the doc-comment of the `load_repo_and_stream` function. Concrete gRPC method implementations will consist mostly of calling it with the closure doing the actual work on the loaded repository.
-
Georges Racinet authored
As the Gitaly Comparison tests showed us in the Python case, the errors to return on wrong `Repository` specifications depend at least on the service, hence we need an added flexibility. For that, we need internally to make a difference between client-side errors (e.g., repo does not exist) and internal errors (failed to load repo, e.g., due to a filesystem issue).
-
- Apr 12, 2023
-
-
Georges Racinet authored
That no response at all should be sent is confirmed by the added assertions in the Gitaly Comparison test
-
- Mar 24, 2023
-
-
Georges Racinet authored
Basic, yet will provide the baseline for RHGitaly development.
-
Georges Racinet authored
not immediately needed, but was overlooked
-
Georges Racinet authored
Reflection with `isinstance` is good enough, but for the sake of symmetry with responses we should perhaps add brackets to the specification, as in `request_sha_attrs='oids[]`
-
- Apr 02, 2023
-
-
Georges Racinet authored
As we don't want to resort ot unstable features of rustfmt, we wrote a simple checker in Python. Also we have to run after the tests, because we need the protocol source files (e.g., `gitaly.rs` to be present). Without them, `cargo fmt` fails at the `use` statement.
-
Georges Racinet authored
Some warnings have been fixed lower in the stack. However, the Camel Case rule for acronyms hurts my eyes in the case of TCP (and leads to plain wrong output when it ends up in startup logs through `fmt::Debug`). We had to change the way the generated code is included. This also has the nice effect of ignoring `gitaly.rs` in `rustfmt`, thus making `cargo fmt --check` work for us. The missing `Eq` impl when deriving `PartialEq` is new in Rust 1.63
-
- Mar 26, 2023
-
-
Georges Racinet authored
Nicer when starting from scratch: we avoid a timeout error. It's still worthwile to have CI do it before hand, because the build log ends up in the job log.
-
- Mar 23, 2023
-
-
Georges Racinet authored
We're defining a new fixture and `RpcHelper` for such comparisons. As expected, this is much simpler than the one for Gitaly Comparison tests. The included test is actually useful: we are sure that we don't have version skew between RHGitaly and HGitaly. Given how it is implemented, this is not a surprise, but it'll have our back in case the implementation has to change.
-
Georges Racinet authored
This new base class does not carry the specificities of the Gitaly comparison, but still has feature flags, default args, and calling primitives. It will be useful to introduce Comparison tests between HGitaly and RHGitaly with room for growth.
-