Skip to content
Snippets Groups Projects
  1. Aug 06, 2023
    • Georges Racinet's avatar
      RHGitaly: bumped MSRV to 1.65 · 2acf3e5f
      Georges Racinet authored
      Rust 1.65 is currently the version generally available in GitLab
      context (upstream usage appeared recently) and it is also able
      to build tokei 12.1.2, whereas 1.61 cannot.
      
      This leads up to fix the following Clippy error (not in 1.61):
      "you are deriving `PartialEq` and can implement `Eq`".
      2acf3e5f
  2. Jul 30, 2023
  3. Jul 29, 2023
    • Georges Racinet's avatar
      RHGitaly: implementing BlobService.GetBlobs · fd8aaff5
      Georges Racinet authored
      This finally puts together all the common logic already used in
      `GetBlob`, and we can switch on the Comparison tests (one single
      test function for both gRPC methods).
      
      Using `blocking_gitlab_revision_node_prefix` is not very elegant,
      we could probably resolve all the revisions before entering the blocking
      thread (and perhaps also avoid scanning the state files many times),
      but this is deemed not so important for the time being, compared to
      the huge efficiency boost provided to Heptapod by
      implementing this method in Rust.
      
      As is now customary, the tracing part has us create a wrapper struct
      and implement `Debug` because of the bytes in `RevisionPath`. At
      some point we'll have to come up with something more generic for this.
      fd8aaff5
    • Georges Racinet's avatar
      RHGitaly: blocking variant of revision resolution · 81cb41f4
      Georges Racinet authored
      This will be used once in a thread created by `spawn_blocking`,
      typically with the closure given to the likes of
      `repository::load_repo_and_stream`.
      
      It is a bit annoying to introduce the dependency to
      `futures::executor` for this, but we don't feel at ease
      using `tokio::runtime::Runtime.block_on`, as this is really
      meant as an entry point (looks like we'd need to create a new
      runtime and it's not clear whether it would interfere with the
      current running one or not).
      81cb41f4
    • Georges Racinet's avatar
      RHGitaly: BlobService.GetBlob · 03355820
      Georges Racinet authored
      This builds on the previously introduced repository file
      extraction and conversion to `GetBlobResponse`.
      
      It is important to emit at least one response, even if the
      resulting data is empty (usually because of `limit=0`).
      03355820
    • Georges Racinet's avatar
      Blob Gitaly comparison tests: testing with limit=0 · 7e37f156
      Georges Racinet authored
      As noted in the comment, this is an important use case.
      7e37f156
    • Georges Racinet's avatar
      RHGitaly: Blob gRPC messages definitions and adapters · d274bd92
      Georges Racinet authored
      This handles the fact that `GetBlobResponse` and `GetBlobsResponse` have
      some similar logic (including metadata only on the first message for a
      given blob) and some differences (more metadata in the latter).
      
      We enclose the common logic in the new `message::BlobResponseChunk` trait,
      and implement it for both response types.
      
      The intermixing with data is not very natural in the Rust ownership
      context, we try and make it not too painful. There is possibly room
      for improvement in reducing cloning/allocations, perhaps using the
      `Bytes`-oriented variant of `prost`.
      d274bd92
    • Georges Racinet's avatar
      RHGitaly: repository files extraction with metadata · 76f69a84
      Georges Racinet authored
      This provides the base logic to access file data, and a common
      representation for metadata that will be enough to implement
      the `GetBlob` and `GetBlobs` gRPC request, and that we hope to be
      reusable for the likes of `TreeEntry` and `GetTreeEntries`.
      
      The `rhgitaly::message` module will provide the necessary conversions
      to actual Gitaly response types.
      76f69a84
    • Georges Racinet's avatar
      RHGitaly WRITE_BUFFER_SIZE lazy static · 93c0c47c
      Georges Racinet authored
      Same as in `hgitaly.streaming`, and ultimately as in Gitaly,
      this is used as a constant that can be overridden by environment
      variable.
      93c0c47c
  4. Jul 28, 2023
  5. Jul 29, 2023
  6. Jul 28, 2023
  7. Jul 25, 2023
  8. Jul 29, 2023
  9. Jul 27, 2023
    • Georges Racinet's avatar
      CI/CD: unblock triggered pipelines · bf041abb
      Georges Racinet authored
      In the stable branches it's probably not so important to run
      triggered HGitaly pipelines, as usually py-heptapod does not change
      at all, but in any case this prevents the triggering job in py-heptapod
      to appear failed.
      bf041abb
  10. Jul 26, 2023
  11. Jul 05, 2023
  12. Jul 19, 2023
  13. Jul 17, 2023
    • Georges Racinet's avatar
      CommitService.FindCommits: implement 'include_referenced_by' · b5e96533
      Georges Racinet authored
      The implementation itself is more general than `FindCommits`: we
      make it an option in `message.commit()`.
      
      For this, we need to read all state files. Because `FindCommits`
      may return lots of commits, we do it once and cache the reverse mapping
      on the repository object. Of course this is somewhat heavy if only
      one commit is being queried, especially if the pattern is about only
      one type of refs (e.g. branch refs, whose numbers are much less than all past
      special refs). An improvement would be to remember which type of file has
      been read already, and load the files on-demand. Another improvement would
      be to have an efficient format allowing to have all refs in one single db
      file.
      
      Since it is unlikely that keep-arounds
      would be asked in this way, we omit them for the time being.
      
      Most of the actual testing is done as a Gitaly Comparison test. We don't
      have a formal definition of the matching at the time being, so this is
      defined by the examples in the test (reading Git source code should provide
      better answers).
      
      Finally, RHGitaly should grow the `message.commit` equivalent, but it would not
      be useful right now. However, if `FindCommit` would turn out to also get this
      option, we'd have to do the Rust version.
      
      Closes #136
      b5e96533
  14. Jul 12, 2023
  15. Jul 11, 2023
    • Georges Racinet's avatar
      RepositoryService: implement RemoveAll · 1a6013fb
      Georges Racinet authored
      Actually, this method is called by `gitaly-backup` as of Gitaly 15.10 only
      (forwarded to the Rake task in upstream Git commit 4be1f22815bed, part of the
      15.11 development cycle, not imported in Heptapod Rails repo yet).
      
      Also, since for the time being an HGitaly storage always comes on top of
      a regular Gitaly storage at the same path, we will actually want *not* to
      call the HGitaly version for lots of time.
      1a6013fb
    • Georges Racinet's avatar
      RepositoryService: implement Get/SetCustomHooks · 9df798b0
      Georges Racinet authored
      According to Gitaly commits a87977cf909a4c2c5a3d7b53e21e3fdf2e5e5048
      and ebc3336284d9ef5f2242e47d433aa94e4093ec2a, this is indeed a simple
      renaming.
      
      We don't bother about deduplication in test fixture wrapping methods.
      After all, deprecation will lead to removal quite soon.
      
      Closes #140
      9df798b0
  16. Jul 08, 2023
    • Georges Racinet's avatar
      Merged stable branch into default · 49074aa6
      Georges Racinet authored
      Developments on stable branch were entirely driven by RHGitaly,
      leading to better matching of Gitaly in corner cases and errors.
      
      A change had to be made for Comparison tests to pass:
      the error message in `FindCommit` with no arguments at all is now
      the one about the `repository` argument missing instead of `revision`.
      This is so much a corner case that we don't bother matching and
      simply add `same_details=False`. The change may have happened anywhere
      between Gitaly v15.5 and v15.9.
      49074aa6
    • Georges Racinet's avatar
      Bump Gitaly protocol to v15.9 · 9898801b
      Georges Racinet authored
      9898801b
    • Georges Racinet's avatar
      CI/CD: work if Cargo is more globally available · da452a65
      Georges Racinet authored
      For GitLab 15.9, we'll take advantage of the fact that
      upstream CI image does have the Rust toolchain ( installed
      to `/usr/local`).
      However it did not keep `/root/.cargo/env`.
      da452a65
  17. Jul 05, 2023
Loading