Skip to content
Snippets Groups Projects
  1. Feb 28, 2024
  2. Jan 15, 2024
  3. Nov 20, 2023
    • Georges Racinet's avatar
      RemoveRepository: better robustness in intermediate situations · 10d6717caedd
      Georges Racinet authored
      It was a mistake to depend on `load_repo`, as it will fail if,
      e.g., the `.hg` sub directory does not exist (seen in Rails
      RSpec tests) and various corrupted cases that a *removal* should
      clean up.
      
      The price to pay is the transactionality of the roster, but we
      cannot lock it properly without a repo. In theory in should be
      possible to instantiate the `Vfs` separately, but it does not seem
      to matter so much.
      
      With this change, if removing the working directories fails, it
      will not prevent removing the trashed repository (but of course
      we risk a later collision in workdirs).
      
      Backs out in particular changeset 1c735d40cfe7
      10d6717caedd
  4. Nov 18, 2023
    • Georges Racinet's avatar
      RepositoryService.RemoveRepository: simplification · 1c735d40cfe7
      Georges Racinet authored
      Now that we need to load the repo for the working directories
      machinery, no need to repeat the exceptions that can be raised
      by the loading.
      
      The only difference lies in the error message on missing repository,
      which we are confortable ignoring.
      1c735d40cfe7
  5. Nov 12, 2023
  6. Nov 10, 2023
  7. Sep 19, 2023
    • Georges Racinet's avatar
      SearchFilesByContent: improving memory efficiency · e40148cc11fe
      Georges Racinet authored
      This iterator-based approach just keeps a small window of up to 3
      lines in RAM, instead of the full list previously obtained with
      splitlines()`).
      
      It uses `collections.deque` (natively implemented), to make sure the
      window will never be reallocated. On the other hand, matching lines
      are allocated afresh, which is expected not to be a problem.
      
      At this point, further improvements should take the form of an implementation
      in RHGitaly.
      e40148cc11fe
  8. Sep 17, 2023
  9. Aug 07, 2023
  10. Jul 31, 2023
    • Georges Racinet's avatar
      RHGitaly implement HasLocalBranches · e0fbe4889654
      Georges Racinet authored
      This method is mostly used to check if the repository is emtpy (and
      that is in the case of Mercurial logically equivalent),
      but it's better to implement it using the GitLab branches file as it
      avoids actually opening the repository, spawning a thread for
      blocking iteration etc.
      e0fbe4889654
  11. Jul 11, 2023
    • Georges Racinet's avatar
      RepositoryService: implement RemoveAll · 1a6013fbd9c5
      Georges Racinet authored
      Actually, this method is called by `gitaly-backup` as of Gitaly 15.10 only
      (forwarded to the Rake task in upstream Git commit 4be1f22815bed, part of the
      15.11 development cycle, not imported in Heptapod Rails repo yet).
      
      Also, since for the time being an HGitaly storage always comes on top of
      a regular Gitaly storage at the same path, we will actually want *not* to
      call the HGitaly version for lots of time.
      1a6013fbd9c5
  12. Mar 19, 2023
    • Georges Racinet's avatar
      RHGitaly: Gitaly comparison tests · 69cb7195481a
      Georges Racinet authored
      We chose to compare RHGitaly with Gitaly rather than with HGitaly
      because it's not impossible that we stop at some point maintaining
      some methods in the Python code (time will tell).
      
      It is likely that we'll end up implementing some of the HGitaly-specific
      methods in RHGitaly (e.g. reading phases, topics and whatnot). In that
      case we'll need comparison between HGitaly and RHGitaly, but that
      will be some much simpler (no correspondence of Git content) that we
      will provide it directly.
      
      This fully hooked in CI/CD. We had to anyway: otherwise the new test would
      not pass.
      
      Implementation details
      ----------------------
      
      We add the capability to start RHGitaly, with a simple `cargo run`.
      This is validated independently by `test_rhgitaly_server` (homologous
      to `test_gitaly_server`).
      
      The `GitalyComparison` class (of which the `gitaly_comparison` fixture
      is an instance) now gains an optional `rhgitaly_channel`, which is set
      and exposed in the new `gitaly_rhgitaly_comparison` fixture.
      
      Finally, using `rhgitaly_channel` rather than `hgitaly_channel` is
      left to the `RPCHelper` class, with its `hg_server` argument.
      Since it is instantiated from inside the tests, it is easy to
      pass `hg_server` down from a parametrization, reaching the end goal: the
      very same test is run for HGitaly and RHGitaly, compared in both
      cases with Gitaly. This is illustrated in the test for `RepositoryExists`.
      
      Two annoyances had to be taken care of:
      
      - we need to ignore the `rust/` directory in `run-all-tests` because
        it makes pytest collection crazy (hangs forever) due at least in part
        to Mercurial full source being present by default in `rust/dependencies`
      - Tonic seems to add trailing metadata that Gitaly does not. We had
        to remove them in `assert_compare_errors()`.
      
      CI/CD assumptions
      -----------------
      
      The base image `heptapod-gitaly` is now expected to provide
      - the Rust toolchain
      - a clone of Mercurial sources. It can be daily or weekly, we
        make a share and update it
      - `rust-protobuf`: could be trimmed, as we only need
        the protocol files and don't need to update them for each build
      69cb7195481a
  13. Apr 12, 2023
  14. Mar 22, 2023
    • Georges Racinet's avatar
      Missing repository argument: proper error codes. · 9d403f00f259
      Georges Racinet authored
      As it was uncovered by experimental implementation in RHGitaly of
      `RepositoryExists`, it's possible that the `repository` argument
      is entirely missing. In that case, we are now raising `ValueError`
      internally, and convert it to the gRPC code
      `HGitalyServicer.STATUS_CODE_STORAGE_NOT_FOUND`.
      9d403f00f259
    • Georges Racinet's avatar
      servicer.load_repo: completed error treatment. · 215369c17798
      Georges Racinet authored
      Up to know, we were thinking that raising `KeyError` and setting
      code and details on the context was enough, but apparently, it
      is at least problematic with streaming gRPC methods.
      
      We're instead using the two-layer organisation that was suggested
      as comment in `service.repository`. As it turns out, the error
      code to set in case of unknown storage depends at least on the
      service. This was already visible in the error treatment of the
      full path methods of `RepositoryService`, which does not need
      to be specific any more.
      
      The new `STATUS_CODE_*`class attributes make it easy to tweak the
      error code in service implementations. Specific methods could override
      them with instance attributes if needed.
      
      In the process, we're basically giving up on providing the same
      `details` as Gitaly: these are highly dependent on implementation
      (it's just stderr if the implementation relies on a
      `git` subprocess) and chances that it may matter are low, given
      that GitLab these days pushes towards "structured errors", i.e. with
      programmatically usable details provided in metadata.
      As a result, the normalization of error details in `assert_compare_errors`
      is not used any more. We're keeping it in the code base, as it may
      turn useful for forthcoming tests were the details would not be a
      simple stderr echo.
      215369c17798
  15. Dec 13, 2022
  16. Oct 25, 2022
    • Georges Racinet's avatar
      Updated protocol as of Gitaly 2b069d8536df98 · b1170af16fc0
      Georges Racinet authored
      See #111, this is shortly after v15.1.0-rc1, but
      not an ancestor of v15.1.0.
      
      Brings in in particular:
      
      - rename of `repository-service.proto` to `repository`.proto, without
        changing the contents, though
      - usage of `errors.proto`
      
      There is a change of wording in the error details for FindChangedPaths
      that we have to follow in Comparison Tests, and don't need to check
      in such detail in ordinary integration tests.
      b1170af16fc0
  17. May 10, 2022
  18. Apr 28, 2022
    • Georges Racinet's avatar
      RepositoryService: implement RemoveRepository · 6e3106e95bc2
      Georges Racinet authored
      Closes #95
      
      This is in advance of the HGitaly3 milestone, but needed for
      gitaly-backup (removal occurs before restoration).
      
      We don't attempt to provide the same level of locking as Gitaly
      does, both because this is not the best time to handle locking
      for mutation methods and because this is a special case, but
      we split the work into atomic rename and cleanup (cheap to
      implement).
      
      See also heptapod#534
      6e3106e95bc2
  19. Oct 09, 2021
  20. Aug 28, 2021
  21. Mar 29, 2022
  22. Feb 15, 2022
    • Georges Racinet's avatar
      RepositoryService.CreateRepository: error if path already taken · f7c06a8948e0
      Georges Racinet authored
      In version 14.6, Gitaly makes it an error to attempt to create
      a repository if anything already exists at the target path
      (even broken symlinks).
      
      This is worth a minor version change.
      
      This deprives us of the way we were triggering actual
      repository creation errors in tests.
      To maintain coverage, we're now using the broken symlink
      trick one step higher in the file hierarchy: on the storage itself.
      Playing with permissions would not work in CI.
      f7c06a8948e0
  23. Oct 20, 2021
    • Georges Racinet's avatar
      RepositoryService: implement SetFullPath · 3ec209c1a656
      Georges Racinet authored
      Perhaps at some point Gitaly will decide that the proper error
      code should be `NotFound` or to stop providing the full path in
      the details, but we are meanhile doing the same.
      
      The only difference in error treatment is the replacement of
      'git repository` by 'Mercurial repository' but as usual I'm betting
      on it not to matter.
      3ec209c1a656
  24. May 31, 2021
  25. Mar 25, 2021
Loading