Skip to content
Snippets Groups Projects
  1. Jun 11, 2023
  2. May 31, 2023
    • Georges Racinet's avatar
      Matching Gitaly v15.8.0 · 7e77711d
      Georges Racinet authored
      The only change in protocol is the addition of `DiffService.GetPatchID`,
      see #131, that does not seem to be used yet in the Rails app.
      
      The only adaptation needed for Gitaly Comparison tests to pass
      against v15.8.0 is the error details in `CommitService.TreeEntry`
      7e77711d
  3. May 30, 2023
  4. May 28, 2023
  5. May 26, 2023
  6. May 24, 2023
  7. May 12, 2023
  8. May 16, 2023
  9. May 15, 2023
    • Georges Racinet's avatar
      revision: support fully qualified tag refs · 88981e79
      Georges Racinet authored
      There was even an old TODO about that
      88981e79
    • Georges Racinet's avatar
      FindCommit: initial Gitaly Comparison test · 364d32a1
      Georges Racinet authored
      This method of the `CommitService` had no Gitaly Comparison test,
      because it was implemented before these tests were even considered.
      
      Such a test will be the main testing of the upcoming RHGitaly
      implementation.
      
      It turns out that we found a missing case: tag in the fully qualified
      ref notation (chances are that clients woud rather use `RefService.FindTag`
      but one never knows.
      364d32a1
    • Georges Racinet's avatar
      Gitaly Comparison tests: avoid side effect in normalization · 82cdf442
      Georges Racinet authored
      In some cases where a message attribute is missing, our normalization
      would have replaced it with a default (empty) value, leading to
      false positives.
      
      This is for instance the case of an empty `FindCommitResponse` (no
      commit attribute) that it would convert to a `FindCommitResponse` with
      an empy `commit` attribute (part of actual tests in the process of
      being written).
      82cdf442
    • Georges Racinet's avatar
      Gitaly Comparison tests: refactor message_to_git · 01c92a1f
      Georges Racinet authored
      A separate method for each "attribute path" gives us a bit more
      clarity (early return if `recurse` rather than a boolean available
      from outside the inner loop), and more room for improvement.
      01c92a1f
  10. May 12, 2023
    • Georges Racinet's avatar
      Matching Gitaly version 15.6 · 803e3656
      Georges Racinet authored
      ## Protocol
      
      The protocol is the same in all Gitaly v15.6 versions,
      including the one used in GitLab v15.6.0-rc42. The protocol changes
      from 15.5 have no impact on HGitaly.
      
      ## Visible method changes
      
      The removal of the feature flag for `FindLocalBranches` has already
      been treated in the parent changeset.
      
      The only remaining change we have to match Gitaly 15.6 is very minor
      (error details wording).
      
      ## Feature flags change
      
      Two changes got our attention:
      
      - [convert_err_to_status](https://gitlab.com/gitlab-org/gitaly/-/issues/4587):
        is actually about internal implementation details on how to generate
        `INTERNAL` status code with less boilerplate
      - [tree_entries_new_page_token_format]: changes the token content only
        because it had collisions, notably in the face of empty blobs (see message of
        Gitaly commit eb82c328b2a3ca312724724eb9849bc25a3b0b24). Our own token
        is the blob id, but is based on path instead of content, so this is
        irrelevant to HGitaly. We'll have not to make the same mistake if some day
        we implement content-based addressing.
      803e3656
  11. May 14, 2023
    • Georges Racinet's avatar
      FindLocalBranches: removing old-style code and tests · 0d836d0f
      Georges Racinet authored
      The `simplify-find-local-branches-response` feature flag
      gets removed in the upcoming Gitaly 15.6
      0d836d0f
    • Georges Racinet's avatar
      Gitaly Comparison test harness: using the bundled Git · 2463d171
      Georges Racinet authored
      Done as a merge from stable branch, before the bump to
      Gitaly 15.6
      2463d171
    • Georges Racinet's avatar
      Gitaly Comparison test harness: always using the bundled Git · 8135fc70
      Georges Racinet authored
      Recent Gitaly versions come with two versions of Git (probably to
      test them in advance), and hence the path that we were trying is
      no longer valid.
      
      It is now mandatory of to the development setup (HDK) and to the CI base
      image to provide Gitaly with the bundled Git.
      The concept of "bundled Git" seems to go beyond vendoring, the
      compilation flag `WITH_BUNDLED_GIT=YesPlease` has the effect of
      putting self-sufficient Git executables in Gitaly bin directory, which
      the main executable knows how to find and select, provided that the
      `bin_dir` configuration item is correct (fixed in a preparation changeset).
      
      Using this bundled Git removes lots of confusion and timebombs
      such as the tests defaulting to a system-wide Git which suddenly can
      become outdated.
      8135fc70
    • Georges Racinet's avatar
      Gitaly Comparison test harness: cleanup refactoring · 5b697cb2
      Georges Racinet authored
      Since we only test one candidate `gitaly` executable, we don't
      need the loop.
      5b697cb2
    • Georges Racinet's avatar
      Gitaly Comparison test harness: fixing Gitaly bin_dir · 4250ce07
      Georges Racinet authored
      In case the `gitaly` executable is to be found in a subdirectory
      of `GITALY_INSTALL_DIR`, then that same subdirectory has to be
      provided as the `bin_dir` configuration item, so that Gitaly can
      find all its non-ruby bundled executables, notably the bundled Git.
      
      It was working until now because Gitaly falls back to a system-wide
      Git if the specified one is not found, whose version is acceptable
      in CI because the `heptapod-gitaly` base image is itself based on the
      `heptapod-base` image, which provides the Git version generally
      accepted by the target GitLab version.
      
      Since there is no doubt as to where the `gitaly` executable is supposed
      to sit, we're only testing one possibility (minimal change to keep the diff
      understable, to be refactored).
      4250ce07
  12. May 03, 2023
  13. May 01, 2023
  14. Apr 30, 2023
    • Georges Racinet's avatar
      workdir: customized roster lock for breaking logic as in reservation. · 5976d391
      Georges Racinet authored
      Again, we don't want the application to be locked for the wrong reason,
      e.g. lock held by a former process, but hostname has changed in the mean time.
      
      We also take the opportunity to use shorter timeout and delay
      betweeni locking attempts.
      
      The customatbility of the breaking logic and of the timeouts could be submitted
      UPSTREAM as options to Mercurial core, although it would be of limited interest to
      Heptapod right now: we cannot readily apply the same logic to other Mercurial locks,
      because HGitaly does not have exclusivity on them.
      Querying the system service manager (systemd or runit) about the Heptapod services
      would be a promising idea to do better, but we'd still have the issue that pushes
      over SSH would create standalone processes.
      5976d391
  15. Apr 26, 2023
    • Georges Racinet's avatar
      workdir: auto-release if held by an invalid process id · fb44e1f5
      Georges Racinet authored
      The rationale is that if a working directory is being held by a process (existing
      or not) that is not considered part of the HGitaly system service, it means that
      the process was killed before it could release the working directory and the pid
      is either not in use or has been reused for an unrelated process. The underlying
      assumption is that only HGitaly is supposed to make such working directories.
      
      This could be further expanded to the roster lock.
      
      Had to put `is_current_service_process` in a separate top level module
      instead of, e.g., `server.mono` to avoid import loop:
        server.mono imports services import workdir import server.mono
      
      We need to use pytest-cov's `cleanup_on_signal` again. This time,
      it's easier as the subprocess startup function is defined by the test itself.
      fb44e1f5
  16. Apr 25, 2023
    • Georges Racinet's avatar
      server.mono: covering the startup function · 57761166
      Georges Racinet authored
      The trick is that we fall precisely in the case where pytest-cov
      needs a hand: termination by signal (SIGTERM), hence we need the
      `cleanup_on_signal` method.
      
      Unfortunately, this does not work with pytest-7.3.1 and pytest-cov-4.0.0
      as the previously lax `test-requirements.txt` would install (detected
      on the `heptapod-gitaly` base image, which does not preinstall these),
      so we fix versions as they are in py-heptapod (hence also as they already
      were in HDK settups, since `py-heptapod` is installed first).
      
      Then, we need the server to actually be running to be sure that
      the handling has been done before we terminate.
      
      Still not covering the case of `SIGINT` while doing some
      Mercurial processing. We'd probably need to launch a long enough
      gRPC method for that.
      57761166
    • Georges Racinet's avatar
      Comparison tests: moved server full startup wait function to hgitaly pkg · 86e2b479
      Georges Racinet authored
      This makes it importable from the whole code base, and is already
      somewhat cleaner in the `tests_with_gitaly.rhgitaly` module.
      86e2b479
  17. Apr 24, 2023
    • Georges Racinet's avatar
      servicer: expose context manager for working directories on self · 15c56172
      Georges Racinet authored
      This wraps things together by passing down the proper root of working
      directories for the given repo (as prescribed in the Gitaly `Repository`
      message.
      
      The test is a simplified version of a service method creating changesets.
      15c56172
    • Georges Racinet's avatar
      servicer: split out relative path normalization · dc2defec
      Georges Racinet authored
      It has already been the case in Heptapod that relative path in
      Gitaly Repository messages were to be passed as if the repository
      was a Git repository (hence ending in `.git`) and then to be
      normalized as `.hg` for Mercurial operation. But this is due
      to the legacy hg-git based integration, and could not be necessary
      in some future.
      
      Meanwhile, we'll need to reuse the logic for working directories.
      dc2defec
    • Georges Racinet's avatar
      working directories: garbage collection · 081e8130
      Georges Racinet authored
      This is to be called as part of ordinary housekeeping.
      
      Another subject to be tackled on would be release of
      working directories held by defunct processes, which would
      introduce a more satisfactory breaking logic, also applicable
      to the roster lock.
      081e8130
  18. Apr 09, 2023
    • Georges Racinet's avatar
      working directories: inner pool handling · b3e7a172
      Georges Racinet authored
      This creates the pool, recorded as a "roster" file
      in the main repository.
      
      To obtain an exclusive working directory, callers should
      use the `working_directory` context manager only.
      
      The roster file is protected by a lock, with input/output
      occurring only at acquire and release times of working
      directories, allowing to acquire working directories freely
      when one is already held. The lock is for now provided by
      Mercurial standard utilities, but we will probably implement
      a more suitable one later on.
      
      A cleanup system for old working directories shall be implemented
      in a following changeset. It will be based on the recorded timestamp.
      b3e7a172
  19. Apr 28, 2023
Loading