1. 02 Nov, 2020 1 commit
  2. 01 Dec, 2020 1 commit
    • Georges Racinet's avatar
      GitLab::Shell: fixed reading Heptapod Version · d36b1d5b79e9
      Georges Racinet authored
      The previous change was only about the required version (the
      one recorded in the Rails checkout), and was still reading the
      upstream version file for the actually running GitLab Shell.
      
      Closes #375
      
      --HG--
      branch : heptapod
      d36b1d5b79e9
  3. 28 Nov, 2020 2 commits
    • Georges Racinet's avatar
      Mercurial native projects: backup default GitLab branch definition · 3038f0db68c4
      Georges Racinet authored
      Before the HGitaly2 milestone, the information is weirdly
      duplicated with Git, so that it doesn't systematically break
      if missing. But it does frequently and this is obviously needed.
      
      We don't need to differentiate between `hg` and `hg_git` repos,
      here, thanks to the `tar` flag to ignore missing files.
      
      --HG--
      branch : heptapod
      3038f0db68c4
    • Georges Racinet's avatar
      Repos backups: clearly identify Wikis in failure logs · 873470b91e12
      Georges Racinet authored
      With the generalization of the hashed storage, it's become
      less immediately visible that the failure is on a Wiki repo,
      to the point that I didn't notice that there was already a
      qualifier at the end of the line in restore case.
      
      Care is taken so that future merges with upstream can conflict
      (good) but not introduce new breakage in error treatment (very
      bad)
      
      UPSTREAM
      
      --HG--
      branch : heptapod
      873470b91e12
  4. 27 Nov, 2020 2 commits
  5. 25 Nov, 2020 3 commits
  6. 18 Nov, 2020 1 commit
  7. 21 Nov, 2020 1 commit
  8. 24 Nov, 2020 1 commit
    • Georges Racinet's avatar
      Setting allowed VCS types via API · e2b0e247a84a
      Georges Racinet authored
      it's necessary for functional tests to pass with
      `--heptapod-hg-native` on a fresh instance, since they aren't
      allowed by default.
      
      I'm not a fan of the comma-separated style, since that's using
      a different type for writes than what gets returned, but it seems
      to be the standard GitLab way. There's even a
      [cop for that](rubocop/cop/api/grape_array_missing_coerce.rb)
      
      --HG--
      branch : heptapod
      e2b0e247a84a
  9. 20 Nov, 2020 1 commit
    • Georges Racinet's avatar
      Re-separate GitLab and Heptapod Workhorse versions · 0fd9ec6fa027
      Georges Racinet authored
      The reason is that the RSpec launch sequence really
      installs GitLab Workhorse (that does not matter for
      the time being), using `Gitlab::Workhorse::version`.
      Of course if that is actually Heptapod Workhorse
      version we have a problem.
      
      In the helper (used only in the admin dashboard), it's
      taken from headers anyway, but still we provide
      an appropriate fallback method
      
      (grafted from 24ca3aba6354,
      this was a last-minute fix to let the CI do its job, that
      was validating the release)
      
      The question of really installing Heptapod Workhorse will
      stay irrelevant for a while: we would need to
      - launch tests that use Workhorse
      - that the difference between GitLab and
        Heptapod Workhorse actually matters.
      
      --HG--
      branch : heptapod
      0fd9ec6fa027
  10. 18 Nov, 2020 1 commit
    • Georges Racinet's avatar
      HgGitRepository: avoid exception on empty repo · dcc83c4464fc
      Georges Racinet authored
      As explained in the comment, we can have some conversion attempts
      while the repo is empty and has no map file.
      
      Specifically, this fixes fil creation from the Rails app on
      empty repositories.
      
      --HG--
      branch : heptapod
      dcc83c4464fc
  11. 17 Nov, 2020 1 commit
    • Georges Racinet's avatar
      BranchCheck: avoid transactional problem with Mercurial · 23bd45dee3c4
      Georges Racinet authored
      With Mercurial, many non-force pushes look like ones for GitLab
      because of obsolescence. On the other hand, the Mercurial backend
      has already applied its relevant protections against that.
      Also, really important: if this is called ultimately from a
      pre-receive hooks, the Mercurial transaction is still ongoing.
      Therefore ancestor? will raise an 'unknown revision' error,
      breaking everything.
      
      --HG--
      branch : heptapod
      23bd45dee3c4
  12. 16 Nov, 2020 1 commit
  13. 17 Nov, 2020 3 commits
  14. 16 Nov, 2020 2 commits
    • Georges Racinet's avatar
      HGitaly RefService client: two calls still going to Gitaly · d97efe17eb31
      Georges Racinet authored
      These were exceptions to the calling style. `find_branch` is the
      one interesting for our current goal: make the current functional tests
      pass.
      
      To UPSTREAM.
      
      --HG--
      branch : heptapod
      d97efe17eb31
    • Georges Racinet's avatar
      Branch acces checks: direct identification of legit merges · 948dac54b949
      Georges Racinet authored
      Closes #368
      
      To let merges done from the Rails application be checked
      against the distinct set of merge rules rather that push
      rules, GitLab looks for MRs matching the pre-receive changes by
      comparing with the merge commit sha that's always available
      (and set by the Repository Model) before the actual branch move.
      
      Because the Mercurial change can be atomic, we have no other
      option than passing a special parameter down from
      `Repository#merge` through the Mercurial backend, with the
      `HEPTAPOD_ACCEPT_MERGE_ID` environment variable.
      In turn the Mercurial process will add it to its pre-receive
      hook payload, and the target branch can be compared to switch
      from (direct) push to (Rails) merge mode.
      
      In the case of merges intended from the very beginning to be
      fast forwards, the commit sha is already known in advance,
      but we're using the same mechanism for consistency in the
      check.
      
      Finally, there was a good dose of wrestling with those RSpec
      examples that care about merging to a protected branch, because
      of the on-demand nature of instantiation.
      
      --HG--
      branch : heptapod-stable
      948dac54b949
  15. 15 Nov, 2020 3 commits
    • Georges Racinet's avatar
      Protected branches: deletion protection when no one allowed to push · 1290d9cfb0b7
      Georges Racinet authored
      As demonstrated by a functional test,
      Maintainers were still able to remove the protected branch even if
      push level was set to `NO_ONE`. The reason is the underlying 'web'
      protocol. The fix is harsh, but removal is a special case
      and the explicit message to unprotect the branch should be
      acceptable.
      
      --HG--
      branch : heptapod-stable
      1290d9cfb0b7
    • Georges Racinet's avatar
      Restore branch protection for Mercurial · b9859c4b49e1
      Georges Racinet authored
      The fix is simple enough and all functional tests are passing.
      This is the occasion to restore/adapt the RSpec tests for
      branch protections.
      
      This still have the unwanted consequence that some MR merges
      will fail, because the related permission checks (occurring
      in pre-receive API endpoint) won't find the matching MR.
      
      This will be handled in a subsequent changeset, for more
      on this, see #368.
      
      --HG--
      branch : heptapod-stable
      b9859c4b49e1
    • Georges Racinet's avatar
      Mercurial native mode: passing environment var to spawned `hg` · 8a49ca1ba8ba
      Georges Racinet authored
      Of course the value is `no` for `HgGitRepository` and `yes` for
      `HgitalyRepository`
      
      Closes #364 and leaving us with lots of small fixes
      to make it work, as the functional tests already told us.
      
      --HG--
      branch : heptapod
      8a49ca1ba8ba
  16. 14 Nov, 2020 3 commits
  17. 09 Nov, 2020 4 commits
    • Georges Racinet's avatar
      HGitayRepository: fixing Diff invocations · 92500bee7f38
      Georges Racinet authored
      Although they go trough `gitaly_commit_client` they actually
      call the `DiffService`, not currently implemented by HGitaly at
      all (obscure generic NoMethod gRPC error, hard to track which
      caller it was).
      
      We need to
      
      - use @git_storage to call the regular Gitaly
      - convert SHAs to Git SHAs
      
      This gives us our first working Commit page in the Web UI, but only
      for Commits without parents. In the (frequent, yes) general case,
      we still need at least BatchByOids to display the Commit parents list
      (must go partly through Markdown rendering). This last part will be
      done after 0.17.0rc1
      
      --HG--
      branch : heptapod
      92500bee7f38
    • Georges Racinet's avatar
      Repository: SHA conversion with fallback · 2605fa2b1a16
      Georges Racinet authored
      This is so often needed that a separate method
      makes sense. Done at generic level for widest
      compat (only `HGitalyRepository` can return `nil`
      by the way).
      
      --HG--
      branch : heptapod
      2605fa2b1a16
    • Georges Racinet's avatar
      HGitalyRepository: fix cached_exists? and conversions for vcs-cloned · b188426a2147
      Georges Racinet authored
      The `#cached_exists` method can issue a Gitaly call. Of course if
      it's running for a temporary vcs-cloned as `GitalyCommitClient` uses,
      that makes for double VCS prefixing and it fails.
      
      We correct this, and hence its various conversion callers by
      subclassing and doing the right thing explicitely.
      
      --HG--
      branch : heptapod
      b188426a2147
    • Georges Racinet's avatar
      Blob,Tree: incoming "sha" can be any revision · a3ed118369a6
      Georges Racinet authored
      That wasn't obvious from the naming, but basic actual browsing
      in the UI failed because these could be just 'branch/default'
      
      --HG--
      branch : heptapod
      a3ed118369a6
  18. 08 Nov, 2020 4 commits
    • Georges Racinet's avatar
      HGitalyRepository: Git-based list_last_commit_for_tree · e77cd6a5f21f
      Georges Racinet authored
      Arguably, this method has nothing to do with Trees – it
      uses paths, not actual Git Tree objects and could therefore
      be implemented directly in HGitaly.
      
      To be frank, if we had come to that conclusion earlier we
      probably would have made it that way, but now that this code
      is there, we can as well turn to other pressing matters.
      
      A consequence is the `hpd_back_convert!` method, but it could
      turn out to be useful for something else. Of course, like its
      cousins in the `Tree` and `Blob` class, we'd prefer not to
      need it at all.
      
      --HG--
      branch : heptapod
      e77cd6a5f21f
    • Georges Racinet's avatar
      Tree: transitional implementation for Mercurial native repo · e77d525616cd
      Georges Racinet authored
      Similar to what's been done previously for Blob, we switch back
      to Git mode and the point is the the Tree OID is the Git one,
      while the corresponding Commit ID is the Mercurial one.
      
      --HG--
      branch : heptapod
      e77d525616cd
    • Georges Racinet's avatar
      Blob: transitional implementation for native Mercurial repos · 6db10a1faece
      Georges Racinet authored
      Git blobs are typically addressed by their OID, meaning that
      we only have to address the few methods that allow to find that
      OID from a Commit and a relative path.
      
      For example, `Gitlab::Workhorse` returns a special payload including
      the blob OID to the Workhorse server in front of the Rails app,
      allowing it to serve the full blob content directly from Gitaly.
      
      We are systematically calling `gitsha_from_sha` and its converse
      here, hoping it won't be too much of an overhead.
      
      The changes to the `Blob` method are tested through methods of
      `Repository` (inherited by `HgitalyRepository`) that in turn call
      them. Comparison with the case of `HgGitRepository` is provided
      for reference.
      
      --HG--
      branch : heptapod
      6db10a1faece
    • Georges Racinet's avatar
      Repository classes: gitsha methods, preparing for HGitaly1 · 312149ad6346
      Georges Racinet authored
      For the integration of native Mercurial repositories to reach
      milestone HGitaly1 (mixed mode still relying on Git conversion for
      some data access), we will need to be able to convert internally
      to Git SHAs. Of course, this conversion is trivial for
      `Git::Repository` and `Mercurial::HgGitRepository`.
      
      Various classes in the `Gitlab::Git` module will make use of
      these methods, `Blob` and `Tree` being the first to come to mind.
      
      --HG--
      branch : heptapod
      312149ad6346
  19. 07 Nov, 2020 2 commits
    • Georges Racinet's avatar
      gitaly_client: plugging in HGitaly in CommitService · d3f9a208d676
      Georges Racinet authored
      This is more complicated than what we've done earlier for
      `RefService`:
      
      - Upstream passes down storage to `GitalyClient.call` as
        `@repository.storage`. We don't want to change all methods
        of CommitService because it wouldn't apply readily to new
        methods from upstream.
        Hence we actually re-instantiate `@repository`,
        with the VCS-qualified storage, with the new `vcs_storage_clone`
        method. On the side, this also requires a small change
        in HgGitRepository to make the value admissible.
      - only a subset of Commit service methods are actually involved
        in our first round of HGitaly implementation (getting rid of
        Git SHAs, see HGitaly1 milestone). We're deciding here that
        the other methods will use directly the original storage name,
        that we store as a new instance variable, `@git_storage`.
        In particular, we're trying to avoid the implementation of the
        various methods for Commit content (files, trees, diffs…) for
        the HGitaly1 milestone. This part is WIP (we'll need to
        convert back to Git revisions to complete it).
      - Finally, this hack also requires to make sure that the resulting
        Commit instances refer to the original Repository, so that further
        (H)Gitaly calls done from these Commits don't reuse a VCS-qualified
        storage, which would defeat the very purpose of separation.
      
      This is all very hacky. The only thing that makes it admissible is
      that it is temporary scaffolding. For the HGitaly2 milestone, all
      methods will go to HGitaly, and perhaps we can convince upstream to
      use a `@storage` instance variable here for the sake of uniformity.
      
      --HG--
      branch : heptapod
      d3f9a208d676
    • Georges Racinet's avatar
      HGitalyRepository: comments about SHA/ID conversion methods · 3639ef277c7e
      Georges Racinet authored
      --HG--
      branch : heptapod
      3639ef277c7e
  20. 01 Nov, 2020 1 commit
    • Georges Racinet's avatar
      Displaying Heptapod versions in admin panel · 10ce53f8da96
      Georges Racinet authored
      While the Shell version is directly taken from the VERSION file
      (now HEPTAPOD_SHELL_VERSION), the Workhorse version is by
      default taken from the HTTP header that Workhorse adds, ultimately
      set at build time from the latest tag, with additional information
      if built on a changeset with no tag.
      We only change the way the header is parsed, to take into account
      the additional `heptapod-` prefix it has.
      
      --HG--
      branch : heptapod
      10ce53f8da96
  21. 29 Oct, 2020 1 commit
  22. 30 Oct, 2020 1 commit