1. 28 May, 2020 2 commits
    • Georges Racinet's avatar
      Squash: honouring squash_in_progress? · cfd7b187b509
      Georges Racinet authored
      This works with the same principle as rebase_in_progress?,
      see parent commit.
      branch : heptapod
    • Georges Racinet's avatar
      Rebase: honouring rebase_in_progress? · d7a0e368784b
      Georges Racinet authored
      We name the temporary working directory according to the given
      rebase_id and the question whether it's in progress boils down
      to the workdir being there.
      Ideally, some other kind of persistent storage should be used
      immediately, but we can use this as a better approximation for
      the time being. In upstream GitLab, this is handled by Gitaly,
      whose rebase RPC call receives the `rebase_id` and then answers
      the question. Later on HGitaly will do the same.
      branch : heptapod
  2. 27 May, 2020 1 commit
    • Georges Racinet's avatar
      Mercurial rebase for topics · 5f6de060df79
      Georges Racinet authored
      Closes #280
      This is enough for merge requests based on topics, which
      currently would display a 'rebase' button if the method is
      For now we add a hard check that the branch being rebased is
      related to a topic, that makes for a simple revset.
      As in the case of squashs, the protection against concurrent
      rebases seems at first sight to be unneeded (the rebase itself
      is atomic enough, and a subsequent would fail because it'd create
      instabilities) but because of #277 we have to be cautious
      with concurrency and avoid potential thundering herds of shares.
      This will be done for both squash and rebase in a follow-up that
      would simply test for presence of a share.
      branch : heptapod
  3. 28 May, 2020 6 commits
  4. 23 May, 2020 1 commit
  5. 19 May, 2020 1 commit
  6. 23 May, 2020 2 commits
    • Georges Racinet's avatar
      Temporary workdirs: switching to the closure style · 8211911f4bca
      Georges Racinet authored
      This is better style and it makes sense for the method that
      create the temporary working directory to take care of its
      cleanup. Overall, we lower the risk to forget an `ensure`
      We change the name of the method in the same move because
      - it's consistent with what the method does
      - it helps us being sure there isn't a leftover call to
        the old `hg_new_workdir` method.
      branch : heptapod
    • Georges Racinet's avatar
      Cleaning up temporary workdirs (shares) if creation fails · ac2102a384f3
      Georges Racinet authored
      Severe leaks of temporary workdirs are reported in #277.
      This change removes a potential source, not claiming to be a
      complete solution: if the update to revision failed, it would
      leave a directory behind, that maybe could be large already.
      That is because the begin … ensure clauses typically happen
      after the return from `hg_new_workdir`
      branch : heptapod
  7. 22 May, 2020 2 commits
    • Georges Racinet's avatar
      wikis: making backups work · 3f8f52d8d321
      Georges Racinet authored
      Closes #275
      We replicate the same logic we've been doing earlier
      for the main repos on Heptapod 0.13: conditional for both
      the Git and the Mercurial repo.
      delegating `vcs_type` and `mercurial?` makes `ProjectWiki` close
      enough to `Project` for the inner method to work (as it does on
      upstream GitLab)
      branch : heptapod
    • Georges Racinet's avatar
      wiki backups: avoid showing of main project config · bfd430d98774
      Georges Racinet authored
      Because `disk_path` doesn't have the `.git` extension,
      the removed `.ext()` would have been called on paths such
      as `my_project.wiki`, returning `my_project.hg` instead of
      the expected `my_project.wiki.hg`.
      branch : heptapod
  8. 21 May, 2020 2 commits
  9. 19 May, 2020 2 commits
    • Georges Racinet's avatar
      Unprotect topic branch that's not default any more · c1f43664c49e
      Georges Racinet authored
      If it stays protected, it won't be pruned when it has to,
      and that's quite unnatural.
      It's ok if it doesn't get pruned immediately, a subsequent push
      to Heptapod will do it.
      We didn't want to go through a separate service, because this
      should be unconditional, and in particular not depend onto who
      did the push that created the new default branch. Very usually,
      that should be the project owner right after creation, and currently
      creating a named branch (hence changing the default) entails be
      Maintainer anyway, but let's be a bit more cautious than that.
      The worst that can happen is a topic branch to be unprotected when
      it shouldn't have in the first place.
      branch : heptapod-stable
    • Georges Racinet's avatar
      Change automatically GitLab default branch if it's a topic · c538af777431
      Georges Racinet authored
      This is a practical solution for #126.
      Because we fear that a non empty repo without a default branch would
      be potentially blocking for GitLab, we prefer to allow the default
      branch to be a topic, provided we override it as soon as a named branch
      is available.
      In GitLab the source of truth about the default branch is the
      repository HEAD.
      One could think that the purpose of BranchHooksService.branch_create_hooks
      is to set the default branch, because it will in turn call
      `project.change_head`. But actually that's just repetition of what's
      already in cache, and the comparison to `nil` in `default_branch?`
      seems to be useless: currently something always writes at least something
      in HEAD, and that something doesn't seem to be part of the Rails app (
      not going through `Git::Repository.write_ref`). It's possible that
      Gitaly would actually check HEAD and set it to an existing branch if
      it is dangling, making this a side effect of a read operation.
      So we decided that Heptapod export code to Git would take care
      of setting a sensible HEAD in case
      - it points to a ref that doesn't exist (refs/heads/master on an empty repo)
      - it points to a topic but the current transaction creates a named branch head
      All we need then is to force GitLab to expire the caches involved, before handing
      over the changed refs to the appropriate services. That's what this patch
      branch : heptapod-stable
  10. 15 May, 2020 1 commit
  11. 11 May, 2020 2 commits
    • Georges Racinet's avatar
      Wikis: add RST to possible formats · ba560959d518
      Georges Racinet authored
      Many of the wikis that we'll need to import from Bitbucket are
      written in reStructuredText (RST), that Gollum supports out of the
      box if the conversion utilities, such as `rst2html` are installed.
      It is important to use the official symbol for the format, `:rest`
      even though extension based rendering would work. For instance that
      is needed for the edit page to keep that format by default instead
      of selecting Markdown for lack of understanding.
      Unfortunately, at this point this is not enough for the preview to
      work in RST, making it only a second class citizen.
      branch : heptapod
    • Georges Racinet's avatar
      Wikis: write actions from the UI. · 1887d8ecf727
      Georges Racinet authored
      We need notably the converse of the hack exposing `branch/default`
      as `master`: the higher parts of Rails will write to
      `wiki.default_branch` which we had to leave at `master` for the
      display to work.
      branch : heptapod
  12. 09 May, 2020 2 commits
    • Georges Racinet's avatar
      Fixed response to workhorse for deploy tokens · 58402491a702
      Georges Racinet authored
      These simply don't have an `email` method. This produces
      a 500 error, which is not what has been reported for #243
      on Heptapod 0.12
      This change is enough to make deploy tokens work on the
      future Heptapod 0.13.
      branch : heptapod-stable
    • Georges Racinet's avatar
      workhorse payload: fixed raise of HgAccess::Unauthorized · 90e5adb7844e
      Georges Racinet authored
      The name of the exception to raise was just wrong. This
      didn't open any door, but produced a 500 error instead of
      the natural 403.
      As the comment explains, reaching this last layer of protection
      is (at least on Heptapod 0.12) to be considered a bug. Indeed
      we found this typo on the future Heptapod 0.13, because of such
      a bug after refactor of `HgAccess`. It's possible that some of
      the finer access control reenabled on Heptapod 0.13 make this
      case less exceptional
      branch : heptapod-stable
  13. 08 May, 2020 1 commit
  14. 06 May, 2020 1 commit
  15. 04 May, 2020 3 commits
    • Georges Racinet's avatar
      Restablish Git hooks from Rails hg operations, triggering pipelines · 3bb507357531
      Georges Racinet authored
      Closes #145
      Pipelines are actually always created from the `post-receive` hook.
      Long ago, we disabled Git hooks in the case of operations like commit
      and publication done from the Rails app. At the time, it was necessary,
      probably because of the race conditions related to pruning topics and
      maybe more. In Heptapod 0.12, we've had to solve most of these anyway.
      We still had a race condition in branch deletions for topics. Since
      the merge is examined both by two workers for branch updates (target and
      source), it was possible for example that
      `PostMergeService`, running from the target branch would delete the
      source topic branch in concurrency with `RefreshService`. Both go through
      `DeleteBranchService` which does a 2 step pass: existence test, then
      deletion. In the deletion we don't have a specific Gitaly error to ignore
      the case of the branch being already deleted, so we do it from the callers.
      In case we perform several operations (such as commit+publish
      for a merge), we avoid launching two set of hooks in fast succession: it
      would be a waste of resources at best (in the merge example,
      twice the same pipeline for the target branch).
      In principle, authentication being already guaranteed, we could skip
      the `pre-receive` hook, but we didn't dare to at this point.
      branch : heptapod-stable
    • Georges Racinet's avatar
      Mercurial repo: make it possible to fire Git hooks · 5b20c8a3cb06
      Georges Racinet authored
      For now, those hooks are disabled, but they wouldn't work
      for lack of `HEPTAPOD_PROJECT_ID` in Heptapod 0.12 or
      `GL_REPOSITORY` in Heptapod 0.13
      branch : heptapod-stable
    • Georges Racinet's avatar
      Pipelines: avoid creation on a trivial branch update · 4fd7b37e7bb5
      Georges Racinet authored
      Our Mercurial backend sends identity changes for topics, so that
      the prune (Git branch deletion) can be done safely after the merge
      detection by the Rails app.
      We don't want a pipeline to be created for this: at best it's a
      waste of resources, potentially delaying an important build/deploy from
      the target branch, and it can fail if the topic head has become
      branch : heptapod-stable
  16. 26 Apr, 2020 1 commit
    • Georges Racinet's avatar
      Wiki: supporting pull/push with HTTP and SSH · db68c092516a
      Georges Racinet authored
      After all the refactoring preparations, all that was left
      was to provide and register the HgAccessWiki class.
      This needs changes in py_heptapod and Heptapod Workhorse
      to actually work. Most notably, the GL_REPOSITORY environment
      variable (WSGI or system) must now be passed over transparently,
      since it's `wiki-${project-id}` in the case of wikis.
      With another change in py_heptapod to duplicate `branch/default`
      as `master`, the web UI can display the wikis correctly, but
      can't update them.
      We'll have to live with that hack for some time, because the
      wiki system is based on Gollum, itself partly exposed through
      Gitaly, which means lots of things are (as good as) hardwired to Git.
      branch : heptapod
  17. 12 May, 2020 1 commit
    • Georges Racinet's avatar
      API projects creation: accepting vcs_type argument · f53cf12ac6ea
      Georges Racinet authored
      The database default is currently 'hg_git' because that was at the time
      of the introduction of this field the simplest way to migrate previously
      existing instances.
      At some point in the future, we'll switch it back to 'git', though, so
      that seeding for upstream unit tests can work. So we're introducing a
      default at the API level, as a constant in `Gitlab::VCS`
      branch : heptapod
  18. 13 May, 2020 1 commit
    • Georges Racinet's avatar
      Making backups work again for Heptapod 0.13 · 6eb63bc56515
      Georges Racinet authored
      In the move from GitLab 12.2 to 12.3, most of the backup/restore
      responsibility has been offloaded to Gitaly. The merge of Heptapod
      from 12.2.0.pre to 12.3.5 did not conflict, but the result didn't
      make any sense.
      This changeset is inevitably ugly, it makes much more sense to
      diff it against v12.3.5.
      We take the opportunity to introduce conditionals based on the
      `project.vcs_type` to prepare for the later HGitaly version and
      dual VCS support.
      The case of wikis, a new feature in Heptapod 0.13 will probably have
      to be treated separately, once we have functional tests for them.
      branch : heptapod
  19. 12 May, 2020 1 commit
    • Georges Racinet's avatar
      Mercurial::Repository.multi_action normalize exception raise · 6fc21d2a7dcb
      Georges Racinet authored
      Higher level methods in GitLab expect Git::Index::IndexError,
      so we have to give them some. For instance, the API will catch
      these and provide appropriate 400 errors from the message.
      We decided to settle for the same messages as provided by Gitlab
      (actually by Gitaly through the `user_commit_files` request of
      the OperationService), even though some are a bit ambiguous,
      especially in the case of renames.
      This doesn't change the fact that the hg calls could still fail,
      even though many cases are covered by the a priori checks. We'll
      still raise HgError for unexpected failures.
      branch : heptapod
  20. 11 May, 2020 7 commits
    • Georges Racinet's avatar
      Mercurial::Repository.multi_action distinction create/update · 52e78e85ca23
      Georges Racinet authored
      What they are supposed to do is written in comment, and
      actually obtained by looking at the Git implementation.
      Notably, subdirectory creation is implicit in create, etc.
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository.multi_action start_branch_name defaulting · 6a19a5f00cd0
      Georges Racinet authored
      Up to now, our implementation behaved as if it were actually
      required, but some callers (e.g., Wikis::CreateAttachmentService)
      don't pass it, and defaulting to `branch_name` is the right way
      to go.
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository.multi_action create_dir · 701a767a9316
      Georges Racinet authored
      It had previously not been implemented
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository.multi_action logging tweaks · e7f932486f0e
      Georges Racinet authored
      Since the content is part of the action Hash, it's important
      to avoid it. Name and path will be enough.
      Also, logging `share_path` is useful to debug (before it
      is destroyed, obviously)
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository multi_action refactor for FS independence · 52136082f11a
      Georges Racinet authored
      The contract for multi_action is that incoming path are specified
      the "web" way (as obvious e.g., in Wiki operations), meaning with
      forward slash as separator.
      In the Git case for GitLab 12 they are passed over as-is to Gitaly.
      In our local Hg implementation, we should now convert with `File.join`
      The odds of using Heptapod on Windows are low, but it's good hygiene.
      Also, factorizing in the `file_path` and `abspath` variables for
      further developments (separation of `create` and `update`, notably).
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository Fixed writing of binary files · ce44e19166a7
      Georges Racinet authored
      This is used, e.g, in Wiki attachments.
      With `File.write` (implicitely text), the binary string
      would give a conversion error to Encoding.default_external
      (usually UTF-8) whereas it goes through in `wb` mode, and I
      checked that actual unicode content is well supported.
      branch : heptapod
    • Georges Racinet's avatar
      Mercurial::Repository using logprefix in multi_action · 1347a8595bdf
      Georges Racinet authored
      The whole point of this prefix is to make logs understandable.
      The mere fact that `Rails.logger` is deprecated in GitLab will
      be adressed another day.
      branch : heptapod