1. 16 Jul, 2020 4 commits
    • Georges Racinet's avatar
      HgGitRepository: make get_hg_heptapod_config work if empty · e70d3a5eb8d7
      Georges Racinet authored
      Because `hg config SECTION` exits with code 1 if no matching
      setting have been found, we previously had an error if this
      method (and the corresponding API endpoint) was called before
      any local setting is set in the `[heptapod]` section.
      
      This can be considered part of #299, too
      
      Also fixed the exception raising, Ruby has its quirks, not the
      same as Python's.
      
      --HG--
      branch : heptapod-stable
      e70d3a5eb8d7
    • Georges Racinet's avatar
      HgGitRepository: don't crash if no managed hgrc exists yet · c23adf095138
      Georges Racinet authored
      Closes #299
      
      We'll return content to be `nil` in that case, which will be
      conveyed as `null` in the JSON API.
      
      --HG--
      branch : heptapod-stable
      c23adf095138
    • Georges Racinet's avatar
      HgGitRepository: taking Mercurial repo size into account · 2d5d3aff9555
      Georges Racinet authored
      Closes #314
      
      Previously only the Git repository was considered.
      
      Note that this is cached in the `projects_statistics` table,
      and can be invalidated by doing
      
      ```
         project.statistics.refresh!
      ```
      
      This seems in turn to be triggered by `UpdateProjectStatisticsWorker`,
      iself called by `ProjectCacheWorker`, from the likes of `PostReceiveWorker`
      etc.
      
      --HG--
      branch : heptapod-stable
      2d5d3aff9555
    • Georges Racinet's avatar
      Bumping Mercurial to 5.4.2 · 79afcc1f4c6a
      Georges Racinet authored
      --HG--
      branch : heptapod-stable
      79afcc1f4c6a
  2. 13 Jul, 2020 1 commit
    • Georges Racinet's avatar
      Bumping heptapod-stable branch to 0.14 series · 2914b99f50b2
      Georges Racinet authored
      This means that releases will be done by merging `heptapod-stable`
      instead of the `heptapod` branch into `heptapod-0-14`.
      
      From now on, the `heptapod` branch will be receive developments and
      upstream merges for Heptapod 0.15.
      
      --HG--
      branch : heptapod-stable
      2914b99f50b2
  3. 11 Jul, 2020 4 commits
    • Georges Racinet's avatar
      pre-receive, Git SSH: a `vcs` param for access class dispatching · 4986a7c018e0
      Georges Racinet authored
      Closes #295
      
      Before this change, both the SSH client operations for Git and
      the pre-receive hook sent by internal Mercurial processes would
      be validated by a `Gitlab::GitAccess` instance, and we were
      using coincidental data to distinguish the cases.
      
      We introduce a new `vcs` param for the `/allowed` internal API
      endpoint and have it control the access checker class: `HgAccess`
      if its value is `hg` and still `GitAccess` in all other cases
      (including if it's not supplied).
      
      This means that internal Mercurial process will have to pass
      a valid Mercurial write action to sanction pushes, i.e `unbundle`
      instead of `git-receive-pack`.
      
      The actual check can now be made much more logical: refuse all
      operations with Git on a Mercurial repository.
      
      --HG--
      branch : heptapod
      4986a7c018e0
    • Georges Racinet's avatar
      Renamed HgAccess::UnauthorizedError into ForbiddenError · 3ee6ed3883b9
      Georges Racinet authored
      Closes #296
      
      --HG--
      branch : heptapod
      3ee6ed3883b9
    • Georges Racinet's avatar
      HgHttpController: frozen string literals · 83a4b62d029c
      Georges Racinet authored
      For consistency with the GitLab code base, and me might even
      gain a few milliseconds thanks to it.
      
      --HG--
      branch : heptapod
      83a4b62d029c
    • Georges Racinet's avatar
      Moved HgHttpController to the Repositories module · 363aa34dc98f
      Georges Racinet authored
      Closes #293
      
      --HG--
      branch : heptapod
      363aa34dc98f
  4. 09 Jul, 2020 2 commits
    • Georges Racinet's avatar
      development: fixing import of testing Git projects · 99d9d4c772ce
      Georges Racinet authored
      With `hg_git` being the default `vcs_type`, of course the bare
      import of Git repositories wouldn't work.
      
      With this change the whole "seeding" part of the HDK setup works
      
      We still have to decide whether we'll make `git` the defaut value
      again in the database in a later Heptapod version. This case here
      is at the service level.
      
      --HG--
      branch : heptapod
      99d9d4c772ce
    • Georges Racinet's avatar
      Fix background jobs startup in HDK context · 680c6aecc56c
      Georges Racinet authored
      With the recent versions of the GDK, those on which HDK is
      currently being reboostrapped, the `Rails` object if not
      available in the startup sequence of the background jobs.
      
      Obviously from how the `Gitlab` module has evolved, the `root`
      method should simply be used instead of `Rails.root`.
      
      --HG--
      branch : heptapod
      680c6aecc56c
  5. 26 Jun, 2020 1 commit
  6. 25 Jun, 2020 7 commits
    • Georges Racinet's avatar
      Bitbucket import: reimporting issue attachments · 378ca92b5e5a
      Georges Racinet authored
      We must content ourselves of a weak detection system, based
      on Markdown links in the issue description.
      
      This will prevent us messing up issues that do have attachments,
      but it's not really reimporting precisely those attachments that
      are missing.
      
      In short it's good enough mostly for projects that have been
      imported before import of attachments was even possible.
      
      --HG--
      branch : heptapod
      378ca92b5e5a
    • Georges Racinet's avatar
      Wikis: support MediaWiki format · 5f191760997b
      Georges Racinet authored
      The GitLab version of Gollum performs a regexp on extension that
      accepts either .mediawiki or .wiki.
      
      Closes #298
      
      --HG--
      branch : heptapod
      5f191760997b
    • Georges Racinet's avatar
      Bitbucket import: robustness in comment.author · 76bebd66f7fa
      Georges Racinet authored
      There are cases where `user` can surprisingly be `nil`, and
      it's probably not worth failing the whole issue or PR for that.
      
      --HG--
      branch : heptapod
      76bebd66f7fa
    • Georges Racinet's avatar
      Typo in comment · 4fb276a63ae3
      Georges Racinet authored
      --HG--
      branch : heptapod
      4fb276a63ae3
    • Georges Racinet's avatar
      Bitbucket importer: catch error for each attachment · 7ff2587a62b2
      Georges Racinet authored
      Previous version was catching errors for all attachments of a given
      issue. It really happens that only one attachment is problematic among
      several of them.
      
      --HG--
      branch : heptapod
      7ff2587a62b2
    • Georges Racinet's avatar
      Bitbucket importer: resist one case of failed attachment · ab946faab5ce
      Georges Racinet authored
      Even with `import_issues_attachment()` catching errors, it's possible
      that the result `Hash` doesn't have a `:markdown` key (would typically
      have an `:error` key)
      
      --HG--
      branch : heptapod
      ab946faab5ce
    • Georges Racinet's avatar
      Bitbucket importer: logs harmonization · 809b771551e0
      Georges Racinet authored
      Rails.logger may be deprecated, but is still a convenient way to
      be really sure to catch messages provided one searches for them explicitely
      
      --HG--
      branch : heptapod
      809b771551e0
  7. 23 Jun, 2020 2 commits
    • Georges Racinet's avatar
      Merged Bitbucket reimport from heptapod-stable · 3dbf737d214e
      Georges Racinet authored
      for all_queues.yml, ran the rake command listed in the file
      itself. We'd probably want to tweak that further the match
      the weight given in sidekiq_queues
      
      --HG--
      branch : heptapod
      3dbf737d214e
    • Georges Racinet's avatar
      Bitbucket import: reimport capability · a7ff2a2d63d2
      Georges Racinet authored
      If using the "free input" form, it's now possible to
      target an existing project, and the importer will try
      and import all missing Issues and Pull Requests.
      
      --HG--
      branch : heptapod-stable
      a7ff2a2d63d2
  8. 22 Jun, 2020 1 commit
    • Georges Racinet's avatar
      Bitbucket import: fixed check that provider is configured · 60579874d98b
      Georges Racinet authored
      At least on GitLab 12.10, there is a precheck for provider
      configuration, that needed adaptation for the Heptapod specific
      `bitbucket_import` alias.
      
      Also found out that we have a tangle of problems with generated
      routes and expected controller names in case a `bitbucket_import`
      but no `bitbucket` is configured. Just reporting an issue telling
      users to have either only `bitbucket` or both will be enough for
      now.
      
      --HG--
      branch : heptapod
      60579874d98b
  9. 21 Jun, 2020 3 commits
  10. 15 Jun, 2020 4 commits
    • Georges Racinet's avatar
      Removed `ignore_git_error` parameter of Branches::DeleteService · 4e7f76755757
      Georges Racinet authored
      This Heptapod specific parameter had been introduced because of
      the inherent race between workers that analyze MRs from the source
      branch and from the target branch points of views.
      
      Now that no GitLab branch deletion can occur in the Rails app due
      to post merge actions in MRs, we don't need it any more.
      
      --HG--
      branch : heptapod
      4e7f76755757
    • Georges Racinet's avatar
      Removed code to prune topic branches GitLab side · 356cd8833e81
      Georges Racinet authored
      Since Heptadpod 0.13.1, the pruning occurs right from the Mercurial
      process, hence that Rails code is at best a waste of resources.
      
      We didn't dare to remove it in the 0.13.x, but we could have to,
      as it could itself be a source of bugs.
      
      --HG--
      branch : heptapod
      356cd8833e81
    • Georges Racinet's avatar
      Fixing some lacks of post receive branch analysis · c71a5000fbee
      Georges Racinet authored
      It's rather obvious that the point of the `merge_request_branches`
      parameter passed to `BranchUpdateService` is to query open MRs
      once for all changes of a given type, hence avoiding lots
      of useless queries.
      
      But it's really wrong to cache the results of `merge_request_branches_for`
      when it's called in a loop with different changes according to action types:
      if there are both removals and pushes (frequent in Mercurial, quite rare in
      Git unless pushes are done by some bots), only the first iteration of
      the loop will pass a meaningful set of branches to `BranchUpdateService`,
      triggering early returns so that MRs don't get updated.
      
      The functional `test_mergerequest_api_stacked_publish_first` triggers such
      bugs systematically: the pruning of the branch for the first (published topic)
      goes together with the move of the target branch, but is evaluated first,
      resulting in an empty set in `merge_request_branches`, so that the second
      MR for `branch/default` is not refreshed.
      
      --HG--
      branch : heptapod
      c71a5000fbee
    • Georges Racinet's avatar
      Merged heptapod-stable into heptapod · 8d10dce3af1a
      Georges Racinet authored
      This brings in the new post-receive system, with separate
      prune reasons, which is more suited anyway for Mercurial to
      go native.
      
      One test fails after this, test_mergerequest_api_stacked_publish_first,
      and we'll commit the fix separately for easy reference, because that's
      probably an upstream GitLab bug
      
      --HG--
      branch : heptapod
      8d10dce3af1a
  11. 14 Jun, 2020 7 commits
    • Georges Racinet's avatar
      Post-receives for repository files and merges with changeset · 376d59c06826
      Georges Racinet authored
      Post-receive notificiations are necessary to trigger pipelines and
      other post branch update actions that must be taken on the GitLab
      side
      
      Previously, we enabled them in merges with changeset performed by the
      Rails app itself, but we had only the publication step send them
      to avoid unnecessary consequences like redundant pipelines, but that
      was wrong: if there's already an added commit on `branch/default`
      a publication doesn't trigger any change on `branch/default`. The
      only post-receive it can send is a topic branch prune.
      
      --HG--
      branch : heptapod-stable
      376d59c06826
    • Georges Racinet's avatar
      Read the Docs: fixing and enabling connection test · 8843a066c5ae
      Georges Racinet authored
      Closes #289
      
      Turns out the test data is perfectly suitable, because it uses the
      real default branch name of the project, BUT it is a regular hash
      whose keys are symbols. Hence our `push_data["ref"]` wouldn't work
      for it
      
      On the other hand, real push data is a `HashWithIndifferentAccess`,
      meaning that we can use symbols in both cases.
      
      Found that logging the ref only after all parsing has succeeded was
      pointless, replaced by resulting Mercurial branch.
      
      --HG--
      branch : heptapod-stable
      8843a066c5ae
    • Georges Racinet's avatar
      Heptapod now based on GitLab 12.10 · 5ebe93d80847
      Georges Racinet authored
      More precisely, the revision chosen for the `heptapod` branch
      is `7e939fce5f42`, common ancestor of the 12-10-stable and
      13-0-stable branches.
      
      --HG--
      branch : heptapod
      5ebe93d80847
    • Georges Racinet's avatar
      Mercurial imports: deferring Git config fixup · 29e5e14fbc44
      Georges Racinet authored
      In `create_service`, the Git repo does not exist yet when
      `project.repository_config` is called, so we defer it to
      the end of the import process.
      
      --HG--
      branch : heptapod
      29e5e14fbc44
    • Georges Racinet's avatar
      HEPTAPOD_CONTRIBUTING: branch layout for Workhorse and Shell · ba8be575eaf7
      Georges Racinet authored
      --HG--
      branch : heptapod
      ba8be575eaf7
    • Georges Racinet's avatar
      internal API: fixed log_user_activity call in Mercurial endpoint · c803b6199c62
      Georges Racinet authored
      The call from the Git endpoint passes `actor.user`, not `actor`.
      
      Checked through internal helpers and the underlying
      `Users::ActivityService` that a `User` instance is indeed
      expected.
      
      --HG--
      branch : heptapod
      c803b6199c62
    • Georges Racinet's avatar
      Passing vcs_type to GitlabShellWorker and fixing repo removal · 1f9e56d13b01
      Georges Racinet authored
      Because sidekiq workers don't seem to take named arguments, we
      pass it as the last required positional one and convert it to
      named argument in the actual call to `Gitlab::Shell`.
      
      Then it was simpler just to let `Gitlab::Shell#rm_namespace`, the
      only method not taking a `vcs_type` named arg to accept it for
      uniformity.
      
      All call sites for GitlabShellWorker are updated. There aren't so
      many, due to the general deprecation of `Gitlab::Shell`, also a
      reason not to worry about new callers appearing in future versions.
      
      Finally, the `remove` method wasn't even implemented in
      `HgGitRepository`.
      
      --HG--
      branch : heptapod
      1f9e56d13b01
  12. 13 Jun, 2020 3 commits
    • Georges Racinet's avatar
      Fixed Mercurial repository HTTP and SSH URLs. · 90c369ed30f8
      Georges Racinet authored
      The generating logic was moved to the separate `HasRepository`
      concern, perhaps another change due to the introduction of
      snippet repositories.
      
      Rather than introducing Mercurial knowledge in the underlying
      lib `RepositoryUrlBuilder` module (would have to carry it over
      in all function calls),
      we preferred this time to change its semantics: the extension
      part (`.git`) is handled by the caller, the concern in our case.
      
      Perhaps in the longer run we'll have to do it in the mixin module
      anyway (another caller in app/helpers/submodule_helper.rb)
      
      --HG--
      branch : heptapod
      90c369ed30f8
    • Georges Racinet's avatar
      Shortcut for mercurial? etc on Repository model · a2a10377274b
      Georges Racinet authored
      A bit tired of these `repository.project.vcs_type` and besides,
      that's bad style.
      
      --HG--
      branch : heptapod
      a2a10377274b
    • Georges Racinet's avatar
      Bitbucket import: fixed after merge with GitLab 12.10 · cf97720379d5
      Georges Racinet authored
      Two merge conflict resolution mistakes.
      
      With the previous fix, Rails would have started, with these
      ones, the import and its UI work.
      
      --HG--
      branch : heptapod
      cf97720379d5
  13. 11 Jun, 2020 1 commit