1. 08 Dec, 2020 5 commits
  2. 06 Dec, 2020 3 commits
    • Georges Racinet's avatar
      Heptapod CI: running RSpec tests in parallel · ddefe30b3c6f
      Georges Racinet authored
      Starting with 6 jobs should give us pipelines around 15 minutes,
      but perhaps we'll need to cache download.heptapod.net
      
      Worth of notice is that for the `parallel` directive, GitLab chose
      the exact environment variables expected by knapsack.
      
      Lately we've needed a tremendous
      amount of retries on one example (#384), that explains
      the very high `RETRIES` setting.
      
      --HG--
      branch : heptapod-stable
      ddefe30b3c6f
    • Georges Racinet's avatar
      HgitalyRepository tests: remove useless includes · 9e6d387532db
      Georges Racinet authored
      These were copied over from other tests and are of no use in
      that one (`Term` is about end-user terms acceptance, just
      to underline how far that is from the concerns of this test).
      
      --HG--
      branch : heptapod-stable
      9e6d387532db
    • Georges Racinet's avatar
      RSpec tests: terminate HGitaly with SIGTERM · c527891b9457
      Georges Racinet authored
      Killing the main process had no other impact on the
      worker processes than to orphan them.
      
      Depends on hgitaly#34 for the wished result.
      
      --HG--
      branch : heptapod-stable
      c527891b9457
  3. 08 Dec, 2020 1 commit
    • Georges Racinet's avatar
      RSpec tests: parallel launch with Knapsack · baa204a5c885
      Georges Racinet authored
      Knapsack is the sharding tool also used by upstream GitLab.
      To balance the shards, it uses a time table (the report file).
      Our main difference with upstream is that we have a checked in
      report file instead of maintaining it from the CI in a S3 bucket.
      
      Updates will have to be done manually. There is a companion
      `heptapod-rspec-parallel` script in the HDK (for lack of a better
      place) that can leverage knapsack to run the tests
      across several entire HDK workspaces.
      
      --HG--
      branch : heptapod-stable
      baa204a5c885
  4. 05 Dec, 2020 4 commits
    • Georges Racinet's avatar
      knapsack: allow report generation if not on CI · f724d7805e7b
      Georges Racinet authored
      First experiences for parallelization of RSpec tests will
      be done on personal workstations. The KNAPSACK_GENERATE_REPORT
      variable should be enough to gate it.
      
      --HG--
      branch : heptapod-stable
      f724d7805e7b
    • Georges Racinet's avatar
      RSpec hg_access_protected_branches: removed lengthy setup · 611aad7f35ed
      Georges Racinet authored
      It turns out that these tests don't need the branches to actually
      exist. This gets the time to run this RSpec file down from 10
      minutes to less than 4 on my development workstation.
      
      That is enough to make sharding meaningful.
      
      Of course that raises the question whether they test anything,
      since some expected Forbidden could actually now be raised for
      the wrong reason. There is the case of the
      `safe_commit_for_new_protected_branch?` where one checks that
      protected branch rules are applied even if the commit actually
      creates the protected branch (through wildcard protection,
      that is indeed possible), but this is not currently covered,
      although it should.
      
      In other words actual coverage is no worse, and we should add
      new tests for the cases that aren't strongly covered,
      (perhaps UPSTREAM).
      
      The empty repo is a special case, but the general factory fixture
      takes care of that.
      
      --HG--
      branch : heptapod-stable
      611aad7f35ed
    • Georges Racinet's avatar
      RSpec tests for hg_access: using an empty repo for main tests · fc0da2f09a87
      Georges Racinet authored
      Thanks to this, the running time goes down from about 3mn to
      1mn and 50s on my system.
      
      The LFS tests are probably irrelevant: the day we activate Mercurial's
      version of LFS, we will certainly have to perform other types of checks.
      But then, they will perhaps serve as reminders that there is
      something to adapt.
      
      --HG--
      branch : heptapod-stable
      fc0da2f09a87
    • Georges Racinet's avatar
      RSpec tests for hg_access: split in two · 84b0544435e7
      Georges Racinet authored
      The tests involving protected branches are likely to have different
      setup needs that the other. The bad news is that it's unbalanced:
      tests for protected branches take 10mn on my system, the others
      `hg_access_main_spec.rb` amount for just 3mn.
      
      --HG--
      branch : heptapod-stable
      84b0544435e7
  5. 07 Dec, 2020 9 commits
    • Georges Racinet's avatar
      Heptapod CI: gate gitlab-prebuilds with variable · 663dfccf451f
      Georges Racinet authored
      We will do something less manual later on, but right now
      this is a major waste of time while tuning the pipeline.
      
      --HG--
      branch : heptapod-stable
      663dfccf451f
    • Georges Racinet's avatar
      Heptapod CI: adding frontend assets to the prebuild · b0f35669d1c9
      Georges Racinet authored
      Closes #382
      
      --HG--
      branch : heptapod-stable
      b0f35669d1c9
    • Georges Racinet's avatar
      Heptapod CI: prebuilding Gitaly and other deps, pushed on download.h.n · 5fc14f94261f
      Georges Racinet authored
      if it works, we'll probably just keep the result and enable the job
      only on GitLab version bumps (perhaps with the `changes` directive)
      
      --HG--
      branch : heptapod-stable
      5fc14f94261f
    • Georges Racinet's avatar
      Heptapod CI: installing proper versions of HGitaly & py-heptapod · bf0b1d7eaa6c
      Georges Racinet authored
      Heads up: in release branches, the tests are expected to pass
      for the versions specified in `python/requirements.txt`.
      
      Was spotted while working on the Gitaly prebuild, but really
      has to be fixed: we were running the tests always against the
      default branch of the Python dependencies (HGitaly and py-heptapod).
      
      There is a complication with tags, and especially release
      branches. For the latter, there is no simple way to know
      which is the appropriate branch of the Python deps. Shortly
      after the beginning of the release branch, it would clearly
      be `default`, but after the swap of branches in the Rails app,
      it would be `stable`.
      
      For tags, the tests are run against the exact versions specified
      in `python/requirements.txt`, that was easy.
      For release branches, we just do the same. This is a lesser evil.
      Hence now, pushes to release branches are expected to make the
      appropriate version bumps (this was already quite often the case).
      
      --HG--
      branch : heptapod-stable
      bf0b1d7eaa6c
    • Georges Racinet's avatar
      Heptapod CI: refactoring of hgitaly-test-build · e9eab32a7d58
      Georges Racinet authored
      Now there's a separate function to update configuration
      files, will be useful for next change.
      
      --HG--
      branch : heptapod-stable
      e9eab32a7d58
    • Georges Racinet's avatar
      Heptapod CI: match HGitaly build/spawn with Gitaly · 56a1a392e87d
      Georges Racinet authored
      This creates and preserves the `tmp/tests/hgitaly` subdir
      and separates for CI the install (build) and startup (spawn)
      phases.
      
      By putting `hgitaly.socket` in that preserved directory,
      it also fixes test runs on developer machines involving
      several spec files (previously, the socket was destroyed
      at the end of each spec file)
      
      This is heavier than strictly needed for HGitaly, but
      matching what is already done for Gitaly will help us
      schedule differently (prepare phase, straight download), and
      prebuilding just Gitaly, but with `TestEnv` that actually starts
      HGitaly is an unmaintanable mess.
      
      --HG--
      branch : heptapod-stable
      56a1a392e87d
    • Georges Racinet's avatar
      Heptapod CI: fixed URL for initial cloning of py-heptapod/hgitaly (!) · 3cf6af5d98ac
      Georges Racinet authored
      The only conclusion is that it worked because almost all the runners
      already had them (spotted on a case where concurrent-1 got used)
      
      --HG--
      branch : heptapod-stable
      3cf6af5d98ac
    • Georges Racinet's avatar
      Heptapod CI/CD: split out push script for reuse · 15b390d2bc98
      Georges Racinet authored
      --HG--
      branch : heptapod-stable
      15b390d2bc98
    • Georges Racinet's avatar
      Heptapod CI/CD: known SSH hosts for sftp · a623efe9f792
      Georges Racinet authored
      about time.
      
      --HG--
      branch : heptapod-stable
      a623efe9f792
  6. 03 Dec, 2020 2 commits
  7. 02 Dec, 2020 6 commits
  8. 26 Nov, 2020 2 commits
  9. 22 Nov, 2020 1 commit
  10. 20 Nov, 2020 1 commit
  11. 16 Nov, 2020 1 commit
    • Georges Racinet's avatar
      Branch acces checks: direct identification of legit merges · 948dac54b949
      Georges Racinet authored
      Closes #368
      
      To let merges done from the Rails application be checked
      against the distinct set of merge rules rather that push
      rules, GitLab looks for MRs matching the pre-receive changes by
      comparing with the merge commit sha that's always available
      (and set by the Repository Model) before the actual branch move.
      
      Because the Mercurial change can be atomic, we have no other
      option than passing a special parameter down from
      `Repository#merge` through the Mercurial backend, with the
      `HEPTAPOD_ACCEPT_MERGE_ID` environment variable.
      In turn the Mercurial process will add it to its pre-receive
      hook payload, and the target branch can be compared to switch
      from (direct) push to (Rails) merge mode.
      
      In the case of merges intended from the very beginning to be
      fast forwards, the commit sha is already known in advance,
      but we're using the same mechanism for consistency in the
      check.
      
      Finally, there was a good dose of wrestling with those RSpec
      examples that care about merging to a protected branch, because
      of the on-demand nature of instantiation.
      
      --HG--
      branch : heptapod-stable
      948dac54b949
  12. 15 Nov, 2020 3 commits
    • Georges Racinet's avatar
      Protected branches: deletion protection when no one allowed to push · 1290d9cfb0b7
      Georges Racinet authored
      As demonstrated by a functional test,
      Maintainers were still able to remove the protected branch even if
      push level was set to `NO_ONE`. The reason is the underlying 'web'
      protocol. The fix is harsh, but removal is a special case
      and the explicit message to unprotect the branch should be
      acceptable.
      
      --HG--
      branch : heptapod-stable
      1290d9cfb0b7
    • Georges Racinet's avatar
      Restore branch protection for Mercurial · b9859c4b49e1
      Georges Racinet authored
      The fix is simple enough and all functional tests are passing.
      This is the occasion to restore/adapt the RSpec tests for
      branch protections.
      
      This still have the unwanted consequence that some MR merges
      will fail, because the related permission checks (occurring
      in pre-receive API endpoint) won't find the matching MR.
      
      This will be handled in a subsequent changeset, for more
      on this, see #368.
      
      --HG--
      branch : heptapod-stable
      b9859c4b49e1
    • Georges Racinet's avatar
      Repository Model: apply Rubocop rules · cdb21e4bda27
      Georges Racinet authored
      More stuff dating back to pre-HDK times (no Rubocop in editor)
      
      --HG--
      branch : heptapod-stable
      cdb21e4bda27
  13. 02 Nov, 2020 1 commit
  14. 01 Dec, 2020 1 commit
    • Georges Racinet's avatar
      GitLab::Shell: fixed reading Heptapod Version · d36b1d5b79e9
      Georges Racinet authored
      The previous change was only about the required version (the
      one recorded in the Rails checkout), and was still reading the
      upstream version file for the actually running GitLab Shell.
      
      Closes #375
      
      --HG--
      branch : heptapod
      d36b1d5b79e9