1. 13 Dec, 2020 3 commits
    • Georges Racinet's avatar
      Bitbucket import: removed most of the code · fdb960fc35b5
      Georges Racinet authored
      There aren't Mercurial repositories to import from Bitbucket
      any more: our divergence with upstream on this is now just a
      Removal is partial in order not to jeopardize the installation
      of 0.18.0rc1 on foss.h.n (second OAuth, worker queue registration
      etc.). In all cases, behaviour on such configurations will be
      tested and adjusted before release.
      branch : heptapod
    • Georges Racinet's avatar
      RSpec tests: introducing "scopes" in parallel runs · e3b221981d54
      Georges Racinet authored
      The new `HEPTAPOD_RSPEC_SCOPE` will be used to define several
      subsets of tests to run.
      The default is what we are currently
      running in CI for everyday development (Merge Requests on
      `heptapod` and `heptapod-stable` branches).
      The 'extended' scope will be used in release preparation, and
      will grow a lot from now on.
      branch : heptapod
    • Georges Racinet's avatar
      RSpec parallel tests: exclusion tags · 20544f19c65b
      Georges Racinet authored
      These are the same tags used by upstream in the "unit" RSpec job
      (with a much wider pattern, of which ours is a small fraction).
      branch : heptapod
  2. 10 Dec, 2020 1 commit
  3. 11 Dec, 2020 2 commits
  4. 10 Dec, 2020 5 commits
  5. 09 Dec, 2020 1 commit
    • Georges Racinet's avatar
      Merge with GitLab v13.5.4 · 3cc386756f43
      Georges Racinet authored
      This result passed all RSpec and functional tests on my
      dev workstation.
      Here follow the most notable changes that will need further
      The upstream backup code has been entirely refactored, with a
      new separation of concerns that has the Repository class take
      care of the actual repo bundle production and restoration.
      This is a breeze for Heptapod and its hierarchy of repository
      classes. Not only is our specific code clearly delimited and
      short (45 lines, including comments), but it is also more
      future proof because we don't rely on spaggheti-like conditionals
      any more. For instance, the day native projects don't include
      a Git repository any more, we can simply provide appropriate
      methods in HGitalyRepository. First shot implementation passed
      the functional tests right away. It will need RSpec tests, though,
      which are by the way now much easier to write as well.
      There's been refactoring around Wikis, probably related to the
      introduction of the Group Wikis. It was enough to pass the
      tests to just delegate the two Heptapod-specific methods needed
      for the raw repository instantiation, but this needs more scrutinity,
      and probably Mercurial options for Group Wikis have to be
      branch : heptapod
  6. 08 Dec, 2020 14 commits
  7. 06 Dec, 2020 3 commits
    • Georges Racinet's avatar
      Heptapod CI: running RSpec tests in parallel · ddefe30b3c6f
      Georges Racinet authored
      Starting with 6 jobs should give us pipelines around 15 minutes,
      but perhaps we'll need to cache download.heptapod.net
      Worth of notice is that for the `parallel` directive, GitLab chose
      the exact environment variables expected by knapsack.
      Lately we've needed a tremendous
      amount of retries on one example (#384), that explains
      the very high `RETRIES` setting.
      branch : heptapod-stable
    • Georges Racinet's avatar
      HgitalyRepository tests: remove useless includes · 9e6d387532db
      Georges Racinet authored
      These were copied over from other tests and are of no use in
      that one (`Term` is about end-user terms acceptance, just
      to underline how far that is from the concerns of this test).
      branch : heptapod-stable
    • Georges Racinet's avatar
      RSpec tests: terminate HGitaly with SIGTERM · c527891b9457
      Georges Racinet authored
      Killing the main process had no other impact on the
      worker processes than to orphan them.
      Depends on hgitaly#34 for the wished result.
      branch : heptapod-stable
  8. 08 Dec, 2020 1 commit
    • Georges Racinet's avatar
      RSpec tests: parallel launch with Knapsack · baa204a5c885
      Georges Racinet authored
      Knapsack is the sharding tool also used by upstream GitLab.
      To balance the shards, it uses a time table (the report file).
      Our main difference with upstream is that we have a checked in
      report file instead of maintaining it from the CI in a S3 bucket.
      Updates will have to be done manually. There is a companion
      `heptapod-rspec-parallel` script in the HDK (for lack of a better
      place) that can leverage knapsack to run the tests
      across several entire HDK workspaces.
      branch : heptapod-stable
  9. 05 Dec, 2020 4 commits
    • Georges Racinet's avatar
      knapsack: allow report generation if not on CI · f724d7805e7b
      Georges Racinet authored
      First experiences for parallelization of RSpec tests will
      be done on personal workstations. The KNAPSACK_GENERATE_REPORT
      variable should be enough to gate it.
      branch : heptapod-stable
    • Georges Racinet's avatar
      RSpec hg_access_protected_branches: removed lengthy setup · 611aad7f35ed
      Georges Racinet authored
      It turns out that these tests don't need the branches to actually
      exist. This gets the time to run this RSpec file down from 10
      minutes to less than 4 on my development workstation.
      That is enough to make sharding meaningful.
      Of course that raises the question whether they test anything,
      since some expected Forbidden could actually now be raised for
      the wrong reason. There is the case of the
      `safe_commit_for_new_protected_branch?` where one checks that
      protected branch rules are applied even if the commit actually
      creates the protected branch (through wildcard protection,
      that is indeed possible), but this is not currently covered,
      although it should.
      In other words actual coverage is no worse, and we should add
      new tests for the cases that aren't strongly covered,
      (perhaps UPSTREAM).
      The empty repo is a special case, but the general factory fixture
      takes care of that.
      branch : heptapod-stable
    • Georges Racinet's avatar
      RSpec tests for hg_access: using an empty repo for main tests · fc0da2f09a87
      Georges Racinet authored
      Thanks to this, the running time goes down from about 3mn to
      1mn and 50s on my system.
      The LFS tests are probably irrelevant: the day we activate Mercurial's
      version of LFS, we will certainly have to perform other types of checks.
      But then, they will perhaps serve as reminders that there is
      something to adapt.
      branch : heptapod-stable
    • Georges Racinet's avatar
      RSpec tests for hg_access: split in two · 84b0544435e7
      Georges Racinet authored
      The tests involving protected branches are likely to have different
      setup needs that the other. The bad news is that it's unbalanced:
      tests for protected branches take 10mn on my system, the others
      `hg_access_main_spec.rb` amount for just 3mn.
      branch : heptapod-stable
  10. 07 Dec, 2020 6 commits
    • Georges Racinet's avatar
      Heptapod CI: gate gitlab-prebuilds with variable · 663dfccf451f
      Georges Racinet authored
      We will do something less manual later on, but right now
      this is a major waste of time while tuning the pipeline.
      branch : heptapod-stable
    • Georges Racinet's avatar
      Heptapod CI: adding frontend assets to the prebuild · b0f35669d1c9
      Georges Racinet authored
      Closes #382
      branch : heptapod-stable
    • Georges Racinet's avatar
      Heptapod CI: prebuilding Gitaly and other deps, pushed on download.h.n · 5fc14f94261f
      Georges Racinet authored
      if it works, we'll probably just keep the result and enable the job
      only on GitLab version bumps (perhaps with the `changes` directive)
      branch : heptapod-stable
    • Georges Racinet's avatar
      Heptapod CI: installing proper versions of HGitaly & py-heptapod · bf0b1d7eaa6c
      Georges Racinet authored
      Heads up: in release branches, the tests are expected to pass
      for the versions specified in `python/requirements.txt`.
      Was spotted while working on the Gitaly prebuild, but really
      has to be fixed: we were running the tests always against the
      default branch of the Python dependencies (HGitaly and py-heptapod).
      There is a complication with tags, and especially release
      branches. For the latter, there is no simple way to know
      which is the appropriate branch of the Python deps. Shortly
      after the beginning of the release branch, it would clearly
      be `default`, but after the swap of branches in the Rails app,
      it would be `stable`.
      For tags, the tests are run against the exact versions specified
      in `python/requirements.txt`, that was easy.
      For release branches, we just do the same. This is a lesser evil.
      Hence now, pushes to release branches are expected to make the
      appropriate version bumps (this was already quite often the case).
      branch : heptapod-stable
    • Georges Racinet's avatar
      Heptapod CI: refactoring of hgitaly-test-build · e9eab32a7d58
      Georges Racinet authored
      Now there's a separate function to update configuration
      files, will be useful for next change.
      branch : heptapod-stable
    • Georges Racinet's avatar
      Heptapod CI: match HGitaly build/spawn with Gitaly · 56a1a392e87d
      Georges Racinet authored
      This creates and preserves the `tmp/tests/hgitaly` subdir
      and separates for CI the install (build) and startup (spawn)
      By putting `hgitaly.socket` in that preserved directory,
      it also fixes test runs on developer machines involving
      several spec files (previously, the socket was destroyed
      at the end of each spec file)
      This is heavier than strictly needed for HGitaly, but
      matching what is already done for Gitaly will help us
      schedule differently (prepare phase, straight download), and
      prebuilding just Gitaly, but with `TestEnv` that actually starts
      HGitaly is an unmaintanable mess.
      branch : heptapod-stable