1. 26 Nov, 2020 1 commit
    • Georges Racinet's avatar
      DockerHeptapod: Fixed SSH params · a9cbd203532d
      Georges Racinet authored
      Now that it inherits from OmnibusHeptapod, it had
      the hardcoded expected `git@localhost`, which isn't
      appropriate because DockerHeptapod is meant to be
      used from outside the container (and OmnibusHeptapod can
      be used from inside the container)
      a9cbd203532d
  2. 25 Nov, 2020 1 commit
  3. 17 Nov, 2020 1 commit
  4. 12 Oct, 2020 1 commit
    • Georges Racinet's avatar
      New `--heptapod-hg-native` option · b2a5da28ceb5
      Georges Racinet authored
      This option makes the functional tests create Mercurial native
      projects instead of the default type (currently `hg_git`).
      
      Hence it will be the main tool to validate HGitaly related
      developments. Later on, we might want to introduce dedicated
      CI jobs. Switching to parametrization can also be considered,
      but it wouldn't change so much in CI, because we now have tests
      sharding.
      
      The new flag is stored on the `Heptapod` object, for lack of
      better choice in the short term (arguably, that's not a property
      of the Heptapod server).
      b2a5da28ceb5
  5. 06 Oct, 2020 3 commits
    • Georges Racinet's avatar
      Fixing flake8 and forcing the job in CI. · 56961c493a41
      Georges Racinet authored
      I *kwew* that this job not running on detached MR pipelines was
      going to be a problem. We can live with duplication of that job.
      56961c493a41
    • Georges Racinet's avatar
      Omnibus: stop hardcoding server-side path to hg · 55b1ee0b3b1b
      Georges Racinet authored
      For server-side operations, we should use the one provided by
      the server, obviously.
      
      The immediate need is that it varies amongst installation types
      With The new Omnibus packaging method (omnibus-heptapod!3)
      for Heptapod 0.17, it changes from /usr/local/bin (expected to be on PATH)
      to /opt/gitlab/embedded/bin.
      
      There are several TODOs:
      
      - read `hg_executable` from config, with remote instances being
        the only exceptions (but these wouldn't call server-side subprocesses
        from the tests).
      - provide a direct helper method on `Heptapod`
      - do the same for Git
      55b1ee0b3b1b
    • Georges Racinet's avatar
      git: new CLI option for client-side executable · ee0c40e91d6a
      Georges Racinet authored
      In 7a354d699944, we had the `heptapod` fixture carry the path
      to the client-side Git executable, but that creates confusion
      between the client and server sides (actually more manifest with
      Mercurial, but both should be handled in the same way).
      
      Now we make the client-side Git executable path work as for
      Mercurial, with a command line option and centralization in the
      `utils.git` module.
      
      Finally, we refactor both `utils.git` and `test_ssh` so that the
      latter can use the standard classmethod of `git.LocalRepo`.
      ee0c40e91d6a
  6. 05 Oct, 2020 4 commits
    • Georges Racinet's avatar
      OmnibusHeptapod: fixing DevToolsActivePort chrome driver problem · fa268a4c73e9
      Georges Racinet authored
      Without this, the tests may crash with the
      "DevToolsActivePort file doesn't exist" error upon initialization
      of ChromeDriver.
      
      Many resources I could find about this are in a context when
      the tests are run as `root`, as I did in the first experiments.
      
      But it can happen also if running a non-privileged user, such
      as `git` in our case.
      
      Of course `--no-sandbox` wouldn't be acceptable for systems
      not entirely dedicated to the task, as is the case with
      `OmnibusHeptapod`, but not `DockerHeptapod` (in the latter,
      the web driver is on the user's system, not in the container).
      
      --HG--
      branch : stable
      fa268a4c73e9
    • Georges Racinet's avatar
      OmnibusHeptapod: new base class to run tests in Omnibus instances · 48a515bc0056
      Georges Racinet authored
      In that case the tests are to be run as the 'git' user, and
      depend on
      
      - availability of sudo
      - a sudoers configuration that gives the 'git' user the
        right to launch `gitlab-ctl` commands.
      
      In particular, this avoids the need to initiate the webdriver
      as root (which can be done but is yet another source of trouble).
      
      --HG--
      branch : stable
      48a515bc0056
    • Georges Racinet's avatar
      Heptapod: made Git executable available as attribute · 7a354d699944
      Georges Racinet authored
      This is from the client side, but still, depending on context,
      the proper one to use may not be on path.
      
      --HG--
      branch : stable
      7a354d699944
    • Georges Racinet's avatar
      Heptapod: factorizing run_shell in base class · 2344f5b0369a
      Georges Racinet authored
      The `execute` method has long been be the fundamental one whose
      implementation varies amongst subclasses.
      
      --HG--
      branch : stable
      2344f5b0369a
  7. 08 Sep, 2020 1 commit
    • Georges Racinet's avatar
      services: GitLab Docker image now uses puma · ff3356bc42d2
      Georges Racinet authored
      This has been true at least since 13.1, meaning in particular
      that we didn't stop Rails properly in the backup tests, and
      that explains the high rate of data corruption I personally had
      with them.
      
      --HG--
      branch : stable
      ff3356bc42d2
  8. 23 Jul, 2020 1 commit
    • Georges Racinet's avatar
      Heptapod: general helpers for API requests · 7ee09803b4ff
      Georges Racinet authored
      These allow notably to send the requests as a given `User`.
      
      Up to now, we had such methods only for projects, and more general
      requests were handcrafted and generally running as `root`.
      
      --HG--
      branch : heptapod-0-14
      7ee09803b4ff
  9. 17 Jul, 2020 1 commit
  10. 21 Jun, 2020 2 commits
  11. 06 Jun, 2020 3 commits
    • Georges Racinet's avatar
      Heptapod: generic implementaion for apply_hashed_storage_setting · 326a7569275a
      Georges Racinet authored
      It will work for any subclass that implements `gitlab_ctl` and
      `rake` (currently corresponding to the `@needs.services` generator),
      hence for `GdkHeptapod` as well as `DockerHeptapod`.
      
      This implementation is tighter in that we make sure that Redis,
      where the annoying cache should reside, is stopped before
      restarting everything. The restart of Rails dominates execution
      time anyway.
      
      Of course, at some point we could find a gentler way to force
      the previous setting out of the caches, but at least we can now
      run hashed storage related tests in GDK context.
      
      We don't need `restart_rails()` anymore, a more controlled style
      is wishable anyway.
      326a7569275a
    • Georges Racinet's avatar
      Heptapod: reworked services management API · a88a65e5c368
      Georges Racinet authored
      Finally settled to `gitlab_ctl()` to express both
      management of all services and of a sublist of services.
      
      The naming and behaviour is more intuitive than the `ctl_services`,
      and it matches the Omnibus (hence Docker) CLI.
      a88a65e5c368
    • Georges Racinet's avatar
      session: avoid a webdriver leak and removed duplication · 9aeb60aea2ea
      Georges Racinet authored
      `login_as_user()` would not store the webdriver anywhere, hence
      it wouldn't be properly closed.
      It is currently used in `test_external_user_can_create_project()`
      
      With this change, it relies on the centralized mechanism currently
      provided by the `Heptapod` object, with the result that the
      webdriver is also stored there and gets automatically cleaned up
      at the end of the session.
      9aeb60aea2ea
  12. 05 Jun, 2020 8 commits
    • Georges Racinet's avatar
      Heptapod: preparing to store User instances · 31054df34fdf
      Georges Racinet authored
      Having real `User` instances on the `Heptapod` object has been
      on the radar since the `User` class was created.
      
      Many places in the test utils and at most a few in the tests
      themselves access the `Heptapod.users` dictionary.
      
      It's therefore a bit too much for now to convert all these to
      fetch a User instance directly, but the new `Heptapod.get_user()`
      method will help us transitioning progressively.
      31054df34fdf
    • Georges Racinet's avatar
      Preparations: simplified root user info and token setup · 0c7dfea61d61
      Georges Racinet authored
      It makes more sense to set the root user once (and in the uniform
      way provided by the User class), to ensure its token immediately
      and to just assume that in the subsequent setups.
      
      Notably `session.login_as_root()` might simply not be called at all
      in the current test run.
      0c7dfea61d61
    • Georges Racinet's avatar
      Preparations: made root webdriver lazy · 7db10f33198d
      Georges Racinet authored
      This is a bit different than for regular users, since
      we have potentially the root password first initialization
      sequence.
      7db10f33198d
    • Georges Racinet's avatar
      Preparations: made webdriver for basic user lazy · f8b865f4cc43
      Georges Racinet authored
      This already saves us 5s at each startup.
      
      - `session.ensure_user()` doesn't initialize a webdriver any more.
      - `Heptapod.get_user_webdriver()` creates the driver if needed
        and stores it
      - `User.ensure_private_token()` now uses `Heptapod.get_user_webdriver()`
        instead of taking a webdriver argument
      f8b865f4cc43
    • Georges Racinet's avatar
      Central method to get a user webdriver · 7fb8d5a12719
      Georges Racinet authored
      After this change, no test gets a webdriver directly
      from the dict on Heptapod object, opening the road to
      make webdriver initialization lazy.
      7fb8d5a12719
    • Georges Racinet's avatar
      Preparations: caching user access tokens · 5f5ce18ee546
      Georges Racinet authored
      These can't be read with a webdriver, as it is a security feature
      of GitLab not to display them, that's why we where creating new
      ones for each session.
      
      Storing them in a cache file will help making the session startup
      time shorter. This is especially useful if launching a few tests
      repeatedly or with the tests that have to redo the preparations after
      an instance restart.
      5f5ce18ee546
    • Georges Racinet's avatar
      Session: moved user personal token generation to User class · b0029b6dec0c
      Georges Racinet authored
      A prelude to forthcoming changes to cache it.
      
      the `create()` class method will automatically store the user
      information in the Heptapod instance, but `search()` won't,
      because it doesn't have the password information (but its caller
      might)
      b0029b6dec0c
    • Georges Racinet's avatar
      Preparations: timed test users management · d0cc38e2434c
      Georges Racinet authored
      These preparations (signed-in webdriver, personal access token)
      are only done once per session, but if the session is comprised of
      only a few tests, they can represent a good chunk of the run time.
      
      For example, launching the only current test in test_wikis.py,
      they represent 45% of the total time (on a warmed up GDK Heptapod
      based on GitLab 12.10).
      d0cc38e2434c
  13. 03 Jun, 2020 5 commits
    • Georges Racinet's avatar
      Heptapod: fixed `restart_rails()` · 1443aca07e8c
      Georges Racinet authored
      It is for now only used in Docker mode, that's why the obviously
      lack of command was there.
      1443aca07e8c
    • Georges Racinet's avatar
      DockerHeptapod: use backup methods from base class · c9100ec2a964
      Georges Racinet authored
      By implementing the new primitives, we can now leverage the base
      implementation of `backup_create()` and `backup_restore()` (which
      were largely derived from the first implementation in
      `DockerHeptapod` anyway)
      c9100ec2a964
    • Georges Racinet's avatar
      Heptapod: providing `restart_rails()` in the base class · daf358925926
      Georges Racinet authored
      The only caller at the time being is its sibling method
      `apply_hashed_storage_setting()`, that will hence become
      available for all concrete subclasses that implement
      `ctl_services()`.
      daf358925926
    • Georges Racinet's avatar
      New GdkHeptapod concrete class · c0a71291b3ac
      Georges Racinet authored
      It derives from SourceHeptapod, only adding service management
      and rake capabilities, hence being able to support backup tests
      and other tests that require service management.
      c0a71291b3ac
    • Georges Racinet's avatar
      Heptapod: backup handling impl in the abstract base class · 4316399e1320
      Georges Racinet authored
      Instead of just declaring `backup_create()` and `backup_restore()`,
      we implement them generically, relying on a new set of primitives
      to be implemented in subclasses:
      
      - `ctl_services()`: service management
      - `RAILS_SERVICES`: class attribute listing the Rails app services
        (web and background). These will be different in the Docker and
        GDK cases.
      - `rake()`
      - `remove_all_backups()`
      4316399e1320
  14. 14 May, 2020 1 commit
    • Georges Racinet's avatar
      SourceHeptapod: make run_shell work · 757d36e282bc
      Georges Racinet authored
      Some callers want to pass a system user name, and we have to
      accept that for uniformity ('git' is been used as meaning
      "the default user", which is lame but that's another story)
      757d36e282bc
  15. 01 May, 2020 1 commit
  16. 02 Apr, 2020 1 commit
  17. 01 Apr, 2020 2 commits
  18. 31 Mar, 2020 1 commit
  19. 29 Mar, 2020 1 commit
  20. 27 Mar, 2020 1 commit