1. 25 Jul, 2021 1 commit
  2. 17 Jul, 2021 2 commits
    • Georges Racinet's avatar
      PAAS Runner job request: adding Runner features, sometimes required. · e16766c024fd
      Georges Racinet authored
      Job acquisition failed for first test run for the Heptapod Rails
      project on foss.h.n, with an error banner on the job page:
      "Your runner is outdated".
      
      After code inspection on the Rails side, it seems that a bunch of
      `features` are required as part of the post data.
      
      I don't know why it didn't show up in earlier tests (also done
      against foss.h.n), it really looks to always be required.
      
      Also it's possible that a wrong list of features may have less
      direct bad impact on the runs, hence this full list, inferred from
      Golang sources.
      
      --HG--
      branch : heptapod
      e16766c024fd
    • Georges Racinet's avatar
      PAAS Runner: adding flavor to the Clever Cloud CI sauce · e81ad8dc4fe9
      Georges Racinet authored
      Instance flavors can be controlled by
      
      - the `CI_CLEVER_CLOUD_FLAVOR` variable (can be set per-job or at
        the Project or Group level)
      - default in PAAS Runner configuration
      
      There is an ultimate hardcoded default (`M`)
      
      --HG--
      branch : heptapod
      e81ad8dc4fe9
  3. 19 Jul, 2021 1 commit
    • Georges Racinet's avatar
      PAAS Dispatcher: tracking jobs progress to limit concurrency · 3e2f4fa66069
      Georges Racinet authored
      To limit concurrency, we have to track jobs progress once they've
      been launched.
      
      This is done via a permanent thread that calls the coordinator
      API endpoint to retrieve job info by job token. As it turns out,
      the endpoint returns a 401 code unless the job is ongoing.
      This feels weird, but given that we are using the token that
      the coordinator itself sent us as part of the job payload, this
      is good enough for now, but it makes debugging of any other
      problem leading to 401 responses really hard.
      
      Later versions of the PAAS dispatcher may have a endpoint on
      the provisioned ressources in order to follow progress directly
      there, which would be less hacky and alleviate the load on
      coordinator a bit (of course it's otherwise hammered, but full
      job details by token are expected to be heavy to generate).
      
      The current design of the `PaasDispatcher` class makes it so
      that progress is fully interpreted only at next polling cycle,
      because there's a reporting thread reading the queue for each
      cycle. We should later make a permanent reporting thread.
      
      --HG--
      branch : heptapod
      3e2f4fa66069
  4. 16 Jul, 2021 3 commits
    • Georges Racinet's avatar
      PAAS Runner: heptapod-paas-runner console script · 202519f0fc8e
      Georges Racinet authored
      This is just exposing the PaasDispatcher
      
      --HG--
      branch : heptapod
      202519f0fc8e
    • Georges Racinet's avatar
      PAAS Runner: main dispatcher loop · e01f5496a4bb
      Georges Racinet authored
      We're using a multi-threaded model:
      
      - the main thread polls the coordinator repeatedly
      - a new thread is spawned for each job to process and
        takes care of provisioning PAAS resources and launch
        the job on them, so that the polling is not blocked by
        waiting for the PAAS provider, the network etc
      - a separate thread and a queue are used to gather information
        about progress in the job related threads. It won't be able to
        know about job completion, only launch completion, but it will
        be able to maintain a list of successfully launched jobs, in
        order to implement concurrency capping and the like.
      
      We don't care about GIL impact on performance for this (more
      details as code comment).
      
      --HG--
      branch : heptapod
      e01f5496a4bb
    • Georges Racinet's avatar
      PAAS Runner: Clever Runner · 73365692ea77
      Georges Racinet authored
      Like the official CLI "Clever Tools" does, we need to
      request the current version of the `docker` instance type
      to create a new one. This should be later on optimized.
      
      Like the `LocalDockerRunner`, the `launch()` method creates
      a Docker build context, but deployment now means to push
      it via Git.
      
      In practice, this means that we have a foreign (non Python)
      dependency: the Git executable.
      
      We should at least later on make its path customizable and
      make a startup error if missing.
      
      --HG--
      branch : heptapod
      73365692ea77
  5. 19 Jul, 2021 1 commit
    • Georges Racinet's avatar
      PAAS Runner: concrete Docker runner on local resources only · cce95f6f51a6
      Georges Racinet authored
      As explained in the docstring, this is mostly for debugging
      purposes and testing.
      
      This doesn't involve any real PAAS provider, as the provisioning
      step is reduced to nothing: all jobs will run on the locally
      controlled Docker host. Still, this allows to play with the
      same execution flow as actual PAAS runners.
      
      The provided unit tests don't prove that it really works, just that
      we are internally consistent (still quite useful).
      
      --HG--
      branch : heptapod
      cce95f6f51a6
  6. 15 Jul, 2021 1 commit
    • Georges Racinet's avatar
      PAAS Runner: helper to build a Docker image for a given job · c91348465170
      Georges Racinet authored
      This is the heart of the distinction between PAAS and other
      types on on-demand provisioning: we need to present the
      job as a standalone, one-shot, application definition and send it to the
      platform, typically with a Git push.
      
      The helper provides a full Docker context to build for the job
      and can push it via Git.
      
      Requires a working `git` executable (also for tests).
      
      --HG--
      branch : heptapod
      c91348465170
  7. 19 Jul, 2021 1 commit
    • Georges Racinet's avatar
      PAAS Runner: PAAS exceptions · cfcd857d023a
      Georges Racinet authored
      This is an attempt to use a single, flexible, base class for
      errors encountered while dealing with PAAS APIs.
      
      Many, but not all interactions go through a transport layer
      (usually HTTP, can also be Git), which has its own, meaningful,
      set of error codes on top of potentially a different one in the
      payload. That's why we have a "main" `code` and an optional
      `transport_code` when distinction is useful.
      
      Similarly, `action_details` can be a HTTP request body, maybe
      something else, and in some cases, nothing.
      
      Typically, the caller would know what to expect in context, while
      an ultimate catcher in top loop wouldn't try to interpret too much.
      
      --HG--
      branch : heptapod
      cfcd857d023a
  8. 11 Jul, 2021 1 commit
    • Georges Racinet's avatar
      PAAS Runner: abstract base class · b9c87329fba3
      Georges Racinet authored
      We're following the multi-runner concept of upstream GitLab Runner:
      the PAAS Runner will have multiple `[[runner]]` configuration sections,
      be it for group/project/instance scopes or to accomodate several
      coordinators.
      
      These runners will have special executor names, for dispatching
      in the main loop, e.g, `clever-docker` for the `docker` executor
      on Clever Code.
      
      For the time being, we have to produce an entire valid Heptapod Runner
      configuration for the PAAS resource, because that's what the
      `exec-fetched-job` command takes.
      
      Now that we have an actual test, we can remove the placeholder tests
      previously introduced just to get a success.
      
      --HG--
      branch : heptapod
      b9c87329fba3
  9. 19 Jul, 2021 1 commit
    • Georges Racinet's avatar
      Python subsystems: empty structure · 9364d141be99
      Georges Racinet authored
      Should be enough to develop, run CI and push to PyPI.
      
      The first target is to develop a Runner leveraging PAAS provider
      resources.
      
      Version starts at 0.2.0 to match existing internal terminology
      (prototype v2 for the Clever Cloud Runner).
      
      As usual, we're committed to a 100% test coverage, meaning
      that every non-tested line has to be justified.
      
      In the case of these Python subsystems, we will be doing
      heavy mocking in tests, with the obvious defect not to actually
      prove so much, but with the length of the feedback loop to test
      anything for real, it's more efficient to prevent self-consistency
      problems than to discover them in the wild, even for prototypes.
      
      --HG--
      branch : heptapod
      9364d141be99
  10. 21 May, 2021 2 commits
  11. 17 May, 2021 4 commits
  12. 27 Apr, 2021 1 commit
  13. 21 Apr, 2021 13 commits
  14. 19 Apr, 2021 7 commits
  15. 18 Apr, 2021 1 commit
    • Georges Racinet's avatar
      Kubernetes executor tests: misplaced condition? · 6477a5843cfa
      Georges Racinet authored
      Perhaps that could depend on some kind of external state, but
      the assertion ties the meaning of `executed` to call to the
      verify function, while the actual test code has it changed
      to `true` at each call of the round-tripper.
      
      Treating as an inconsistency, and this was the last failing
      test in a full `make test` run without an actual k8s installation.
      
      --HG--
      branch : heptapod
      6477a5843cfa