Read about our upcoming Code of Conduct on this issue

  1. 07 May, 2018 1 commit
  2. 02 Feb, 2018 1 commit
  3. 06 Jan, 2018 1 commit
  4. 29 Aug, 2017 1 commit
  5. 16 Aug, 2017 1 commit
  6. 28 Jul, 2017 1 commit
  7. 27 Jul, 2017 1 commit
  8. 21 Jun, 2017 1 commit
  9. 13 Jun, 2017 2 commits
  10. 31 May, 2017 1 commit
  11. 09 May, 2017 1 commit
  12. 04 May, 2017 1 commit
  13. 03 Apr, 2017 1 commit
  14. 07 Mar, 2017 1 commit
  15. 23 Feb, 2017 4 commits
  16. 23 Dec, 2016 1 commit
  17. 21 Dec, 2016 1 commit
    • Markus Koller's avatar
      Add more storage statistics · 4571941f6307
      Markus Koller authored
      This adds counters for build artifacts and LFS objects, and moves
      the preexisting repository_size and commit_count from the projects
      table into a new project_statistics table.
      
      The counters are displayed in the administration area for projects
      and groups, and also available through the API for admins (on */all)
      and normal users (on */owned)
      
      The statistics are updated through ProjectCacheWorker, which can now
      do more granular updates with the new :statistics argument.
      4571941f6307
  18. 01 Dec, 2016 1 commit
    • Yorick Peterse's avatar
      Pass commit data to ProcessCommitWorker · 22942061ddc3
      Yorick Peterse authored
      By passing commit data to this worker we remove the need for querying
      the Git repository for every job. This in turn reduces the time spent
      processing each job.
      
      The migration included migrates jobs from the old format to the new
      format. For this to work properly it requires downtime as otherwise
      workers may start producing errors until they're using a newer version
      of the worker code.
      22942061ddc3
  19. 21 Nov, 2016 1 commit
    • Yorick Peterse's avatar
      Refactor cache refreshing/expiring · f10e7ac7c903
      Yorick Peterse authored
      This refactors repository caching so it's possible to selectively
      refresh certain caches, instead of just expiring and refreshing
      everything.
      
      To allow this the various methods that were cached (e.g. "tag_count" and
      "readme") use a similar pattern that makes expiring and refreshing
      their data much easier.
      
      In this new setup caches are refreshed as follows:
      
      1. After a commit (but before running ProjectCacheWorker) we expire some
         basic caches such as the commit count and repository size.
      
      2. ProjectCacheWorker will recalculate the commit count, repository
         size, then refresh a specific set of caches based on the list of
         files changed in a push payload.
      
      This requires a bunch of changes to the various methods that may be
      cached. For one, data should not be cached if a branch used or the
      entire repository does not exist. To prevent all these methods from
      handling this manually this is taken care of in
      Repository#cache_method_output. Some methods still manually check for
      the existence of a repository but this result is also cached.
      
      With selective flushing implemented ProjectCacheWorker no longer uses an
      exclusive lease for all of its work. Instead this worker only uses a
      lease to limit the number of times the repository size is updated as
      this is a fairly expensive operation.
      f10e7ac7c903
  20. 11 Nov, 2016 2 commits
  21. 09 Nov, 2016 1 commit
  22. 07 Nov, 2016 1 commit
    • Yorick Peterse's avatar
      Process commits in a separate worker · e0fe6e6916ff
      Yorick Peterse authored
      This moves the code used for processing commits from GitPushService to
      its own Sidekiq worker: ProcessCommitWorker.
      
      Using a Sidekiq worker allows us to process multiple commits in
      parallel. This in turn will lead to issues being closed faster and cross
      references being created faster. Furthermore by isolating this code into
      a separate class it's easier to test and maintain the code.
      
      The new worker also ensures it can efficiently check which issues can be
      closed, without having to run numerous SQL queries for every issue.
      e0fe6e6916ff
  23. 13 Oct, 2016 1 commit
  24. 20 Sep, 2016 1 commit
    • Timothy Andrew's avatar
      Implement review comments from @yorickpeterse · 57d5f49c9457
      Timothy Andrew authored
      1. Change multiple updates to a single `update_all`
      
      2. Use cascading deletes
      
      3. Extract an average function for the database median.
      
      4. Move database median to `lib/gitlab/database`
      
      5. Use `delete_all` instead of `destroy_all`
      
      6. Minor refactoring
      57d5f49c9457
  25. 19 Sep, 2016 1 commit
    • Timothy Andrew's avatar
      Test all cycle analytics pre-calculation code. · fbf82d1d74ae
      Timothy Andrew authored
      All the code that pre-calculates metrics for use in the cycle analytics
      page.
      
      - Ci::Pipeline -> build start/finish
      - Ci::Pipeline#merge_requests
      - Issue -> record default metrics after save
      - MergeRequest -> record default metrics after save
      - Deployment -> Update "first_deployed_to_production_at" for MR metrics
      - Git Push -> Update "first commit mention" for issue metrics
      - Merge request create/update/refresh -> Update "merge requests closing issues"
      fbf82d1d74ae
  26. 18 Sep, 2016 1 commit
  27. 15 Sep, 2016 1 commit
    • Timothy Andrew's avatar
      Improve performance of the cycle analytics page. · 52e536c75ee6
      Timothy Andrew authored
      1. These changes bring down page load time for 100 issues from more than
         a minute to about 1.5 seconds.
      
      2. This entire commit is composed of these types of performance
         enhancements:
      
           - Cache relevant data in `IssueMetrics` wherever possible.
           - Cache relevant data in `MergeRequestMetrics` wherever possible.
           - Preload metrics
      
      3. Given these improvements, we now only need to make 4 SQL calls:
      
          - Load all issues
          - Load all merge requests
          - Load all metrics for the issues
          - Load all metrics for the merge requests
      
      4. A list of all the data points that are now being pre-calculated:
      
          a. The first time an issue is mentioned in a commit
      
            - In `GitPushService`, find all issues mentioned by the given commit
              using `ReferenceExtractor`. Set the `first_mentioned_in_commit_at`
              flag for each of them.
      
            - There seems to be a (pre-existing) bug here - files (and
              therefore commits) created using the Web CI don't have
              cross-references created, and issues are not closed even when
              the commit title is "Fixes #xx".
      
          b. The first time a merge request is deployed to production
      
            When a `Deployment` is created, find all merge requests that
            were merged in before the deployment, and set the
            `first_deployed_to_production_at` flag for each of them.
      
          c. The start / end time for a merge request pipeline
      
            Hook into the `Pipeline` state machine. When the `status` moves to
            `running`, find the merge requests whose tip commit matches the
            pipeline, and record the `latest_build_started_at` time for each
            of them. When the `status` moves to `success`, record the
            `latest_build_finished_at` time.
      
          d. The merge requests that close an issue
      
            - This was a big cause of the performance problems we were having
              with Cycle Analytics. We need to use `ReferenceExtractor` to make
              this calculation, which is slow when we have to run it on a large
              number of merge requests.
      
            - When a merge request is created, updated, or refreshed, find the
              issues it closes, and create an instance of
              `MergeRequestsClosingIssues`, which acts as a join model between
              merge requests and issues.
      
            - If a `MergeRequestsClosingIssues` instance links a merge request
              and an issue, that issue closes that merge request.
      
      5. The `Queries` module was changed into a class, so we can cache the
         results of `issues` and `merge_requests_closing_issues` across
         various cycle analytics stages.
      
      6. The code added in this commit is untested. Tests will be added in the
         next commit.
      52e536c75ee6
  28. 16 Aug, 2016 1 commit
    • Timothy Andrew's avatar
      Fix failing tests relating to backporting ee!581. · 5f2a1dd22382
      Timothy Andrew authored
      1. `GitPushService` was still using `{merge,push}_access_level_attributes` instead
         of `{merge,push}_access_levels_attributes`.
      
      2. The branches API creates access levels regardless of the state of the
         `developers_can_{push,merge}` parameters. This is in line with the UI,
         where Master access is the default for a new protected branch.
      
      3. Use `after(:build)` to create access levels in the
         `protected_branches` factory, so that `factories_spec` passes. It
         only builds records, so we need to create access levels on `build` as
         well.
      5f2a1dd22382
  29. 12 Aug, 2016 1 commit
  30. 11 Aug, 2016 1 commit
    • Kamil Trzcinski's avatar
      Pre-create all builds for Pipeline when a trigger is received · 9c9a8bc446d7
      Kamil Trzcinski authored
      This change simplifies a Pipeline processing by introducing a special new status: created.
      This status is used for all builds that are created for a pipeline.
      We are then processing next stages and queueing some of the builds (created -> pending) or skipping them (created -> skipped).
      This makes it possible to simplify and solve a few ordering problems with how previously builds were scheduled.
      This also allows us to visualise a full pipeline (with created builds).
      
      This also removes an after_touch used for updating a pipeline state parameters.
      Right now in various places we explicitly call a reload_status! on pipeline to force it to be updated and saved.
      9c9a8bc446d7
  31. 04 Aug, 2016 1 commit
  32. 29 Jul, 2016 2 commits
    • Timothy Andrew's avatar
      Use `Gitlab::Access` to protected branch access levels. · 88fe211e233f
      Timothy Andrew authored
      1. It makes sense to reuse these constants since we had them duplicated
         in the previous enum implementation. This also simplifies our
         `check_access` implementation, because we can use
         `project.team.max_member_access` directly.
      
      2. Use `accepts_nested_attributes_for` to create push/merge access
         levels. This was a bit fiddly to set up, but this simplifies our code
         by quite a large amount. We can even get rid of
         `ProtectedBranches::BaseService`.
      
      3. Move API handling back into the API (previously in
         `ProtectedBranches::BaseService#translate_api_params`.
      
      4. The protected branch services now return a `ProtectedBranch` rather
         than `true/false`.
      
      5. Run `load_protected_branches` on-demand in the `create` action, to
         prevent it being called unneccessarily.
      
      6. "Masters" is pre-selected as the default option for "Allowed to Push"
         and "Allowed to Merge".
      
      7. These changes were based on a review from @rymai in !5081.
      88fe211e233f
    • Timothy Andrew's avatar
      Fix default branch protection. · a3cdb25bdf9e
      Timothy Andrew authored
      1. So it works with the new data model for protected branch access levels.
      a3cdb25bdf9e
  33. 18 Jul, 2016 1 commit
  34. 13 Jul, 2016 1 commit