1. 08 Dec, 2016 1 commit
  2. 02 Dec, 2016 1 commit
  3. 29 Nov, 2016 1 commit
  4. 23 Nov, 2016 1 commit
  5. 21 Nov, 2016 1 commit
    • Yorick Peterse's avatar
      Refactor cache refreshing/expiring · f10e7ac7c903
      Yorick Peterse authored
      This refactors repository caching so it's possible to selectively
      refresh certain caches, instead of just expiring and refreshing
      To allow this the various methods that were cached (e.g. "tag_count" and
      "readme") use a similar pattern that makes expiring and refreshing
      their data much easier.
      In this new setup caches are refreshed as follows:
      1. After a commit (but before running ProjectCacheWorker) we expire some
         basic caches such as the commit count and repository size.
      2. ProjectCacheWorker will recalculate the commit count, repository
         size, then refresh a specific set of caches based on the list of
         files changed in a push payload.
      This requires a bunch of changes to the various methods that may be
      cached. For one, data should not be cached if a branch used or the
      entire repository does not exist. To prevent all these methods from
      handling this manually this is taken care of in
      Repository#cache_method_output. Some methods still manually check for
      the existence of a repository but this result is also cached.
      With selective flushing implemented ProjectCacheWorker no longer uses an
      exclusive lease for all of its work. Instead this worker only uses a
      lease to limit the number of times the repository size is updated as
      this is a fairly expensive operation.
  6. 18 Nov, 2016 6 commits
  7. 17 Nov, 2016 1 commit
  8. 16 Nov, 2016 2 commits
  9. 11 Nov, 2016 1 commit
  10. 09 Nov, 2016 1 commit
  11. 07 Nov, 2016 1 commit
  12. 04 Nov, 2016 1 commit
  13. 01 Nov, 2016 1 commit
  14. 19 Oct, 2016 2 commits
  15. 13 Oct, 2016 1 commit
  16. 11 Oct, 2016 3 commits
  17. 10 Oct, 2016 1 commit
    • Yorick Peterse's avatar
      Precalculate trending projects · 8d729570767c
      Yorick Peterse authored
      This commit introduces a Sidekiq worker that precalculates the list of
      trending projects on a daily basis. The resulting set is stored in a
      database table that is then queried by Project.trending.
      This setup means that Unicorn workers no longer _may_ have to calculate
      the list of trending projects. Furthermore it supports filtering without
      any complex caching mechanisms.
      The data in the "trending_projects" table is inserted in the same order
      as the project ranking. This means that getting the projects in the
      correct order is simply a matter of:
          SELECT projects.*
          FROM projects
          INNER JOIN trending_projects ON trending_projects.project_id = projects.id
          ORDER BY trending_projects.id ASC;
      Such a query will only take a few milliseconds at most (as measured on
      GitLab.com), opposed to a few seconds for the query used for calculating
      the project ranks.
      The migration in this commit does not require downtime and takes care of
      populating an initial list of trending projects.
  18. 07 Oct, 2016 2 commits
    • Stan Hu's avatar
      Fix Event#reset_project_activity updates · 48860fdc0747
      Stan Hu authored
      !6678 removed the lease from Event#reset_project_activity, but it wasn't
      actually updating the project's last_activity_at timestamp properly.
      The WHERE clause would always return no matching projects. The spec
      passed occasionally because the created_at timestamp was automatically
      set to last_activity_at.
    • Nick Thomas's avatar
      Add markdown cache columns to the database, but don't use them yet · af001f6c6325
      Nick Thomas authored
      This commit adds a number of _html columns and, with the exception of Note,
      starts updating them whenever the content of their partner fields changes.
      Note has a collision with the note_html attr_accessor; that will be fixed later
      A background worker for clearing these cache columns is also introduced - use
      `rake cache:clear` to set it off. You can clear the database or Redis caches
      separately by running `rake cache:clear:db` or `rake cache:clear:redis`,
  19. 04 Oct, 2016 2 commits
    • Yorick Peterse's avatar
      Remove lease from Event#reset_project_activity · dbdf1c5da92f
      Yorick Peterse authored
      Per GitLab.com's performance metrics this method could take up to 5
      seconds of wall time to complete, while only taking 1-2 milliseconds of
      CPU time. Removing the Redis lease in favour of conditional updates
      allows us to work around this.
      A slight drawback is that this allows for multiple threads/processes to
      try and update the same row. However, only a single thread/process will
      ever win since the UPDATE query uses a WHERE condition to only update
      rows that were not updated in the last hour.
      Fixes gitlab-org/gitlab-ce#22473
    • Yorick Peterse's avatar
      Exclude system notes from Project.trending · cdcc4493726c
      Yorick Peterse authored
      Having many system notes isn't really an indication of a project being
      trending. Including these notes would lead to projects with lots of
      commit cross references (for example) showing up in the trending
      projects list.
  20. 28 Sep, 2016 1 commit
    • Rémy Coutable's avatar
      Allow Member.add_user to handle access requesters · 44fe4f60aea6
      Rémy Coutable authored
      Changes include:
      - Ensure Member.add_user is not called directly when not necessary
      - New GroupMember.add_users_to_group to have the same abstraction level as for Project
      - Refactor Member.add_user to take a source instead of an array of members
      - Fix Rubocop offenses
      - Always use Project#add_user instead of project.team.add_user
      - Factorize users addition as members in Member.add_users_to_source
      - Make access_level a keyword argument in GroupMember.add_users_to_group and ProjectMember.add_users_to_projects
      - Destroy any requester before adding them as a member
      - Improve the way we handle access requesters in Member.add_user
        Instead of removing the requester and creating a new member,
        we now simply accepts their access request. This way, they will
        receive a "access request granted" email.
      - Fix error that was previously silently ignored
      - Stop raising when access level is invalid in Member, let Rails validation do their work
      Signed-off-by: default avatarRémy Coutable <remy@rymai.me>
  21. 22 Sep, 2016 3 commits
  22. 21 Sep, 2016 2 commits
  23. 20 Sep, 2016 1 commit
  24. 15 Sep, 2016 1 commit
  25. 14 Sep, 2016 1 commit
    • ubudzisz's avatar
      add test about flash alert · 4e1cf9d2f6a0
      ubudzisz authored
      add tests with dependent destroy
      add tests with dependent destroy
      add tests
      add tests to projects spec
      update it title  and remove let
      update it title  and remove let
      remove changes after rebase
      remove changes after rebase
      update changelog
  26. 13 Sep, 2016 1 commit
    • Yorick Peterse's avatar
      Move pushes_since_gc to Redis · 27f7b8eda3a7
      Yorick Peterse authored
      This moves tracking of the pushes since the last Git GC from PostgreSQL
      to Redis. This reduces the number of writes on the "projects" table.
      This in turn reduces the vacuuming overhead.
      The lease used for incrementing the counter has been removed. This lease
      was mostly put in place to prevent high database load but this isn't
      needed anymore due to the counter now being stored in Redis.
      Fixes gitlab-org/gitlab-ce#22125