Commit 84561349 authored by Adam Niedzielski's avatar Adam Niedzielski
Browse files

Describe polling with ETag caching

parent 3ba89cf0ec7d
......@@ -403,8 +403,8 @@ There are a few rules to get your merge request accepted:
- Avoid repeated polling of endpoints that require a significant amount of overhead
- Check for N+1 queries via the SQL log or [`QueryRecorder`](
- Avoid repeated access of filesystem
1. If you need polling to support real-time features, consider using this [described long
polling approach](
1. If you need polling to support real-time features, please use
[polling with ETag caching][polling-etag].
1. Changes after submitting the merge request should be in separate commits
(no squashing). If necessary, you will be asked to squash when the review is
over, before merging.
......@@ -547,6 +547,7 @@ available at [](http://contributor
[UX Guide for GitLab]:
[license-finder-doc]: doc/development/
[GitLab Inc engineering workflow]:
[^1]: Specs other than JavaScript specs are considered backend code. Haml
changes are considered backend code if they include Ruby code other than just
# Polling with ETag caching
Polling for changes (repeatedly asking server if there are any new changes)
introduces high load on a GitLab instance, because it usually requires
executing at least a few SQL queries. This makes scaling large GitLab
instances (like very difficult so we do not allow adding new
features that require polling and hit the database.
Instead you should use polling mechanism with ETag caching in Redis.
## How to use it
1. Add the path of the endpoint which you want to poll to
1. Implement cache invalidation for the path of your endpoint using
`Gitlab::EtagCaching::Store`. Whenever a resource changes you
have to invalidate the ETag for the path that depends on this
1. Check that the mechanism works:
- requests should return status code 304
- there should be no SQL queries logged in `log/development.log`
## How it works
1. Whenever a resource changes we generate a random value and store it in
1. When a client makes a request we set the `ETag` response header to the value
from Redis.
1. The client caches the response (client-side caching) and sends the ETag as
the `If-None-Modified` header with every subsequent request for the same
1. If the `If-None-Modified` header matches the current value in Redis we know
that the resource did not change so we can send 304 response immediately,
without querying the database at all. The client's browser will use the
cached response.
1. If the `If-None-Modified` header does not match the current value in Redis
we have to generate a new response, because the resource changed.
For more information see:
- [RFC 7232](
- [ETag proposal](
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment