Cache stampede, also known as dogpile or thundering herd problem, occurs when multiple requests for the same data hit the cache simultaneously, leading to a surge in backend requests. Here are some common techniques to mitigate cache stampede:
1. Locking
Locking ensures that only one request regenerates the cache while others wait. When a cache miss occurs, a lock is acquired for the cache key, and only the process holding the lock recomputes the data. Other processes either wait for the lock to be released or handle the absence of the value in other ways, such as returning a "not-found" response or using stale data[2][3][4].
2. External Recomputation
This technique involves using an external process to regenerate cache values independently of the requests. The external process can be triggered periodically, when the cache value is about to expire, or upon a cache miss. This approach offloads the recomputation from the main processes, but it requires additional resources and maintenance[1][2][3][6].
3. Probabilistic Early Expiration
In this method, each process makes an independent probabilistic decision to recompute the cache value before it expires. The probability increases as the expiration time approaches, spreading out the recomputation load over time. This technique helps avoid simultaneous cache misses but requires at least one request per time-to-live (TTL) period to be effective[3][6].
4. Adding Jitter to TTL
Adding random variations (jitter) to the TTL of cache entries ensures that they do not all expire at the same time. This spreads out the cache regeneration requests over a period, reducing the likelihood of a stampede[8].
5. Asynchronous Caching
Using asynchronous mechanisms, such as promises, allows the cache regeneration process to notify clients once the data is ready. This helps manage the load more efficiently by avoiding multiple simultaneous recomputation attempts[1].
6. Circuit Breaker Pattern
A circuit breaker acts as a kill-switch that trips when a certai...
