Stop Cache Stampede: Protect Your DB from Thundering Herd

Imagine this all-too-common production scenario: your application is serving peak traffic smoothly. Your Redis cache holds hot data—say, the homepage product list or top trending posts—reducing database load from tens of thousands of queries per second down to single digits. Everything feels fast and rock-solid.

Then, at precisely 14:00:00, that key expires.

In a fraction of a millisecond, 2,000 concurrent user requests hit your API servers. They all check Redis, find a cache miss, and immediately attempt to query your PostgreSQL database to recalculate the exact same dataset. Your database connection pool is instantly exhausted, CPU utilization spikes to 100%, request timeouts cascade across your microservices, and your app crashes.

Welcome to the Cache Stampede (also known as the Thundering Herd problem). In this article, let's explore why this happens and three practical patterns to eliminate it from your production systems.

The Naive Caching Trap

Most developers start caching with a simple read-through pattern. It usually looks something like this:

async function getTrendingProducts(): Promise<Product[]> {
  const cacheKey = "products:trending";
  const cachedData = await redis.get(cacheKey);

  if (cachedData) {
    return JSON.parse(cachedData);
  }

  // CACHE MISS: Fetch from DB and populate cache
  const products = await db.products.findMany({ where: { isTrending: true } });
  await redis.set(cacheKey, JSON.stringify(products), "EX", 300); // 5 mins

  return products;
}

This code works perfectly in local testing and low-traffic environments. However, under high concurrency, when products:trending expires, dozens or hundreds of requests execute db.products.findMany() simultaneously before the first request finishes and populates Redis.

If the DB query takes 300ms to complete, every single request arriving during those 300ms becomes an unbuffered database hit.

Solution 1: Distributed Mutex Locking (Single Flight)

The most direct solution is ensuring that only one process recalculates the cache at any given time, while all other concurrent requests either wait for that single calculation to finish or receive a fallback.

In TypeScript/Node.js or Go, this is sometimes called the "SingleFlight" pattern. Using Redis, we can achieve this with atomic locks using SET key value NX PX ttl:

async function getTrendingProductsLocked(): Promise<Product[]> {
  const cacheKey = "products:trending";
  const lockKey = "lock:products:trending";

  // 1. Check cache first
  const cachedData = await redis.get(cacheKey);
  if (cachedData) return JSON.parse(cachedData);

  // 2. Try to acquire lock (valid for 5 seconds)
  const acquired = await redis.set(lockKey, "1", "NX", "PX", 5000);

  if (!acquired) {
    // 3. Couldn't get lock -> Wait briefly and retry reading cache
    await new Promise((resolve) => setTimeout(resolve, 100));
    return getTrendingProductsLocked();
  }

  try {
    // 4. Acquired lock: We are the sole worker building the cache
    const products = await db.products.findMany({ where: { isTrending: true } });
    await redis.set(cacheKey, JSON.stringify(products), "EX", 300);
    return products;
  } finally {
    // 5. Always release lock when done
    await redis.del(lockKey);
  }
}

Trade-offs

• Pros: Absolutely guarantees that only one DB query executes during a cache miss.
• Cons: Waiting requests experience increased response latency. You must also set lock TTLs carefully to prevent deadlocks if the worker crashes.

Solution 2: Stale-While-Revalidate (SWR) with Soft Expiration

Instead of letting the cache key hard-expire in Redis, we store a payload containing both the data and an explicit soft_expire_at timestamp.

When a request reads the cache:

• If now < soft_expire_at: Return data immediately (fresh cache).
• If now >= soft_expire_at: Return the stale data immediately, but trigger an asynchronous background task to recompute the fresh data and update Redis.

interface CachePayload<T> {
  data: T;
  softExpireAt: number;
}

async function getTrendingProductsSWR(): Promise<Product[]> {
  const cacheKey = "products:trending";
  const raw = await redis.get(cacheKey);

  if (raw) {
    const payload: CachePayload<Product[]> = JSON.parse(raw);
    const isStale = Date.now() > payload.softExpireAt;

    if (isStale) {
      // Trigger background refresh non-blockingly
      refreshCacheInBackground(cacheKey).catch(console.error);
    }

    // Return cached data instantly, even if slightly stale!
    return payload.data;
  }

  // Fallback for cold start (no cache exists at all)
  return rebuildCache(cacheKey);
}

Trade-offs

• Pros: Zero latency impact for end users—reads are always fast because they serve stale cached data while updating in the background.
• Cons: Requires tolerating slightly outdated data for a brief window during revalidation.

Solution 3: Probabilistic Early Expiration (XFetch)

If stale data is unacceptable and background workers add operational complexity, consider Probabilistic Early Expiration (often implemented via the XFetch algorithm).

Instead of waiting for TTL to hit zero, requests randomly choose to recompute the cache before it actually expires. The closer the key is to expiring, and the longer the computation takes, the higher the probability that a request will trigger a refresh.

The core evaluation decision uses this formula:

now - (delta * beta * log(rand())) > expiry

Where:

• delta: Time (in seconds/ms) it took to compute the value originally.
• beta: Aggressiveness multiplier (usually 1.0).
• rand(): Pseudo-random float between 0 and 1.

function shouldRecompute(expiryTimestamp: number, delta: number, beta = 1.0): boolean {
  const now = Date.now();
  const randomFactor = -Math.log(Math.random());
  return now - delta * beta * randomFactor > expiryTimestamp;
}

Because rand() introduces randomness, only one or two incoming requests will decide to early-refresh the cache shortly before actual expiration, seamlessly keeping the cache fresh without any herd effect.

Which Pattern Should You Pick?

Here is a quick cheat sheet for production decisions:

• E-commerce Product Pages / Feed Lists: Use Stale-While-Revalidate (SWR). Users prefer a 5-second old product list delivered in 10ms over a fresh list delivered in 2,000ms.
• Financial / Inventory Data: Use Distributed Mutex Lock. You cannot afford stale balances or incorrect stock counts, so forcing single-flight execution is worth the small latency penalty.
• High-throughput API Gateways: Use Probabilistic Early Expiration (XFetch). It requires no background workers or lock management overhead while delivering high stability.

Wrapping Up

Cache stampede is one of those latent bugs that stays silent in staging and destroys your database on peak production traffic. By moving away from naive cache-miss logic and adopting locking, SWR, or probabilistic early expiration, your backend will remain resilient no matter how heavy the traffic gets.

What caching strategy are you currently using in your stack? Let's discuss in the comments!