Request Coalescing: Stop Cache Stampedes with Singleflight

A caching layer that protects your database all day can become the exact mechanism that destroys it the moment a hot key expires. When thousands of concurrent clients request an expired key simultaneously, every single worker misses the cache and runs the expensive database query at the exact same millisecond.

This phenomenon—known as a cache stampede or thundering herd—turns routine key invalidation into database outages. Distributed locks like Redis mutexes can prevent redundant reads, but they introduce network overhead, lock contention, and cascading timeout failures. Request coalescing (often referred to as the singleflight pattern) solves this problem in memory with minimal complexity.

The Anatomy of a Cache Stampede

Most web services implement caching through a standard read-through pattern:

1. Check the in-memory or Redis cache.
2. If present, return the cached value immediately.
3. If absent, execute an SQL query or call a third-party API, write the result to the cache, and return.

Under low traffic, this works seamlessly. However, imagine an e-commerce flash sale where a product endpoint receives 3,000 requests per second. If that product cache key has a 5-minute TTL, request 3,001 encounters an expired key. While that request is busy querying PostgreSQL and serializing the response, 200 other requests arrive in the next 50 milliseconds.

Because none of those subsequent requests see data in the cache yet, all 200 fire identical heavy queries to PostgreSQL. Connection pools exhaust, database CPU saturates at 100%, query latency spikes from 10ms to 5,000ms, and your upstream application begins throwing 504 Gateway Timeouts.

The Singleflight Solution

Request coalescing eliminates redundant inflight work. The concept was popularized by Go's golang.org/x/sync/singleflight package [0].

Instead of letting every cache miss execute independently, the application gates the fetch behind an in-memory execution registry:

• The first request arrives, sees a cache miss, registers an inflight execution token (a Promise or Goroutine channel) for key product:101, and starts the database query.
• Requests 2 through 200 arrive for product:101. Seeing that an execution token already exists for that key, they do not touch the database. Instead, they subscribe to the first request's pending result.
• Once the database query completes, the original request publishes the payload to the cache and resolves all 200 waiting callers with the identical response.
• The key is cleared from the inflight registry, ready for future misses.

Rather than executing 200 database queries, the database executes exactly one.

A Minimal Implementation in TypeScript

You do not need a complex distributed coordination framework to implement request coalescing. In a Node.js or Bun service, you can build a robust in-memory coalescer in under twenty lines:

export class RequestCoalescer<T> {
  private inflight = new Map<string, Promise<T>>();

  async run(key: string, fn: () => Promise<T>): Promise<T> {
    const existing = this.inflight.get(key);
    if (existing) {
      return existing;
    }

    const promise = (async () => {
      try {
        return await fn();
      } finally {
        this.inflight.delete(key);
      }
    })();

    this.inflight.set(key, promise);
    return promise;
  }
}

Integrating this into your data access layer wraps your cache-miss handler cleanly:

const coalescer = new RequestCoalescer<Product>();

async function getProduct(id: string): Promise<Product> {
  const cached = await redis.get(`product:${id}`);
  if (cached) return JSON.parse(cached);

  return coalescer.run(`product:${id}`, async () => {
    // Double-check cache inside coalescer boundary to handle rapid sequential runs
    const doubleCheck = await redis.get(`product:${id}`);
    if (doubleCheck) return JSON.parse(doubleCheck);

    const product = await db.product.findUnique({ where: { id } });
    await redis.set(`product:${id}`, JSON.stringify(product), "EX", 300);
    return product;
  });
}

Practical Traps to Avoid

While the concept is straightforward, production deployments require attention to three specific failure modes.

1. Error Propagation

If the underlying fetcher throws an error (such as a database connection timeout), that rejection will propagate to every caller waiting on that promise. Make sure temporary network blips do not permanently poison downstream callers, and verify that the finally block always deletes the inflight entry so future callers can retry.

2. Context and Cancellation Traps

In Go or async runtimes with cancellation tokens, never tie the lifecycle of the shared background fetch to the first caller's request context. If the first user closes their browser tab, their HTTP context is cancelled. If the coalesced operation inherits that context, it will abort midway, failing all other 199 active requests waiting on that same key. Always run the shared fetching task on a detached context or supervisor signal.

3. Object Mutation

In runtime environments that pass data by reference (like Node.js or Go pointer structs), all waiting callers receive references to the exact same object in memory. If one downstream middleware mutates a field on that returned object, it inadvertently alters the response for every other concurrent user. Always treat coalesced returns as immutable, or deep-clone before mutation.

Small Footprint, Massive Stability

Request coalescing acts as a high-density shock absorber. It does not replace Redis or edge caching; rather, it bridges the perilous sub-second gap when keys expire under heavy traffic. Adding this lightweight primitive to your data tier ensures your database load remains predictable regardless of concurrent spike volume.

Sources

1. pkg.go.dev