Mastering Singleflight: Stop Cache Stampedes Safely

When a hot cache key expires in a high-throughput system, hundreds of concurrent requests often hit the backend at the same millisecond to rebuild it. The Singleflight pattern—also known as request coalescing—eliminates this cache stampede by merging duplicate in-flight calls into a single execution without the overhead of distributed locks.

While the concept is straightforward, implementing it in production exposes several subtle concurrency traps around context lifecycles, memory mutations, and error blast radiuses.

Anatomy of a Cache Stampede

Under normal conditions, an in-memory cache like Redis shields your database by serving reads in sub-millisecond time. The danger appears during key expiration or sudden cache evictions.

If a key receiving 2,000 requests per second expires, every incoming worker discovers a cache miss almost simultaneously. Before the first worker can execute the database query and repopulate the cache, hundreds of subsequent workers launch identical queries.

This phenomenon—known as a cache stampede or thundering herd—quickly exhausts database connection pools, drives CPU usage to 100%, and increases latency across the entire application.

Many teams attempt to solve this with distributed mutexes (such as Redis-based locks). However, distributed locks add network round-trips, require TTL tuning to avoid deadlocks, and complicate failure recovery. In contrast, request coalescing solves duplicate work directly inside the application process.

How Singleflight Works Under the Hood

Diagram illustrating multiple concurrent requests coalescing into a single database query

Go provides a canonical implementation in golang.org/x/sync/singleflight. The core structure maintains a mutex-protected map of in-flight call descriptors, each containing a sync.WaitGroup and the result payload.

package repository

import (
	"context"
	"fmt"
	"golang.org/x/sync/singleflight"
)

var group singleflight.Group

func FetchProduct(ctx context.Context, id string) (*Product, error) {
	key := fmt.Sprintf("product:%s", id)

	v, err, shared := group.Do(key, func() (any, error) {
		// Only one goroutine reaches this expensive call
		return db.QueryProduct(context.Background(), id)
	})
	if err != nil {
		return nil, err
	}

	// Deep copy to prevent data races on shared pointer
	p := v.(*Product)
	return p.Clone(), nil
}

When goroutine A calls group.Do(key, fn), the group registers the key and executes fn. If goroutines B and C arrive while fn is still executing, they see the existing key, skip executing fn, and block on WaitGroup.Wait(). Once fn finishes, the result is delivered to all three callers, and the entry is removed from the map.

Trap 1: The First-Caller Context Trap

The most common production bug with Singleflight occurs when passing the HTTP request context into the shared worker function.

If Goroutine A initiates the flight with its own r.Context(), and that client cancels the request or encounters a client-side timeout after 20ms, the context gets canceled. If the underlying database driver respects context cancellation, the query aborts immediately. Goroutines B and C—which might have generous timeouts—now receive an unexpected context canceled error instead of valid data.

To prevent this cascading failure, decouple the shared workload from any individual caller's lifecycle:

Use a detached background context with an explicit, bounded operational timeout inside the Singleflight closure.
Alternatively, use group.DoChan to listen on the caller context and the result channel simultaneously, allowing early exits without terminating the shared execution.

Trap 2: The Shared Pointer Data Race

Because Singleflight distributes the exact same any return value to all waiting goroutines, returning mutable pointers creates hidden data races.

Suppose the shared function returns *UserProfile. Caller B receives the pointer and updates a localized property before serializing to JSON. Meanwhile, Caller C reads from the same pointer. This leads to silent memory corruption or concurrent read/write panics.

Always enforce strict immutability on coalesced outputs:

Return value types (structs) rather than pointer types when payloads are small.
Implement an explicit .Clone() method for reference types before mutating fields.
Treat Singleflight responses as strictly read-only before caching them.

Trap 3: Unbounded Hangs and Error Blast Radius

Coalescing concentrates risk: if the single upstream call hangs on a stalled socket, every waiting caller hangs with it. If the upstream returns an error, all waiters receive that same error at once.

To build a resilient data tier:

Combine Singleflight with a short circuit breaker so repeated downstream failures fail fast.
Use group.Forget(key) immediately inside the callback if you do not want long-running background routines to block incoming traffic for that key.
Pair Singleflight with Stale-While-Revalidate caching: serve expired data instantly while a background singleflight repopulates the fresh entry.

Summary

Request coalescing provides immense leverage, shrinking hundreds of database hits down to a single upstream query during traffic spikes. By detaching caller contexts, cloning shared pointers, and bounding execution timeouts, you get the full resilience of distributed locking with zero network overhead.