Cache Stampede: When Your Cache Expiring Takes Down the Database

RedBite caches its homepage “trending products” list in Redis, TTL 60 seconds, because computing it means scanning order volume across the last 24 hours — an expensive aggregate query. Traffic is steady: a few hundred requests a second, all reading the same cache key, all cheap.

At 14:32:00 the key expires. In the next 40 milliseconds, roughly 2,000 requests for the homepage arrive — normal traffic, nothing unusual. Every single one of them checks Redis, gets a cache miss, and falls through to the same expensive Postgres aggregate query. Postgres, which was happily serving a few hundred queries a second, now gets 2,000 copies of the exact same query in under a second. CPU spikes, connection pool saturates, unrelated requests start timing out.

Nothing changed about demand. The cache didn’t get slower. One key expired, and every request that would normally have been answered by Redis in under a millisecond instead went to the database — at the same instant. This is a cache stampede (also called the thundering herd problem), and it happens to every cache with a TTL, the first time it gets enough concurrent traffic on one key.

The naive cache wrapper, and why it stampedes

Here’s the obvious way to write “check cache, fall back to the database”:

async function getTrendingProducts() {
  const cached = await redis.get('trending:products');
  if (cached) return JSON.parse(cached);

  const products = await computeTrendingFromPostgres(); // expensive
  await redis.set('trending:products', JSON.stringify(products), 'EX', 60);
  return products;
}

This is correct for one request at a time. It is not correct for 2,000 requests arriving within the same 40ms window. Walk through what each of those 2,000 calls actually does:

1. redis.get('trending:products') — all 2,000 run this within microseconds of each other, right after the key expired. All 2,000 get null.
2. Every single one of them independently decides “cache miss, I need to compute this myself,” and calls computeTrendingFromPostgres().
3. 2,000 copies of the same expensive query hit Postgres simultaneously.

Nothing in step 1 or step 2 ever asked “is someone else already computing this?” Each request only knows about itself — same root shape as the checkout race condition: a decision made from stale shared state, with no coordination between the requests making it.

Fix: request coalescing — only the first request computes, the rest wait on it

The fix doesn’t touch the TTL or the query. It adds one rule: while a value is being computed, every other request for that same key waits for the result instead of starting its own computation. Of 2,000 requests, exactly one talks to Postgres; the other 1,999 get the same answer for free.

The coordination point is an in-process map of “is anyone already computing this key,” keyed by cache key, holding a Promise:

const inFlight = new Map(); // key -> Promise<value>

async function getTrendingProducts() {
  const cached = await redis.get('trending:products');
  if (cached) return JSON.parse(cached);

  if (inFlight.has('trending:products')) {
    // someone else is already computing this — wait for their result
    return inFlight.get('trending:products');
  }

  const promise = (async () => {
    try {
      const products = await computeTrendingFromPostgres();
      await redis.set('trending:products', JSON.stringify(products), 'EX', 60);
      return products;
    } finally {
      inFlight.delete('trending:products');
    }
  })();

  inFlight.set('trending:products', promise);
  return promise;
}

Walk through the same 2,000 requests with this version. The first one to run, call it request #1, finds no cached value and no in-flight entry — it starts the computation and stores the Promise in inFlight before awaiting anything. Requests #2 through #2,000 arrive microseconds later, also miss the cache, but now find inFlight.has('trending:products') is true — they grab request #1’s Promise and await it instead of calling Postgres themselves. When request #1’s query finishes, all 1,999 waiters resolve with the same result, and the finally block clears the entry for the next expiry cycle.

Caveat that matters: this Map lives in process memory. It coalesces requests within one server process. If RedBite runs 8 API instances behind a load balancer, you still get up to 8 concurrent Postgres queries on expiry — one per instance — instead of 2,000. That’s a 250x reduction, usually enough, but if you need it down to exactly 1 across the whole fleet, the in-process map isn’t sufficient on its own.

Taking it further: a distributed lock for multi-instance coalescing

To get to exactly one computation across every instance, move the coordination into Redis itself with a short-lived lock:

async function getTrendingProducts() {
  const cached = await redis.get('trending:products');
  if (cached) return JSON.parse(cached);

  const gotLock = await redis.set(
    'trending:products:lock', '1', 'NX', 'EX', 10
  );

  if (!gotLock) {
    // another instance is computing — poll briefly for its result
    for (let i = 0; i < 20; i++) {
      await new Promise((r) => setTimeout(r, 100));
      const result = await redis.get('trending:products');
      if (result) return JSON.parse(result);
    }
    // lock-holder stalled past 2s — fall back to computing anyway
  }

  try {
    const products = await computeTrendingFromPostgres();
    await redis.set('trending:products', JSON.stringify(products), 'EX', 60);
    return products;
  } finally {
    if (gotLock) await redis.del('trending:products:lock');
  }
}

SET ... NX is atomic in Redis, so exactly one instance across the whole fleet wins the lock; every other instance — on any server — polls Redis for the result the winner writes instead of querying Postgres itself. The 10-second lock TTL is the safety net for a crashed lock-holder: without it, a process that dies mid-computation leaves the lock held forever and every other instance stalls out its poll loop and falls back to computing anyway, which is correct but defeats the point until the lock expires.

Which one RedBite should ship

Start with the in-process map — it’s a 15-line change, no new infrastructure, and it already turns a 2,000x spike into an N-instance spike where N is your server count. Reach for the Redis lock only once you’ve measured that N concurrent queries on expiry still hurts, not as a reflexive first move — it adds a lock key, a poll loop, and a TTL you have to get right, for a problem the in-process version may have already solved.

Hit a stampede that neither of these covers? Drop it in #scaling on Discord.