Caching Strategies

Why cache at all

A cache is a copy of data kept somewhere cheaper to reach than the source. Every cache decision is really a bet: that the same value will be requested again before it changes, and that serving a slightly old copy is acceptable. When both hold, the payoff is enormous.

Three effects follow from a single hit:

• Latency. A Redis lookup answers in well under a millisecond; the same row from PostgreSQL costs several milliseconds of network, planning and I/O. On a page that reads twenty things, that difference is the whole experience.
• Load. If 95% of reads are served from cache, the database sees one query where it used to see twenty. You can survive a traffic spike, or run a smaller instance, without changing a line of query logic.
• Cost. Fewer database reads mean smaller instances, fewer read replicas and less cross-region traffic. At scale, caching is one of the few optimisations that pays for itself in dollars.

The trade is complexity. A cache is a second copy of the truth, and every copy can disagree. Most of this guide is about keeping that disagreement small and temporary.

Where a cache can live

Caching is not one system but a stack of layers, each closer to the reader and each with a different invalidation story.

• The browser. HTTP responses marked Cache-Control: max-age are reused without a request. This is free, private and largely under the client’s control.
• The CDN or edge. A shared cache in front of your origin absorbs traffic for public responses. It is fast and global, but a mistake here is visible to everyone.
• The application. An in-process cache (a Map, an LRU) is the fastest tier because it avoids the network entirely. It is also per-instance, so it must be small and disposable.
• A shared store. Redis or Memcached sits beside the application and serves every instance from one consistent copy. This is where most application caching lives.
• The database. Buffer pools, materialized views, prepared plans and query result caches keep the source fast on its own. They are the last line before disk.

A request can be answered at any of these layers. The closer the answer, the faster it is and the harder it is to invalidate, which is the central tension of the whole subject.

Cache hit rate: the metric that decides

The health of a cache is its hit rate: hits divided by hits plus misses. It is the only number that tells you whether the cache is doing anything.

hit rate = hits / (hits + misses)

The arithmetic is dramatic. At a 90% hit rate the database sees one in ten requests, a 10x reduction. At 50% it sees one in two, only a 2x reduction, and the cache has added a network hop to half the traffic. A cache with a low hit rate can be slower than no cache at all.

Measure it from the store itself. Redis reports keyspace_hits and keyspace_misses in INFO stats, and client libraries expose the same counters. Watch the hit rate after every change to a key, a TTL or a query; a small key change can quietly drop a 95% cache to 40%.

The corollary is that you must cache things that are actually repeated. A unique-per-request key has a 0% hit rate by construction and only wastes memory.

The four core patterns

Almost every cache implementation is one of four shapes. They differ in who fills the cache and when the cache is updated.

Cache-aside (lazy loading)

The application owns the logic. On a read it checks the cache, and on a miss it loads from the source and writes the result back. This is the default pattern because it is simple, works with any store, and only caches data that is actually requested.

async function getUser(id: string) {
  const key = `user:${id}`;

  const hit = await redis.get(key);
  if (hit) return JSON.parse(hit);

  const user = await db.user.findUniqueOrThrow({ where: { id } });
  await redis.set(key, JSON.stringify(user), "EX", 600);
  return user;
}

The cost is that the first reader always pays full price, and the cache can hold stale data until its TTL expires or a write removes it.

Read-through

Read-through moves the fallback into the cache layer itself: the application always asks the cache, and the cache is configured with a loader that runs on a miss. Some libraries and proxies implement this, so application code has no explicit miss path. The behaviour is the same as cache-aside; only the location of the logic changes.

Write-through

Write-through updates the cache and the source in the same operation. Reads are always hot and never see data older than the last successful write.

async function updateUser(id: string, patch: Partial<User>) {
  const user = await db.user.update({ where: { id }, data: patch });
  await redis.set(`user:${id}`, JSON.stringify(user), "EX", 600);
  return user;
}

It costs write latency and caches data that may never be read, but it keeps the cache consistent with the write path, which is exactly what user-facing updates need.

Write-behind (write-back)

Write-behind writes to the cache immediately and flushes to the source asynchronously. Writes are very fast and can be batched, which is valuable for counters and telemetry. The risk is severe: if the cache dies before the flush, the write is gone. Use it only where losing the last few writes is acceptable, and never for money or records of record.

TTLs and freshness

Every cached value should have a time to live. The TTL is a staleness budget: it is the maximum time a reader can see a value that is no longer true in the source. Choosing it is a product decision as much as a technical one.

• Prices and inventory: seconds. A wrong price is a support ticket.
• Profiles, feeds and listings: minutes. Small drift is invisible.
• Reference data, feature flags and configuration: minutes to hours.
• Static assets and rarely changing content: days, versioned by filename.

Add jitter to the TTL. If a thousand keys are written at the same moment with the same TTL, they expire together and produce a spike. Randomising by a few percent spreads the reloads out.

const ttl = 600 + Math.floor(Math.random() * 60);
await redis.set(key, JSON.stringify(value), "EX", ttl);

Even with explicit invalidation, keep a TTL as a safety net. A bug that forgets to delete a key should cost you minutes of staleness, not a permanent lie.

Invalidation: the hard part

There is a reason cache invalidation is the punchline of the oldest joke in computer science. The value is trivial to store and genuinely hard to remove at the right moment, from every layer, without races.

Three techniques cover almost every case.

Key versioning. Instead of deleting, change the key. Prefix keys with a version that you bump when the underlying data changes, so old entries become unreachable and expire on their own.

const version = (await redis.get(`user:${id}:v`)) ?? "1";
const key = `user:${id}:v${version}`;

This is race-free and works across instances, which is why it is the preferred approach for anything shared.

Explicit bust. Delete the key on write. It is the most obvious approach and the easiest to get wrong, because every write path must remember to do it.

await db.user.update({ where: { id }, data: patch });
await redis.del(`user:${id}`);

Event-driven invalidation. Publish a change event and let every instance and layer react. This is how a CDN purge or a multi-service cache is kept honest, and it scales to systems you do not own.

Whichever you choose, be consistent about ordering, and prefer deleting to overwriting on update. Deleting is idempotent; overwriting can resurrect a value that a concurrent writer already superseded.

Cache stampede and thundering herd

When a popular key expires, every request in flight misses at the same instant. If a thousand requests arrive per second and the key takes 200 ms to rebuild, hundreds of identical queries hit the database together. That is a cache stampede, also called the thundering herd, and it can take down a service that was healthy a moment earlier.

Four mitigations, in rough order of preference:

• Stale-while-revalidate. Serve the stale value immediately and refresh it in the background. Readers never wait and the source sees one refresh. HTTP has a header for exactly this.
• Locking or single-flight. Only the first caller recomputes; everyone else waits briefly or serves stale data. Redis SET key value NX EX is a simple distributed lock.
• Jittered TTLs. Spread expiries so all keys do not die together.
• Probabilistic early expiration. Each read has a small, increasing chance of refreshing before the key actually expires, so the refresh is spread across many requests.

The lock must always have an expiry, or a crashed worker leaves the key locked forever.

What to cache and what never to

Cache data that is expensive to produce, read far more often than it changes, tolerant of small staleness, and shared across many readers. Product listings, public profiles, configuration, rendered fragments and expensive aggregates are all good candidates.

Never cache:

• Authorization decisions in a shared cache. A role change or logout must take effect immediately, and a CDN must never serve one user’s private response to another.
• Per-user sensitive data under a shared key. Every user-specific value needs a key that includes the user, and a private cache directive if it ever reaches an edge.
• Secrets and credentials. A cache is another place they can leak from.
• Data you cannot invalidate. If a value has no natural key and no event, a cache will eventually serve something wrong with no way to fix it.

A useful test: if you cannot describe how a value leaves the cache, do not put it in.

HTTP caching at the edge

The cheapest cache is the one that never reaches your server. HTTP gives you precise control over it.

Cache-Control: public, max-age=60, s-maxage=600, stale-while-revalidate=30
ETag: "33a64df551425fcc55e4d42a148795d9f25f89d4"
Vary: Accept-Encoding

max-age is how long the browser may reuse the response. s-maxage overrides it for shared caches such as a CDN. stale-while-revalidate lets the edge serve a stale copy while it fetches a fresh one in the background, which removes stampedes for public content. private and no-store keep sensitive responses out of shared caches entirely.

An ETag enables conditional requests. The client sends If-None-Match, and if the tag still matches you answer 304 Not Modified with no body, saving bandwidth while guaranteeing freshness.

app.get("/posts", async (req, res) => {
  const body = JSON.stringify(await listPosts());
  const tag = `"${createHash("sha256").update(body).digest("hex")}"`;

  res.set("Cache-Control", "public, max-age=30, stale-while-revalidate=60");
  res.set("ETag", tag);

  if (req.headers["if-none-match"] === tag) return res.status(304).end();
  res.type("application/json").send(body);
});

Vary is easy to forget and important: if a response depends on Accept-Encoding or Accept-Language, say so, or a cache will hand the wrong variant to the wrong client.

In-memory vs Redis vs CDN

The three shared layers are not competitors; they are a hierarchy.

An in-process cache is the fastest possible lookup because it never crosses the network. It is ideal for small, hot, read-mostly data such as configuration or a permission map. Its limits are that each instance has its own copy, so memory is multiplied and values can diverge, and it disappears on deploy.

Redis is the shared application cache. One logical copy serves every instance, it survives restarts, and it supports TTLs, atomic operations and data structures beyond simple strings. It costs a network round-trip, usually well under a millisecond on the same network.

A CDN is the outermost layer. It caches public responses close to users around the world and can absorb enormous traffic before it reaches you. It only works for responses that are identical for many users, and purging it is a deliberate, sometimes slow, operation.

A common production shape is a tiny in-process cache in front of Redis for the hottest keys, with a CDN in front of the whole API for public GETs.

Negative caching

Caches are usually described as storing values, but storing the absence of a value is just as useful. If a query for a missing record is expensive and repeated, cache the miss.

const hit = await redis.get(key);
if (hit === "__miss__") return null;

const user = await db.user.findUnique({ where: { id } });
if (!user) {
  await redis.set(key, "__miss__", "EX", 60);
  return null;
}

Keep negative TTLs short, because a record that did not exist a minute ago may exist now. The pattern defends against cache penetration, where a flood of requests for nonexistent keys bypasses the cache and hits the database every time. Guard against an attacker generating endless unique missing keys by validating input and rate limiting before the cache.

Consistency and the two hard things

A cache makes a system eventually consistent: for a short window, different readers can see different values. That is usually fine, but some situations are not.

• Read-your-writes. A user who just updated their profile expects to see the change. Invalidate on write, or read from the source for a short period after a write by the same user.
• Replication lag. If reads go to a replica, a cache filled from that replica can lag the primary. Invalidate from the primary’s write path.
• Cross-region caches. A purge in one region does not instantly reach another. Version keys or accept the propagation delay.

The honest framing is that caching trades consistency for speed, and the only way to do it safely is to decide explicitly which staleness is acceptable and to build invalidation into the write path rather than bolting it on.

Cache keys are an API

A cache key looks like an implementation detail and behaves like a public interface. It is what every reader and writer must agree on, and it is visible in Redis, in slow logs and on dashboards. Design keys deliberately.

Three rules keep them sane:

• Namespace by version and environment. v1:user:42 lets you change the shape later and keeps staging from sharing keys with production.
• Be deterministic. The same logical lookup must always produce the same string. Normalise case, trim input and never include a value that changes per call.
• Never put secrets or personal data in a key. Keys are logged, exported and shown to operators.

export const cacheKeys = {
  user: (id: string) => `v1:user:${id}`,
  userPosts: (id: string, cursor: string) => `v1:user:${id}:posts:${cursor}`,
  productBySku: (sku: string) => `v1:product:sku:${sku.trim().toLowerCase()}`,
};

Centralising key construction in one module is what makes invalidation possible. When a write needs to drop a key, it calls the same function the read used; when the shape changes, there is one place to bump the version.

Eviction policies

A cache that is allowed to grow without bound is a memory leak with extra steps. Both Redis and in-process caches need a ceiling and a policy for what to remove when it is reached.

Redis exposes maxmemory and maxmemory-policy. The common choice for a pure cache is allkeys-lru, which evicts the least recently used key, or allkeys-lfu, which favours frequently used ones when access patterns are skewed.

redis-cli CONFIG SET maxmemory 2gb
redis-cli CONFIG SET maxmemory-policy allkeys-lru

Never use noeviction for a cache: once memory is full, writes fail and your application starts throwing errors instead of simply missing. For in-process caches, use a bounded LRU library rather than a plain Map, and set both a maximum entry count and a maximum size.

Evictions are not failures, but a rising eviction count is a signal. It means the working set no longer fits and the hit rate is about to fall. Either give the cache more memory or cache less.

Cache warming

A cache is coldest the moment it is most dangerous: right after a deploy, a restart or a scale-out. Every instance starts empty, traffic arrives at full volume, and the source sees the full load until the cache fills. This is a stampede caused by your own deployment.

Two approaches work. Lazy warming accepts the cold window and relies on locking and stale-while-revalidate to survive it. It is simple and needs no extra machinery. Proactive warming runs a job that loads the known hot keys before the new version takes traffic.

async function warmCache() {
  const popular = await db.product.findMany({
    orderBy: { views: "desc" },
    take: 500,
    select: { id: true },
  });

  for (const { id } of popular) {
    await cached(`product:${id}`, 300, () =>
      db.product.findUniqueOrThrow({ where: { id } }),
    );
  }
}

Warm only what you know is hot. Warming everything is just a slow, expensive copy of the database that will mostly be evicted unused.

Observability for caches

A cache you do not measure is a cache you cannot trust. Four numbers tell the story.

• Hit rate — are reads actually being served from the cache?
• Latency — is a hit meaningfully cheaper than the source, including the network hop?
• Evictions — is the working set outgrowing the memory?
• Errors and timeouts — is the cache becoming a source of failures?

redis-cli INFO stats | grep -E 'keyspace_(hits|misses)|evicted_keys'
redis-cli INFO memory | grep used_memory_human

Emit a counter for hits and misses from the application as well, tagged by cache name. That is what lets you correlate a hit-rate drop with a deploy, and it is the first thing to check when database load rises for no obvious reason.

Graceful degradation when the cache is down

If losing the cache takes down the API, the cache has become a single point of failure for a system whose whole purpose is to be optional. The source of truth still exists; the application should be able to reach it.

Wrap every cache operation so a failure becomes a miss rather than an error, keep timeouts short, and consider a circuit breaker that stops calling a failing cache for a cooldown period.

async function safeGet(key: string) {
  try {
    return await redis.get(key);
  } catch (err) {
    logger.warn({ err, key }, "cache_read_failed");
    return null;
  }
}

The same applies to writes to the cache: a failed set should never fail the request. The one exception is write-behind, where the cache is part of the write path; that is another reason to reserve it for data you can afford to lose.

Caching expensive queries and aggregates

Not everything worth caching is a single row. Expensive joins, dashboard aggregates and search results are often the best candidates, because they are slow to compute and change slowly.

A materialized view is a cache that lives in the database: it stores the result of a query and is refreshed on a schedule or on demand. It gives you the freshness of a batch job and the read speed of a table.

CREATE MATERIALIZED VIEW daily_revenue AS
SELECT date_trunc('day', created_at) AS day,
       sum(total_cents)             AS revenue_cents
FROM orders
WHERE status = 'paid'
GROUP BY 1;

REFRESH MATERIALIZED VIEW CONCURRENTLY daily_revenue;

For results that are too dynamic for a view, cache the serialized response in Redis under a key that includes every input that affects it — filters, sort order and page. Two requests for different pages are different values, so they must be different keys.

Testing a cache

Cache bugs are the kind that pass tests and fail in production, so test the behaviours that matter explicitly: the miss path, the hit path, invalidation and failure.

test("loads once and serves the second read from cache", async () => {
  let calls = 0;
  const load = async () => {
    calls += 1;
    return { id: "1" };
  };

  await cached("user:1", 60, load);
  await cached("user:1", 60, load);

  expect(calls).toBe(1);
});

test("falls back to the source when the cache is unavailable", async () => {
  redis.get = async () => {
    throw new Error("connection_refused");
  };

  await expect(cached("user:1", 60, load)).resolves.toEqual({ id: "1" });
});

Use a real Redis in a container for integration tests, or an in-memory fake that implements get, set and del. Also test the negative cases: an update that should invalidate, a TTL that should expire, and a cache outage that should degrade instead of failing.

Best practices

• Cache only after measuring; add a cache to a slow, repeated read, not by default.
• Give every key a TTL, even when you also invalidate explicitly.
• Use deterministic, namespaced keys such as user:42:profile and version them for shared data.
• Prefer deleting or versioning keys over overwriting them on update.
• Add jitter to TTLs and use stale-while-revalidate to survive stampedes.
• Keep per-user and authorization data out of shared caches; mark responses private.
• Treat the cache as optional in code so an outage degrades instead of failing.
• Watch the hit rate and eviction count, not just latency.
• Cache misses as well as hits when lookups of absent data are expensive.

Common mistakes

• Caching without a TTL and relying on a manual purge that never comes.
• Sharing one key across users and leaking one person’s data to another.
• Invalidating in some write paths but not others.
• Setting the same TTL on every key so they all expire together.
• Caching an authorization decision that a role change should have revoked.
• Using a cache as a database and losing writes when it restarts.
• Forgetting Vary and serving the wrong content encoding from a CDN.
• Measuring nothing, so a key change silently drops the hit rate.
• Caching something unique per request, which guarantees a 0% hit rate.

Where to go next

Caching is one tool for controlling load; its natural companions are the others that bound work. Redis goes deep on the store most caches are built on, and Pagination is the request-level version of the same idea: never do more work than you must. If the database behind the cache is the bottleneck, Connection Pooling reduces the cost of each connection, and PostgreSQL covers the source of truth you are protecting.