Caching is storing a copy of data somewhere fast so you don't have to fetch it from somewhere slow on every request. The basics — using Redis, setting TTLs, cache-aside pattern — are covered in Caching Fundamentals. This article goes deeper into the parts that cause problems at scale: how caches decide what to throw out when they're full, how distributed caches work across multiple machines, the three classic failure modes that suddenly slam your database, and the genuinely hard problem of keeping cached data consistent with the real thing. If Cartara flagged something related to cache configuration, eviction policies, or Redis cluster setup, this is the context you need.
Read Patterns, One Level Deeper
Cache-aside (the most common pattern) means your application manages the cache: check the cache first, and if it's a miss, load from the database and store the result. The cache doesn't know about your database — your code does.
Read-through means the cache library itself handles misses by fetching from the database. Your code just asks the cache and doesn't think about misses. Less code, but you're tied to a cache layer that supports it and you have less control.
In practice, most Redis-based caching in web applications is cache-aside — you write the miss logic explicitly.
Eviction Policies: What Gets Thrown Out First
A cache has finite memory. When it fills up, it must evict something to make room. Which thing it evicts is the eviction policy, and it meaningfully affects your cache hit rate:
LRU (Least Recently Used) evicts whatever hasn't been touched in the longest time. The sensible default for most workloads — assumes recently used data will be used again soon.
LFU (Least Frequently Used) evicts whatever is accessed least often overall. Better when some items are persistently hot regardless of recency.
FIFO evicts the oldest item inserted, regardless of how often it's been accessed. Simple, rarely optimal.
TTL-based evicts whatever is closest to its expiration time.
In Redis, you choose a policy via configuration (allkeys-lru, allkeys-lfu, volatile-ttl, etc.). The volatile-* variants only evict keys that have a TTL set — useful when you're mixing cached data (which should be evicted) with persistent data (which should never be evicted) in the same Redis instance.
A rising eviction rate is a signal that your cache is too small for its working set — either grow it or cache less data.
The Three Classic Cache Failure Modes
These three problems have similar symptoms (your database suddenly gets hammered), but different causes and different fixes. Worth knowing by name.
Cache Stampede (also called Cache Breakdown or Dogpile)
A single hot key expires. Dozens or hundreds of concurrent requests that wanted that key all miss the cache at the same moment and hit the database simultaneously. The database gets slammed by a sudden spike from a single cache miss.
Fixes:
- TTL jitter — add a small random amount to each TTL so keys expire at slightly different times
- Mutex/lock — only one request rebuilds the cache entry; other requests wait for it
- Probabilistic early refresh — start refreshing a key a bit before it expires, rather than after
Cache Penetration
Requests for keys that don't exist anywhere — not in the cache, not in the database. Every request misses the cache and misses the database. An attacker sending random user IDs or product codes can trigger this deliberately, bypassing your cache entirely and hammering your database with useless queries.
Fixes:
- Cache the "not found" result too — store a short-TTL negative cache entry when a key doesn't exist in the database
- Bloom filter — a compact data structure that can cheaply answer "this key definitely doesn't exist" before hitting the cache or database at all
Cache Avalanche
A large number of keys expire at the same moment (perhaps you warmed them all at startup with identical TTLs), or the cache server itself goes down. The full traffic load falls on the database at once.
Fixes:
- Spread TTLs with jitter so expirations scatter over time rather than clustering
- Run the cache with redundancy (Redis Sentinel or Redis Cluster) so a single node failure doesn't take down everything
- Database-side protection — circuit breakers and rate limiting so the database doesn't collapse under the surge
What You'll See in Your Code
Redis configuration for eviction policy and max memory (in redis.conf or passed as arguments):
maxmemory 512mb
maxmemory-policy allkeys-lruImplementing a mutex to prevent cache stampede — only one worker rebuilds a hot key:
async function getWithLock(key: string, build: () => Promise<string>): Promise<string> {
// Check cache first
const cached = await redis.get(key);
if (cached) return cached;
// Try to acquire lock (SET NX = only set if not exists)
const lockKey = `lock:${key}`;
const acquired = await redis.set(lockKey, '1', 'NX', 'EX', 10); // 10s lock timeout
if (acquired) {
// We have the lock — build the value
try {
const value = await build();
// Add jitter to TTL (300s ± up to 60s)
const ttl = 300 + Math.floor(Math.random() * 60);
await redis.setex(key, ttl, value);
return value;
} finally {
await redis.del(lockKey);
}
} else {
// Someone else is building it — wait and retry
await new Promise(r => setTimeout(r, 100));
return redis.get(key) ?? build(); // fallback if still not ready
}
}Caching a negative result to prevent cache penetration:
const CACHE_MISS_SENTINEL = 'NOT_FOUND';
async function getUser(id: string): Promise<User | null> {
const cached = await redis.get(`user:${id}`);
if (cached === CACHE_MISS_SENTINEL) return null; // Cached "not found"
if (cached) return JSON.parse(cached);
const user = await db.users.findById(id);
if (!user) {
// Cache the "not found" for 60s to prevent penetration
await redis.setex(`user:${id}`, 60, CACHE_MISS_SENTINEL);
return null;
}
await redis.setex(`user:${id}`, 300, JSON.stringify(user));
return user;
}Distributed Caching
A single Redis server has a memory ceiling and is a single point of failure. At scale, you run a cluster of cache nodes — which raises a question: which node holds a given key?
Naive approach: hash(key) % number_of_nodes. The problem: add or remove one node and almost every key remaps to a different node — a mass cache miss that can trigger an avalanche.
Consistent hashing solves this by mapping keys and nodes onto a ring. Adding or removing a node only relocates the keys near it on the ring; everything else stays put. Redis Cluster uses this approach internally.
Two-tier caching is a powerful pattern for very high throughput: a small, ultra-fast local in-process cache on each app server (zero network latency) backed by a larger shared Redis cache. The local tier absorbs your hottest keys at no network cost; Redis provides capacity and coherence across servers. The cost is that local caches on different servers can briefly disagree — which leads to the hardest problem in caching.
Cache Consistency: The Hard Part
A cache is a copy of data that lives somewhere else. The moment the source changes, the copy is potentially wrong. Phil Karlton's famous observation — "there are only two hard things in Computer Science: cache invalidation and naming things" — is a joke, but it's a joke about a real problem.
Key realities to design around:
You're choosing staleness, not avoiding it. Even with event-driven invalidation, there's a window between the write and the cache update where reads return stale data. The honest question is: how much staleness is acceptable for this specific piece of data? Seconds for a product listing is fine. Zero for an account balance is required.
Invalidate rather than update. When the source data changes, delete the cache entry so the next read repopulates it from the database. Trying to write the new value into the cache races with other writers and can leave the cache in a wrong state.
Distributed caches make it worse. With local in-memory caches on many app servers, a database write on one server doesn't automatically invalidate the copies on the others. Solutions include broadcasting invalidation messages via Redis pub/sub, or keeping local TTLs short so they self-heal quickly.
Some data shouldn't be cached. Account balances, inventory counts, auth tokens, and anything where serving a stale value causes a real problem — don't cache these, or cache with very short TTLs and accept the cache barely helps.
The pragmatic approach: use short TTLs as a backstop even when you also do explicit invalidation. If an invalidation event is ever missed, the short TTL means the cache self-corrects within a bounded time window.
Measuring and Tuning
- Hit rate tells you how often the cache is actually helping. Below 80-90% and your cache may be undersized or caching the wrong things.
- Eviction rate tells you whether your cache is full — high evictions mean you're cycling through data faster than it stays useful.
- A high hit rate isn't automatically good if you're caching data that changes frequently and serving staleness. Measure correctness complaints alongside hit rate.
- Right-size TTLs to the data's actual change rate. Different data deserves different TTLs — configuration might be cached for hours, user-generated content for minutes.
For a Small Team
You almost certainly don't need distributed caching, consistent hashing, or two-tier caches yet. A single managed Redis instance (Upstash, Redis Cloud, or Railway Redis) with cache-aside and sensible TTLs handles an enormous amount of traffic.
What's worth internalizing now, even before you need it: the consistency mindset (you're trading freshness for speed, intentionally), TTL jitter, and negative caching. These are cheap to add early and protect you from the day a key gets hot or someone starts probing your API with random IDs.