Caching Strategies

TL;DR: A cache keeps a copy of hot data close to the request to cut latency and load on the backing store, trading some freshness for speed. The write pattern is the main decision: cache-aside (app reads cache, fills it on a miss), write-through (writes go to cache and store together, always fresh), write-back (writes hit cache and flush to the store later, fast but risks loss). Eviction (LRU/LFU) and TTL bound what you keep; stampede protection stops a hot expired key from dogpiling the store. A CDN is this same idea pushed to the network edge.

The three patterns

• Cache-aside (lazy loading). The app checks the cache; on a miss it reads the store, writes the value into the cache, and returns it. The cache only holds what was actually requested. Simple and the most common; the downsides are the extra round trip on a miss and the chance of stale data until TTL or explicit invalidation.
• Write-through. Every write goes to the cache and the store synchronously. Reads are always fresh (no stale window) but writes are slower (two hops), and you cache data that may never be read.
• Write-back (write-behind). Writes go to the cache and are flushed to the store asynchronously in batches. Very fast writes and great for write-heavy bursts, but a crash before flush loses data, so it suits tolerant workloads (counters, metrics), not payments.

Eviction and TTL

A cache is bounded, so when it fills you evict. LRU (least recently used) drops the entry untouched longest, a good default when recency predicts reuse. LFU (least frequently used) drops the entry hit least often, better when a stable set of items is popular regardless of recency. Real systems often use approximations (sampled LRU, or W-TinyLFU as in Caffeine) because exact ordering is expensive at scale. TTL is the orthogonal control: every entry expires after a fixed time so stale data self-heals even without explicit invalidation. Short TTL means fresher but lower hit rate; long TTL means higher hit rate but staler data. You tune the two against your tolerance for stale reads.

Cache stampede (the dogpile)

When a hot key expires, every concurrent request misses at once and they all hit the backing store to recompute the same value, a self-inflicted spike that can knock the store over right when it is busiest. Three standard defenses:

• Locking / single-flight. Only the first miss recomputes; the rest wait for that result (one DB call, not 10,000).
• Stale-while-revalidate. Serve the slightly stale value and refresh in the background, so users never wait on a recompute.
• Jittered / early expiry. Randomize TTLs (or refresh probabilistically before expiry) so many keys do not expire in lockstep, the same anti-synchronization logic as backoff jitter.

CDNs: caching at the edge

A CDN caches static and cacheable content on servers physically near users, so a request is served from a nearby edge instead of crossing the planet to your origin. It cuts latency (fewer network hops), offloads the origin, and absorbs traffic spikes. Cache-control headers and TTLs govern freshness, and you invalidate via purge or content-hashed URLs. For applied-AI systems the same instinct applies to expensive computed artifacts: cache LLM responses, embeddings, and retrieval results because recomputing them is far costlier than a cache lookup.

Why interviewers probe this

Caching is the cheapest large latency and cost win in most systems, so "where and how do you cache" comes up in nearly every design. The signal is whether you pick a write pattern deliberately (cache-aside for reads, write-through when staleness is unacceptable, write-back only when loss is tolerable), reason about eviction and TTL against staleness tolerance, and anticipate stampede. For applied-AI, knowing that LLM calls are expensive enough to cache aggressively (and how to invalidate when prompts or models change) is the senior move.

Common misconceptions

• "Caching is just put data in Redis." The write pattern (aside vs through vs back) and invalidation strategy are the actual design; the store is incidental.
• "TTL handles freshness, so I never need to invalidate." TTL bounds staleness but can serve stale data for the whole window; explicit invalidation is needed when correctness matters.
• "LRU is always the right eviction policy." LFU or W-TinyLFU beats LRU when a stable popular set matters more than recency; pick by access pattern.
• "A cache miss is harmless." A hot expired key causes a stampede onto the store; defend with single-flight, stale-while-revalidate, or jittered TTLs.
• "Write-back is just faster write-through." Write-back can lose unflushed data on a crash; never use it where durability is required.

Key takeaways

• A cache trades freshness for speed; choose the write pattern deliberately: cache-aside (reads), write-through (always fresh), write-back (fast, lossy).
• Eviction (LRU for recency, LFU/W-TinyLFU for popularity) plus TTL bound what you keep and how stale it gets.
• Cache stampede on a hot expired key is a real outage cause; use single-flight, stale-while-revalidate, or jittered TTLs.
• CDNs cache at the network edge to cut latency and offload origin; the same logic justifies caching expensive LLM outputs.
• Tune TTL against staleness tolerance: shorter is fresher but lower hit rate, longer is the reverse.