Design a distributed cache like Redis or Memcached that serves millions of reads per second.

A cache is easy until you scale it across many nodes. Then you face consistent hashing, eviction policy, the thundering herd on a cache miss, hot keys, and how stale you can tolerate. Here is the design that holds up under real traffic.

Updated Aug 2026 · Grounded in real Applied AI Engineer interview loops and written to a senior-engineer editorial bar.