Lesson 58 — System Design II: caching

~20 minutes · Day 76 · system design · caching

Caching is the first serious move in many system-design interviews. The interviewer says, “reads are slow” or “the database is hot,” and a cache sounds obvious. The real interview is whether you can name the trade: faster reads, but more decisions about stale data, writes, misses, and evictions.

The invariant

A cache trades freshness for speed — so decide explicitly what happens on a miss, on a write (write-through vs write-back), and on eviction (LRU vs LFU).

That sentence is the design. Do not just say “put Redis in front.” Say what the cache stores, when it is trusted, when it is refreshed, and what happens when memory is full.

The script

System design uses the same script every time: requirements → estimate scale → API → data model → high-level boxes → deep-dive the bottleneck → failure modes. For caching, apply it like this:

  1. Requirements. Design fast reads for product pages. Slightly stale price or description data is acceptable for a few minutes; checkout prices still come from the source of truth.
  2. Estimate scale. Suppose 10,000 reads/sec and 100 writes/sec. That read-heavy shape is why caching matters.
  3. API. Reads look like GET /products/{id}. Writes look like PATCH /products/{id} from an internal admin service.
  4. Data model. Cache key: product:{id}. Value: serialized product summary plus updated_at. Add a TTL so forgotten keys expire.
  5. High-level boxes. Client → application service → cache first; database only on miss or write.
  6. Deep-dive the bottleneck. The bottleneck is not drawing Redis. It is the policy: miss behavior, write behavior, eviction behavior.
  7. Failure modes. Cache down, cache stampede, stale values, hot keys, and evicting the wrong data.

The architecture diagram

Cache-aside architecture for product reads The application checks the cache before the database, fills the cache on misses, and updates or evicts the cache after database writes. Client Application read/write logic Cache Redis / Memcached Database source of truth request get key hit: return cached value miss: read source fill cache write: DB first then update or evict Key insight: the cache short-circuits reads, but the database remains the truth.

This is a cache-aside shape: the application checks the cache first, reads the database on a miss, then fills the cache. AWS describes cache-aside and write-through as common database caching patterns in its Redis caching strategies whitepaper.

The bottleneck deep-dive

In a caching interview, the interesting component is the policy around the cache. Make three decisions out loud:

DecisionOptionTrade-off
MissCache-asideSimple and cheap: only requested data enters cache. First request is slower because it hits both cache and database.
WriteWrite-throughUpdate database, then update or evict cache. Safer freshness, but every write does extra work.
WriteWrite-backWrite cache first and flush later. Faster writes, but more danger if the cache fails before flush.
EvictionLRUEvict least recently used keys. A strong default when recent access predicts near-future access.
EvictionLFUEvict least frequently used keys. Better when a few hot products are repeatedly requested.

For product pages, a conservative answer is: cache-aside on reads, database-first write-through or cache invalidation on writes, TTLs for bounded staleness, and LRU unless metrics show stable hot keys. Redis documents LRU and LFU as built-in eviction policies; the important interview move is choosing based on the access pattern.

Numbers that matter

FigureWhy you say itSource
Hit rate = hits / (hits + misses)This is the first health metric. If hit rate is low, the cache is not protecting the database.Redis INFO stats
80% hit rate at 10,000 reads/secThe database now sees about 2,000 reads/sec instead of 10,000. That arithmetic is the point.Derived from hit rate
Sub-millisecond cache readsMemory-backed caches can make common reads much faster than disk-backed database reads.AWS caching overview
~200 microseconds network latencyA remote cache is fast, but not free. Too many round trips can erase part of the win.Redis latency guide
Hundreds of thousands of requests/secA side cache can absorb hot reads that would otherwise overload the database.AWS caching overview

Practice — answer before you scroll past

1. A product key is absent from cache. What should cache-aside do next?

2. Which write policy keeps the database authoritative?

3. Celebrity product pages keep getting reread daily. Which eviction choice protects them best?

4. The cache is down during a read-heavy spike. What is the safest first response?

Your move (tonight)

Redraw the architecture diagram from memory on paper: client, application, cache, database, hit path, miss path, write path. Then explain the design out loud in 5 minutes as if to an interviewer. Your explanation must include these exact decisions: miss policy, write policy, eviction policy, and one failure mode.

For the encode step in your Notion tracker, write the invariant from this page in your own words. Keep it to one sentence.

One primary source

Read AWS — Database Caching Strategies Using Redis: Caching patterns after your redraw, as the compare step. Focus on cache-aside and write-through; those are the two policies interviewers expect you to reason about first.

Stuck or curious? Ask your teacher (the agent) — that's what it's for.