System Design · Chapter 10

Answering before the database wakes up

Every trip to a disk-based database costs time and money. A cache keeps the hot data in memory, intercepting requests so most of them never reach the database at all.

🧭
How to read this

Short chapter, clean build: why a cache matters (latency and throughput), what you can cache (static vs. dynamic, and the TTL that governs it), where to place one across the stack, a real cost-saving story, and finally the four strategies — cache-aside, read-through, write-through, write-back — with the use case each wins. Anything marked [+] is context I added; everything else is rewritten from the four teaching lessons.

01

Users bounce, so we answer from memory

Caching copies frequently accessed data out of the disk-based database and into RAM, so requests get answered fast instead of waiting on the database. The motivation is human impatience — wait 15, 30, 45 seconds for a page and you leave for another site. A cache keeps latency low so users don't bounce.

Two payoffs, both from the same move:

The cache intercepts requests darting toward the database, which frees the database's resources for the work only it can do — uncached queries, open connections, write operations. It's especially powerful for data assembled from expensive multi-table joins: cache the result once and skip re-running the join every time.

Client Cache RAM · key-value cache hit → instant return miss Database disk-based populate cache
Figure 1 — A cache sits in front of the database: hits return instantly from RAM; misses fall through to disk and populate the cache. Original diagram after Illustration 1.71.
02

Static vs. dynamic data, governed by TTL

You can cache both kinds of data — the difference is how they expire.

Kind What it is Caching behavior
Static Rarely changes — images, fonts, CSS, plus customer data like name, age, address, social id, photos. Cache easily on the client browser, a CDN, or the server, depending on sensitivity.
Dynamic Changes more often; carries an expiry. Given a TTL (Time To Live). When the TTL ends, the data is purged and refreshed — this is cache invalidation.
⏱️
The TTL sweet spot

A TTL should be long enough to make caching worthwhile — but caching barely helps for data that changes constantly, like a stock price or a live cricket/baseball score. If it churns every second, you're invalidating faster than you're serving.

03

A cache can live almost anywhere

There are no ground rules for where caching goes — it can sit at any layer, with any component, from the browser to the database. Using one is almost always better than not, especially with static data; you'll find few use cases where it doesn't help. The caveat: implement it wisely, or you invite data-inconsistency issues.

Browser REST API Service ↔ Service Database cache cache cache cache
Figure 2 — Caching applied across the stack: browser (static data), REST API, inter-service communication, and the database (intercepting queries). Original diagram after Illustration 1.72.

The most common use is database caching — intercepting requests to relieve the database. Beyond that: the browser caches static data, the REST API layer caches responses, and microservices cache cross-module communication. The guiding rule is to look for patterns — any frequently accessed content is a caching candidate; never poll a component repeatedly for the same data.

Two more benefits worth naming: even if the database goes down briefly, users won't notice while the cache keeps serving; and caching is core to the HTTP protocol. User sessions are commonly stored in a cache, and key-value stores (from the database chapter) are the primary tool for implementing caching.

04

Caching as a cost lever, not just a speed lever

Caching isn't only about latency — it can slash hosting costs by keeping expensive writes and storage off the database. Two real cases from the course author.

A stock-market multiplayer game

The game moved stock prices every second (or faster) by algorithm. Persisting every price change to the database immediately produced a huge write volume — expensive. The fix: write updated prices to a cache (Memcache) instead, then run a scheduled batch job every few hours to flush prices into the database. Writing to Memcache in the cloud was far cheaper than writing to the database; the cache served all reads and writes, and the DB caught up on the batch run. Not ideal for a real fintech app, but it saved a fortune and kept the game running longer.

💰
Polyhaven: static data on a shoestring

Polyhaven, a 3D-asset library, handled 5 million page views and 80TB of traffic a month for under $400 by leaning on caching. Without it, storing all that data on cloud object storage would have cost roughly $4,000/month — a 10× difference.

The lesson generalizes: for not-so-mission-critical information, you don't have to persist every little change to the database — let the cache absorb it and reconcile later.

05

Four strategies, four use cases

How the cache and database stay coordinated defines the strategy. There are four common ones, often combined.

Cache aside — the default for read-heavy loads

The most common strategy. The cache sits beside the database and data is lazy-loaded. On a read, the app checks the cache first: found = cache hit, return it; not found = cache miss, fetch from the database, return it, and populate the cache for next time. Writes go directly to the database, which risks cache/DB drift — so cached entries carry a TTL and expire. Best for read-heavy workloads on data that changes infrequently (e.g. customer name, account number), which can take a long TTL.

    App Cache DB 1 · check 2 · hit → return 3 · miss → read 4 · populate cache
    Figure 3 — Cache-aside: check cache → hit returns; miss reads the DB and back-fills the cache. Writes bypass the cache and go to the DB. [+] Flow diagram of the lesson's description.

    Read-through — cache-aside, but automatic

    Nearly identical to cache-aside, with one difference: the cache library or framework keeps the cache consistent with the database automatically, so you don't write explicit update logic. Data is still lazy-loaded on request, but the cache's data model must match the database since the library updates it for you.

    Write-through — consistency first

    The cache sits in line with the database: every write goes through the cache before reaching the database. This keeps cache and DB tightly consistent, at the cost of a little extra write latency (you're writing twice). Best when you need strict cache/DB consistency; usually paired with another strategy for optimized performance.

    Write-back — throughput and cost first

    Writes go straight to the cache, and the cache flushes to the database later, after a delay set by business logic. This is exactly the stock-game trick from Section 04 — batch the DB writes to cut load and cost. The risk: if the cache fails before flushing, unwritten data can be lost. Also typically combined with other strategies to balance the trade-offs.

    Write-through App Cache DB write immediately Write-back App Cache DB write delayed batch
    Figure 4 — Write-through writes to cache and DB together (consistency); write-back writes to cache now and flushes to the DB later (throughput and cost). [+] Comparison diagram of the two write strategies.
    Strategy How writes/reads flow Best for
    Cache aside Read from cache; miss reads DB & back-fills. Writes go to DB; TTL expires stale entries. Read-heavy loads, infrequently changing data.
    Read-through Like cache-aside but the library keeps cache/DB consistent automatically. Read-heavy loads where you'd rather not write cache logic.
    Write-through Writes pass through cache → DB together. Strict cache/DB consistency.
    Write-back Writes hit cache; flushed to DB later in a batch. Write-heavy loads; cutting DB load and cost.
    06

    Recap

    Cache = data in RAM Copies hot data off disk into memory; intercepts DB requests for low latency and high throughput.
    Static vs. dynamic Static (images, CSS, customer data) caches easily; dynamic needs a TTL and cache invalidation.
    Cache anywhere Browser, CDN, REST API, inter-service, database. Most common use: database caching.
    A cost lever too Write-back + batch flush cut DB write costs; Polyhaven served 80TB/month for <$400.
    Four strategies Cache-aside & read-through (reads), write-through (consistency), write-back (cost/throughput).
    Key-value stores power it Redis/Memcached are the primary tools; caching is also core to HTTP.
    💡
    The single idea underneath it all

    A cache is a bet that the same data will be asked for again. Every strategy just decides when the database finds out about a change — instantly (write-through), never on the read path (cache-aside), or lazily on your schedule (write-back). Pick the timing that matches how much staleness, and how much cost, your use case can absorb.