The previous lesson left Kilometre Zero's catalogue in PostgreSQL with jsonb and a promise: that Redis would be put in front of it. This lesson keeps the promise and closes the module. A cache keeps a copy of data that is expensive to obtain (a database query, a gRPC call to inventory, a resized image) somewhere faster and closer, so as not to pay that cost again while the copy is still useful. It sounds simple, and in its basic form it is; the hard part is everything surrounding "while it is still useful": when the copy expires, how it is invalidated when the data changes, what happens when thousands of requests discover at once that it has expired, what to do about the key everybody asks for, and how to stop the cache returning something the database no longer says. We will look at the patterns (cache-aside, read-through, write-through, write-behind, refresh-ahead), the classic problems and their solutions, the Memcached/Redis comparison, and Redis Cluster with its 16,384 slots as a reincarnation of the partitioning of 04-01. The hands-on part puts it all into services/catalog/cache.py, with invalidation driven by the stock.updated events from Kafka, and measures the difference in latency with and without the cache during "Artisan Cheese Week".

1. Why cache: latency and load on the database
2. Where to cache: client, CDN, application and distributed cache
3. Caching patterns
4. Invalidation and TTL: the two hard problems
5. Classic problems: stampede, hot keys, penetration
6. Consistency between cache and database
7. Memcached versus Redis
8. Redis Cluster: slots, replicas and failover
9. Hands-on: services/catalog/cache.py and Redis Cluster
10. Common Mistakes and Tips
11. Exercises
12. Conclusion

Caching addresses two different problems that tend to show up together: latency (the request takes too long) and load (the origin cannot cope with that many requests). A few numbers to get a feel for the difference:

Kilometre Zero's sums for "Artisan Cheese Week": the website serves 6 million product views a day, concentrated in 12 hours, with a peak of 3 times the average: about 420 views/s at peak. Each view builds the product page with one catalogue query (product + producer + photos + variants, about 8 ms in PostgreSQL) and one gRPC call to inventory for the stock per market (another 4 ms). Without a cache: 420 queries/s of 8 ms each on PostgreSQL (taking up 3.4 s of CPU for every second: three or four cores just for the catalogue, and growing with the campaign) and a minimum latency of 12 ms per page. But the catalogue changes about 300 times a day (prices, descriptions, photos), that is, one write for every 20,000 reads. With a cache serving 98% of the views, PostgreSQL receives 8 queries/s instead of 420, and the page is built in under 1 ms. The read/write ratio is the first indicator that a piece of data is cacheable; the second is how much staleness it can tolerate, and the catalogue tolerates seconds.

A request passes through several layers, and each of them can cache:

In practice they are combined: a small local cache with a very short TTL (1–5 s) for the hottest keys, in front of Redis for everything, in front of the database. This lesson focuses on the distributed cache, which is the one that solves the problem of load on the database in a way that is shared by all the instances of catalog.

How the application, the cache and the database relate to each other defines the pattern:

Kilometre Zero uses cache-aside for the catalogue, with the event-driven invalidation variant of section 6, and refresh-ahead for the 200 most visited product pages during campaigns. Stock, which changes with every reservation, is not cached in the same way: the page shows "available / only a few left / sold out" derived from a value with a 10 s TTL, and the actual reservation always goes to inventory; it is an example of how staleness tolerance is decided per piece of data, not per service.

"There are only two hard things in computer science: cache invalidation and naming things" (Phil Karlton). The line is a joke because it is true: deciding when a copy stops being valid requires knowing when the original changed, and in a distributed system nobody has that information completely and in time. There are two mechanisms, and they are used together:

• TTL (time to live): each key expires after N seconds. It is simple, robust (the cache cleans itself up even if invalidation fails), and it bounds the maximum staleness. Its limit: it is a blind compromise. A 5-minute TTL on the price of aged-cheese means a price rise may take 5 minutes to show up; a 5-second one means 12 misses per minute per key even if the price does not change for a month.
• Explicit invalidation: when the data changes, somebody deletes (or updates) the key. It is precise, but it requires every write path to perform it (the producer dashboard, the bulk importer, the fix-up script an administrator runs by hand…) and the delete to get through (if Redis is unreachable at that instant, the old key survives). With several caches (local + Redis + CDN), you have to invalidate in all of them.

The practical combination: explicit invalidation as the main mechanism and TTL as the safety net, with a TTL long enough for misses to be rare and short enough that an invalidation failure does not last for hours. For Kilometre Zero's catalogue: a 10-minute TTL and event-driven invalidation.

A rule that follows from this: invalidating (deleting) is safer than updating the cache on a write. If two concurrent writes update the cache in a different order from the database, the cache is left with the old value until the TTL; if both delete, the next read reloads the correct value. We will look at this in detail in section 6.

Cache stampede / thundering herd. The key product:aged-cheese expires at 12:00:00 in the middle of a campaign. In the next 50 ms, 40 requests discover the miss at the same time, all 40 fire the same 8 ms query at PostgreSQL, and all 40 write the same value to Redis. With thousands of keys expiring in the same window (because they were all loaded together on deployment), the database is hit by an avalanche that can bring it down, which makes queries take longer, which piles up more requests on a miss. Solutions, which can be combined:

Hot keys. aged-cheese during Artisan Cheese Week gets 100 times more reads than the average product. In a partitioned distributed cache (section 8), that key lives on one node, which becomes the bottleneck while the others sit idle. Solutions: a local cache in each instance with a short TTL for the hot keys (it absorbs 99% before anything reaches Redis), read replicas of the node that holds it, or sharding the key (product:aged-cheese:0 … :9 with the same content, picked at random on reads, all invalidated on writes) to spread it over 10 nodes.

Cache penetration. A bot (or a broken link) asks for product:aged-chese, which does not exist. Cache-aside looks in Redis (miss), queries PostgreSQL (no row) and writes nothing to the cache because there is no value: every request for a non-existent key reaches the database. Under an attack with a million made-up slugs, the cache is useless. Solutions: cache the absence (SET product:aged-chese "" EX 60, a sentinel value with a short TTL) and, for enormous key sets, a Bloom filter in front: a compact probabilistic structure that answers "definitely does not exist" or "may exist" with no false negatives, so that keys that definitely do not exist are rejected without touching either the cache or the database (Redis offers it through the RedisBloom module; Cassandra uses it internally for SSTables, as we saw in 04-04).

The cache is a replica of the database (03-04) with no replication protocol whatsoever: the application maintains it, by hand, and so the anomalies are the code's responsibility. The two questions are what to do on a write (update or invalidate) and in what order.

Consider updating the cache on a write, with two concurrent price writes (A: 15.20; B: 15.90):

With invalidation (deleting), the same interleaving leaves the key deleted and the next read loads 15.90: correct. That leaves the order between writing to the DB and invalidating:

The usual answer is "write to the DB, then invalidate", with the TTL as a safety net and, if the data is sensitive, a double delete. And there is a structurally better solution, which picks up from 02-05: invalidate from the change events. inventory already publishes stock.updated to orders.events through the outbox table, in the same transaction as the change, and that event is guaranteed to arrive (the relay sees to that) and arrives after the commit (by construction). A catalog consumer deletes the key on receiving it. With that:

• Invalidation does not depend on every write path remembering to delete: any change that goes through the outbox invalidates.
• If Redis is down when the event arrives, the consumer does not commit the offset and retries: the invalidation is durable.
• The order is always "commit, then invalidate".
• The price is the pipeline delay (tens of milliseconds, sometimes seconds), during which the cache serves the old value: eventual consistency with a bounded window, acceptable for the catalogue and for the stock indicator, not for the reservation (which never reads from the cache).

The two most widely used distributed caching systems are alike in the basics (in-memory key-value, networked, sub-millisecond latency) and differ in almost everything else:

Kilometre Zero chooses Redis: it needs the stampede lock (SET NX EX), hashes for the product pages, sorted sets for the most-viewed products ranking (refresh-ahead), and optionally streams. And it will need Redis Cluster when a single instance is no longer enough.

A single-instance Redis serves hundreds of thousands of operations per second and tens of GB; to go beyond that, or to tolerate a node going down, there is Redis Cluster, which is the partitioning of 04-01 in its fixed number of partitions variant:

• The key space is divided into 16,384 slots. A key is assigned with slot = CRC16(key) mod 16384. There is no ring and there are no vnodes: there are 16,384 fixed partitions shared out among the masters (with 3 masters: 0–5460, 5461–10922, 10923–16383).
• Each master owns a range of slots and has one or more replicas (asynchronous replication, 03-04). The nodes learn about each other by gossip and share the slot map.
• The client is partition-aware (option (c) from 04-01): it downloads the slot map and sends each command to the right master. If the map is out of date, the node replies -MOVED 10215 172.22.0.4:6379 and the client updates and retries; during a slot migration it replies -ASK. redis-py handles this with RedisCluster.
• Hash tags: only the part between braces is hashed: {aged-cheese}:page and {aged-cheese}:stock:Girona fall in the same slot, which allows multi-key operations (MGET, transactions, Lua scripts) on them. Without a hash tag, an MGET of keys in different slots fails with CROSSSLOT.
• Failover: if a master stops responding for cluster-node-timeout (15 s by default), the majority of masters declare it FAIL and one of its replicas is promoted (an election with an epoch, along the lines of Raft in 03-03). Writes not yet replicated are lost: Redis Cluster is AP with windows of loss, suitable for a cache, not for data that cannot be rebuilt.
• Rebalancing: redis-cli --cluster rebalance or reshard move whole slots between masters, key by key with MIGRATE, while the cluster keeps serving: the fixed number of partitions of 04-01 in action.

As an alternative to Cluster when there is no need to partition, Redis Sentinel watches over a primary with replicas and performs automatic failover without splitting up the data.

A plain Redis in docker-compose.yml for the first part:

allkeys-lru means that, once the 512 MB fill up, Redis evicts the least recently used keys: cache behaviour. appendonly no because a cache does not need persistence.

The cache module (pip install redis). It implements cache-aside with TTL and jitter, a lock against stampedes, caching of absences, and a stock.updated consumer that invalidates:

The important points in each part:

• get_or_load is cache-aside with a per-key lock: SET lock:<key> <token> NX PX 3000 succeeds only for the first process; the others wait 20 ms and re-read. The lock expires after 3 s in case its owner dies. _RELEASE is a Lua script so that "if the token is mine, delete" is atomic: without it, a slow process could delete someone else's lock. (This is a cache lock: if it fails, the cost is a duplicate query; a correctness lock, like the relay's in 03-03, is done with etcd.)
• The double check after acquiring the lock stops the second arrival from reloading what the first has just written.
• If the wait exceeds max_wait, it degrades to querying the origin directly without caching: better one extra query than a hung request.
• _ttl_with_jitter desynchronises the expiries; SENTINEL caches absences to counter penetration.
• The consumer reuses the orders.events topic and the event_id/type/version/timestamp_ms/source/data envelope from 02-05. enable_auto_commit=False and the commit() after invalidating make the invalidation at-least-once: deleting twice is harmless (idempotent), whereas not deleting is the error we want to avoid. Note that the Kafka partition keys are the order id, so the stock.updated events for a given product may arrive through different partitions and in a different order; since we invalidate (rather than update), the order does not matter: this is the underlying reason for the "delete, don't update" rule.

Measuring latency (simulations/cache_latency.py): we simulate the aged-cheese product page with a "database" that takes 8 ms and compare:

Forty times less latency and (what matters to PostgreSQL) one query instead of 2,000. Afterwards, run python -c "from services.catalog.cache import *; [invalidate_product('aged-cheese') for _ in range(1)]" from another terminal while the measurement is running with 20 threads, and in redis-cli's MONITOR you will see a single SET product:aged-cheese after each DEL, with the other 19 threads reading the freshly loaded value: the lock at work.

Redis Cluster with three masters and three replicas, for the second part. Six nodes with cluster-enabled yes and a container that joins them together:

--cluster-replicas 1 assigns one replica to each master (the first three are masters, the last three replicas, paired so as to avoid the same host where possible). Checks:

CLUSTER KEYSLOT shows the CRC16 mod 16384 of each key and demonstrates that hash tags in braces place related keys in the same slot. The MOVED reply without -c is the partition map talking to the client. And CROSSSLOT is a reminder that multi-key operations, in a partitioned cache, require deliberate co-location.

Failover: docker compose stop redis-1; after 5 s (cluster-node-timeout), cluster nodes shows redis-1 as master,fail and its replica promoted to master with slots 0–5460; writes to those keys keep working. When redis-1 is started again, it joins as a replica of the new master. From Python:

redis-py's RedisCluster downloads the slot map, routes each command and follows MOVED/ASK automatically: the cache.py module works unchanged if you replace redis.Redis with RedisCluster, except for scan_iter (which walks through all the masters) and the multi-key operations, which need hash tags. A natural improvement to the module would be to name the keys {aged-cheese}:product and {aged-cheese}:stock:Girona so that everything belonging to a product can be invalidated with a single multi-key DEL.

• Caching without a TTL "because we already invalidate". Invalidation will fail at some point (a forgotten write path, Redis down at just the wrong instant). The TTL is the safety net; always set one.
• Updating the cache on a write instead of invalidating. Two concurrent writes leave the cache with the wrong value until the TTL. Delete; the next read loads the right value.
• Invalidating before writing to the database. A large window for caching the old value. Write first, invalidate afterwards; better still, invalidate from the outbox events.
• All keys with the same TTL, loaded at once after a deployment. They expire together: a stampede. Jitter and, for what is hot, refresh-ahead.
• No caching of absences. A bot with made-up slugs goes straight through the cache and reaches the database on every request. A sentinel with a short TTL, or a Bloom filter.
• A lock with no expiry, or released without checking the owner. A dead process blocks the key for ever; a slow process releases someone else's lock. NX PX and release through a Lua script.
• Confusing the cache lock with a correctness lock. The Redis one with SET NX can fail during a failover (asynchronous writes are lost); if not selling the last piece twice depends on it, use etcd (03-03) or the database.
• Caching the response of the stock reservation. The "only a few left" indicator may lag 10 s behind; the actual reservation never reads from the cache. Decide the staleness tolerance per piece of data.
• Multi-key operations in Redis Cluster without hash tags. CROSSSLOT. Design your keys with {tag} from the start; changing them later means flushing the cache.
• Treating Redis Cluster as a database. Failover loses unreplicated writes. It is a cache (or a rebuildable store), and orders and inventory live where 04-04 decided.

Exercise 1. Implement in cache.py the stale-while-revalidate variant of get_or_load: store in Redis a hash with the fields value and logical_expiry (a timestamp), with a physical TTL equal to twice the logical one. If on reading the logical value has expired, return the old value immediately and kick off the reload (in a thread, protected by the same lock). Explain what the website gains during Artisan Cheese Week compared with the lock-based version of section 9, and what is lost.

Exercise 2. The stock.updated event is published to orders.events with partition key = order id. Two reservations of aged-cheese in Girona (orders P-2026-000125 and P-2026-000126) generate two events in different partitions, which the catalog-cache consumer may process in reverse order. Explain (a) why invalidation by deletion is correct in any order; (b) what would happen if, instead of deleting, the consumer did SET stock:aged-cheese:Girona <available from the event>; (c) how the situation would change if inventory published the stock events with key = product, and what is lost with that change (hint: 02-04 and the relationship with order.created).

Exercise 3. During the campaign, product:aged-cheese receives 15,000 reads/s and its slot lives on redis-2, which saturates (the rest of the masters are at 10%). Propose two combined solutions, write the code for sharding the key into N copies ({aged-cheese}:product:0..N-1) with random reads and invalidation of all of them, and reason out why the hash tag {aged-cheese} means the sharding does not help in Redis Cluster, and how you would name the copies so that it does.

Solution 1:

What the website gains: during the campaign, no request for a hot key waits for the database (not even the one that triggers the reload), so the p99 stays at Redis's sub-millisecond level instead of jumping to 8 ms every 10 minutes; and a stampede is impossible because only one reload runs at a time while the rest serve the old value. What is lost: additional staleness (the old value is served until the reload finishes and, if the origin fails, until the physical TTL), memory (one hash per key) and complexity (background threads, which on an asynchronous server would be done with a task). For the price on a product page it is a good deal; for the stock indicator with its 10 s TTL, debatable.

Solution 2:

(a) DEL is idempotent and carries no value: it makes no difference which of the two events is processed first, the end result is "the key is not there", and the next read reloads it from inventory, which has the correct value (what is available after both reservations). Invalidation turns an ordering problem into an existence problem, which has no order.

(b) With SET <available from the event>, if the event for P-2026-000126 (available = 1) is processed before the one for P-2026-000125 (available = 2), the cache ends up at 2 while the real stock is 1: wrong until the 10 s TTL. You would need to compare timestamp_ms or a version and discard old events (the last-writer-wins pattern of 03-04, with its clock risks from 01-05), which is more code and more failure modes than a DEL.

(c) With key = product, all the stock.updated events for aged-cheese would go to the same partition and arrive in order (02-04), and then SET would be safe. But co-location with the rest of the order's events is lost: the order.created, stock.reserved and payment.confirmed of P-2026-000125 would no longer be in the same partition as their stock.updated, and the consumers that rebuild an order's history in order (the choreographed saga of 03-05, analytics) would lose that guarantee. A clean alternative: a separate topic inventory.stock with key = product for the stock events, which is what a mature design would do; in the meantime, the DEL works with any key.

Solution 3:

Combined solutions: (1) a local cache in each instance of catalog (cachetools.TTLCache(maxsize=500, ttl=2)) for the most-read keys, which absorbs the vast majority of the 15,000 reads/s before they reach Redis, and which is also invalidated by the event consumer (each instance consumes with its own group_id, or a Redis PUBLISH is sent to all of them); (2) sharding the key into N copies to spread it across masters:

With the hash tag {aged-cheese} all the copies would have the same slot (only what is between the braces is hashed) and therefore the same master: the load would be spread across keys, not across nodes, which is exactly what is of no use. Without a hash tag (product:aged-cheese:c0…c7), the CRC16 of each full name falls in different slots and, with high probability, on different masters; you can check this with CLUSTER KEYSLOT and tweak the suffixes until the 8 copies cover all 3 masters. The cost: 8 misses instead of 1 after each invalidation (irrelevant with 300 changes a day) and 8 DELs per invalidation, which can no longer be multi-key (they go to different slots); Redis Cluster's read replicas (READONLY on the client) are the third option, which spreads reads of the same slot between master and replica without changing the keys, at the price of reading with replication lag.

A distributed cache is a replica of convenience maintained by hand by the application, which is why it concentrates in the code all the replication problems that databases solve by protocol. Caching pays off when the read/write ratio is high and the data tolerates some staleness, like Kilometre Zero's catalogue with its 20,000 reads per write: the measurement brought the aged-cheese product page down from 8 ms to 0.2 ms and took 98% of the Artisan Cheese Week queries off PostgreSQL. We have placed the cache in its layers (client, CDN, local, distributed), gone through the patterns (cache-aside as the choice, refresh-ahead for what is hot, write-through and write-behind for other cases), and dealt with invalidation through the rule "TTL as the net, explicit invalidation as the mechanism, delete rather than update, after writing to the database". Against the stampede: a per-key lock with a token and a Lua script, coalescing, jitter and early reloading; against hot keys: a local cache, replicas and key sharding; against penetration: sentinels and Bloom filters. Invalidation through the stock.updated events in orders.events has turned the outbox of 02-05 into the mechanism that keeps inventory and the catalog cache coherent, durable and correct in any order. Redis has beaten Memcached thanks to its structures and its locks, and Redis Cluster has brought the module full circle: 16,384 fixed slots, MOVED as the partition map in the client's hands, hash tags for co-location, and replicas with asynchronous failover that make it suitable as a cache and not as a database.

This lesson closes Module 4, and every piece of Kilometre Zero's data now has its place:

The data is spread over many nodes and we know why each piece is where it is. The next question is how to process it in bulk now that it no longer fits on one machine: how to compute Grape Harvest Week sales by producer and market over the hundreds of millions of events in the lake, how to train recommendations on a year's worth of clicks, how to feed the delivery dashboard in real time. That is the territory of Module 5, distributed computing, which begins with distributed computing models: what it means to spread a computation, and not just a piece of data, across many nodes.