So far, all of Kilometre Zero's communication has been synchronous: orders calls inventory over gRPC, waits and receives. That is the right thing when the answer is needed at that very moment, and the wrong thing for everything else. Remember the second symptom of the monolith (lesson 01-06): the external payment gateway slowed down and, because the charge was inside the same transaction as the stock reservation and the creation of the order, every blocked order held on to connections and threads until the whole platform went down. Turning that chain into gRPC calls does not fix it: it would still be a chain of multiplied availabilities and added latencies. What is needed is for orders to be able to say "this has happened" and move on, and for whoever has to react to do so when they can.

That is the job of messaging: an intermediary (the broker) that stores messages and delivers them to their recipients, decoupling the sender from the receiver in time, in space and in pace. In this lesson we will pin down the concepts (producer, consumer, queue, topic, ack, persistence), the two models (point-to-point and publish/subscribe), and the two technologies that dominate the industry, RabbitMQ and Apache Kafka, with their fundamental differences. We will build Kilometre Zero's first asynchronous flow: orders publishes the order.created event, and inventory and analytics each consume it at their own pace. The delivery guarantees and the patterns that make them safe (idempotency, outbox, dead letter queues) are named here and developed in lesson 02-05.

1. Why decouple in time
2. The vocabulary of messaging
3. Two models: point-to-point and publish/subscribe
4. RabbitMQ and AMQP: exchanges, queues, bindings and routing keys
5. order.created with RabbitMQ and pika
6. Apache Kafka: the distributed log
7. order.created with Kafka
8. RabbitMQ versus Kafka: selection criteria
9. Extending the km0/ docker-compose.yml
10. Common mistakes and tips
11. Exercises
12. Conclusion

A synchronous call couples the two participants in three ways:

• In time: both must be alive and available at the same instant. If analytics is restarting when orders tries to notify it of a sale, the notification fails or orders waits.
• In space: the sender needs to know who the receiver is and where it is (address, port). If tomorrow delivery also wants to know about orders, orders has to change.
• In pace: the sender cannot go faster than the slowest receiver. During "Grape Harvest Week", orders produces 1,200 orders per second; if analytics can only process 300, orders slows down to 300.

A broker breaks all three couplings: orders hands the message to the broker (which is available, because it is replicated infrastructure) and carries on; the broker stores it; the consumers pick it up whenever they like and at whatever pace they can manage; and adding a new consumer does not touch the producer.

The cost is just as clear and has to be accepted with open eyes: orders no longer knows whether the message has been processed, or when. Decisions that require an immediate answer (is there stock?) remain synchronous; asynchronous communication is for the consequences of a decision already taken. And one more critical piece of infrastructure appears, with its own availability and its own failure model.

Point-to-point (work queue): producers leave messages on a queue and one or more consumers pull them off; each message is processed by exactly one consumer. This is the model for distributing work: generating the PDF invoices for orders, sending emails, calculating routes. Adding consumers increases the processing rate (competing consumers, which we will cover in 02-05).

Publish/subscribe (pub/sub): producers publish to a topic without knowing who is listening; each subscriber receives its own copy of every message. This is the model for events: "an order has been placed" is of interest to inventory, to analytics and to delivery, and each one does something different with it.

In practice the two models are combined: each subscriber to a topic is usually a group of instances that compete with each other for that subscriber's messages (three replicas of analytics share out the events between them, but analytics as a whole receives all of them). Both RabbitMQ and Kafka support this combination, with different mechanisms.

RabbitMQ is a broker that implements AMQP 0-9-1 (Advanced Message Queuing Protocol), a binary protocol over TCP with a very flexible routing model. The key to understanding it is that producers never publish to queues: they publish to an exchange, and the exchange decides which queues to copy the message to according to a set of rules (bindings) and a label on the message (routing key).

The four exchange types:

Other RabbitMQ concepts: each consumer keeps a TCP connection and, inside it, one or more channels (lightweight multiplexing); messages are delivered to the consumer by push and remain "unacknowledged" until the ack; if the consumer dies without acknowledging, the broker requeues the message for another consumer; and once acknowledged, the message disappears from the queue. That last property is the fundamental difference from Kafka.

pika is the official Python client for RabbitMQ (pip install pika). First, the producer in orders:

Notice the event envelope: besides the order data it carries a unique identifier, a type, a version, a timestamp and a source. It is a convention we will keep for every Kilometre Zero event, and each field has a use that will show up in 02-05 (the event_id for detecting duplicates, the version for evolving the schema).

Now the inventory consumer. Its job, in this first version, is to deduct the reserved stock (later on we will decide whether the synchronous gRPC reservation and the event-driven deduction coexist or one replaces the other; for now, what matters is the mechanism):

And the analytics one, which wants all order events, not just the creation ones, to accumulate statistics:

Key points about the three programs:

• Each consumer declares its own queue and binds it to the exchange with the pattern it cares about. inventory only wants order.created; analytics wants order.# (created, paid, cancelled...). The producer knows nothing about either queue: that is decoupling in space.
• The message is copied to every bound queue: inventory and analytics each receive their own copy (pub/sub). If you start two instances of inventory's rabbit_consumer.py, they will share the inventory.orders queue and each message will go to one of them (point-to-point within the subscriber).
• Start the consumers after publishing and you will see that they receive the message all the same: it was stored in the queue. But if the queue did not exist when the message was published (because the consumer had never started), the message was discarded: the exchange had nowhere to copy it. That is why the queues of critical consumers must be declared at deployment time, not when the first consumer starts up.
• durable=True + delivery_mode=2 are the two halves of surviving a broker restart: the queue and the message. One without the other is no use.
• prefetch_count=1 in inventory: the broker does not send it the next message until it acknowledges the current one, which stops an instance from hoarding messages that it cannot process if it dies. It costs throughput; analytics, which can afford to reprocess batches, uses 50.

The management web interface (http://localhost:15672, username and password km0) shows exchanges, queues, pending messages and connected consumers: it is the first diagnostic tool to reach for.

Kafka was born at LinkedIn (2011) with a different idea: instead of a queue from which messages disappear once consumed, a log, that is, an append-only file in which each message occupies a fixed position (offset) and stays there for a retention period (7 days by default) regardless of whether anyone has read it. Consumers do not pull messages off: they read the log from a position and remember how far they have got.

The concepts you need to master:

• Topic: the logical name of the stream (orders.events).
• Partition: each topic is split into N independent logs. It is the unit of parallelism (each partition is read by a single instance of each group) and of ordering (messages are ordered within a partition, not across partitions).
• Partition key: the producer can assign a key to each message; Kafka computes hash(key) mod N to choose the partition. All messages with the same key go to the same partition and are therefore read in order. Using the order identifier as the key, order.created, order.paid and order.cancelled for P-2026-000123 always reach the same consumer in that order. Without a key, they are spread around and the order is lost. (Hashing to distribute keys across partitions is the same problem as data partitioning, lesson 04-01.)
• Offset: the position of a message within its partition. It is an increasing integer; consumers use it as a bookmark.
• Consumer group: instances that share a group.id split the topic's partitions between them (point-to-point within the group); different groups each read the whole topic (pub/sub between groups). The offset commit is the equivalent of the ack: "the inventory group has processed up to offset 4 of partition 0".
• Retention: messages are deleted by age or by size, not by consumption. A new consumer can read the full history (auto.offset.reset=earliest); one that was down for three days catches up on what it missed; and analytics can re-read the whole month if its logic changes. This is a capability that RabbitMQ does not have.
• Replicas: each partition is replicated across several brokers (replication factor 3 in production); one is the leader and serves reads and writes. Replication and its guarantees are studied in 03-04.

We will use confluent-kafka (pip install confluent-kafka), the most efficient Python client (it wraps the C library librdkafka); kafka-python is a pure-Python alternative with a similar API. First we create the topic with 6 partitions (with a single development broker, replication factor 1):

The producer:

produce does not send: it queues the message in an internal buffer that a background thread sends in batches (which is how Kafka achieves hundreds of thousands of messages per second). flush() blocks until all of them are confirmed; acks=all makes "confirmed" mean written to the leader and to the in-sync replicas. Without flush at the end of the program, the messages in the buffer would be lost.

The inventory consumer:

For analytics, the same code with "group.id": "analytics" and its own logic: since it is a different group, it receives all the messages again, regardless of what inventory has committed. Try starting two instances of the inventory consumer: you will see in the logs how Kafka reassigns the 6 partitions (3 and 3) between them, and how each order always goes to the same instance because of its key. Stop one and you will see the partitions go back to the other: this is group rebalancing, which gives consumers fault tolerance without the producer ever noticing.

One detail about the commit: commit(message=msg) marks up to that offset in that partition, not "this message". If messages 3, 4 and 5 are processed and 5 is committed, after a restart consumption resumes from 6. If 5 is committed without 4 having been processed (for example, when processing in parallel), 4 is lost. The commit is a position marker, and the consequences of committing before or after processing (losing or duplicating) are exactly the delivery guarantees that 02-05 analyses.

Decision criteria:

1. Are the messages events that several systems will want, perhaps in the future, perhaps re-read? Kafka. This is why the target architecture in 01-06 chooses Kafka for domain events: analytics will want to reprocess the month; a new service will want the history.
2. Are they tasks to be run once and forgotten, with fine-grained routing, priorities or a reply? RabbitMQ.
3. Does per-entity ordering matter (all the events of one order, in order)? Kafka with a partition key provides it naturally.
4. Volume? Below a few thousand messages per second, either will do; far above that, Kafka.
5. Who is going to operate it? A badly operated broker is worse than none. If the team is small, start with just one, and with a managed service.

Kilometre Zero, following the target architecture, will use Kafka for domain events and keeps RabbitMQ for work queues and for the request/reply pattern that we will see in 02-05. Having both is not mandatory; many systems live happily with just one.

We add the two brokers to the environment from 01-06. Kafka runs in KRaft mode (without ZooKeeper, the standard mode since version 3.x), with two listeners: an internal one for the services on the Compose network and another for the scripts we run from the host.

Services running inside Compose (such as inventory from 02-03 onwards) will use kafka:19092 and rabbitmq:5672; scripts launched from the host, localhost:9092 and localhost:5672. The plain-text passwords are for development only: secrets management is the subject of 06-04. With docker compose up -d rabbitmq kafka and the three scripts from this chapter, you have Kilometre Zero's first asynchronous flow up and running.

• Using messaging for something that needs an immediate answer. "Is there stock?" cannot wait for a consumer to process an event. Synchronous for decisions, asynchronous for consequences.
• Non-durable queues or non-persistent messages in critical flows. A broker restart takes the orders with it. Durable + persistent, always, except for disposable telemetry.
• Acknowledging (ack/commit) before processing. If the consumer dies after acknowledging and before finishing, the message is lost. Acknowledging after processing causes duplicates in the opposite case, and 02-05 explains how to live with that.
• Forgetting flush() in the Kafka producer. produce only queues; a process that exits without flush loses the buffer, with no error.
• Publishing without a partition key and expecting ordering. Without a key, the events of a single order are spread across partitions and order.paid may be processed before order.created.
• One topic per event type in Kafka. It breaks the ordering between events of the same entity (they are in different topics) and multiplies the partitions. One topic per entity (orders.events) with the type in a header is usually better.
• Consumers that declare queues that "ought to exist". If inventory never started, its RabbitMQ queue does not exist and the events published in the meantime are lost. Declare the topology (exchanges, queues, bindings, topics) at deployment time.
• Huge messages. Neither RabbitMQ nor Kafka is built to carry 5 MB product photos. The message carries a reference to the object store (04-03).
• Tip: always put a standard envelope on your events (event_id, type, version, timestamp_ms, source, data). It costs five lines and you will be grateful for it in every pattern in 02-05 and in every investigation in 07-02.
• Tip: keep an eye on the RabbitMQ interface and on the lag of the Kafka groups (kafka-consumer-groups.sh --describe) from day one. A queue that keeps growing or a lag that does not go down is the first symptom of a sick consumer, long before anyone complains.

delivery wants to receive only orders that have already been paid (order.paid), and customer-support wants all the cancellation events of any entity (order.cancelled, delivery.cancelled, ...). Write the queue and binding declarations for both on the km0.orders exchange (assume that delivery events are also published to it with routing keys delivery.<type>), and say what each one would receive if the following were published, in order: order.created, order.paid, delivery.assigned, delivery.cancelled, order.cancelled.

The orders.events topic has 6 partitions and inventory has 3 instances in the same group. (a) How many partitions does each instance read? (b) If 4 more instances are added (7 in total), what happens? (c) orders publishes order.created and order.cancelled for P-2026-000123 with key P-2026-000123, and order.created for P-2026-000124. Is it guaranteed that inventory processes the creation of 123 before its cancellation? And before the creation of 124? (d) A developer suggests using customer as the key instead of the order id, "so that all of Anna's orders go to the same consumer". What is gained and what is put at risk?

For each Kilometre Zero flow, say RabbitMQ or Kafka and justify it using the table in section 8:

1. Generating the PDF invoice for each paid order (heavy task, once, no ordering).
2. The stock events (inventory.events) that catalog consumes for the "only a few left" indicator and that analytics wants to reprocess every month to study stock-outs.
3. Positions from 400 couriers, one per second each, for the live map and for later route analysis.
4. An internal service that needs to ask payments whether a card is blocked, and wait for the answer.

Solution 1:

With the five events published: delivery.orders receives only order.paid. support.cancellations receives delivery.cancelled and order.cancelled (the * wildcard matches exactly one word: order or delivery). Neither of the two receives order.created or delivery.assigned. And the existing queues keep receiving what is theirs: inventory.orders only order.created; analytics.orders (order.#) the three order events, but not the delivery ones. Each consumer decides what it wants; the producer does not change.

Solution 2:

(a) Kafka splits the 6 partitions among the 3 instances: 2 each. (b) With 7 instances and 6 partitions, 6 instances read one partition each and the seventh sits idle: the number of partitions is the upper limit on a group's parallelism. To make use of more instances, the topic would have to be created with more partitions (and they cannot be added without changing the key→partition assignment of future messages, which temporarily breaks per-key ordering). (c) Yes for 123: both messages have the same key, they go to the same partition, and a partition is read by a single instance in order. No with respect to 124: it may be in another partition read by another instance; their relative order is undefined, and it does not matter, because they are independent orders. (d) The gain is that all the events of a given customer are processed in order and on the same instance (useful if there were per-customer rules, such as a limit on orders per hour). Two things are put at risk: unbalanced partitions (a business customer who places 30% of the orders concentrates 30% of the load on one partition and one instance) and, in a sense, unnecessary ordering: Anna's orders are independent of each other, so serialising them adds nothing and reduces parallelism. The key should be the entity whose ordering matters, neither finer nor coarser.

Solution 3:

1. RabbitMQ: a classic work queue; each invoice is generated by one instance and disappears; no ordering or re-reading; with priorities if needed (campaign invoices first). Kafka could do it too, but it adds nothing here.
2. Kafka: domain events with two consumers with different needs (one in real time, the other re-reading a month), per-product ordering (key = product slug) and long retention. It is exactly the use case it was designed for.
3. Kafka (with an MQTT bridge in front for the mobile leg, 08-02): high volume (400 messages/s sustained, far more during a campaign), two consumers (live map and route analysis), per-courier ordering (key = van-3), and retention for the later analysis, which is a batch job from Module 5. RabbitMQ would pile up the positions and would not allow them to be re-read.
4. RabbitMQ, if you decide to do it through messaging: the request/reply pattern with reply_to and correlation_id (02-05) is natural in AMQP and awkward in Kafka. But the prior question is whether it should be messaging at all: it is a synchronous query that needs an immediate answer, and a gRPC call to payments (02-03) is simpler and faster. Messaging would only be justified if payments were slow or intermittent and you wanted to absorb spikes, or if the answer could take a while and the requester could wait without blocking.

This lesson has introduced the second half of communication between services. A synchronous call couples in time, in space and in pace; a broker breaks all three couplings in exchange for the sender no longer knowing when, or whether, its message will be processed. With that vocabulary (producer, consumer, queue, topic, ack, persistence, durability, retention) we have distinguished the point-to-point model, for distributing work, from the publish/subscribe model, for broadcasting events, and we have seen how each broker combines them. RabbitMQ is a smart queue with flexible routing (exchanges, bindings, routing keys with wildcards) that deletes messages once they are acknowledged; Kafka is a distributed log in which messages stay put, consumers remember their offset, partitions provide parallelism and keys provide per-entity ordering, and any group can re-read the history. Kilometre Zero now has its first asynchronous flow: orders publishes order.created with a standard envelope, inventory and analytics consume it at their own pace, and both brokers are in the docker-compose.yml.

But we have deliberately left several questions open. What happens if inventory dies after deducting the stock and before acknowledging the message? The broker will redeliver it and the stock will be deducted twice, the same problem we left pending in 01-04 and in 02-02. What happens if orders saves the order in PostgreSQL and crashes before publishing the event, or publishes the event and then the COMMIT fails? And what about a malformed message that makes the consumer fail over and over again, blocking everything queued behind it? These questions are the delivery guarantees and the patterns that make them manageable (idempotent consumers, transactional outbox, dead letter queues), and they are the subject of the module's last lesson: Asynchronous Communication Patterns.