The Project | About Us | Contribute | Donations | License

HOME

In the early nineties, Peter Deutsch, an engineer at Sun Microsystems, compiled a list of false assumptions that programmers make, almost without noticing, the first time they write software that communicates over a network. James Gosling (the creator of Java) rounded the list out to the eight fallacies of distributed computing that are now a classic. Their value lies in the fact that each of them is so natural that we take it for granted when writing code, and only find out it was false when the system fails in production.

This lesson goes through the eight fallacies one by one. For each we will see what it means, how it shows up specifically at Kilometre Zero and which design measure counters it, pointing to the lesson in the course where that measure is developed. We will finish with a code example in which a "naive" call from the orders service to the inventory service comes up against a network that loses packets and takes a variable amount of time, and we will see how a simple timeout completely changes its behaviour.

Origin and purpose of the fallacies
Fallacy 1: the network is reliable
Fallacy 2: latency is zero
Fallacy 3: bandwidth is infinite
Fallacy 4: the network is secure
Fallacy 5: topology doesn't change
Fallacy 6: there is one administrator
Fallacy 7: transport cost is zero
Fallacy 8: the network is homogeneous
Summary table: fallacy, symptom, countermeasure, lesson
Code example: orders calls inventory over a flaky network
Common mistakes and tips
Exercises
Conclusion

Origin and purpose of the fallacies

The fallacies are not programming errors: they are mental model errors. When, in the Kilometre Zero monolith, the orders module called inventory.deduct_stock(product_id, 1), that call was a Python function: it always arrived, it always answered within microseconds, nobody could intercept it and it cost no money. Once orders and inventory are split into two services, the line of code can stay almost identical (inventory.deduct_stock(...) through an RPC client), but all of those properties are gone. If the programmer does not change their mental model, they will write code that is correct for the monolith and incorrect for the distributed system.

That is why it is worth going through the eight fallacies with a specific example in mind. Our example will mainly be the orders → inventory interaction when confirming an order, and the stream of positions that couriers send to the delivery service over 4G.

Fallacy 1: the network is reliable

What believing it means. That every message sent reaches its destination, exactly once and uncorrupted.

The reality. Packets get lost (congestion, faulty cables, switches rebooting), duplicated (retries by lower layers) and delivered out of order. TCP hides part of this (it retransmits and reorders), but it can do nothing if the link goes down altogether, if the other end restarts or if a firewall starts dropping connections. And, most importantly, when a request gets no response, the sender cannot know whether it was the request or the response that was lost.

At Kilometre Zero. When confirming Anna's order, orders sends inventory "deduct one unit of aged cheese". If the request is lost, the stock is not deducted and the order is confirmed with nothing in stock. If the request arrives and inventory deducts the stock, but the response is lost, orders believes it has failed; if it retries, two units will be deducted.

Countermeasures. Acknowledgements and retries, but with idempotent operations (ones that can be repeated with no additional effects) and unique request identifiers (lesson 02-05); message queues with guaranteed delivery (02-04); and retry strategies with exponential backoff (07-04).

Fallacy 2: latency is zero

What believing it means. That a remote call takes the same time as a local call, that is, practically none.

The reality. An in-memory function call takes nanoseconds. A network call within the same data centre takes between 0.5 and 2 ms; between European cities, 20-40 ms; between continents, 100-200 ms; over 4G from a phone, between 50 and 500 ms, with enormous variability. In other words, a remote call is between 10,000 and 1,000,000 times slower than a local one. And the problem gets worse when calls are chained: the "N+1" pattern (making one call for each item in a list), which in the monolith was a tolerable oversight, turns a page into a disaster in a distributed system.

At Kilometre Zero. Lucy's basket page shows 15 products. If the new version of orders makes one call to catalog per product to get its name and price, that is 15 calls × 20 ms = 300 ms of network waiting alone, in series. And couriers' positions arrive with variable delays of 80 to 900 ms, so the "current" position on the map is always somewhat stale and sometimes arrives out of order.

Countermeasures. Design coarse-grained interfaces (one call that returns 15 products, not 15 calls); parallelise independent calls; cache (04-05); place data close to whoever uses it (geographic replication, 03-04); use asynchronous communication for whatever does not need an immediate response (02-04). And, in any case, measure the latency of every call (07-01).

Fallacy 3: bandwidth is infinite

What believing it means. That you can send any amount of data and size does not matter.

The reality. Although bandwidth has grown enormously, it is still finite and shared. Moreover, latency and bandwidth interact: sending 10 MB over a 100 Mb/s link takes almost a second, however low the latency. And on mobile networks, bandwidth is scarce, variable and often charged by volume.

At Kilometre Zero. The first version of the catalog service returned, on every search, the full products with their photos base64-encoded inside the JSON: 2 MB per response. With 400 searches/s during the campaign, that is 800 MB/s, more than the network interface can deliver. Another case: on every position update, the courier app sent the whole day's history instead of just the new position.

Countermeasures. Send only what is needed (pagination, selectable fields); compact serialization formats such as Protocol Buffers (02-03); compression; move large data (photos) into object storage and send only references (04-03); batch small messages together when the volume justifies it.

Fallacy 4: the network is secure

What believing it means. That only legitimate components can read or send messages on the network.

The reality. Any message crossing a network can be read, modified or forged by whoever has access to that network. And "the internal network" is an increasingly blurry concept: containers, public clouds, provider networks, employees' laptops, mobile devices. In a monolith, the call from orders to inventory could not be intercepted because it never left the process. Now it can.

At Kilometre Zero. If inventory accepts any "deduct stock" request without verifying who sent it, anyone who gets onto the internal network (a compromised container, a disgruntled employee) can empty Roble Alto Winery's stock. If couriers' positions travel unencrypted, an attacker on the same Wi-Fi can follow their movements, or inject fake positions.

Countermeasures. Encryption in transit (TLS) for all communications, internal ones included (06-02); mutual authentication between services with mTLS (06-04); user authentication and authorization with tokens (06-01); secure secrets management (06-04); a gateway that centralises controls (06-05). General principle: zero trust, a request is not trusted just because it comes from "inside".

Fallacy 5: topology doesn't change

What believing it means. That machines always sit at the same IP address, that the same node is always in the same place and that the map of the network is stable.

The reality. In any modern system, nodes appear and disappear constantly: autoscaling, restarts, deployments, hardware failures, migrations between zones. A container may have a different IP address every time it starts. Network links change routes. Code with hard-coded addresses ("inventory is at 10.0.3.17") works until the first redeployment.

At Kilometre Zero. During Artisan Cheese Week, 6 instances of catalog are added and, when it ends, removed. orders needs to find the live instances at any given moment. And when inventory is redeployed with a new version, its containers change IP: if orders was holding the old address, it starts failing without anything being "broken".

Countermeasures. Service discovery (dynamic registration of instances and resolution by name, not by address) and orchestrators that manage it automatically (07-05); load balancers that route to healthy instances; health checks that take unresponsive instances out of rotation (07-03); designing services so that they do not depend on the identity of the machine they run on.

Fallacy 6: there is one administrator

What believing it means. That a single person (or team) knows, controls and configures the whole network and all the systems involved.

The reality. As soon as the system grows, multiple parties are involved: the infrastructure team, each service team, the cloud provider, the external payment gateway, the couriers' mobile network operators. Nobody has the full picture. A configuration change made by one team can break another; an update by the external provider can change the behaviour of an API; a cloud network policy can block a port.

At Kilometre Zero. The external payment gateway announces that it is dropping support for an old version of TLS, and payments stops working one Tuesday morning without anyone at Kilometre Zero having touched anything. The delivery team changes the format of an event, and analytics (owned by another team) starts discarding messages.

Countermeasures. Explicit, versioned contracts between services (02-03); centralised, audited configuration; infrastructure automation so that configuration is reproducible (07-05); observability to detect changes in behaviour quickly (Module 7); service level agreements with external providers and a design that tolerates their failures (07-04).

Fallacy 7: transport cost is zero

What believing it means. That moving data over the network costs nothing, either in money or in computing resources.

The reality. There are two costs. The first is computational: to send an object over the network you have to serialize it (turn it into bytes) and, on receiving it, deserialize it; for large data sets, this work can use more CPU than the business logic itself. The second is financial: cloud providers charge for traffic leaving their data centres (and sometimes between zones), and mobile networks charge by volume. What was free in the monolith (passing a Python object from one module to another) now has a price.

At Kilometre Zero. The analytics service is initially designed to query all of the day's orders every night by requesting them from orders as JSON: 2 million records serialized, transmitted and deserialized, every night. A month later, the bill for traffic between cloud zones takes management by surprise, and the process spends longer serializing than calculating.

Countermeasures. Efficient binary formats (02-03); taking the computation to where the data is instead of moving the data to the computation, which is the principle behind MapReduce and Spark (05-02, 05-03); event streams processed as they happen instead of bulk dumps (05-04); placing the services that talk to each other most in the same zone.

Fallacy 8: the network is homogeneous

What believing it means. That all nodes use the same hardware, the same operating system, the same language, the same library versions and the same data format.

The reality. Any real system is a mixture: different languages, different versions of the same service coexisting during a deployment, different representations of numbers (how is a decimal encoded? and a date?), different character sets, different network capacities (a 10 Gb/s interface in the data centre versus 4G on a phone).

At Kilometre Zero. The delivery service receives positions from an Android app (which sends the timestamp in milliseconds), from an iOS app (which sends it in seconds with decimals) and from a van's GPS device (which sends it as text in a proprietary format). During a rolling deployment, version 1 and version 2 of inventory coexist, and version 2 returns a new field that the old version of orders does not expect. A price of €9.90 travels as 9.9 (floating point) and arrives as 9.899999.

Countermeasures. Language-independent interchange formats with an explicit schema (02-03); forward and backward compatibility rules in APIs (adding fields without breaking old clients); precise data types for money (decimals, not floats) and dates (UTC with an explicit zone); containers to make the runtime environment uniform (07-05).

Summary table: fallacy, symptom, countermeasure, lesson

#	Fallacy	Symptom at Kilometre Zero	Main countermeasure	Lesson
1	The network is reliable	Orders confirmed with no stock, or stock deducted twice	Retries with idempotency, queues with guaranteed delivery	02-04, 02-05, 07-04
2	Latency is zero	A basket that takes 300 ms because of 15 chained calls; courier positions out of order	Coarse-grained interfaces, caching, asynchrony, measurement	02-04, 04-05, 07-01
3	Bandwidth is infinite	2 MB responses with base64 photos	Pagination, compact serialization, object storage	02-03, 04-03
4	The network is secure	Anyone on the internal network can empty the stock	TLS, mTLS, service-to-service authentication, zero trust	06-01, 06-02, 06-04, 06-05
5	Topology doesn't change	Hard-coded IP addresses that fail after every deployment	Service discovery, health checks, orchestration	07-03, 07-05
6	There is one administrator	The payment gateway changes and `payments` stops working without anything being touched	Versioned contracts, infrastructure as code, observability	02-03, 07-05, Module 7
7	Transport cost is zero	Traffic bill and CPU through the roof because of nightly JSON dumps	Binary formats, taking the computation to the data, event streams	02-03, 05-02, 05-04
8	The network is homogeneous	Timestamps in three formats; prices with rounding errors; incompatible versions	Explicit schemas, API compatibility, precise types	02-03, 07-05

Code example: `orders` calls `inventory` over a flaky network

Let's simulate fallacies 1 and 2 without real sockets: a network that loses requests with a certain probability and takes a variable amount of time to deliver them. Over that network, the orders service will try to deduct stock in inventory. First with a naive client (which assumes the network is reliable and fast) and then with one that uses a timeout.

We will use asyncio to model the waiting: asyncio.sleep will stand for the message's travel time.

import asyncio
import random
import time


class FlakyNetwork:
    """Simulates a network with packet loss and variable latency.

    - loss_prob: probability that a message never arrives.
    - min_latency / max_latency: seconds a message takes to arrive
      (picked at random within that range, on every send).
    """

    def __init__(self, loss_prob: float, min_latency: float, max_latency: float):
        self.loss_prob = loss_prob
        self.min_latency = min_latency
        self.max_latency = max_latency

    async def transmit(self, message: dict) -> dict:
        """Delivers the message after a random latency... or never."""
        if random.random() < self.loss_prob:
            # The packet has been lost. On a real network NOTHING happens: no
            # error, no exception, the response simply never arrives.
            await asyncio.sleep(float("inf"))
        await asyncio.sleep(random.uniform(self.min_latency, self.max_latency))
        return message


class InventoryService:
    """A very simple inventory, holding the stock of each product."""

    def __init__(self):
        self.stock = {"aged-cheese": 5, "pink-tomato": 20, "crianza-wine": 12}
        self.requests_served = 0

    async def deduct(self, product: str, quantity: int) -> dict:
        self.requests_served += 1
        if self.stock.get(product, 0) < quantity:
            return {"ok": False, "reason": "out of stock"}
        self.stock[product] -= quantity
        return {"ok": True, "remaining_stock": self.stock[product]}


class NaiveOrdersClient:
    """Calls inventory as if it were a local function."""

    def __init__(self, network: FlakyNetwork, inventory: InventoryService):
        self.network = network
        self.inventory = inventory

    async def confirm_order(self, customer: str, product: str) -> str:
        request = {"op": "deduct", "product": product, "quantity": 1}
        # Outbound trip over the network, processing, and return trip.
        await self.network.transmit(request)
        response = await self.inventory.deduct(product, 1)
        await self.network.transmit(response)
        if response["ok"]:
            return f"Order for {customer} confirmed ({response['remaining_stock']} units left)"
        return f"Order for {customer} rejected: {response['reason']}"


class TimeoutOrdersClient(NaiveOrdersClient):
    """Same as the naive one, but never waits longer than 'timeout' seconds."""

    def __init__(self, network, inventory, timeout: float):
        super().__init__(network, inventory)
        self.timeout = timeout

    async def confirm_order(self, customer: str, product: str) -> str:
        try:
            return await asyncio.wait_for(
                super().confirm_order(customer, product), timeout=self.timeout
            )
        except asyncio.TimeoutError:
            return (f"Order for {customer}: NO RESPONSE from inventory within "
                    f"{self.timeout} s (was the stock deducted? we don't know)")


async def process_batch(orders_client, orders: list[tuple[str, str]], name: str):
    """Processes several orders, one after another, and measures how long it takes."""
    print(f"\n--- {name} ---")
    start = time.monotonic()
    for customer, product in orders:
        t0 = time.monotonic()
        result = await orders_client.confirm_order(customer, product)
        print(f"[{time.monotonic() - t0:5.2f} s] {result}")
    print(f"Total: {time.monotonic() - start:.2f} s; "
          f"inventory served {orders_client.inventory.requests_served} requests")


async def main():
    random.seed(16)
    orders = [("Anna", "aged-cheese"), ("Mark", "aged-cheese"),
              ("Lucy", "crianza-wine"), ("Anna", "pink-tomato")]

    # Scenario A: near-perfect network. The naive client seems to work fine.
    good_network = FlakyNetwork(loss_prob=0.0, min_latency=0.01, max_latency=0.03)
    await process_batch(NaiveOrdersClient(good_network, InventoryService()),
                        orders, "A: naive client, good network")

    # Scenario B: network with 25% loss and latency of up to 1.5 s.
    bad_network = FlakyNetwork(loss_prob=0.25, min_latency=0.05, max_latency=1.5)

    # B1: naive client. We guard it with an overall 6 s limit because otherwise
    # the program would hang FOREVER on the first lost packet.
    try:
        await asyncio.wait_for(
            process_batch(NaiveOrdersClient(bad_network, InventoryService()),
                          orders, "B1: naive client, bad network"),
            timeout=6.0,
        )
    except asyncio.TimeoutError:
        print("!!! The whole batch has hung: the naive client waits "
              "indefinitely for a packet that will never arrive")

    # B2: client with a 1 s timeout per call.
    inventory = InventoryService()
    await process_batch(TimeoutOrdersClient(bad_network, inventory, timeout=1.0),
                        orders, "B2: client with timeout, bad network")
    print(f"Final stock in inventory: {inventory.stock}")


if __name__ == "__main__":
    asyncio.run(main())

Let's run it and analyse the output (the exact timings vary slightly from machine to machine):

--- A: naive client, good network ---
[ 0.04 s] Order for Anna confirmed (4 units left)
[ 0.05 s] Order for Mark confirmed (3 units left)
[ 0.03 s] Order for Lucy confirmed (11 units left)
[ 0.05 s] Order for Anna confirmed (19 units left)
Total: 0.17 s; inventory served 4 requests

--- B1: naive client, bad network ---
[ 2.37 s] Order for Anna confirmed (4 units left)
!!! The whole batch has hung: the naive client waits indefinitely for a packet that will never arrive

--- B2: client with timeout, bad network ---
[ 1.00 s] Order for Anna: NO RESPONSE from inventory within 1.0 s (was the stock deducted? we don't know)
[ 0.98 s] Order for Mark confirmed (3 units left)
[ 1.00 s] Order for Lucy: NO RESPONSE from inventory within 1.0 s (was the stock deducted? we don't know)
[ 1.00 s] Order for Anna: NO RESPONSE from inventory within 1.0 s (was the stock deducted? we don't know)
Total: 3.98 s; inventory served 3 requests
Final stock in inventory: {'aged-cheese': 3, 'pink-tomato': 20, 'crianza-wine': 11}

What each part teaches us:

FlakyNetwork.transmit is the heart of the simulation. Notice how loss is modelled: not with an exception, but with await asyncio.sleep(float("inf")), an infinite wait. This is true to life: the network does not tell you it has lost a packet. It is fallacy 1 in its purest form.
Scenario A. With a near-perfect network, the naive client works fine, and that is precisely the danger: the code passes every test on the developer's laptop and in the test environment, where the network is good.
Scenario B1. With 25% loss, the first order takes almost 2.4 seconds (fallacy 2: latency is not zero, and here it is up to 1.5 s each way) and the second hangs forever. We had to wrap the batch in a 6-second wait_for just so that the program would finish. On a real server, this translates into blocked threads or connections piling up until resources run out, which is exactly what happened to the monolith during Grape Harvest Week.
Scenario B2. The client with a timeout never hangs: each call takes 1 second at most, and the batch finishes in 4 seconds. But look closely at the last lines: orders is only aware of one confirmed order (Mark's), and yet inventory served 3 requests: the aged cheese stock went down from 5 to 3 (two deductions, not one) and the wine stock from 12 to 11, even though Lucy's order was written off as failed. In other words: Anna's cheese order and Lucy's wine order did reach inventory and did deduct stock; what was lost was the response, not the request. Anna's tomato order, on the other hand, never arrived. From the point of view of orders, the three cases are indistinguishable.

The timeout solves the blocking problem, but it leaves the most important question open: when it expires, was the operation carried out or not? The message "was the stock deducted? we don't know" is literally true. Resolving that ambiguity requires idempotency and safe retries (02-05), and deciding when to stop trying so as not to overload an ailing service is the job of the circuit breaker (07-04). This lesson stops at the timeout, which is the minimum, indispensable countermeasure: no remote call should ever be made without a time limit.

Common Mistakes and Tips

Testing only on good networks. Scenario A shows that code which ignores the fallacies works perfectly in development. You have to test with injected packet loss and latency (lesson 07-06 covers chaos engineering for precisely this purpose).
Setting "generous" timeouts to avoid false failures. A 60-second timeout protects you from nothing: 60 seconds of blocked connections under load is an outage. The timeout should be somewhat higher than the normal latency of the operation (the 99th percentile, for example), not "big enough never to fire".
Treating a timeout as "the operation was not carried out". As scenario B2 shows, after a timeout the operation may well have been executed. Any retry must be safe against duplicates.
Hard-coding IP addresses or host names. Fallacy 5. Every address should come from configuration or from a discovery system.
Assuming that "internal" means "secure". Fallacy 4. Encrypt and authenticate between your own services too.
Using float for money, or timestamps without a time zone. Fallacy 8. These are the two most common sources of discrepancies between services written by different teams.
Tip: when you review code that makes a remote call, run through the "three Ts" checklist: does it have a Timeout? Is it Tolerant of duplicates (idempotent)? Is it Traced (is it recorded how long it took and whether it failed)? If any of them is missing, the call is not ready for production.

Exercises

Exercise 1: Diagnosing fallacies

For each of the following Kilometre Zero incidents, say which fallacy (or fallacies) was believed to be true and which countermeasure you would apply:

After migrating inventory to containers, orders fails every morning at 6:00, just when inventory is redeployed with the producers' updated data.
In a rural market, the courier app uses 400 MB of mobile data a day and the battery lasts half as long.
A developer writes a loop that, for each of the day's 300 orders, calls payments to check its status. The report takes 2 minutes.
A security review shows that an analytics container (which should only read) can send "deduct stock" requests to inventory and they are accepted.

Exercise 2: Tuning the timeout

Using the simulation from section 11, with bad_network (25% loss, latency of 0.05-1.5 s each way), reason it out and then check experimentally:

What timeout is needed so that no request that does arrive is given up for lost (that is, so that the timeout only fires because of real loss)?
What proportion of calls will fail because of real loss, bearing in mind that each call makes two trips (there and back)?
Modify main to run 200 orders with the timeout client and count how many are confirmed, how many time out and how many requests inventory actually serves. Compare the last two numbers and explain the difference.

Exercise 3: A naive retry

Add a retries parameter to TimeoutOrdersClient so that, after a timeout, it tries the same call again up to that number of times. Run the batch of 4 orders with retries=3 and look at the final stock of aged-cheese. What has happened and why? (You do not need to fix it: just diagnose it. The solution is covered in 02-05.)

Solutions

Solution 1:

Fallacy 5 (topology doesn't change). orders has cached (or configured) the address of the old inventory containers, which disappear on redeployment. Countermeasure: service discovery and name resolution on every call, with health checks (07-05, 07-03).
Fallacies 3 and 7 (bandwidth is infinite, transport cost is zero), and probably 2 as well (it sends too often). The app sends too much data or sends it too many times. Countermeasure: send only the new position (not the history), in a compact format, at an adaptive rate (lower when the courier is stationary), and batch positions when there is no urgency (02-03, 08-02).
Fallacy 2 (latency is zero). This is the N+1 pattern: 300 calls × 400 ms = 2 minutes. Countermeasure: one coarse-grained call to payments that returns the status of all 300 orders at once or, better still, have payments publish status changes as events that orders already keeps stored locally (02-04).
Fallacy 4 (the network is secure). inventory does not authenticate the caller or check its permissions. Countermeasure: service-to-service authentication with mTLS and authorization based on the service's identity (06-04); the principle of least privilege.

Solution 2:

Each call makes two trips, each of up to 1.5 s, so the worst case for a call that does arrive is 3 s. With a 3 s timeout (or slightly more, to allow for processing time), no timeout would be a "false positive". But notice the price: every real loss blocks the client for 3 seconds. This tension (short timeout = false failures; long timeout = long blocks) has no perfect solution.
A call completes only if both trips succeed: 0.75 × 0.75 = 0.5625. That is, 43.75% of calls will fail because of real loss. Of those, roughly half (0.25 × 0.75 / 0.4375 ≈ 43%) will have reached inventory and lost only the response.

async def experiment(n: int = 200):
    random.seed(11)
    network = FlakyNetwork(loss_prob=0.25, min_latency=0.05, max_latency=1.5)
    inventory = InventoryService()
    inventory.stock["pink-tomato"] = 10_000        # so that it does not run out
    client = TimeoutOrdersClient(network, inventory, timeout=3.0)
    confirmed = timeouts = 0
    for _ in range(n):
        r = await client.confirm_order("Anna", "pink-tomato")
        if "confirmed" in r:
            confirmed += 1
        else:
            timeouts += 1
    print(f"confirmed={confirmed} timeouts={timeouts} "
          f"served by inventory={inventory.requests_served} "
          f"stock deducted={10_000 - inventory.stock['pink-tomato']}")

With 200 orders you get confirmed=109 timeouts=91 served by inventory=144 stock deducted=144 (the specific values may vary slightly depending on each machine's timing). The difference between the requests served (144) and those confirmed (109) is the 35 orders that did deduct stock but whose client believes they failed; and the 45.5% of timeouts observed matches the theoretical 43.75%. In a real system, those 35 customers would see an error, would probably try again, and the stock would be deducted twice. (The experiment takes a few minutes because the waits are real; you can scale the latencies down proportionally to speed it up.)

Solution 3:

class RetryingOrdersClient(TimeoutOrdersClient):
    def __init__(self, network, inventory, timeout: float, retries: int):
        super().__init__(network, inventory, timeout)
        self.retries = retries

    async def confirm_order(self, customer: str, product: str) -> str:
        for attempt in range(1, self.retries + 1):
            result = await super().confirm_order(customer, product)
            if "NO RESPONSE" not in result:
                return result + f" (attempt {attempt})"
        return f"Order for {customer}: failed after {self.retries} attempts"

When you run the batch of 4 orders with retries=3, you will commonly see that the stock deducted from aged-cheese is higher than the number of confirmed cheese orders: for example, with seed 16, both cheese orders end up as "failed after 3 attempts" and yet the stock has gone down from 5 to 2 (three deductions that nobody has confirmed). Every time a request reaches inventory and the response is lost, the retry deducts again. The retry turns message loss into duplicated effects. The solution (having each request carry a unique identifier and having inventory remember which ones it has already processed, that is, idempotency) is developed in lesson 02-05.

Conclusion

Deutsch and Gosling's eight fallacies are a catalogue of the mental model errors we make when moving from local calls to network calls: believing that the network is reliable, that latency is zero, that bandwidth is infinite, that the network is secure, that topology doesn't change, that there is one administrator, that transporting data costs nothing and that everything is homogeneous. For each one we have seen a specific symptom at Kilometre Zero and the countermeasure the course will develop later on, summarised in the table in section 10.

The simulation has made the most important lesson tangible: code that ignores the fallacies works perfectly on a good network and hangs forever on a bad one; a timeout prevents the blocking but leaves unanswered the question of whether the operation was executed, and a naive retry turns that doubt into duplicates. No remote call should be made without a timeout, and no operation with side effects should be retried unless it is idempotent.

There is an implicit fallacy that is not on Deutsch's list but lies beneath many problems: the belief that all nodes share the same notion of time. When Anna and Mark buy the last unit of cheese from two different cities, who was first? The next lesson, Time, Clocks and Event Ordering, shows why that question has no obvious answer and which tools (logical and vector clocks) let us answer it in a useful way.

The Fallacies of Distributed Computing

Contents

Origin and purpose of the fallacies

Fallacy 1: the network is reliable

Fallacy 2: latency is zero

Fallacy 3: bandwidth is infinite

Fallacy 4: the network is secure

Fallacy 5: topology doesn't change

Fallacy 6: there is one administrator

Fallacy 7: transport cost is zero

Fallacy 8: the network is homogeneous

Summary table: fallacy, symptom, countermeasure, lesson

Code example: `orders` calls `inventory` over a flaky network

Common Mistakes and Tips

Exercises

Exercise 1: Diagnosing fallacies

Exercise 2: Tuning the timeout

Exercise 3: A naive retry

Solutions

Conclusion

Distributed Architectures Course

Module 1: Introduction to Distributed Systems

Module 2: Communication in Distributed Systems

Module 3: Consistency and Replication

Module 4: Distributed Storage

Module 5: Distributed Computing

Module 6: Security in Distributed Systems

Module 7: Monitoring and Maintenance

Module 8: Case Studies and Applications

The Fallacies of Distributed Computing

Contents

Origin and purpose of the fallacies

Fallacy 1: the network is reliable

Fallacy 2: latency is zero

Fallacy 3: bandwidth is infinite

Fallacy 4: the network is secure

Fallacy 5: topology doesn't change

Fallacy 6: there is one administrator

Fallacy 7: transport cost is zero

Fallacy 8: the network is homogeneous

Summary table: fallacy, symptom, countermeasure, lesson

Code example: orders calls inventory over a flaky network

Common Mistakes and Tips

Exercises

Exercise 1: Diagnosing fallacies

Exercise 2: Tuning the timeout

Exercise 3: A naive retry

Solutions

Conclusion

Distributed Architectures Course

Module 1: Introduction to Distributed Systems

Module 2: Communication in Distributed Systems

Module 3: Consistency and Replication

Module 4: Distributed Storage

Module 5: Distributed Computing

Module 6: Security in Distributed Systems

Module 7: Monitoring and Maintenance

Module 8: Case Studies and Applications

Code example: `orders` calls `inventory` over a flaky network