The Project | About Us | Contribute | Donations | License

HOME

The previous lesson ended with an inventory TCP server that understood STOCK aged-cheese\n, and with a nuisance: every new operation meant inventing a text format, writing a parser on each side and translating errors by hand. What an orders developer would like to write is inventory.reserve_stock("aged-cheese", 2) and get a result back, without thinking about bytes or sockets. That aspiration has had a name since 1984: the remote procedure call (RPC), and its object-oriented version in Java is RMI (Remote Method Invocation).

In this lesson we will look at what is inside a remote call (stubs, marshalling, IDL), why "as if it were local" is a promise the network cannot fully keep (picking up the fallacies from 01-04), which invocation semantics exist and why "exactly once" is a myth. We will implement inventory as an XML-RPC server in Python, call it from orders and deal with the errors that cross the network; then we will see the same service in Java RMI, to understand what changes when the remote things are objects. We will finish by comparing RPC, RMI and REST. gRPC, the modern RPC that Kilometre Zero will use, is left for the next lesson: here we lay down the concepts that gRPC takes for granted.

The idea: calling a procedure as if it were right here
Anatomy of a remote call: stubs, marshalling and IDL
Invocation semantics: what happens when something fails
Transparency and its limits
Classic RPC in Python: inventory with XML-RPC
Errors that cross the network
RMI: remote objects in Java
RPC versus RMI versus REST
Common mistakes and tips
Exercises
Conclusion

The idea: calling a procedure as if it were right here

In the Kilometre Zero monolith, confirming an order is a Python function call: inventory.reserve_stock(lines) (lesson 01-06). The orders module neither knows nor cares how it is implemented. RPC proposes keeping exactly that experience when inventory becomes another process on another machine: the programmer writes the same call, and an intermediate layer (the middleware from lesson 01-01) takes care of turning the arguments into bytes, sending them, waiting, and turning the response into a value.

The original proposal comes from Birrell and Nelson (1984), and it has had descendants in every decade: Sun RPC (the basis of NFS), DCE RPC (the basis of Windows remote calls), CORBA (multi-language remote objects in the 90s), Java RMI (1997), XML-RPC and SOAP (RPC over HTTP with XML, 1998-2000), and finally Thrift and gRPC (binary RPC with schemas, 2007-2015). Formats and transports change; the internal structure is always the same, and that is what we are going to dissect.

Anatomy of a remote call: stubs, marshalling and IDL

An RPC call goes through ten steps, five at each end:

sequenceDiagram
    participant O as orders (business code)
    participant CS as Client stub
    participant NET as Network (TCP/HTTP)
    participant SK as Server skeleton
    participant I as inventory (implementation)
    O->>CS: reserve_stock("aged-cheese", 2)
    CS->>CS: marshalling: args → bytes
    CS->>NET: send request
    NET->>SK: receive request
    SK->>SK: unmarshalling: bytes → args
    SK->>I: reserve_stock("aged-cheese", 2)
    I-->>SK: {"remaining": 3}
    SK->>SK: marshalling of the result
    SK-->>NET: send response
    NET-->>CS: receive response
    CS->>CS: unmarshalling
    CS-->>O: {"remaining": 3}

Each piece has a name worth remembering, because we will meet them all again in gRPC:

Client stub (or proxy): a local object with the same signature as the remote procedure. orders calls it as if it were the real implementation. Its job is to package the call and send it.
Marshalling and unmarshalling: converting arguments and results between the language's in-memory representation and a sequence of bytes that can travel over the network. This includes resolving differences in representation between machines (integer byte order, string encoding) and deciding what to do with pointers or references, which make no sense on another machine. The term serialization is practically a synonym, and the specific formats (JSON, XML, Protocol Buffers) are the subject of lesson 02-03.
Server skeleton (or dispatcher): receives the bytes, unpacks them, works out which procedure is being requested, invokes it and packs the response.
Transport: the protocol from the previous lesson (raw TCP, HTTP...). RPC does not care which one it is, but its properties (timeouts, ordering, reliability) do shape the guarantees.
IDL (Interface Definition Language): a neutral language for describing the contract, that is, which procedures exist, with which parameters and types, and what they return. From the IDL, a generator automatically produces the stubs and skeletons in each language. Sun RPC had .x, CORBA had its own IDL, gRPC uses .proto. XML-RPC and Java RMI have no separate IDL: the contract is simply the signature of the Python functions or the Java interface.

The IDL is conceptually the most important piece. Without it, the contract between orders and inventory exists only in the developers' heads and in the code on each side, and it breaks silently when one of them changes. With it, the contract is a versioned file that both teams share and from which code is generated: the contract-first approach that lesson 02-03 develops.

Invocation semantics: what happens when something fails

A local call either runs or throws an exception; there are no other options. A remote call has a third outcome, one we already came across in lesson 01-04: nobody knows. There are three points at which a message can be lost, and the client cannot tell them apart:

flowchart LR
    A[1. The request is lost] --> X[All the client sees: no response arrives]
    B[2. inventory crashes after executing] --> X
    C[3. The response is lost] --> X

In case 1 the reservation was not made; in cases 2 and 3 it was. The client sees the same thing: silence until the timeout. What the stub does in the face of that silence defines the invocation semantics:

Semantics	What the stub does	Possible outcome for `reserve_stock`	When it is acceptable
Maybe	Sends once, does not retry, does not wait for confirmation	Executed 0 or 1 times; the client does not know which	Telemetry, unimportant notifications
At-least-once	Retries until it gets a response	Executed 1 or more times: the stock may be deducted twice	Idempotent operations (querying stock, setting an absolute value)
At-most-once	Retries, but the server filters out duplicates by request identifier	Executed 0 or 1 times; if there is a response, it comes from the single execution	Operations with side effects (reserving, charging) when it is tolerable for them not to happen
Exactly-once	—	Executed once, guaranteed	Does not exist end to end; see below

Why exactly-once does not exist in practice. To guarantee that an operation with side effects runs exactly once no matter what, the server would have to execute the operation and record "I have already executed it" atomically, that record would have to survive the server's own crash, and the client would always have to be able to find out what state things were left in. You can get very close (a durable log of processed requests, unique identifiers, retries), but there is always a window (the server crashes between executing and recording; the disk fails) in which the guarantee turns into "at least once" or "at most once". What you can achieve is for the operation to be effectively once: running it twice has the same result as running it once (idempotency). That is the practical approach, and lesson 02-05 implements it with a table of processed requests in inventory. For now it is enough to understand that the question "did my call execute?" does not always have an answer, and that the design of the contract must assume so.

One useful detail: at-most-once is usually implemented with a request identifier generated by the client and a response cache on the server. If a repeated request arrives, the server does not re-execute it: it returns the stored response. Keep this mechanism in mind; it is the same one that will reappear in 02-05 under a different name.

Transparency and its limits

RPC sells access transparency (lesson 01-01): the remote call is written just like the local one. It is a promise that is both useful and dangerous. In 1994, Waldo, Wyant, Wollrath and Kendall (the designers of what would become Java RMI) published "A Note on Distributed Computing", whose thesis is that there are four differences between local and remote that no middleware can hide, and that pretending they do not exist produces fragile systems:

Latency. A local call costs nanoseconds; a remote one, milliseconds: between four and six orders of magnitude more. A loop that calls get_stock for each of the 40 products in a basket is harmless in the monolith and disastrous with RPC (40 round trips). Fallacy 2 from 01-04. Design consequence: coarse-grained remote interfaces (get_stock(list_of_products)), not fine-grained ones.
Memory access. A pointer or a reference to an object means nothing on another machine. Arguments travel by value (copied); modifying the copy on the server does not modify the original. This changes the semantics of the code in subtle ways.
Partial failures. We have already seen it: "nobody knows". Locally, if the function does not return it is because the whole process died. Remotely, the client is still alive and has to decide what to do without any information. Fallacy 1.
Concurrency. The server handles many clients at once without the client being aware of it; shared state on the server needs protecting (the lock in exercise 1 of the previous lesson), and the arrival order of calls from different clients is undefined.

The conclusion reached by Waldo and his colleagues, which gRPC and the whole industry have ended up adopting, is that remote interfaces must be designed as remote: explicit signatures, types that travel well, mandatory timeouts, network errors distinguishable from application errors, and no pretending that a remote object is a local object. Keep this in mind as you read the code in the following sections: XML-RPC and RMI make the call look local, but at every step we will see where the network pokes through.

Classic RPC in Python: `inventory` with XML-RPC

XML-RPC is the simplest RPC there is: calls are encoded in XML and travel as the body of an HTTP POST. Python ships it in the standard library (xmlrpc.server and xmlrpc.client), which makes it perfect for seeing the concepts without installing anything. We would not use it in production (it is slow, verbose and schemaless), but every piece has its counterpart in gRPC.

The server

# km0/services/inventory/rpc_server.py
import threading
from socketserver import ThreadingMixIn
from xmlrpc.client import Fault
from xmlrpc.server import SimpleXMLRPCRequestHandler, SimpleXMLRPCServer

# Error codes of the inventory contract. They are part of the interface:
# the client needs them to tell causes apart without parsing text.
ERR_UNKNOWN_PRODUCT = 100
ERR_INSUFFICIENT_STOCK = 101
ERR_INVALID_QUANTITY = 102


class Inventory:
    """Implementation of the service. All its public methods are remote."""

    def __init__(self):
        self._stock = {
            "pink-tomato": 120, "zucchini": 80, "aged-cheese": 5,
            "fresh-cheese": 30, "crianza-wine": 200,
        }
        self._lock = threading.Lock()

    def get_stock(self, product):
        with self._lock:
            units = self._stock.get(product)
        if units is None:
            raise Fault(ERR_UNKNOWN_PRODUCT, f"unknown product: {product}")
        return units

    def reserve_stock(self, product, quantity):
        if not isinstance(quantity, int) or quantity <= 0:
            raise Fault(ERR_INVALID_QUANTITY, "quantity must be a positive integer")
        with self._lock:
            available = self._stock.get(product)
            if available is None:
                raise Fault(ERR_UNKNOWN_PRODUCT, f"unknown product: {product}")
            if available < quantity:
                raise Fault(ERR_INSUFFICIENT_STOCK,
                            f"only {available} units of {product} left")
            self._stock[product] = available - quantity
            remaining = self._stock[product]
        print(f"[inventory] reserved {quantity} of {product}, {remaining} left")
        return {"product": product, "reserved": quantity, "remaining": remaining}


class ThreadedServer(ThreadingMixIn, SimpleXMLRPCServer):
    """SimpleXMLRPCServer serves requests one by one; with ThreadingMixIn, one thread per request."""
    daemon_threads = True


class Handler(SimpleXMLRPCRequestHandler):
    rpc_paths = ("/rpc",)     # only accepts POSTs to this path


if __name__ == "__main__":
    server = ThreadedServer(("0.0.0.0", 8001), requestHandler=Handler,
                            allow_none=True, logRequests=False)
    server.register_introspection_functions()   # system.listMethods, etc.
    server.register_instance(Inventory())        # exposes its public methods
    print("[inventory] XML-RPC listening on :8001/rpc")
    server.serve_forever()

Key points:

register_instance plays the role of the skeleton: it receives the XML, looks for a method with that name on the instance, invokes it with the unpacked arguments and packs the result. There is no IDL: the contract is the list of public methods of Inventory. Methods that start with _ are not exposed (which is why _stock and _lock carry an underscore, and not just by convention).
Fault is the exception that is part of the protocol: XML-RPC defines how it is encoded in the response XML (an integer faultCode and a faultString). We define our own codes because the client needs to know why the call failed without analysing the text. If the method raised any ordinary Python exception (ValueError), the server would turn it into a generic Fault with code 1 and a text such as <class 'ValueError'>:...: useful for debugging, useless for programming against.
ThreadingMixIn is needed for the same reason as the threads in the previous lesson: without it, one slow request blocks all the others. And with it, _lock is essential.
XML-RPC can only carry a handful of types: 32-bit integers, doubles, strings, booleans, dates, binaries, lists and dictionaries with string keys. A Decimal for the price will not travel; neither will a 64-bit integer (in the standard implementation). These marshalling limitations are one of the reasons for preferring formats with a schema (02-03).

The client in `orders`

# km0/services/orders/rpc_client.py
import socket
import xmlrpc.client


class TimeoutTransport(xmlrpc.client.Transport):
    """The default Transport has no timeout: fallacy 1 in its purest form."""

    def __init__(self, timeout):
        super().__init__()
        self._timeout = timeout

    def make_connection(self, host):
        connection = super().make_connection(host)   # http.client.HTTPConnection
        connection.timeout = self._timeout
        return connection


# The stub: a local object with the same "shape" as the remote service.
inventory = xmlrpc.client.ServerProxy("http://localhost:8001/rpc",
                                      transport=TimeoutTransport(2.0),
                                      allow_none=True)

if __name__ == "__main__":
    print("Remote methods:", inventory.system.listMethods())
    print("Aged cheese stock:", inventory.get_stock("aged-cheese"))
    print("Anna's reservation:", inventory.reserve_stock("aged-cheese", 2))
    print("Mark's reservation:", inventory.reserve_stock("aged-cheese", 4))

Output:

Remote methods: ['get_stock', 'reserve_stock', 'system.listMethods', ...]
Aged cheese stock: 5
Anna's reservation: {'product': 'aged-cheese', 'reserved': 2, 'remaining': 3}
Traceback (most recent call last):
  ...
xmlrpc.client.Fault: <Fault 101: 'only 3 units of aged-cheese left'>

ServerProxy is the stub: inventory.reserve_stock(...) does not exist as a Python method; ServerProxy intercepts the attribute access, builds an XML document with the method name and the arguments, makes the POST, and unpacks the response. For the orders developer, it is just a call. To see what actually travels, this is the body of Anna's request:

<?xml version='1.0'?>
<methodCall>
  <methodName>reserve_stock</methodName>
  <params>
    <param><value><string>aged-cheese</string></value></param>
    <param><value><int>2</int></value></param>
  </params>
</methodCall>

About 230 bytes to transmit two arguments: marshalling to XML is easy to read and very expensive to transmit and to parse. And Mark's error response:

<methodResponse>
  <fault>
    <value><struct>
      <member><name>faultCode</name><value><int>101</int></value></member>
      <member><name>faultString</name><value><string>only 3 units of aged-cheese left</string></value></member>
    </struct></value>
  </fault>
</methodResponse>

Errors that cross the network

The previous example ends with a traceback because we do not catch the Fault. A serious RPC client must distinguish four families of error, because each one calls for a different reaction:

Family	Example	Exception in `xmlrpc.client`	Did the operation execute?	Sensible reaction
Application error	Insufficient stock, unknown product	`Fault` (with our `faultCode`)	Yes, and it decided to fail	Treat it as a business rule: tell the customer, do not retry
Could not connect	`inventory` down, wrong DNS	`ConnectionRefusedError`, `socket.gaierror`	No	Retry after a wait, or degrade (02-05 and 07-04)
Timeout	Slow network, overloaded server	`socket.timeout` / `TimeoutError`	Nobody knows	Only retry if the operation is idempotent (02-05)
Protocol error	Wrong path, proxy returning 502, non-XML response	`ProtocolError`, `xmlrpc.client.ResponseError`	Usually not, but a 502 from a proxy does not guarantee it	Log and alert: it is usually a deployment or configuration error

The orders client with full error handling:

# km0/services/orders/confirm_order.py
import socket
import xmlrpc.client
from rpc_client import inventory, TimeoutTransport  # noqa: F401

ERR_INSUFFICIENT_STOCK = 101


def reserve_for_order(product, quantity):
    """Returns (ok, message). Never raises: it turns each family of error
    into an explicit business decision."""
    try:
        r = inventory.reserve_stock(product, quantity)
        return True, f"reserved {r['reserved']} of {product}, {r['remaining']} left"
    except xmlrpc.client.Fault as f:
        if f.faultCode == ERR_INSUFFICIENT_STOCK:
            return False, f"not enough stock: {f.faultString}"
        return False, f"inventory rejected the request ({f.faultCode}): {f.faultString}"
    except (ConnectionRefusedError, socket.gaierror) as e:
        # It definitely did not execute: retrying is risk-free.
        return False, f"inventory unavailable ({e}); order on hold"
    except (socket.timeout, TimeoutError):
        # AMBIGUOUS: it may have executed. Retrying here would duplicate reservations (01-04).
        return False, "inventory did not respond in time; reservation status unknown"
    except xmlrpc.client.ProtocolError as e:
        return False, f"protocol error {e.errcode} at {e.url}: {e.errmsg}"


if __name__ == "__main__":
    for customer, product, quantity in [("Anna", "aged-cheese", 2),
                                        ("Mark", "aged-cheese", 4),
                                        ("Lucy", "crianza-wine", 6)]:
        ok, msg = reserve_for_order(product, quantity)
        print(f"{customer}: {'OK' if ok else 'NO'} - {msg}")

Two observations. First: the timeout branch is the only one that cannot be resolved in this lesson. With what we know so far, the most honest thing to do is leave the order in a "reservation pending confirmation" state and not retry; the complete solution (having the request carry an identifier and having inventory remember which ones it has processed) is the idempotency of 02-05. Second: the Fault exception travels over the network; the others originate in the client. This distinction holds in every RPC system: gRPC formalises it with status codes (02-03), and REST with HTTP 4xx codes (application) versus connection errors and 5xx (infrastructure).

RMI: remote objects in Java

Java RMI takes the idea one step further: what is remote is not a procedure but an object, with state and identity. The client obtains a remote reference to an object that lives in another JVM and invokes methods on it. Although Kilometre Zero is written in Python, RMI is worth a look because it shows the machinery (remote interface, stub, registry, serialization) with complete clarity, and because it is still alive in a great many enterprise Java systems that an architect will run into.

The pieces

Remote interface: extends java.rmi.Remote; every method declares throws RemoteException. This is the contract (it plays the role of the IDL).
Implementation: extends UnicastRemoteObject, which on construction exports the object: it makes it reachable over the network on a port and creates its skeleton.
Registry (rmiregistry): a naming service where the server publishes the object under a name and the client looks it up. It is a primitive form of service discovery.
Stub: since Java 5 it is generated dynamically (with java.lang.reflect.Proxy); the client receives one when it calls lookup and uses it as if it were the object.
Serialization: arguments and results must implement java.io.Serializable (they travel by value) or be remote objects (they travel by reference: the stub is sent).

The contract

// km0/services/inventory-java/Inventory.java
import java.rmi.Remote;
import java.rmi.RemoteException;

public interface Inventory extends Remote {
    int getStock(String product) throws RemoteException, UnknownProductException;
    Reservation reserveStock(String product, int quantity)
            throws RemoteException, UnknownProductException, InsufficientStockException;
}

RemoteException in every signature is RMI refusing to hide fallacy 1: every remote call can fail because of the network, and the compiler forces you to deal with it. The other two exceptions are application exceptions (the equivalent of our Faults with a code).

// km0/services/inventory-java/Reservation.java
import java.io.Serializable;

public class Reservation implements Serializable {
    private static final long serialVersionUID = 1L;   // version of the serialized format
    public final String product;
    public final int reserved;
    public final int remaining;

    public Reservation(String product, int reserved, int remaining) {
        this.product = product; this.reserved = reserved; this.remaining = remaining;
    }
}

// km0/services/inventory-java/InsufficientStockException.java
public class InsufficientStockException extends Exception {
    public InsufficientStockException(String message) { super(message); }
}

// km0/services/inventory-java/UnknownProductException.java
public class UnknownProductException extends Exception {
    public UnknownProductException(String message) { super(message); }
}

Reservation travels by value: the server builds one, RMI serializes it to bytes, and the client receives a copy. serialVersionUID is the version of the format: if the server adds a field and does not update it (or updates it without the client being recompiled), deserialization fails. This is the problem of schema evolution in its crudest form, which 02-03 solves with explicit rules. Exceptions are Serializable through Throwable, so they cross the network with no extra work.

The implementation and the server

// km0/services/inventory-java/InventoryImpl.java
import java.rmi.RemoteException;
import java.rmi.server.UnicastRemoteObject;
import java.util.HashMap;
import java.util.Map;

public class InventoryImpl extends UnicastRemoteObject implements Inventory {
    private final Map<String, Integer> stock = new HashMap<>();

    public InventoryImpl() throws RemoteException {
        super();   // exports the object: from here on it is reachable over the network
        stock.put("pink-tomato", 120); stock.put("zucchini", 80);
        stock.put("aged-cheese", 5);   stock.put("fresh-cheese", 30);
        stock.put("crianza-wine", 200);
    }

    // synchronized: RMI serves each call on its own thread; the state is shared
    public synchronized int getStock(String product) throws UnknownProductException {
        Integer u = stock.get(product);
        if (u == null) throw new UnknownProductException("unknown product: " + product);
        return u;
    }

    public synchronized Reservation reserveStock(String product, int quantity)
            throws UnknownProductException, InsufficientStockException {
        int available = getStock(product);
        if (available < quantity)
            throw new InsufficientStockException("only " + available + " of " + product + " left");
        stock.put(product, available - quantity);
        System.out.println("[inventory] reserved " + quantity + " of " + product);
        return new Reservation(product, quantity, available - quantity);
    }
}

// km0/services/inventory-java/InventoryServer.java
import java.rmi.Naming;
import java.rmi.registry.LocateRegistry;

public class InventoryServer {
    public static void main(String[] args) throws Exception {
        LocateRegistry.createRegistry(1099);             // starts the registry inside this process
        Inventory inventory = new InventoryImpl();        // exported on construction
        Naming.rebind("rmi://localhost:1099/inventory", inventory);
        System.out.println("[inventory] RMI published as 'inventory'");
        // The process does not exit: the exported object keeps the JVM alive
    }
}

The client in `orders`

// km0/services/orders-java/OrdersClient.java
import java.rmi.Naming;
import java.rmi.RemoteException;

public class OrdersClient {
    public static void main(String[] args) {
        try {
            // lookup returns the STUB: a proxy that implements Inventory
            Inventory inventory = (Inventory) Naming.lookup("rmi://localhost:1099/inventory");
            System.out.println("Stock: " + inventory.getStock("aged-cheese"));
            Reservation anna = inventory.reserveStock("aged-cheese", 2);
            System.out.println("Anna: " + anna.remaining + " left");
            Reservation mark = inventory.reserveStock("aged-cheese", 4);
            System.out.println("Mark: " + mark.remaining + " left");
        } catch (InsufficientStockException e) {
            System.out.println("Out of stock: " + e.getMessage());        // application error
        } catch (UnknownProductException e) {
            System.out.println("Unknown product: " + e.getMessage());
        } catch (RemoteException e) {
            System.out.println("Network or server failure: " + e);        // did it execute? nobody knows
        } catch (java.net.MalformedURLException | java.rmi.NotBoundException e) {
            System.out.println("Configuration error: " + e);              // wrong name or URL
        }
    }
}

To run it (with the interface and the serializable classes shared between both programs):

javac *.java
java InventoryServer &
# 2 s response timeout: RMI does not have one by default either
java -Dsun.rmi.transport.tcp.responseTimeout=2000 OrdersClient

Expected output:

Stock: 5
Anna: 3 left
Out of stock: only 3 of aged-cheese left

What RMI adds on top of RPC, and what it costs:

Remote references. If a method returned another Remote object (for example, if Reservation were remote so that you could cancel() it later), the client would receive a stub for that object, which would go on living on the server. This allows natural object-oriented designs... and creates server-side state tied to clients, which has to be cleaned up when they disappear (RMI has a distributed garbage collector based on leases). It is exactly the kind of coupling that modern services avoid: gRPC and REST are deliberately stateless per call.
Coupling to Java. Both ends must be JVMs and share the classes and their serialVersionUIDs. With no neutral IDL, a Python client is out of the question.
Native Java serialization, historically a source of serious vulnerabilities (deserialization of untrusted objects). One more reason for the schema-based formats of 02-03.

RPC versus RMI versus REST

REST is not RPC: it is an architectural style on top of HTTP in which resources identified by URLs are manipulated through the standard verbs (GET, POST, PUT, DELETE), with well-defined semantics for each verb (for example, GET and PUT are idempotent by contract). But it competes with RPC for the same job (synchronous communication between services), so it is worth comparing them:

Criterion	RPC (XML-RPC, gRPC)	RMI	REST
Unit of design	Procedure / function	Object with methods and state	Resource with a representation
Contract	IDL (gRPC) or function signature (XML-RPC)	Java interface	URL + HTTP verbs + body schema (OpenAPI, optional)
`reserve_stock` example	`reserve_stock("aged-cheese", 2)`	`inventory.reserveStock("aged-cheese", 2)`	`POST /products/aged-cheese/reservations {"quantity": 2}`
Languages	Multi-language (gRPC), Python (XML-RPC)	JVM only	Anything that speaks HTTP
Errors	Custom codes / remote exceptions	Serialized Java exceptions	HTTP status codes + body
Server-side state between calls	No (by design)	Possible (remote references)	No (one of its principles)
Idempotency	Defined by convention per operation	Defined by convention	Defined by the verb (`GET`, `PUT`, `DELETE` yes; `POST` no)
Performance	High (binary gRPC, HTTP/2) / low (XML)	Medium (Java serialization)	Medium (JSON over HTTP/1.1, usually)
Intermediate caching (proxies, CDN)	No	No	Yes, native to HTTP for `GET`
Typical use	Service to service, internal	Legacy enterprise Java systems	Public APIs, browsers, third parties
At Kilometre Zero	`orders` → `inventory` (gRPC, 02-03)	Not applicable (Python)	Public API for apps and producers (06-05)

The rule of thumb that Kilometre Zero will adopt: REST on the outside, RPC on the inside. Anna's, Mark's and Lucy's apps and the producers speak REST to a gateway (lesson 06-05), because it is universal, cacheable and does not require generating clients. Between services, where we control both ends and the volume is high, RPC with a schema is more efficient and safer against contract errors.

Common Mistakes and Tips

Designing the remote interface as if it were local. Forty calls with one item each instead of one call with forty items. Every remote call costs a round trip; batch them.
Treating a timeout as "it did not execute". This is the most common way of duplicating reservations and charges. A timeout means "I don't know"; retrying is only safe if the operation is idempotent (02-05).
Using a generic exception for everything. If the client cannot tell "out of stock" from "server down", it cannot react well to either. Explicit error codes in the contract, always.
Forgetting the timeout on the client. Neither ServerProxy nor RMI sets one by default. We have already seen in 01-04 and 02-01 what happens.
Exposing stateful objects by reference (RMI). Every remote reference is state on the server tied to a client that may disappear. Prefer stateless calls with explicit identifiers.
Changing the signature of a remote method without versioning. With XML-RPC, the old client will fail at runtime with a cryptic error; with RMI, with a deserialization exception. This is the problem that the IDL and the evolution rules solve (02-03).
Tip: when you write an RPC client, first write out the table of the four error families (application, connection, timeout, protocol) and what your code will do for each one. If a row is left empty, the design is not finished.
Tip: generate request identifiers on the client from day one, even if you do not use them yet. When 02-05 comes along and you want idempotency, they will already be in the logs and the contracts.

Exercises

Exercise 1: Invocation semantics

An orders client calls reserve_stock("aged-cheese", 2) through a stub that retries up to 3 times on timeout. For each of these scenarios, say how many times the reservation is executed in inventory, what the client sees, and which semantics are being applied: (a) the first request is lost in the network, and the second arrives and gets a response; (b) the first arrives and is executed, but the response is lost; the second arrives and gets a response; (c) all three requests arrive and are executed, and all three responses are lost. Then describe what the server would need to add so that scenario (b) did not duplicate the reservation.

Exercise 2: A coarse-grained method

Add to Inventory (XML-RPC version) a reserve_lines(lines) method that receives a list of dictionaries {"product": ..., "quantity": ...} and reserves all or none: if any line has no stock, nothing is deducted and a Fault with code 101 is returned saying which product failed. Explain why this method is preferable to calling reserve_stock in a loop from orders, in terms of latency and partial failures.

Exercise 3: Classifying errors

A developer sees these five messages in the orders logs during "Artisan Cheese Week". For each one, give the error family (application, connection, timeout, protocol), whether the operation was executed in inventory, and whether it is right to retry automatically:

xmlrpc.client.Fault: <Fault 101: 'only 1 units of aged-cheese left'>
ConnectionRefusedError: [Errno 111] Connection refused
socket.timeout: timed out
xmlrpc.client.ProtocolError: <ProtocolError for localhost:8001/rpc: 502 Bad Gateway>
xmlrpc.client.Fault: <Fault 1: "<class 'TypeError'>:unsupported operand type(s)">

Solutions

Solution 1:

(a) The reservation is executed once (only the second request arrived). The client sees a successful response after one timeout. Effective semantics: at-least-once, which in this case coincides with exactly once.

(b) The reservation is executed twice: the stock drops by 4 units even though Anna wanted 2. The client sees a successful response (from the second execution) and has no way of knowing there was a first one. Semantics: at-least-once, with its typical consequence of duplication.

(c) It is executed three times (the stock drops by 6) and the client sees a definitive failure after three timeouts: it believes that nothing has been reserved. This is the worst case: multiple execution plus false information on the client. It is what the simulation in 01-04 showed with seed 16.

For (b) not to duplicate, the server would need at-most-once semantics: the client includes a unique request identifier (for example, a UUID generated before the first attempt and reused in the retries), and the server stores, by identifier, the response to each request it has already executed. When the retry arrives with the same identifier, it does not re-execute: it returns the stored response. The reservation happens once and the client receives its result. The implementation with persistence in PostgreSQL is the subject of 02-05.

Solution 2:

def reserve_lines(self, lines):
    with self._lock:
        # Phase 1: check everything without touching anything
        for line in lines:
            product, quantity = line["product"], line["quantity"]
            if not isinstance(quantity, int) or quantity <= 0:
                raise Fault(ERR_INVALID_QUANTITY, f"invalid quantity for {product}")
            available = self._stock.get(product)
            if available is None:
                raise Fault(ERR_UNKNOWN_PRODUCT, f"unknown product: {product}")
            if available < quantity:
                raise Fault(ERR_INSUFFICIENT_STOCK,
                            f"{product}: only {available} left, {quantity} requested")
        # Phase 2: apply everything (nothing can fail now, and the lock is still held)
        result = []
        for line in lines:
            self._stock[line["product"]] -= line["quantity"]
            result.append({"product": line["product"],
                           "remaining": self._stock[line["product"]]})
    return result

Advantages over the client-side loop: (1) Latency: a basket with 6 lines costs one round trip instead of six (with a 2 ms RTT between containers that seems little; during a campaign, at 1,200 orders per second, it is 6,000 calls versus 1,200). (2) Partial failures: with the loop, if the fourth line fails because of stock or a timeout, the first three are already reserved and orders has to undo them with more remote calls, which can also fail. With the atomic operation on the server, either everything is reserved or nothing is, and the lock guarantees that no other order slips in between the check and the application. Atomicity is easy here because all the stock lives in one process; once it is spread across inv-bcn and inv-vlc, this same operation becomes a distributed transaction (lesson 03-05).

Solution 3:

Application (code 101). The check was executed and inventory decided to reject. Do not retry: the result will be the same; tell the customer there is no stock left.
Connection. It was not executed (the request never went out). Retrying is safe, but it is wise to wait (the service is down or restarting) and to cap the retries; see 07-04 for the full strategy.
Timeout. Nobody knows. Do not automatically retry a reservation without idempotency (02-05); leave the order pending and check later.
Protocol (a proxy or load balancer answered 502). The request almost certainly never reached inventory, but a 502 can occur if the backend closed the connection halfway through the response, so it is as ambiguous as a timeout. Do not retry without idempotency; raise an alert: it usually points to a faulty deployment or configuration.
Application, but caused by a bug: the generic code 1 and the TypeError indicate that the server raised an unhandled exception (someone probably passed "2" as a string instead of 2). It was partially executed up to the error; with the lock and no earlier side effects, the stock did not change. Do not retry (it would fail in the same way); fix the client and, on the server, validate types before touching state.

Conclusion

We have opened up the box of a remote call. A stub on the client packs (marshalling) the arguments and sends them; a skeleton on the server unpacks them, invokes the implementation and returns the result; an IDL, when there is one, fixes the contract between the two and allows the stubs to be generated. RPC's promise is that all of this is invisible, and we have seen why it cannot be entirely so: latency forces us to design coarse-grained interfaces, arguments travel by value, and partial failures introduce the "nobody knows" outcome, which the invocation semantics (maybe, at-least-once, at-most-once) handle in different ways; exactly-once does not exist end to end, and what we pursue instead is idempotency. With XML-RPC we have implemented inventory.get_stock and reserve_stock, called them from orders with a timeout and classified the errors into four families with different reactions; with Java RMI we have seen the same machinery applied to remote objects, with its references, its registry and its coupling to the JVM. REST, RPC and RMI occupy different places: REST on the outside, RPC on the inside.

What XML-RPC and RMI do not solve well is plain to see in the code: 230 bytes of XML for two arguments, limited types, implicit contracts that break silently as they evolve, and no streaming. The next lesson, gRPC and Data Serialization, tackles exactly that: an explicit IDL (.proto), a compact binary format with evolution rules, and an RPC over HTTP/2 with deadlines, status codes and four kinds of call. It will be the definitive version of Kilometre Zero's orders → inventory interface.

RPC and RMI

Contents

The idea: calling a procedure as if it were right here

Anatomy of a remote call: stubs, marshalling and IDL

Invocation semantics: what happens when something fails

Transparency and its limits

Classic RPC in Python: `inventory` with XML-RPC

The server

The client in `orders`

Errors that cross the network

RMI: remote objects in Java

The pieces

The contract

The implementation and the server

The client in `orders`

RPC versus RMI versus REST

Common Mistakes and Tips

Exercises

Exercise 1: Invocation semantics

Exercise 2: A coarse-grained method

Exercise 3: Classifying errors

Solutions

Conclusion

Distributed Architectures Course

Module 1: Introduction to Distributed Systems

Module 2: Communication in Distributed Systems

Module 3: Consistency and Replication

Module 4: Distributed Storage

Module 5: Distributed Computing

Module 6: Security in Distributed Systems

Module 7: Monitoring and Maintenance

Module 8: Case Studies and Applications

RPC and RMI

Contents

The idea: calling a procedure as if it were right here

Anatomy of a remote call: stubs, marshalling and IDL

Invocation semantics: what happens when something fails

Transparency and its limits

Classic RPC in Python: inventory with XML-RPC

The server

The client in orders

Errors that cross the network

RMI: remote objects in Java

The pieces

The contract

The implementation and the server

The client in orders

RPC versus RMI versus REST

Common Mistakes and Tips

Exercises

Exercise 1: Invocation semantics

Exercise 2: A coarse-grained method

Exercise 3: Classifying errors

Solutions

Conclusion

Distributed Architectures Course

Module 1: Introduction to Distributed Systems

Module 2: Communication in Distributed Systems

Module 3: Consistency and Replication

Module 4: Distributed Storage

Module 5: Distributed Computing

Module 6: Security in Distributed Systems

Module 7: Monitoring and Maintenance

Module 8: Case Studies and Applications

Classic RPC in Python: `inventory` with XML-RPC

The client in `orders`

The client in `orders`