By the close of module 8, BiblioTech had stopped waiting: it imports the catalogue without blocking the menu, sends two hundred notices with a bounded pool, protects its state with ConcurrentHashMap and produces reports with chains of CompletableFuture. And yet it still had a limit that no amount of threads can break: everything happens inside a single machine. Marta Ruiz can only consult the catalogue if she sits in front of that computer. Diego Alonso, from another floor, cannot. Nuria Vidal cannot ask an external service for the metadata of a new title.

This module changes that. But before writing the first line of networking code it is worth understanding what lies underneath a connection, because networking code is the only code you will write in which failure is not a remote possibility: it is the system's normal state. A local file is almost always where you left it; a machine at the other end of the network may be switched off, saturated, behind a firewall, at a changed address or simply slow, and your program has to behave sensibly in every one of those cases.

This lesson has hardly any networking code —that starts in 09-02— but it is the one that makes the rest of the module make sense. You are going to understand the client-server model, the layered model, what an IP address and a port really are, how a name becomes an address, how TCP and UDP differ and why that difference decides the design of your application, and how a protocol is designed. At the end we will design —on paper— BiblioTech's catalogue query protocol, which you will implement in the next two lessons.

A note on the approach. This lesson explains networking concepts from the point of view of somebody who is going to program them, not somebody who is going to administer an infrastructure. You will not find routing tables or subnetting in detail here; you will find exactly what you need to know so that your networking code never surprises you.

1. What it means for two programs to communicate
2. The client-server model and the peer-to-peer model
3. The layered model: the envelope inside the envelope
4. IP addresses: IPv4, IPv6, localhost, private and public
5. Ports: why the address is not enough
6. What a socket is
7. DNS: from a name to an address
8. InetAddress in practice
9. TCP versus UDP
10. The three-way handshake, flow control and retransmission
11. When to choose each one
12. What an application protocol is and how it is designed
13. The BiblioTech protocol
14. Latency versus bandwidth
15. Diagnostic tools
16. What can go wrong on a network
17. Common Mistakes and Tips
18. Exercises

Until now, whenever two parts of your program exchanged information, they shared memory. One thread wrote into a ConcurrentHashMap and another read from it; the problem was coordination (locks, visibility, atomicity), but the data was right there, in the same memory, reachable in nanoseconds.

When two programs communicate over a network, none of that holds:

• They do not share memory. There are no common objects. Only bytes can be sent.
• They do not share time. What you send takes a while to arrive, and sometimes never does.
• They do not share trust. The other end may be a different program, written in another language, or an attacker outright.
• They do not share a lifetime. The other end may die halfway through the conversation without telling you.

Of these four, the first has the most day-to-day consequences: only bytes travel over the network. A BiblioTech Book does not travel. What travels is a sequence of bytes that, by prior agreement between the two programs, both know how to interpret as a book. That prior agreement has a name: a protocol.

The whole of module 9 boils down to learning how to turn information into bytes, push them down a pipe, and rebuild it at the other end knowing that the pipe can break at any moment.

There are two basic ways of organising who talks to whom.

One program —the server— sits waiting for requests at a well-known address. Other programs —the clients— take the initiative, connect to that address, ask for something and receive a reply.

Characteristics:

• Asymmetric. The server does not call the clients; it waits. The client does not wait; it calls.
• The server has a stable, well-known address. The client does not need one.
• The server usually serves many clients at once. This is where module 8 becomes indispensable.
• It is the model of the entire web, of email, of databases and of 95 % of what you will ever program.

BiblioTech is going to be a server: one Nexus Software machine with the catalogue, and every workstation as a client.

Every program is both client and server: each one can start a conversation and each one can serve.

Characteristics:

• Symmetric. There is no central point.
• Resilient: if one node goes down, the rest carry on.
• Complex: node discovery, consistency, security. Far harder to program.
• Real uses: BitTorrent, blockchains, some messaging systems and —on a small scale— the local network discovery you will see in 09-04, where workstations find each other without anybody having told them where the server is.

In Java both models are built from the same pieces (Socket and ServerSocket): a P2P node is simply a program that has both.

When you write a line of text into a socket, that line does not travel down the wire as it is. It travels wrapped, several times over, like a letter inside an envelope that itself goes inside another envelope, which goes inside a mailbag.

Imagine Marta Ruiz wants to send Diego a note that says QUERY 978-0000000001:

1. She writes the note (your application's data).
2. She puts it in an envelope marked "To: Systems department, office 12" — that identifies the destination program inside the building. This is the transport layer, and the office number is the port.
3. She puts that envelope inside another marked "To: Nexus building, 3 High Street" — that identifies the machine. This is the network layer, and the address is the IP.
4. And that envelope goes into the mailbag of the van that covers the leg to the next sorting office. This is the link layer, and the bag only serves that particular leg: at each office the bag is changed, but the envelopes inside are never touched.

On arrival the bag is opened, the outer envelope is opened, the inner one is opened and the note is read. Each layer only understands its own envelope and treats everything inside as opaque content.

About the seven-layer OSI model. You may have heard of seven layers (physical, link, network, transport, session, presentation, application). OSI is an academic reference model; what the Internet actually implements is the four-layer TCP/IP model in the table. Knowing OSI is useful for interviews and for talking to infrastructure people, but for programming the four are enough.

The practical consequence of all this is enormous and worth stating explicitly: you only program the application layer. When in 09-02 you write writer.println("QUERY " + isbn), the operating system takes care of everything else. You will not see a single TCP header or a single MAC address. What you will see are the consequences of those layers: data arriving in pieces, delays, a connection cut in half. Understanding the model is understanding why those things happen.

An IP address identifies a machine's network interface. There are two versions in use.

Four numbers from 0 to 255 separated by dots: 192.168.1.50. That is 32 bits, about 4,300 million possible addresses — which ran out years ago, and that is why IPv6 exists (and NAT, below).

Eight groups of four hexadecimal digits separated by colons: 2001:0db8:85a3:0000:0000:8a2e:0370:7334. That is 128 bits, an absurdly large number of addresses. Zeros may be abbreviated:

In Java, InetAddress handles both transparently: Inet4Address and Inet6Address are subclasses of it and you will almost never need to tell them apart.

A practical detail. In a URL an IPv6 address is written in square brackets so the : are not confused with the port separator: http://[::1]:8080/catalogue.

The address 127.0.0.1 (and its IPv6 equivalent, ::1) is the loopback interface: it always refers to the machine itself. The name localhost points at it.

What matters to you: packets sent to localhost never leave the machine. They do not even touch the network card; the operating system returns them internally. That is why you can try out this whole module on your own computer, with no second machine, no Internet connection and almost zero latency. Every example in the module runs against localhost.

Certain ranges are reserved for private networks and are not routable on the Internet: any public router discards packets addressed to them.

Nexus Software's internal network is 192.168.1.0/24, and the BiblioTech server will live at 192.168.1.50. That means any employee inside the office can connect to it, but nobody from outside — which, for an internal catalogue, is exactly what is wanted.

A note on NAT. Your computer at home has a private address (192.168.1.23), yet you browse the Internet. That works thanks to NAT (network address translation): the router replaces your private address with its single public address on the way out, and notes in a table which connection belongs to whom so it can hand the replies back. The consequence that will affect you as a programmer is this: behind a NAT you can start outbound connections, but nobody outside can start a connection towards you, because the router would not know which internal machine to deliver it to. That is why servers need a public address or explicit port-forwarding rules, and why P2P systems have to jump through hoops to traverse NAT.

If the IP identifies the machine, why is anything else needed?

Because a machine runs many programs that use the network at once. Right now your computer may have a browser with fifteen tabs, a mail client, a development server and BiblioTech. When a packet arrives at the machine's address, the operating system needs to know which of them all to deliver it to.

The port is that number. Back to the analogy: the IP is the street and the building number; the port is the office number.

• It is an unsigned 16-bit number: from 0 to 65535.
• Ports 0-1023 are well known and on Unix-like systems require administrator privileges to listen on.
• Ports 1024-49151 are registered: assigned to particular applications but without privileges.
• Ports 49152-65535 are ephemeral: the ones the system assigns automatically to your outbound connections.

The fact that HTTP lives on 80 is what lets you write http://example.com with no port: the browser assumes 80 because the http scheme implies it. If you write http://example.com:8080, you are explicitly naming a different office.

BiblioTech will use port 9090 for its catalogue server (TCP) and 9091 for its discovery service (UDP). They are high ports, unprivileged, and clash with nothing common.

Port 0 is special. If you ask for port 0 when creating a ServerSocket, the system assigns you any free one and you can then read it back with getLocalPort(). It is the standard technique in automated tests for not clashing with busy ports — you will see it in module 11 when we cover JUnit.

Now the word that gives the next lesson its name can be defined precisely.

A socket is one endpoint of a communication, identified by the pair (IP address, port). A TCP connection is uniquely identified by four values: source IP, source port, destination IP, destination port.

This is why a thousand clients can connect to port 9090 of the same server without conflict: even though the destination is identical in all thousand cases, the source (IP + ephemeral port) is different for each, so the four-tuples differ and the operating system never confuses one connection with another. It is a very common doubt when starting out: no, the server does not need "one port per client".

In Java a TCP socket is represented by the java.net.Socket class, and the object that accepts incoming connections on the server is java.net.ServerSocket. The pleasant surprise —and the reason module 7 was a prerequisite— is what a Socket offers you once connected:

An InputStream and an OutputStream. Exactly the same ones you used to read and write files. The same decorators (BufferedReader, InputStreamReader, PrintWriter), the same need to declare the charset, the same try-with-resources. Once connected, the network is I/O you already know how to do. The only new part is how that connection is set up and how it breaks.

Nobody types 142.250.185.78 into a browser. They type a name. DNS (the Domain Name System) is the distributed phone book that translates names into addresses.

Points that matter when programming:

• Resolution takes time. It is a network call that may add tens or hundreds of milliseconds the first time. If you call getByName inside a loop, you pay it every time.
• It is cached. The operating system and the JVM itself cache results. The JVM has the networkaddress.cache.ttl property to control this; the default value in applications with no security manager is usually 30 seconds.
• It can fail. A non-existent name produces UnknownHostException, which is an IOException. You have to catch it.
• A name may have several addresses (load balancing): that is why getAllByName exists, returning an array.
• The /etc/hosts file (on Windows, C:\Windows\System32\drivers\etc\hosts) takes priority over DNS. It is a common trick for testing a production name locally.

java.net.InetAddress is the class that represents an IP address in Java. It has no public constructor: you obtain one through static factory methods.

Let us write the module's first networking class: a diagnostic tool that Nexus Software will use to check where each workstation is and what it can reach.

Typical output:

Three details worth fixing in your mind:

1. getLocalHost() can throw UnknownHostException on badly configured machines (containers with no /etc/hosts entry, above all). This is not a theoretical case; it happens often in Docker.
2. isReachable is not ping. It is an attempted check that depends on privileges and on firewalls. A false means "I did not manage it", not "the machine is down". Never base an important decision on its result alone.
3. Resolving a non-existent name can take seconds, not milliseconds, because the resolver retries. Another reason not to put it in a hot loop.

Two transport protocols live on top of the network layer (IP), and choosing between them is the first design decision of any network application.

TCP (Transmission Control Protocol) offers your program the illusion of a continuous, reliable pipe between the two ends, as if it were a file open at both sides.

It guarantees:

• Delivery: if a segment is lost, it is retransmitted. If delivery proves impossible, you are notified with an error.
• Ordering: the bytes arrive in the same order in which they were sent.
• No duplicates: repeated segments are discarded.
• Flow control: if the receiver is slow, the sender automatically throttles back.
• Congestion control: if the network is saturated, TCP slows down so as not to make it worse.

In exchange: the connection has to be established before anything is sent (which costs a round trip), the headers are bigger, and the guarantees introduce delays — if one segment is lost, everything after it waits for the retransmission to arrive (head-of-line blocking).

UDP (User Datagram Protocol) is almost transparent: it takes your block of bytes, puts ports and a checksum on it, and drops it onto the network.

It guarantees nothing:

• A datagram may be lost and nobody tells you.
• They may arrive out of order.
• One may arrive duplicated.
• There is no flow control: you can swamp the receiver without noticing.

In exchange: there is no setup (you send instantly), the header is 8 bytes against TCP's 20+, and each message is preserved as a unit (if you send 100 bytes, the other end receives 100 bytes or nothing, never 60). And it allows sending to many at once with broadcast and multicast, which TCP cannot do by definition.

Before a single byte of your data travels, TCP establishes the connection with three messages:

Practical consequences:

• Opening a connection costs a full round trip. If the server is 80 ms away, setup alone is 80 ms before a single useful byte is sent. That is why modern HTTP libraries reuse connections (you will see this in 09-06 with HttpClient) and why opening a new socket for every request is a classic performance mistake.
• The server replies SYN+ACK even if your program has never called accept(). The operating system keeps a queue of already-established connections waiting for your code to pick them up: it is the backlog of 09-03.
• If the port is closed, the system replies with RST instead of SYN+ACK, and in Java that becomes ConnectException: Connection refused — immediate, not a timeout. Telling "refused" (there is a machine, nobody is listening) apart from "timed out" (no reply at all: machine down or a firewall dropping silently) is diagnosis, pure and simple.
• Teardown is four messages, not three, because each direction is closed separately. That is where the concept of a half-close (shutdownOutput) that you will see in 09-02 comes from.

Every segment sent must be acknowledged. If the sender receives no acknowledgement within an estimated deadline, it resends. That deadline is recalculated constantly by measuring the round-trip time.

What you see from Java: nothing. Your read() simply takes a little longer. That is exactly TCP's value proposition, and also the reason why a read() without setSoTimeout can sit still for far longer than you imagine.

The receiver announces in every acknowledgement how much room is left in its buffer (the window). If it announces zero, the sender stops until space is freed.

What you see from Java: if the other end does not read, your write() will eventually block. It is a common surprise: many people believe that writing to a socket never blocks because "it is only copied into a buffer". It is copied into a buffer, yes, but that buffer is finite and fills up if the other end does not consume.

The mental rule that works: is the data from a moment ago still worth anything, or must it be waited for no matter what? If retransmitting stale data is worse than losing it, UDP. If losing it breaks the meaning, TCP.

The hybrid case. HTTP/3 and QUIC, the most recent web protocols, run on top of UDP and implement reliability, ordering and congestion control themselves, precisely to escape TCP's head-of-line blocking. It is the proof that "unreliable" does not mean "worse": it means "the guarantees are yours to define, tailored to your needs".

BiblioTech will use both: TCP for the catalogue (09-02 and 09-03) and UDP for discovery and telemetry (09-04).

An application protocol is the agreement about which bytes mean what. TCP gives you a reliable pipe of bytes, but a pipe of bytes does not say where one message ends and the next begins, nor what is a request and what is a reply. That part is yours.

For learning and for most internal applications, text. That is what we will do: you will be able to talk to the BiblioTech server using telnet, which is an invaluable teaching aid.

TCP is a byte stream with no boundaries. If the client performs two writes, the server may receive them in a single read, or split into three. This is sometimes called "the message-sticking problem" and it is the number-one cause of subtle bugs in home-made protocols.

There are three classic solutions:

HTTP uses all three at once: a delimiter (\r\n) for the headers, Content-Length (a length prefix) for the body, and in HTTP/1.0 the connection close.

In Java the line-delimiter solution is almost free: BufferedReader.readLine() already takes care of accumulating until it finds the line break, and PrintWriter.println() of adding it. That is why line protocols are so convenient and why BiblioTech's will be one of them.

Careful with readLine(). It returns null when the other end closes. That null is the only end-of-conversation signal you are going to get, and not checking it produces a NullPointerException on the first client that disconnects. You will see it demonstrated in 09-02.

When you design one, answer explicitly:

1. Who speaks first? (does the client ask, or does the server greet?)
2. Where does each message end? (delimiter, length, close)
3. Which character encoding? (UTF-8, always, explicitly)
4. How is an error signalled? (codes, as in HTTP; or free text, which is worse)
5. How does the conversation end? (a farewell command, a close, an idle timeout)

With that, we can now design the protocol you will implement in 09-02 (client) and 09-03 (server). It will be called BTCP (BiblioTech Catalog Protocol), version 1.

Notice the decisions and why they are taken:

• Three-digit numeric codes. Shamelessly copied from HTTP and SMTP, and for good reasons: the client can decide on the first digit alone (2 = fine, 4 = your fault, 5 = my fault) without parsing the text, and the text can be changed or translated without breaking anybody.
• | as the field separator. Chosen because it appears neither in titles nor in ISBNs. If it could appear, it would have to be escaped — the same problem you already solved with CSV in 07-07.
• One reply per request. It simplifies the client enormously: write one line, read one reply.
• The multi-line reply announces its size and is also closed with .. The number allows you to allocate and to detect truncation; the final full stop allows you to finish even if the number was wrong. It is redundant on purpose.
• The server greets first. That way the client confirms it has reached the right place, and a server that speaks its version of the protocol, before sending anything.
• There is a farewell command. Without it, the only way to finish cleanly would be to close the socket, and the server could not tell an orderly close from a crash.

This is the contract. In 09-02 you will write the client against it (testing it with nc while there is no server), and in 09-03 the multi-client server that implements it.

Two different quantities that are constantly confused and that govern the performance of everything you program on a network.

The key point: bandwidth can be bought; latency cannot. It is limited by the speed of light. London to New York is about 5,600 km; a round trip over fibre is at least about 60 ms, and in practice 90-120 ms. No optimisation of your code is going to go below that.

Orders of magnitude worth internalising:

Compare the first row with the last: the network is a million times slower than memory. Three design rules follow from this, and you will see them applied all through the module:

1. Minimise the number of round trips, not the number of bytes. One request returning 100 records beats a hundred single-record requests, even though they move the same data.
2. Overlap the waits. If you have to make ten independent queries, make them in parallel. This is exactly what you learned in module 8 and what you will do in 09-06 with sendAsync and allOf.
3. Reuse connections. Opening one costs a full round trip. It is the reason HttpClient has an internal pool.

Note: network byte order. When you send a 4-byte int, does the most significant byte go first or the least significant one? Different CPU architectures choose differently (big-endian versus little-endian), so network protocols fix one: big-endian, also called network byte order. The good news for you is that Java's DataOutputStream.writeInt() and DataInputStream.readInt() already write and read in big-endian, and ByteBuffer uses big-endian by default, so if you use those classes at both ends you never have to think about it. It only matters if you are talking to a C program that has written raw bytes, in which case it will have to use htonl/ntohl.

Programming networks without diagnostic tools is programming blind. These five answer most questions, and you will use them all through the module.

It measures latency and loss. A ping that fails does not prove the machine is down: a great many firewalls block ICMP as a matter of policy. And a ping that works does not prove your service is alive: only that the machine replies.

It shows every intermediate hop with its latency. Useful for locating where a connection degrades.

This is the tool that solves the module's most frequent error: BindException: Address already in use. It tells you exactly which process is holding your port. If it is an earlier run of your own server that you did not shut down, you kill it and you are done.

It acts as a generic TCP/UDP client or server. It is like having a Socket and a ServerSocket on the command line.

In 09-02 you will write the BiblioTech client before having a server, and you will test it against nc -l 9090 playing the server by hand. It is the best way to understand a protocol.

The same as nc for TCP, but present for decades and with local echo, which makes typing more comfortable. In 09-03 you will test the BiblioTech server with it.

In 09-05 and 09-06 you will use it to compare what your Java code does with what a reference client does. The -v option shows the headers sent and received, which is exactly what you need to see.

This lesson ends with what will shape you most as a network programmer. There is a famous list of fallacies of distributed computing: assumptions everybody makes at the start, all of them false.

Of all of them, the first is the one that must change the way you write code. Network code does not fail occasionally: it always fails, and the only question is how often.

This connects directly with module 6. Every networking class in the JDK throws IOException or subclasses of it, and they are checked exceptions — the compiler forces you to deal with them. That is not bureaucratic annoyance: it is the language design telling you the truth about the world.

Three ideas you will carry through the whole module:

1. Tell transient apart from permanent. A timeout deserves a retry; a refused connection or a non-existent name does not. Retrying the unretryable is an expensive anti-pattern.
2. Always set a time limit. With no setSoTimeout and no setConnectTimeout, a read() can sit still until the operating system gives up, and that can be hours. A thread blocked forever is a thread lost, and with a bounded pool, a pool that runs dry.
3. Translate the failure into your domain at the boundary. Having RemoteCatalog throw BiblioTechException and not SocketException is what lets the menu know nothing about sockets — exactly the layered strategy of 06-07.

Confusing IP with port, or believing the server needs one port per client. We have already seen it: the connection is identified by the complete four-tuple. A thousand clients, a single server port.

Caching an InetAddress for the whole life of the program. Addresses change: DHCP, migrations, load balancers. Resolve the name when you need it and let the system cache do its job, instead of storing the IP yourself in a static final.

Using getLocalHost() to find out "my public address". It is not. It returns the machine's address on its network, which behind a NAT is private. To know your public address, somebody outside has to tell you.

Assuming that isReachable() is the same as ping. It depends on privileges and firewalls. Use it as a hint, never as a decision.

Choosing UDP "because it is faster". UDP is faster in the happy case. If you need reliability and implement it on top, you will end up writing a worse version of TCP. Choose UDP when you do not need the guarantees, not when you want to save the 20 bytes of header.

Designing a protocol without deciding where a message ends. The bug will show up the day a message is split across two reads, which is the day you have real traffic. Decide the delimiter before writing the first line.

Not fixing the encoding in the protocol. If the client writes in UTF-8 and the server reads in the platform's default encoding, Refactoring – Improving Design will arrive as Refactoring – Improving Design. It is documented in the protocol, and made explicit in the code. Always.

Writing network code with no time limits. The case that proves it: a server whose read() blocks because the client went away without closing (their laptop battery ran out). Without setSoTimeout, that pool thread never comes back. With ten clients like that, the ten-thread pool is dead and the server stops accepting anybody, without a single error in the log.

Testing only on localhost and assuming it works. On localhost latency is zero, there are no losses, there is no small MTU and there is no firewall. Many network bugs only appear outside. Test between two real machines as well when you can, and at the very least keep in mind what loopback is hiding from you.

A cross-cutting tip. When something does not work, diagnose layer by layer, from the bottom up: does the machine reply (ping)? is the port listening (ss -tlnp)? can you connect (nc -zv)? does the protocol reply what you expect (telnet or curl -v)? Each question rules out a whole layer and the problem shows up in two minutes instead of two hours.

Write a class NetworkInventory with a method analyse(String... names) that, for each name received:

• resolves it to one or more addresses, measuring the resolution time in milliseconds;
• states for each address whether it is IPv4 or IPv6, whether it is loopback and whether it is private;
• if it does not resolve, says so without throwing an exception.

It must print an aligned table and finish with a summary of how many names resolved and how many did not. Try it with localhost, 127.0.0.1, your own machine and a made-up name.

Write a class BtcpAnalyser with a static method String describe(String line) that takes one line of the BTCP/1 protocol defined in this lesson and returns a readable description. It must:

• tell requests (QUERY, LIST, LEND, QUIT) apart from replies (which start with three digits);
• for a reply, state the family according to the first digit (2 = success, 4 = client error, 5 = server error);
• validate that QUERY carries exactly one argument and LEND at least two;
• return an error description for anything that does not fit.

Write a main that tests it with the sample session from section 13. Do not use the Streams API.

Write a class NetworkEstimator with a method estimate(int numberOfRequests, int bytesPerRequest, int latencyMs, double bandwidthMbps) that calculates and compares three strategies for fetching the data of N catalogue materials:

1. Sequential: N requests, one after another.
2. Parallel with 8 connections: N requests spread across 8 simultaneous lanes.
3. Batched: 1 single request returning the N materials.

For each one, work out the total time (latency + transfer) and show it in a table with the improvement factor relative to the sequential case. Assume each sequential request costs one full round trip (the latency) plus the time to transfer its bytes.

Solution 1

Typical output:

Comments. Notice three things in the real output: 127.0.0.1 resolves in 0 ms because there is no DNS query (it is a literal IP, it is merely parsed); the non-existent name takes 41 ms, far longer than the ones that do resolve, because the resolver asks and waits before giving up — on a badly configured network this can be seconds; and localhost is loopback but not private, because isSiteLocalAddress() only looks at the 10/8, 172.16/12 and 192.168/16 ranges.

Solution 2

Comments. Three decisions deserve attention. The first: isReplyCode does not just look for three digits, it also demands that a space or the end of the line follows; without that, the data line 978-0000000001|Effective Java|... starts with three digits and would be classified as a reply. The second: LEND does not use split(" ") because the employee name contains spaces; the first separator is found by hand and the remainder is taken whole. And the third: notice that the data line 978-...|Effective Java|... falls into the default of describeRequest and is flagged as an unknown command. That is correct: outside the context of a 201 reply, that line is not valid. Parsing a stateful protocol requires remembering which point of the conversation you are at, and that is exactly what the real client of 09-02 will do.

Solution 3

Output:

Comments. This is the most important exercise in the lesson, because the result is counter-intuitive and decides designs. The three cases move exactly the same 20,000 bytes, and yet the slowest takes 12 seconds and the fastest 123 milliseconds: a difference of almost a hundredfold. What changes is not the volume, but the number of round trips.

Compare the two scenarios as well. On the local network the sequential strategy takes 100 ms — annoying but bearable. Against a service 120 ms away, the same strategy takes twelve seconds, and no increase in bandwidth fixes it: the real transfer is 0.2 ms out of the 120.2 of each round trip. That is why the two techniques that work are batching (one request instead of a hundred) and overlapping (eight at a time), and that is why module 8 was a prerequisite for this one. In 09-06 you will implement exactly the parallel strategy with sendAsync and allOf.

This lesson has laid the foundations of the whole module. You are no longer going to write networking code blind.

You know what it really means for two programs to communicate: they share no memory, no time, no trust and no lifetime, and only bytes travel over the network, bytes that both ends have to know how to interpret. You know the two ways of organising them —client-server, asymmetric and with a well-known address, which is BiblioTech's and 95 % of what you will ever program; and peer-to-peer, symmetric and resilient but far more complex— and you know that in Java both are built from the same pieces.

You understand the layered model as envelopes inside envelopes: the application layer carries your data, the transport layer identifies the program with the port, the network layer identifies the machine with the IP, and the link layer only covers the next leg. And you know the practical consequence: you only program the application layer, but you suffer the consequences of all the others.

You can handle IP addresses —32-bit IPv4, 128-bit IPv6, 127.0.0.1 as the loopback that never even touches the network card, and the private ranges that let Nexus Software run on 192.168.1.0/24 with NAT to get out— and ports: 16 bits, from 0 to 65535, with the well-known ones below 1024 and the ephemeral ones above 49152. Along with the idea that settles the most doubts: a socket is the (IP, port) pair, a TCP connection is a four-tuple, and that is why a thousand clients fit on a single server port.

You know how a name becomes an address through DNS, which takes time, is cached and can fail with UnknownHostException; and you have written your first networking class, NetworkDiagnostics, which resolves names, lists interfaces and checks reachability, with the three warnings that come with it: getLocalHost() fails in containers, isReachable() is not ping, and resolving a non-existent name can take seconds.

And above all you have understood the decision that opens this module: TCP versus UDP. TCP gives you a reliable, ordered pipe in exchange for a three-way handshake that costs a whole round trip, retransmissions that make your reads take longer without you noticing, and flow control that can end up blocking your writes if the other end does not read. UDP gives you none of that, which is why it is the right choice when stale data is worthless: video, games, telemetry, DNS and —what BiblioTech will use— local network discovery by broadcast, something TCP simply cannot do.

You know what an application protocol is, why text wins on clarity and binary on size, and what its central problem is: TCP has no message boundaries, so you have to add them with a delimiter, with a length prefix or with the connection close. And you have designed a complete one, BTCP/1: a server greeting, the commands QUERY, LIST, LEND and QUIT, replies with three-digit codes whose first digit already says whether it went well, multi-line replies that announce their size and close with a full stop, explicit UTF-8 and a line delimiter.

You have internalised the difference between latency —which cannot be bought because the speed of light limits it— and bandwidth, with the figure that sums it all up: the network is a million times slower than memory, and from that come the three rules that will govern your code: minimise round trips, overlap waits, reuse connections. Your own estimator has shown you that the same 20,000 bytes take twelve seconds or one hundred and twenty-three milliseconds depending on how you ask for them.

You know the five tools with which real diagnosis is done —ping, traceroute, ss, nc and curl— and the method: layer by layer, from the bottom up. And you accept the premise that makes a good network programmer: network code does not fail occasionally, it always fails, so you write it assuming failure, telling the transient from the permanent, putting a time limit on everything and translating the network failure into a BiblioTechException at the boundary, exactly as you learned in 06-07.

In the next lesson, Sockets, the theory ends. You are going to open your first connection from Java with the Socket class, and you are going to discover something you already suspect: once connected, getInputStream() and getOutputStream() hand you exactly the streams of module 7, with the same decorators, the same explicit charset and the same try-with-resources. You will see how to connect with a time limit, how to close half a connection, what each network exception means and what options a socket has. And you will see demonstrated, live, the number-one bug of everybody starting with sockets: forgetting to flush the buffer and leaving the other end waiting forever. By the end you will have the BiblioTech client speaking BTCP/1 — tested against an nc -l 9090 in which you play the server by hand, while the real server arrives in 09-03.