BiblioTech's catalogue server already works. Marta, Diego and Nuria connect at the same time from their desks and query the catalogue with BTCP/1 over TCP. But there is one detail that spoils the party: every workstation has the server's address written by hand in its bibliotech.properties. The day the server machine changes IP, somebody will have to go desk by desk correcting a configuration file.

There is an elegant solution: let the workstations ask the whole local network where BiblioTech is and let the server reply. But TCP cannot do that. TCP demands that you know who you want to connect to before connecting; there is no "send this to everybody and see who answers" operation.

UDP can. And that is the subject of this lesson: the other transport protocol, the one that guarantees nothing —not delivery, not ordering, not uniqueness, not flow control— and that precisely because of it can do things TCP cannot. You are going to understand when that lack of guarantees is a problem and when it is exactly what you want, you are going to make (and avoid) the two classic mistakes everybody makes, and you are going to give BiblioTech two new capabilities: automatic discovery of the server on the local network and continuous telemetry that blocks nobody and does not care about losing the odd packet.

Contents

  1. UDP in practice: what you lose and what you gain
  2. When UDP is the right choice
  3. DatagramPacket: the envelope
  4. DatagramSocket: the letterbox
  5. Sending and receiving: the echo client-server
  6. Classic mistake 1: reusing the receive packet without restoring the length
  7. Classic mistake 2: using buffer.length instead of packet.getLength()
  8. Datagram size, MTU and fragmentation
  9. Loss, duplication and disorder: the demonstration
  10. Reliability by hand: sequence, time limit and retry
  11. Broadcast
  12. Multicast with MulticastSocket
  13. BiblioTech: the discovery service
  14. BiblioTech: the telemetry publisher
  15. The same case solved with TCP and with UDP
  16. Common Mistakes and Tips
  17. Exercises

  1. UDP in practice: what you lose and what you gain

With TCP you had a pipe. With UDP you have a postbox: you put an envelope in, you drop it, and you hope it arrives.

graph LR
    subgraph TCP
    A["Client"] ---|established connection| B["Server"]
    end
    subgraph UDP
    C["Sender"] -.->|envelope 1| D["Receiver"]
    C -.->|envelope 2 - lost| E["X"]
    C -.->|envelope 3| D
    end

What you lose compared with TCP:

Guarantee lost What it means in your code
Delivery A datagram can vanish and nobody tells you. No exception, no warning
Ordering Packet 3 can arrive before packet 2
Uniqueness A packet can arrive twice (retransmitted by a router)
Flow control You can swamp the receiver without noticing
Congestion control You can make a congested network worse
Detecting the peer going down There is no connection, so there is nothing to break

What you gain:

Advantage Detail
No setup You send instantly. TCP spends a round trip before the first useful byte
Minimal header 8 bytes against TCP's 20 or more
Message boundaries If you send 100 bytes, the other end receives 100 bytes or nothing. Never 60. The delimiter problem is gone
Stateless The server keeps nothing per client: it can serve thousands with a single socket
One to many Broadcast and multicast, impossible in TCP
No head-of-line blocking A lost packet does not delay the ones after it

Of all of them, the two that most change the design are the message boundaries —which make all the delimiter work of 09-02 unnecessary— and the one-to-many capability, which is literally impossible with TCP.

  1. When UDP is the right choice

The deciding question is always the same: is stale data still worth anything, or must it be waited for no matter what?

Real case Why UDP
DNS One question, one answer, fits in a datagram. If it is lost, ask again. Establishing a TCP connection for 60 bytes would be absurd
Live video and audio A lost frame is a 40 ms flicker. Retransmitting it would arrive late and would also hold up everything else
Action games The player's position 200 ms ago is of no interest: the one now is. Retransmitting an old position is worse than losing it
Telemetry and metrics Thousands of measurements per second; losing one changes nothing. And the sender must never block because of the receiver
Local network discovery It requires broadcast. TCP cannot
Clock synchronisation (NTP) The latency must be minimal and predictable; the protocol already tolerates losses
Remote logging (syslog) A log server that is down must not block the applications writing to it
QUIC and HTTP/3 They implement their own reliability over UDP to avoid TCP's head-of-line blocking

And the negative criterion, just as important: if you end up implementing acknowledgements, retransmissions, sequence numbers and flow control over UDP, you have written a worse version of TCP. Choose UDP when you do not need the guarantees, not when you want to save twelve bytes of header.

There is one legitimate exception to that rule, and it is when you need partial, tailored reliability: for example, retransmitting only the key frames of a video, or guaranteeing ordering but not delivery. TCP does not offer that —it is all or nothing— and it is the reason QUIC exists.

  1. DatagramPacket: the envelope

A DatagramPacket is an envelope: it holds the data, its length, and —depending on what you use it for— an address and a port.

The class is used in two completely different ways, and confusing them is the source of most problems.

For sending: data + destination

byte[] data = "WHERE IS BIBLIOTECH?".getBytes(StandardCharsets.UTF_8);

DatagramPacket packet = new DatagramPacket(
        data,                                    // the content
        data.length,                             // how many bytes of that array
        InetAddress.getByName("192.168.1.50"),   // where it goes
        9091);                                   // to which port

For receiving: an empty buffer

byte[] buffer = new byte[1024];

// No address or port: receive() will fill them in with the SENDER's.
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

socket.receive(packet);         // blocks until something arrives

// Now the packet DOES have a sender, and we know how many bytes arrived.
InetAddress sender = packet.getAddress();
int senderPort = packet.getPort();
int bytesReceived = packet.getLength();

DatagramPacket methods

Method Meaning
getData() The byte array. Careful: the whole array, not just what was received
getLength() How many bytes are valid. The piece everybody forgets
getOffset() From which position in the array the valid data starts
getAddress() / getPort() The destination (when sending) or the sender (when receiving)
setData(byte[]) Changes the array
setLength(int) Changes the valid length. Key to reusing packets
getSocketAddress() Address and port together, as a SocketAddress

The distinction between getData().length (the buffer size) and getLength() (the bytes that actually arrived) is the source of the second classic mistake, and we cover it in section 7.

  1. DatagramSocket: the letterbox

A DatagramSocket is the letterbox through which the envelopes go out and come in. Unlike Socket, it does not represent a connection with anybody: it represents a point on the machine through which you send and receive.

// EPHEMERAL socket: the system assigns a free port.
// It serves to send and to receive replies to what was sent.
DatagramSocket client = new DatagramSocket();

// Socket bound to a WELL-KNOWN port: this is what a UDP server does.
DatagramSocket server = new DatagramSocket(9091);

// Bound to a port and to a particular interface.
DatagramSocket local = new DatagramSocket(9091, InetAddress.getByName("127.0.0.1"));

The essential difference from Socket and ServerSocket

TCP UDP
Client class Socket DatagramSocket
Server class ServerSocket and a Socket per client DatagramSocket (just one, for everybody)
Objects per client One None: there is no per-client state
How you know who is talking It is the connection It is in each packet: getAddress()
Threads needed One per connection One, or a few

A UDP server does not need a thread pool per client, because there are no clients: there are loose packets. A single thread in a receive() loop can serve thousands of machines. It is a radically simpler architecture, at the price of your doing everything TCP used to do for you.

Main methods

Method What it does
send(DatagramPacket) Sends. It almost never blocks and almost never fails, even if the destination does not exist
receive(DatagramPacket) Blocks until a datagram arrives
setSoTimeout(int ms) Makes receive throw SocketTimeoutException. Indispensable
connect(InetAddress, int) Filters: it only sends to and receives from that pair. It establishes no connection
disconnect() Undoes the filter
setBroadcast(boolean) Allows sending to the broadcast address
getLocalPort() The port you were assigned
close() Releases the port. It implements Closeable

About send: the absolute silence. This surprises everybody. send() normally throws no exception even if the destination is switched off, does not exist or drops the packet. The system hands the datagram to the network and forgets about it. If you send to 192.168.1.250 and there is nothing there, your code learns nothing. The only possible signal is an ICMP "port unreachable" message that may arrive and cause a PortUnreachableException on the next operation — but only if the socket is connect-ed, and not even then is it guaranteed. With UDP, the success of send() means absolutely nothing.

About connect in UDP: it is not what it looks like. DatagramSocket.connect(addr, port) performs no handshake, establishes nothing and sends not a single packet. It only installs a local filter: from then on send may omit the destination and receive discards packets from any other sender. It is useful for two reasons: it stops a third party injecting fake packets, and it allows receiving PortUnreachableException. It is a convenience and a defence, not a connection.

  1. Sending and receiving: the echo client-server

Let us start with the simplest pair that works, to fix the mechanics.

The echo server

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.SocketException;
import java.nio.charset.StandardCharsets;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * UDP echo server. One thread, one socket, every client.
 * No connections, no pool, no per-client state.
 */
public class UdpEchoServer implements AutoCloseable {

    private static final Logger LOG = Logger.getLogger(UdpEchoServer.class.getName());

    /** Size of the receive buffer. See section 8 about the MTU. */
    private static final int MAX_DATAGRAM = 1400;

    private final int port;
    private DatagramSocket socket;
    private volatile boolean running = false;

    public UdpEchoServer(int port) {
        this.port = port;
    }

    public void run() throws SocketException {
        socket = new DatagramSocket(port);
        running = true;
        LOG.info(() -> "UDP echo server on port " + port);

        // ONE buffer and ONE packet, reused throughout the loop:
        // creating a 1400-byte array for every received packet
        // would generate needless garbage thousands of times a second.
        byte[] buffer = new byte[MAX_DATAGRAM];
        DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

        while (running) {
            try {
                // ESSENTIAL before every receive: restore the length.
                // If the previous packet had 20 bytes, getLength() is 20
                // and only 20 bytes of the next one would be received. Section 6.
                packet.setLength(buffer.length);

                socket.receive(packet);         // BLOCKS until something arrives

                // getLength() -> what REALLY arrived, not buffer.length.
                String text = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);

                LOG.info(() -> "From " + packet.getSocketAddress() + " ("
                        + packet.getLength() + " bytes): " + text);

                // We reply TO THE SENDER, which comes in the packet itself.
                byte[] reply = ("ECHO: " + text).getBytes(StandardCharsets.UTF_8);
                DatagramPacket back = new DatagramPacket(
                        reply, reply.length,
                        packet.getAddress(), packet.getPort());
                socket.send(back);

            } catch (SocketException e) {
                if (!running) {
                    LOG.info("UDP echo server stopped");
                    return;
                }
                LOG.log(Level.SEVERE, "UDP socket failure", e);
                return;

            } catch (IOException e) {
                // A failure with ONE packet cannot bring the server down.
                LOG.log(Level.WARNING, "Error processing a datagram", e);
            }
        }
    }

    @Override
    public void close() {
        running = false;
        if (socket != null) {
            // Closing the socket unblocks the receive(), just as closing
            // the ServerSocket unblocked the accept() in 09-03.
            socket.close();
        }
    }

    public static void main(String[] args) throws SocketException {
        UdpEchoServer server = new UdpEchoServer(9095);
        Runtime.getRuntime().addShutdownHook(new Thread(server::close));
        server.run();
    }
}

The echo client

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.charset.StandardCharsets;

/** UDP echo client with a time limit. */
public class UdpEchoClient {

    private static final int MAX_DATAGRAM = 1400;

    public static void main(String[] args) throws IOException {
        String host = args.length > 0 ? args[0] : "localhost";
        int port = args.length > 1 ? Integer.parseInt(args[1]) : 9095;

        // Ephemeral socket: the system gives us a free port.
        // try-with-resources: DatagramSocket is Closeable.
        try (DatagramSocket socket = new DatagramSocket()) {

            // MANDATORY. Without this, if the reply is lost
            // -and with UDP it can be- the receive() never returns.
            socket.setSoTimeout(2000);

            InetAddress destination = InetAddress.getByName(host);

            for (String message : new String[]{"Hello", "BiblioTech", "Naïve Set Theory"}) {
                byte[] data = message.getBytes(StandardCharsets.UTF_8);
                socket.send(new DatagramPacket(data, data.length, destination, port));
                System.out.println("-> " + message + " (" + data.length + " bytes)");

                byte[] buffer = new byte[MAX_DATAGRAM];
                DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
                try {
                    socket.receive(reply);
                    System.out.println("<- " + new String(reply.getData(),
                            reply.getOffset(), reply.getLength(),
                            StandardCharsets.UTF_8));
                } catch (SocketTimeoutException e) {
                    // With UDP this is NOT a program error: it is the
                    // normal behaviour when something gets lost.
                    System.out.println("<- (no reply in 2 s: packet lost"
                            + " or server down)");
                }
            }
        }
    }
}

Test in two terminals:

# Terminal 1
java -cp classes com.nexussoftware.bibliotech.network.UdpEchoServer

# Terminal 2
java -cp classes com.nexussoftware.bibliotech.network.UdpEchoClient
Terminal 2:
-> Hello (5 bytes)
<- ECHO: Hello
-> BiblioTech (10 bytes)
<- ECHO: BiblioTech
-> Naïve Set Theory (17 bytes)
<- ECHO: Naïve Set Theory

Notice Naïve Set Theory: it is 16 characters but 17 bytes, because the ï takes two bytes in UTF-8. With UDP, size is measured in bytes, not in characters, and that difference matters when you work out whether something fits in a datagram.

You can also test it with nc in UDP mode:

echo "Hello from nc" | nc -u -w1 localhost 9095
ECHO: Hello from nc

And something that would not happen with TCP: switch the server off and run the client again. You get no ConnectException and no error at all; you get three two-second timeouts. send() succeeded all three times. That is the nature of UDP in one line.

  1. Classic mistake 1: reusing the receive packet without restoring the length

This is the most quoted bug of Java's UDP API, and its symptom is baffling: messages start arriving truncated from the second one onwards.

// BROKEN CODE. Do not copy it.
byte[] buffer = new byte[1400];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

while (true) {
    socket.receive(packet);         // <-- the setLength before it is missing
    process(packet);
}

What happens

receive() modifies the packet's length to state how many bytes arrived. And at the same time, that very length is what receive() uses as the maximum limit of what it will accept on the next call.

Initial state:  packet.getLength() = 1400   (the whole buffer)

1st receive:  "Hello" arrives (5 bytes)
              -> receive writes 5 bytes and sets getLength() = 5

2nd receive:  the limit is now 5 bytes.
              "BiblioTech" arrives (10 bytes)
              -> ONLY 5 are accepted: "Bibli"
              -> the remaining 5 ARE DISCARDED WITH NO WARNING
              -> getLength() = 5

3rd receive:  the limit is still 5. And so on forever.

Real server output with the bug:

From /127.0.0.1:51234 (5 bytes): Hello
From /127.0.0.1:51234 (5 bytes): Bibli
From /127.0.0.1:51234 (5 bytes): Refac

The first message arrives perfectly. Every one after it is trimmed to the length of the first. And there is no exception, no warning, nothing in the log: the surplus bytes are simply thrown away.

The solution

// CORRECT: restore the length BEFORE every receive.
while (true) {
    packet.setLength(buffer.length);        // <-- the missing line
    socket.receive(packet);
    process(packet);
}

A single line. And if you would rather not have to remember:

// Alternative: a new packet on every turn. Correct, but it generates
// garbage. In a server receiving thousands of packets per second,
// reuse with setLength is markedly better.
while (true) {
    DatagramPacket packet = new DatagramPacket(new byte[1400], 1400);
    socket.receive(packet);
    process(packet);
}

Rule: if you reuse the packet —and you should—, setLength(buffer.length) before every receive(). No exceptions.

  1. Classic mistake 2: using buffer.length instead of packet.getLength()

The second mistake has the opposite effect: instead of losing data, you drag rubbish along.

// BROKEN CODE. Do not copy it.
socket.receive(packet);
String text = new String(packet.getData(), StandardCharsets.UTF_8);
//                       ^^^^^^^^^^^^^^^^ the WHOLE array, 1400 bytes

getData() returns the complete array, of 1400 bytes. If only 5 arrived, the other 1395 are whatever was in memory before: zeros the first time, and remains of the previous message from then on.

With the bug, receiving "Hello" and then "Hi":

  1st message: "Hello" + 1395 bytes of zeros
               -> "Hello\0\0\0\0\0\0..."  (seems to work, deceptive)

  2nd message: "Hi" + "llo" (remains of the previous one) + zeros
               -> "Hillo\0\0\0..."   <-- MIXED-UP DATA!

That "Hillo" is corrupt data that no log will expose. In a real case —receiving the number of active loans, or an identifier— it produces wrong values that look legitimate.

The solution

// CORRECT: always offset and getLength().
String text = new String(
        packet.getData(),
        packet.getOffset(),         // from where
        packet.getLength(),         // how many bytes are valid
        StandardCharsets.UTF_8);    // explicit charset, as always

And to copy the bytes into an array of exactly the right size:

byte[] useful = Arrays.copyOfRange(
        packet.getData(),
        packet.getOffset(),
        packet.getOffset() + packet.getLength());
Method Returns When to use it
packet.getData().length The buffer size (1400) Almost never. It is the source of the bug
packet.getLength() The bytes received (5) Always
packet.getOffset() Where they start Always, together with the previous one

The two mistakes of these two sections are, by a distance, the most frequent ones with DatagramPacket. If you avoid them, 80 % of UDP problems in Java disappear.

  1. Datagram size, MTU and fragmentation

How big can a datagram be? The answer has three levels.

The theoretical limit

The length field of the UDP header is 16 bits, so the absolute maximum is 65,535 bytes, minus 8 of UDP header and 20 of IP header: 65,507 bytes of data with IPv4.

The practical limit: the MTU

The MTU (maximum transmission unit) is the largest frame size a link accepts. On Ethernet it is 1500 bytes, including the IP and UDP headers.

  1500 bytes of Ethernet MTU
-   20 bytes of IPv4 header   (40 if it is IPv6)
-    8 bytes of UDP header
= 1472 bytes of data that fit in ONE Ethernet frame

If you send more, IP fragments the datagram into several frames and the receiver reassembles them. And here is the problem:

If a single fragment is lost, the whole datagram is lost. There is no fragment retransmission: the receiver discards what it had and your application receives nothing.

With 1 % loss per packet, a datagram of 10 fragments has nearly a 10 % chance of being lost. Fragmentation multiplies the loss rate.

Besides, many firewalls drop IP fragments as a matter of security policy, so your large datagrams simply never arrive — and the diagnosis is hellish because the small ones work.

The practical recommendation

Size Assessment
512 bytes Very safe. It is classic DNS's limit, chosen to work on any network
1400 bytes Prudent. It fits in Ethernet even with VPN tunnels, which take a few bytes
1472 bytes The exact Ethernet maximum without fragmenting. No margin
More than 1500 It fragments. Only on controlled networks
More than 8192 On top of that, some systems reject it outright

BiblioTech will use 1400 bytes as the buffer size and will keep its messages well below that. If a piece of data does not fit in one datagram, the right answer is almost never "make the datagram bigger": it is to split the data yourself with sequence numbers, or to use TCP, which already knows how.

A detail that bites

If a datagram arrives and does not fit in your buffer, Java truncates it with no warning. There is no exception. You receive the first N bytes and the rest is lost.

// Useful trick for detecting truncation: ask for one byte more.
// If getLength() == buffer.length, it is VERY likely that a bigger
// packet arrived and was truncated.
byte[] buffer = new byte[MAX_DATAGRAM + 1];
DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
socket.receive(packet);

if (packet.getLength() > MAX_DATAGRAM) {
    LOG.warning("Datagram of " + packet.getLength()
            + " bytes: it exceeds the expected maximum. Discarded.");
    return;
}

  1. Loss, duplication and disorder: the demonstration

On localhost UDP looks perfect: nothing gets lost. That is deceptive, because loopback does not cross a real network. Let us check that all three phenomena have to be tolerated.

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.charset.StandardCharsets;
import java.util.HashSet;
import java.util.Set;

/**
 * Shows that a UDP receiver must tolerate loss, duplication and disorder.
 * The sender sends N numbered packets; the receiver analyses what arrived.
 */
public class UdpReliabilityDemo {

    private static final int PORT = 9096;
    private static final int TOTAL = 20;

    /** Receiver: counts what arrives and detects gaps, duplicates and disorder. */
    static void receiver() throws IOException {
        try (DatagramSocket socket = new DatagramSocket(PORT)) {
            socket.setSoTimeout(3000);      // without this, if the last one is lost,
                                            // the loop would never finish

            byte[] buffer = new byte[512];
            DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

            Set<Integer> seen = new HashSet<>();
            int duplicates = 0;
            int outOfOrder = 0;
            int lastSeen = -1;

            while (true) {
                packet.setLength(buffer.length);        // classic mistake 1
                try {
                    socket.receive(packet);
                } catch (SocketTimeoutException e) {
                    break;      // 3 s with nothing: we call it finished
                }

                String text = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);   // classic mistake 2
                int number = Integer.parseInt(text.substring(text.indexOf('#') + 1));

                if (!seen.add(number)) {
                    // add() returns false if it was already there: a DUPLICATE.
                    duplicates++;
                    System.out.println("  DUPLICATE: #" + number);
                    continue;
                }
                if (number < lastSeen) {
                    // It arrived after a higher one: DISORDER.
                    outOfOrder++;
                    System.out.println("  OUT OF ORDER: #" + number
                            + " after #" + lastSeen);
                }
                lastSeen = Math.max(lastSeen, number);
            }

            // The missing ones are LOST.
            StringBuilder lost = new StringBuilder();
            for (int i = 0; i < TOTAL; i++) {
                if (!seen.contains(i)) {
                    lost.append('#').append(i).append(' ');
                }
            }

            System.out.println();
            System.out.println("=== ANALYSIS OF " + TOTAL + " PACKETS ===");
            System.out.println("Unique received  : " + seen.size());
            System.out.println("Lost             : " + (TOTAL - seen.size())
                    + (lost.length() == 0 ? "" : "  -> " + lost));
            System.out.println("Duplicated       : " + duplicates);
            System.out.println("Out of order     : " + outOfOrder);
        }
    }

    /** Sender: sends TOTAL numbered packets as fast as possible. */
    static void sender() throws IOException, InterruptedException {
        try (DatagramSocket socket = new DatagramSocket()) {
            InetAddress destination = InetAddress.getByName("localhost");

            for (int i = 0; i < TOTAL; i++) {
                byte[] data = ("PACKET#" + i).getBytes(StandardCharsets.UTF_8);
                socket.send(new DatagramPacket(data, data.length, destination, PORT));
                // send() has "succeeded". That does NOT mean it arrived.
            }
            System.out.println("Sent " + TOTAL + " packets");
        }
    }

    public static void main(String[] args) throws Exception {
        if (args.length > 0 && args[0].equals("sender")) {
            sender();
        } else {
            receiver();
        }
    }
}

On localhost you will normally see zero loss. To see the real behaviour there are two routes:

Route 1: swamp the receive buffer. Raise TOTAL to 100,000 and remove any pause. The sender will generate packets faster than the receiver consumes them, the system buffer will fill up and the system will discard the ones that do not fit, without telling anybody:

=== ANALYSIS OF 100000 PACKETS ===
Unique received  : 73412
Lost             : 26588
Duplicated       : 0
Out of order     : 0

A 26 % loss, on localhost, with no network in between. That is the absence of flow control: TCP would have throttled the sender automatically; UDP throttles nobody and the surplus datagrams are thrown away.

Route 2: simulate a bad network. On Linux, with tc (needs administrator rights):

# Add 10% loss, 5% duplication and variable delay to loopback
sudo tc qdisc add dev lo root netem loss 10% duplicate 5% delay 20ms 10ms

# ... run the test ...

# ALWAYS remove it when you finish
sudo tc qdisc del dev lo root
  DUPLICATE: #3
  OUT OF ORDER: #7 after #8
  DUPLICATE: #11

=== ANALYSIS OF 20 PACKETS ===
Unique received  : 18
Lost             : 2  -> #5 #14
Duplicated       : 2
Out of order     : 1

There are the three phenomena. Any serious UDP receiver has to tolerate them, and the pattern is always the same: number the messages, discard repeats with a set or a window, and consciously decide what to do with the out-of-order ones (for telemetry, ignore the old ones; for a request-reply protocol, discard those that do not match the current request).

  1. Reliability by hand: sequence, time limit and retry

When you need some reliability but not all of TCP's, it is built by hand. The skeleton is always this:

sequenceDiagram
    participant C as Client
    participant S as Server
    C->>S: REQUEST id=42
    Note over C: setSoTimeout(500) and wait
    Note over S: (packet lost)
    Note over C: timed out -> retry 1
    C->>S: REQUEST id=42 (same id)
    S-->>C: REPLY id=42
    Note over C: the id matches: accepted

The four pieces:

  1. A unique identifier per request. It lets you match replies and discard those that do not correspond — because the reply of an earlier attempt already given up for lost may arrive.
  2. setSoTimeout + retry. With increasing backoff, so as not to make a congested network worse: 200 ms, 400, 800...
  3. A retry limit. Retrying forever is an infinite loop in disguise.
  4. Idempotence. If the server can receive the same request twice, the operation must be repeatable without harm. QUERY is idempotent; LEND is not, and that is a solid argument for leaving loans on TCP.
package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.ThreadLocalRandom;
import java.util.logging.Logger;

/**
 * Reliable request-reply over UDP: identifier, time limit,
 * retry with increasing backoff and discarding of unmatched replies.
 *
 * WARNING: if you need this for all your traffic, use TCP. This makes
 * sense for short, isolated exchanges (as DNS does), not as a general
 * replacement for TCP.
 */
public class ReliableUdpRequest {

    private static final Logger LOG = Logger.getLogger(ReliableUdpRequest.class.getName());
    private static final int MAX_DATAGRAM = 1400;

    private final InetAddress destination;
    private final int port;
    private final int maxAttempts;
    private final int initialWaitMs;

    public ReliableUdpRequest(InetAddress destination, int port,
                              int maxAttempts, int initialWaitMs) {
        this.destination = destination;
        this.port = port;
        this.maxAttempts = maxAttempts;
        this.initialWaitMs = initialWaitMs;
    }

    /**
     * Sends a request and waits for a reply, retrying.
     * Returns null if there was no reply after all the attempts.
     *
     * Format: "<id> <request>"  ->  "<id> <reply>"
     */
    public String ask(String request) throws IOException {
        // Random identifier: to match replies AND to make it hard for
        // a third party to guess the id and inject a fake reply.
        int id = ThreadLocalRandom.current().nextInt(1, Integer.MAX_VALUE);
        byte[] data = (id + " " + request).getBytes(StandardCharsets.UTF_8);

        try (DatagramSocket socket = new DatagramSocket()) {
            // connect() in UDP does not connect: it filters. We will only
            // accept packets from this destination, which rules out injections.
            socket.connect(destination, port);

            int wait = initialWaitMs;
            byte[] buffer = new byte[MAX_DATAGRAM];
            DatagramPacket reply = new DatagramPacket(buffer, buffer.length);

            for (int attempt = 1; attempt <= maxAttempts; attempt++) {
                socket.send(new DatagramPacket(data, data.length, destination, port));
                socket.setSoTimeout(wait);

                // Inner loop: replies from earlier attempts we had given
                // up for lost may arrive. They have to be discarded and
                // we must keep waiting for ours within the same deadline.
                long limit = System.currentTimeMillis() + wait;
                while (System.currentTimeMillis() < limit) {
                    try {
                        reply.setLength(buffer.length);         // classic mistake 1
                        socket.receive(reply);

                        String text = new String(reply.getData(),
                                reply.getOffset(), reply.getLength(),
                                StandardCharsets.UTF_8);        // classic mistake 2

                        int sep = text.indexOf(' ');
                        if (sep < 0) {
                            LOG.fine("Reply with an invalid format; discarded");
                            continue;
                        }
                        int receivedId = Integer.parseInt(text.substring(0, sep));
                        if (receivedId != id) {
                            // Reply to another request: discarded.
                            LOG.fine("Reply with id " + receivedId
                                    + ", we expected " + id + "; discarded");
                            continue;
                        }
                        return text.substring(sep + 1);         // ours!

                    } catch (SocketTimeoutException e) {
                        break;      // this attempt's deadline expired
                    } catch (NumberFormatException e) {
                        LOG.fine("Non-numeric identifier; discarded");
                    }
                }

                LOG.info("Attempt " + attempt + "/" + maxAttempts
                        + " with no reply (wait " + wait + " ms)");

                // INCREASING BACKOFF: double it on every attempt. Retrying
                // at the same rate over a congested network makes it worse.
                wait *= 2;
            }
            return null;    // no reply after all the attempts
        }
    }
}

Three decisions deserve an explanation:

The inner receive loop. When the first attempt runs out of time and you launch the second, the reply to the first may arrive late. If you simply accepted the first packet to arrive, you would swallow it as though it were the second attempt's. Using an identifier and discarding the ones that do not match solves this; but you have to keep waiting within the same deadline after discarding one, hence the inner loop with a time limit.

The increasing backoff. It is the difference between a polite client and one that helps bring a network down. If a thousand clients retry every 200 ms against a saturated server, they guarantee it stays saturated. Doubling the deadline spreads the load.

The random identifier instead of a counter. A counter is predictable: a third party who knows you are on 43 can send you a fake reply with id 44 before the real one arrives. It is exactly the DNS cache-poisoning attack. A random 31-bit value makes it impractical.

  1. Broadcast

Broadcast lets you send one datagram that every machine on the local network receives. It is the capability TCP does not have and the one BiblioTech needs.

Broadcast addresses

Address Reach
255.255.255.255 Limited broadcast: the whole local network. It never crosses a router
192.168.1.255 Broadcast directed to the 192.168.1.0/24 subnet
10.0.255.255 Broadcast directed to 10.0.0.0/16

The directed address is worked out by setting all the host bits to 1. On Nexus Software's network, 192.168.1.0/24, the broadcast is 192.168.1.255.

Sending

try (DatagramSocket socket = new DatagramSocket()) {
    // MANDATORY. Without this, sending to a broadcast address
    // throws SocketException: Permission denied.
    socket.setBroadcast(true);

    byte[] data = "WHERE IS BIBLIOTECH?".getBytes(StandardCharsets.UTF_8);
    socket.send(new DatagramPacket(data, data.length,
            InetAddress.getByName("255.255.255.255"), 9091));
}

Receiving

There is nothing special: being bound to the destination port is enough.

try (DatagramSocket socket = new DatagramSocket(9091)) {
    byte[] buffer = new byte[1400];
    DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
    socket.receive(packet);         // it also receives broadcasts to 9091
}

Limits of broadcast

Limit Consequence
It does not cross routers It only works within the same local network. It is a design limit, not a fault
It bothers everybody Every machine on the network processes the packet even if it has no interest in it
Many Wi-Fi networks filter it Access points often limit or block it
It does not exist in IPv6 IPv6 removed it: its replacement is multicast

That last point is important: broadcast is legacy technology. It works perfectly on an IPv4 office network, and that is why BiblioTech will use it; but for something new meant to last, multicast is the right answer.

  1. Multicast with MulticastSocket

Multicast is broadcast done properly: instead of bothering the whole network, a group is defined and the interested machines subscribe to it voluntarily.

Group addresses

IPv4 range Use
224.0.0.0 – 224.0.0.255 Reserved for network protocols. Does not cross routers
224.0.1.0 – 238.255.255.255 Global groups, assigned by the IANA
239.0.0.0 – 239.255.255.255 Administrative scope: for private use. The one you should use

BiblioTech will use 239.10.10.10, which is in the private range.

The TTL: how far it goes

socket.setTimeToLive(1);        // it does not leave the local network
TTL Reach
0 The machine itself only
1 The local network only. The default and the prudent value
32 The site
255 Unrestricted (routers usually block it)

The modern API

MulticastSocket had joinGroup(InetAddress) and leaveGroup(InetAddress) methods that are deprecated since Java 14, because they do not let you state which network interface to join through — and on a machine with Wi-Fi, Ethernet and Docker virtual adapters, the interface the system chooses is almost never the one you want.

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.MulticastSocket;
import java.net.NetworkInterface;
import java.nio.charset.StandardCharsets;
import java.util.Enumeration;
import java.util.logging.Logger;

/**
 * Multicast sender and receiver for BiblioTech's internal notices
 * ("the catalogue has been updated", "maintenance in 10 minutes").
 */
public class MulticastNotices implements AutoCloseable {

    private static final Logger LOG = Logger.getLogger(MulticastNotices.class.getName());

    /** Range 239.x.x.x: administrative scope, reserved for private use. */
    private static final String GROUP = "239.10.10.10";
    private static final int PORT = 9097;
    private static final int MAX_DATAGRAM = 1400;

    private MulticastSocket socket;
    private InetSocketAddress group;
    private NetworkInterface networkInterface;
    private volatile boolean running = false;

    /** Joins the group and gets ready to receive. */
    public void join() throws IOException {
        socket = new MulticastSocket(PORT);
        socket.setTimeToLive(1);            // it does not leave the local network

        group = new InetSocketAddress(InetAddress.getByName(GROUP), PORT);
        networkInterface = chooseInterface();

        // MODERN API (Java 14+): joinGroup(SocketAddress, NetworkInterface).
        // The old joinGroup(InetAddress) is deprecated because it does not let
        // you state the interface, and on a machine with Wi-Fi + Ethernet + Docker
        // the system almost never picks the one you want.
        socket.joinGroup(group, networkInterface);
        running = true;

        LOG.info(() -> "Joined group " + GROUP + ":" + PORT
                + " through interface " + networkInterface.getName());
    }

    /**
     * Chooses an interface that is up, not loopback and supports multicast.
     * In production this ought to be configurable in bibliotech.properties.
     */
    private NetworkInterface chooseInterface() throws IOException {
        Enumeration<NetworkInterface> interfaces = NetworkInterface.getNetworkInterfaces();
        while (interfaces.hasMoreElements()) {
            NetworkInterface ni = interfaces.nextElement();
            if (ni.isUp() && !ni.isLoopback() && ni.supportsMulticast()) {
                return ni;
            }
        }
        // Fallback: on a machine with no network, loopback at least lets
        // a sender and a receiver on the same machine talk to each other.
        return NetworkInterface.getByName("lo");
    }

    /** Sends a notice to everybody subscribed to the group. */
    public void announce(String message) throws IOException {
        byte[] data = message.getBytes(StandardCharsets.UTF_8);
        if (data.length > MAX_DATAGRAM) {
            throw new IOException("Notice too long: " + data.length + " bytes");
        }
        socket.send(new DatagramPacket(data, data.length,
                InetAddress.getByName(GROUP), PORT));
        LOG.info(() -> "Notice sent to the group: " + message);
    }

    /** Listening loop. Call it from a thread of its own. */
    public void listen(java.util.function.Consumer<String> onReceive) {
        byte[] buffer = new byte[MAX_DATAGRAM];
        DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

        while (running) {
            try {
                packet.setLength(buffer.length);        // classic mistake 1
                socket.receive(packet);

                String message = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);    // classic mistake 2

                // CAREFUL: we also receive OUR OWN notices. If that bothers you,
                // it is disabled with socket.setOption(StandardSocketOptions.IP_MULTICAST_LOOP, false)
                // or filtered by sender, comparing against the local addresses.
                onReceive.accept(message);

            } catch (IOException e) {
                if (!running) {
                    return;     // orderly close
                }
                LOG.warning("Error receiving a notice: " + e.getMessage());
            }
        }
    }

    @Override
    public void close() {
        running = false;
        if (socket != null) {
            try {
                socket.leaveGroup(group, networkInterface);     // modern API here too
            } catch (IOException e) {
                LOG.fine("Failure leaving the group: " + e.getMessage());
            }
            socket.close();     // unblocks the receive()
        }
    }
}

Broadcast versus multicast

Broadcast Multicast
Who receives Every machine on the network Only those subscribed to the group
Crosses routers No Yes, if they are configured
IPv6 Does not exist Yes
Configuration None Choosing a group and a TTL
Cost to the network High: it bothers everybody Low
When to use it Simple discovery on an IPv4 LAN Everything else

  1. BiblioTech: the discovery service

Now we solve the problem the lesson opened with: every workstation having the server's IP written by hand.

The discovery protocol

BTDP/1 - BiblioTech Discovery Protocol
======================================
Transport : UDP, broadcast to port 9091
Encoding : UTF-8
Maximum size : 512 bytes (well below any MTU)

Request (workstation -> broadcast):
  BTDP/1 WHERE <requestId>

Reply (server -> sender, unicast):
  BTDP/1 HERE <requestId> <ip> <tcpPort> <serverName>

Example:
  -> BTDP/1 WHERE 748291
  <- BTDP/1 HERE 748291 192.168.1.50 9090 bibliotech-central

Notice two decisions. The request goes by broadcast because we do not know who to ask; the reply goes by unicast to the sender, because we do know who to answer and there is no need to bother the whole network. And the identifier lets us discard replies that are not to our question.

sequenceDiagram
    participant P as Marta's desk
    participant N as Local network (broadcast)
    participant S as BiblioTech server
    participant O as Other machines
    P->>N: BTDP/1 WHERE 748291 (255.255.255.255:9091)
    N->>S: (arrives)
    N->>O: (arrives, they ignore it)
    S-->>P: BTDP/1 HERE 748291 192.168.1.50 9090 (unicast)
    Note over P: Now I know where to connect by TCP
    P->>S: TCP connection to 9090 (BTCP/1 of 09-03)

The responder, on the server

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.atomic.AtomicLong;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * Replies to BiblioTech's discovery requests.
 * It runs alongside the CatalogServer of 09-03, in its own thread.
 *
 * One socket, one thread, every desk in the office:
 * that is what a UDP server is like, with no pool and no per-client state.
 */
public class DiscoveryResponder implements AutoCloseable, Runnable {

    private static final Logger LOG =
            Logger.getLogger(DiscoveryResponder.class.getName());

    private static final int DISCOVERY_PORT = 9091;
    /** 512 bytes: the universal prudent limit, the same one DNS uses. */
    private static final int MAX_DATAGRAM = 512;
    private static final String VERSION = "BTDP/1";

    private final int tcpPort;
    private final String serverName;

    private DatagramSocket socket;
    private volatile boolean running = false;
    private final AtomicLong answered = new AtomicLong();
    private final AtomicLong discarded = new AtomicLong();

    public DiscoveryResponder(int tcpPort, String serverName) {
        this.tcpPort = tcpPort;
        this.serverName = serverName;
    }

    public void start() throws SocketException {
        socket = new DatagramSocket(DISCOVERY_PORT);
        running = true;
        LOG.info(() -> "Discovery responder on UDP:" + DISCOVERY_PORT);
    }

    @Override
    public void run() {
        // A buffer one byte bigger than the maximum, to detect truncation.
        byte[] buffer = new byte[MAX_DATAGRAM + 1];
        DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

        while (running) {
            try {
                packet.setLength(buffer.length);        // CLASSIC MISTAKE 1
                socket.receive(packet);

                if (packet.getLength() > MAX_DATAGRAM) {
                    // A datagram bigger than expected: suspicious. Discarded.
                    discarded.incrementAndGet();
                    LOG.warning("Datagram of " + packet.getLength()
                            + " bytes discarded because of its size");
                    continue;
                }

                String request = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);    // CLASSIC MISTAKE 2

                process(request.strip(), packet.getAddress(), packet.getPort());

            } catch (SocketException e) {
                if (!running) {
                    LOG.info("Discovery responder stopped");
                    return;
                }
                LOG.log(Level.SEVERE, "Discovery socket failure", e);
                return;

            } catch (IOException e) {
                // A bad packet cannot bring the service down.
                LOG.log(Level.WARNING, "Error processing a discovery request", e);
            }
        }
    }

    private void process(String request, InetAddress sender, int senderPort) {
        // EVERYTHING arriving over the network is untrusted, and here it
        // arrives from ANYBODY on the local network, with no connection to identify it.
        String[] parts = request.split(" ");

        if (parts.length != 3 || !parts[0].equals(VERSION)
                || !parts[1].equals("WHERE")) {
            discarded.incrementAndGet();
            // We do not reply to what we do not understand: replying to arbitrary
            // packets would turn this service into an AMPLIFIER for reflected
            // denial-of-service attacks.
            LOG.fine(() -> "Unrecognised discovery request from " + sender);
            return;
        }

        String id = parts[2];
        if (!validIdentifier(id)) {
            discarded.incrementAndGet();
            return;
        }

        try {
            String myIp = InetAddress.getLocalHost().getHostAddress();
            String reply = VERSION + " HERE " + id + " " + myIp + " "
                    + tcpPort + " " + serverName;

            byte[] data = reply.getBytes(StandardCharsets.UTF_8);

            // UNICAST to the sender: there is no need to bother the whole
            // network with the reply. The question broadcasts; the answer does not.
            socket.send(new DatagramPacket(data, data.length,
                    sender, senderPort));

            answered.incrementAndGet();
            LOG.info(() -> "Discovery answered to " + sender.getHostAddress()
                    + " (id " + id + ")");

        } catch (IOException e) {
            // If the send fails, nothing serious happens: the client will retry.
            LOG.log(Level.WARNING, "Could not answer the discovery request", e);
        }
    }

    /** Allow list: digits only, 10 at most. No arbitrary data. */
    private boolean validIdentifier(String id) {
        if (id.isEmpty() || id.length() > 10) {
            return false;
        }
        for (int i = 0; i < id.length(); i++) {
            if (!Character.isDigit(id.charAt(i))) {
                return false;
            }
        }
        return true;
    }

    @Override
    public void close() {
        running = false;
        if (socket != null) {
            socket.close();     // unblocks the receive()
        }
        LOG.info(() -> "Discovery: " + answered.get() + " answered, "
                + discarded.get() + " discarded");
    }
}

The finder, on the workstation

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.ThreadLocalRandom;
import java.util.logging.Logger;

/**
 * Looks for the BiblioTech server on the local network by broadcast.
 * Removes the need to configure the server's IP on every desk.
 */
public class ServerFinder {

    private static final Logger LOG = Logger.getLogger(ServerFinder.class.getName());

    private static final int DISCOVERY_PORT = 9091;
    private static final int MAX_DATAGRAM = 512;
    private static final String VERSION = "BTDP/1";
    private static final int MAX_ATTEMPTS = 3;

    /** What is discovered: where the server is. A record from 04-07. */
    public record FoundServer(String ip, int tcpPort, String name) {
    }

    /**
     * Looks for the server. Returns null if nobody replies after the retries.
     * @param initialWaitMs deadline of the first attempt; it doubles on each one.
     */
    public FoundServer find(int initialWaitMs) throws IOException {
        int id = ThreadLocalRandom.current().nextInt(1, 1_000_000);
        String request = VERSION + " WHERE " + id;
        byte[] data = request.getBytes(StandardCharsets.UTF_8);

        try (DatagramSocket socket = new DatagramSocket()) {
            // Without this, sending to 255.255.255.255 throws
            // SocketException: Permission denied.
            socket.setBroadcast(true);

            // CAREFUL: connect() cannot be used here, because the reply
            // will come from the server's IP and not from the broadcast
            // address we sent to. connect() would filter it and we would discard it.

            byte[] buffer = new byte[MAX_DATAGRAM + 1];
            DatagramPacket reply = new DatagramPacket(buffer, buffer.length);
            InetAddress broadcast = InetAddress.getByName("255.255.255.255");

            int wait = initialWaitMs;

            for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
                socket.send(new DatagramPacket(data, data.length,
                        broadcast, DISCOVERY_PORT));
                LOG.info("Looking for BiblioTech on the local network (attempt "
                        + attempt + "/" + MAX_ATTEMPTS + ")...");

                socket.setSoTimeout(wait);
                long limit = System.currentTimeMillis() + wait;

                while (System.currentTimeMillis() < limit) {
                    try {
                        reply.setLength(buffer.length);         // classic mistake 1
                        socket.receive(reply);

                        String text = new String(reply.getData(),
                                reply.getOffset(), reply.getLength(),
                                StandardCharsets.UTF_8);        // classic mistake 2

                        FoundServer found = parse(text, id);
                        if (found != null) {
                            LOG.info(() -> "BiblioTech found: " + found);
                            return found;
                        }
                        // A reply with another id or malformed: keep waiting.

                    } catch (SocketTimeoutException e) {
                        break;      // deadline expired, next attempt
                    }
                }
                wait *= 2;          // increasing backoff
            }

            LOG.warning("No BiblioTech server has replied on the local network");
            return null;
        }
    }

    /** Parses the reply VALIDATING everything: it comes from a stranger on the network. */
    private FoundServer parse(String text, int expectedId) {
        String[] p = text.strip().split(" ");
        if (p.length != 6 || !p[0].equals(VERSION) || !p[1].equals("HERE")) {
            return null;
        }
        try {
            if (Integer.parseInt(p[2]) != expectedId) {
                return null;    // a reply to another question: discarded
            }
            int port = Integer.parseInt(p[4]);
            if (port < 1 || port > 65535) {
                LOG.warning("Invalid port in the reply: " + port);
                return null;
            }
            // The IP is validated by trying to parse it: if it is not valid,
            // getByName will throw and we will not connect to anything odd.
            InetAddress.getByName(p[3]);

            return new FoundServer(p[3], port, p[5]);

        } catch (NumberFormatException | IOException e) {
            LOG.fine("Invalid discovery reply: " + text);
            return null;
        }
    }

    public static void main(String[] args) throws IOException {
        FoundServer server = new ServerFinder().find(500);
        if (server == null) {
            System.out.println("No server was found.");
            System.out.println("Set the IP by hand in bibliotech.properties.");
        } else {
            System.out.println("Server: " + server.name());
            System.out.println("Connect to " + server.ip() + ":" + server.tcpPort());
            // And from here on, the CatalogClient of 09-02 over TCP.
        }
    }
}

How to test it

# Terminal 1: the server (or the responder on its own)
java -cp classes com.nexussoftware.bibliotech.network.DiscoveryResponder

# Terminal 2: the workstation
java -cp classes com.nexussoftware.bibliotech.network.ServerFinder
Terminal 2:
INFO: Looking for BiblioTech on the local network (attempt 1/3)...
INFO: BiblioTech found: FoundServer[ip=192.168.1.50, tcpPort=9090, name=bibliotech-central]
Server: bibliotech-central
Connect to 192.168.1.50:9090

Terminal 1:
INFO: Discovery answered to 192.168.1.23 (id 748291)

With this, the manual configuration disappears. Marta's desk starts up, asks the network and finds the server. If tomorrow the machine changes IP, nobody has to touch anything. This is the kind of thing TCP cannot do.

A security note. Broadcast discovery is convenient and trusts anybody on the local network. A malicious machine can reply before the real server and make Marta connect to it. Notice too that the responder does not answer what it does not understand: answering arbitrary packets would turn it into an amplifier for reflected denial-of-service attacks, in which the attacker forges the source IP so that the replies land on their victim. In a real environment, discovery serves to find candidates and the server is then verified with TLS and a certificate (12-07). Without verification, automatic discovery is an internal-network convenience, not a security mechanism.

  1. BiblioTech: the telemetry publisher

The second case where UDP is the right choice: sending the server's state every few seconds to a metrics collector.

The requirements make it clear:

  • It must never block. The server is serving Marta and Diego; sending metrics cannot get in the way.
  • Losing the odd measurement does not matter. One is sent every 5 seconds: if one is lost, the next arrives right away.
  • The collector may be down. And the server must carry on working exactly the same, with no errors and no retries.

With TCP, a collector that is down would mean connection attempts that take time, retries, and a thread busy with something that does not matter. With UDP, you drop the envelope in the postbox and move on.

package com.nexussoftware.bibliotech.network;

import com.nexussoftware.bibliotech.service.BiblioTechStatistics;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketException;
import java.nio.charset.StandardCharsets;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
import java.util.concurrent.TimeUnit;
import java.util.concurrent.atomic.AtomicLong;
import java.util.logging.Level;
import java.util.logging.Logger;

/**
 * BiblioTech's telemetry publisher.
 *
 * Sends the state of the service over UDP every N seconds. It uses the
 * ScheduledExecutorService of 08-05 and NEVER blocks the server:
 * if the collector is down, the datagrams are lost and nothing happens.
 *
 * Format (one-line text, StatsD style):
 *   bibliotech.loans.active:12|g
 *   bibliotech.queries.total:8471|c
 */
public class TelemetryPublisher implements AutoCloseable {

    private static final Logger LOG = Logger.getLogger(TelemetryPublisher.class.getName());
    private static final int MAX_DATAGRAM = 512;

    private final String collectorHost;
    private final int collectorPort;
    private final int intervalSeconds;
    private final BiblioTechStatistics statistics;
    private final CatalogServer server;

    private DatagramSocket socket;
    private InetAddress destination;
    private ScheduledExecutorService scheduler;

    private final AtomicLong sent = new AtomicLong();
    private final AtomicLong failed = new AtomicLong();

    public TelemetryPublisher(String collectorHost, int collectorPort,
                              int intervalSeconds,
                              BiblioTechStatistics statistics,
                              CatalogServer server) {
        this.collectorHost = collectorHost;
        this.collectorPort = collectorPort;
        this.intervalSeconds = intervalSeconds;
        this.statistics = statistics;
        this.server = server;
    }

    public void start() throws IOException {
        socket = new DatagramSocket();       // ephemeral: we only send

        // The name is resolved ONCE at startup, not on every send:
        // a DNS lookup every 5 seconds is a waste and could also
        // block the scheduler's thread.
        destination = InetAddress.getByName(collectorHost);

        scheduler = Executors.newSingleThreadScheduledExecutor(r -> {
            Thread t = new Thread(r, "bibliotech-telemetry");
            t.setDaemon(true);      // telemetry must not prevent exiting
            return t;
        });

        // scheduleAtFixedRate from 08-05: every intervalSeconds, without piling up.
        scheduler.scheduleAtFixedRate(this::publish,
                intervalSeconds, intervalSeconds, TimeUnit.SECONDS);

        LOG.info(() -> "Telemetry to " + collectorHost + ":" + collectorPort
                + " every " + intervalSeconds + " s");
    }

    /**
     * Sends a batch of measurements. Runs in the scheduler's thread.
     *
     * CRITICAL: this method must NOT throw anything. An exception escaping
     * a scheduleAtFixedRate task CANCELS the task silently and the
     * telemetry stops publishing without anybody noticing (08-05).
     */
    private void publish() {
        try {
            send("bibliotech.loans.active:"
                    + statistics.activeLoans() + "|g");
            send("bibliotech.queries.total:"
                    + statistics.totalQueries() + "|c");
            send("bibliotech.connections.active:"
                    + server.activeConnections() + "|g");
            send("bibliotech.requests.total:"
                    + server.servedRequests() + "|c");

            Runtime rt = Runtime.getRuntime();
            long memoryMb = (rt.totalMemory() - rt.freeMemory()) / (1024 * 1024);
            send("bibliotech.memory.mb:" + memoryMb + "|g");

        } catch (RuntimeException e) {
            // Logged and we carry on: the next batch will try again.
            LOG.log(Level.WARNING, "Failure publishing telemetry", e);
        }
    }

    private void send(String measurement) {
        byte[] data = measurement.getBytes(StandardCharsets.UTF_8);
        if (data.length > MAX_DATAGRAM) {
            LOG.warning("Measurement too long; discarded: " + measurement);
            return;
        }
        try {
            socket.send(new DatagramPacket(data, data.length,
                    destination, collectorPort));
            sent.incrementAndGet();

            // Remember: this "success" does NOT mean it arrived.
            // With UDP, send() succeeds even if the collector is switched off.

        } catch (IOException e) {
            // It almost never happens. And if it does, it does not matter: it is telemetry.
            failed.incrementAndGet();
            LOG.fine("Telemetry datagram not sent: " + e.getMessage());
        }
    }

    @Override
    public void close() {
        if (scheduler != null) {
            scheduler.shutdown();           // two-phase shutdown (08-05)
            try {
                if (!scheduler.awaitTermination(2, TimeUnit.SECONDS)) {
                    scheduler.shutdownNow();
                }
            } catch (InterruptedException e) {
                scheduler.shutdownNow();
                Thread.currentThread().interrupt();
            }
        }
        if (socket != null) {
            socket.close();
        }
        LOG.info(() -> "Telemetry stopped: " + sent.get() + " datagrams sent, "
                + failed.get() + " failed");
    }
}

A minimal collector to test it

package com.nexussoftware.bibliotech.network;

import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.nio.charset.StandardCharsets;
import java.util.Map;
import java.util.TreeMap;

/** Minimal telemetry collector: shows a dashboard on the console. */
public class TelemetryCollector {

    public static void main(String[] args) throws Exception {
        int port = args.length > 0 ? Integer.parseInt(args[0]) : 9098;

        try (DatagramSocket socket = new DatagramSocket(port)) {
            System.out.println("Collector listening on UDP:" + port);

            byte[] buffer = new byte[512];
            DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
            Map<String, String> dashboard = new TreeMap<>();    // sorted by key
            long received = 0;

            while (true) {
                packet.setLength(buffer.length);
                socket.receive(packet);
                received++;

                String measurement = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);

                int colon = measurement.indexOf(':');
                int bar = measurement.indexOf('|');
                if (colon < 0 || bar < 0 || bar < colon) {
                    continue;       // malformed measurement: ignored
                }
                dashboard.put(measurement.substring(0, colon),
                        measurement.substring(colon + 1, bar));

                // We repaint the whole dashboard with every measurement.
                System.out.print("\033[H\033[2J");      // clear the screen
                System.out.println("=== BIBLIOTECH TELEMETRY ===");
                System.out.println("Datagrams received: " + received);
                System.out.println();
                for (Map.Entry<String, String> e : dashboard.entrySet()) {
                    System.out.printf("  %-38s %s%n", e.getKey(), e.getValue());
                }
            }
        }
    }
}
=== BIBLIOTECH TELEMETRY ===
Datagrams received: 145

  bibliotech.connections.active          3
  bibliotech.loans.active                12
  bibliotech.memory.mb                   34
  bibliotech.queries.total               8471
  bibliotech.requests.total              12093

The key test: stop the collector with Ctrl+C and look at the server. Absolutely nothing happens. Not an error, not a delay, not a stack trace. The server carries on serving the desks exactly as before. Start the collector again and the measurements reappear within five seconds.

That is impossible to achieve with TCP without writing a fair amount of reconnection code, and it is precisely why the telemetry of almost the whole industry —StatsD, the metrics part of Datadog, Graphite's metrics— travels over UDP.

  1. The same case solved with TCP and with UDP

Let us close with the direct comparison. The case: querying a book's availability.

With TCP (what we did in 09-02 and 09-03)

try (CatalogClient client = new CatalogClient("192.168.1.50", 9090)) {
    client.connect();                                       // 3-way handshake + BTCP greeting
    ProtocolCard card = client.query("978-0000000001");     // request + reply
    System.out.println(card.available());
}                                                           // QUIT + 4-message close

With UDP

ReliableUdpRequest request = new ReliableUdpRequest(
        InetAddress.getByName("192.168.1.50"), 9099, 3, 300);
String reply = request.ask("QUERY 978-0000000001");
if (reply == null) {
    System.out.println("No reply after 3 attempts");
} else {
    System.out.println(reply);
}

The comparison

Criterion TCP UDP
Network round trips 3 (setup) + 1 (query) + 2 (close) = 6 1
Total latency on a LAN (0.5 ms/trip) ~3 ms ~0.5 ms
Total latency 100 ms away ~600 ms ~100 ms
Header bytes 20+ per segment, many segments 8, a single datagram
State on the server A socket and a pool thread None
Simultaneous clients it withstands Those of the pool (16) Thousands with one thread
If a packet is lost TCP retransmits by itself You retry
If the reply does not fit in 1400 bytes No problem It has to be split by hand
Reliability From the protocol Yours
Code needed More on the client, far more on the server Less overall

The verdict for BiblioTech

Operation Transport Reason
QUERY of an ISBN TCP It fits in UDP, but the catalogue is already on TCP and the query happens inside a session with several operations
Full LIST TCP The reply easily exceeds 1400 bytes: splitting it by hand would be reinventing TCP
LEND TCP It is not idempotent. A retry could register two loans
Discovery UDP It needs broadcast. TCP cannot
Telemetry UDP It must not block and loss does not matter
Internal notices UDP multicast One to many, with no connections

The general conclusion, which holds beyond BiblioTech: UDP does not replace TCP, it complements TCP. A real system uses both, each where it fits. And the decision criterion is not speed, but three questions: is stale data still worth anything?, do I need to talk to several at once? and is the operation idempotent?

Common Mistakes and Tips

Not restoring setLength() before every receive(). Mistake number one. The first message arrives fine and every one after it is truncated to that same length, with no warning at all.

Using packet.getData() without getOffset() and getLength(). Mistake number two. You drag the previous content of the buffer along and produce corrupt data that looks legitimate.

Believing that send() with no exception means it arrived. It means nothing. send() succeeds even if the destination is switched off. If you need to know whether it arrived, the receiver has to confirm it.

Not setting setSoTimeout before a receive(). With UDP the reply can legitimately be lost. Without a time limit, your thread waits forever for a reply that is never coming.

Sending large datagrams. Above the MTU they fragment, and losing a single fragment loses the whole datagram: fragmentation multiplies the loss rate. Besides, many firewalls drop fragments. Stay at 1400 bytes or, if you want to be very safe, at 512.

Not allowing for truncation. If a datagram bigger than your buffer arrives, Java cuts it silently. Ask for one byte more and check whether getLength() reached the maximum.

Forgetting setBroadcast(true). Sending to a broadcast address without it throws SocketException: Permission denied, and the message is not very informative.

Using connect() on a socket that expects a reply to a broadcast. The reply comes from the server's real IP, not from the broadcast address, and connect() would filter it out. In the discovery finder it cannot be used.

Using joinGroup(InetAddress). It has been deprecated since Java 14. On a machine with several interfaces —and today almost all of them have several, with Docker, VPN or Wi-Fi plus Ethernet— the system picks an interface that is probably not yours. Use joinGroup(SocketAddress, NetworkInterface).

Choosing any old multicast address. Use the 239.x.x.x range, which is reserved for private use. The other ranges are assigned to particular protocols.

Reimplementing TCP over UDP. If you end up writing acknowledgements, windows, retransmissions and congestion control, use TCP: it is better tested than whatever you are going to write.

Retrying without increasing backoff. Retrying at the same rate against a saturated server guarantees it stays saturated. Double the deadline on each attempt and set a limit.

Using a predictable counter as the request identifier. It lets a third party guess it and send a fake reply before the real one. Use a random value.

Replying to packets you do not understand. It turns your service into an amplifier for reflected denial-of-service attacks: the attacker forges the source IP and the replies land on their victim. If you do not recognise the format, stay quiet.

Letting an exception escape a scheduleAtFixedRate task. The task is cancelled silently and the telemetry stops publishing without anybody noticing. Wrap the whole body in a try/catch.

A diagnostic tip. UDP is hard to debug precisely because it does not fail noisily. Three tools: nc -u -l 9095 acts as a UDP receiver and shows you exactly what arrives; ss -ulnp lists the listening UDP sockets (the u instead of the t); and tcpdump -i any -n udp port 9091 -A shows you the datagrams in flight with their content as text. If with tcpdump you see the packet leave and not arrive, the problem is in the network or in a firewall, not in your code.

Exercises

Exercise 1: UDP loss and latency meter

Write a pair UdpProbe (client) and UdpReflector (server) that measure the quality of a UDP link, in the style of a network diagnostic tool.

Requirements:

  • The reflector returns every datagram as it is, untouched, and keeps no state at all.
  • The probe sends N numbered datagrams with a timestamp in nanoseconds, at a configurable rate (packets per second), without waiting for the reply of each one — it sends and receives at different moments.
  • A separate thread receives the replies and works out the latency of each from the timestamp that comes back.
  • Report: sent, received, lost and loss percentage; minimum, mean, median, p95 and maximum latency; and jitter (mean variation between consecutive latencies), which is the metric that decides whether a link is usable for voice or video.
  • Detection of duplicates and of disorder.
  • Test with datagram sizes of 64, 512 and 1400 bytes, and comment on the differences.

Exercise 2: Local network chat over multicast

Write BiblioTechChat, an internal Nexus Software chat over multicast, with no central server.

Requirements:

  • Group 239.10.10.20, port 9099, TTL 1, joinGroup with the modern API and an explicit interface choice.
  • One thread receives and shows the messages; the main one reads from the console and sends.
  • Format: <mark> <user> <message>, with the mark in milliseconds since the application started (no java.time, which is 10-05).
  • It must filter out your own messages so as not to see them duplicated. Investigate the two ways of doing it (through a socket option and by comparing the sender against the local addresses) and implement one, explaining the choice.
  • An announcement on joining and leaving the group (*** Marta Ruiz has joined ***).
  • Strict validation of everything received: length, control characters and sanitising before showing it on the console.
  • Test with three instances on the same machine and, if you can, with two machines.

Exercise 3: Reliable file transfer over UDP

Implement ReliableUdpSender and ReliableUdpReceiver to transfer BiblioTech's catalog.csv over UDP with reliability built by hand, using the stop and wait scheme.

Requirements:

  • Split the file into blocks of 1024 bytes of data.
  • A binary header per datagram with DataOutputStream over a ByteArrayOutputStream: sequence number (int), last-block indicator (boolean) and data length (int).
  • The receiver acknowledges each block with an acknowledgement datagram carrying the sequence number received.
  • The sender waits for the acknowledgement with setSoTimeout and retries up to 5 times with increasing backoff; if they run out, it aborts with an error.
  • The receiver must detect duplicate blocks (a lost acknowledgement makes the sender resend a block that has already been received) and re-acknowledge them without writing them twice.
  • An integrity check at the end with a simple sum computed over all the bytes.
  • Simulate 20 % artificial loss in the sender (discarding datagrams before sending them) and check that the transfer completes all the same.
  • Final report: blocks, retransmissions, duplicates detected, bytes and time.
  • Write a final reflection comparing the result with doing it in three lines over TCP.

Solutions

Solution 1

package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;

/**
 * UDP reflector: returns every datagram as it is, without interpreting it.
 * No state, no threads, no connections. A UDP server in its pure form.
 */
public class UdpReflector {

    public static void main(String[] args) throws IOException {
        int port = args.length > 0 ? Integer.parseInt(args[0]) : 9100;

        try (DatagramSocket socket = new DatagramSocket(port)) {
            // Generous buffer: we accept up to 1500 bytes without truncating.
            byte[] buffer = new byte[1500];
            DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
            long reflected = 0;

            System.out.println("UDP reflector on port " + port);

            while (true) {
                packet.setLength(buffer.length);        // classic mistake 1
                socket.receive(packet);

                // We return EXACTLY the bytes received, not one more:
                // the packet already carries the sender's address and port,
                // so send() returns it to its origin without touching anything.
                socket.send(packet);

                if (++reflected % 1000 == 0) {
                    System.out.println("Reflected: " + reflected);
                }
            }
        }
    }
}
package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.ByteBuffer;
import java.util.Arrays;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.CountDownLatch;
import java.util.concurrent.TimeUnit;

/**
 * UDP probe: measures loss, latency and jitter of a link.
 * It sends and receives in separate threads, without waiting for a reply
 * per packet, which is how a link is really measured.
 */
public class UdpProbe {

    private final String host;
    private final int port;
    private final int total;
    private final int datagramSize;
    private final int perSecond;

    // State shared between the sending thread and the receiving one.
    private final long[] latenciesUs;
    private final Set<Integer> received = new HashSet<>();
    private volatile int duplicates = 0;
    private volatile int outOfOrder = 0;
    private volatile int receivedTotal = 0;

    public UdpProbe(String host, int port, int total,
                    int datagramSize, int perSecond) {
        this.host = host;
        this.port = port;
        this.total = total;
        this.datagramSize = Math.max(datagramSize, 16);     // minimum header
        this.perSecond = perSecond;
        this.latenciesUs = new long[total];
    }

    public void measure() throws IOException, InterruptedException {
        try (DatagramSocket socket = new DatagramSocket()) {
            socket.setSoTimeout(2000);
            InetAddress destination = InetAddress.getByName(host);

            CountDownLatch sendingFinished = new CountDownLatch(1);

            // --- Receiving thread ---
            // It receives in parallel with the sending: that way the measurement
            // does not include the time of "waiting my turn to send the next one".
            Thread receiver = new Thread(() -> receive(socket, sendingFinished),
                    "probe-receiver");
            receiver.start();

            // --- Sender, in this thread ---
            long intervalNs = 1_000_000_000L / perSecond;
            long next = System.nanoTime();

            for (int i = 0; i < total; i++) {
                // Binary header: sequence number + timestamp.
                // ByteBuffer uses big-endian by default: the network order.
                ByteBuffer bb = ByteBuffer.allocate(datagramSize);
                bb.putInt(i);
                bb.putLong(System.nanoTime());
                // The rest of the datagram is padding up to the requested size.

                socket.send(new DatagramPacket(bb.array(), datagramSize,
                        destination, port));

                // Controlled rate: without this we would swamp the receiver's
                // buffer and measure our own saturation, not the network.
                next += intervalNs;
                long waitNs = next - System.nanoTime();
                if (waitNs > 0) {
                    TimeUnit.NANOSECONDS.sleep(waitNs);
                }
            }

            sendingFinished.countDown();
            // Room for the last replies in flight to arrive.
            receiver.join(3000);
            socket.close();     // unblocks the receiver's receive

            report();
        }
    }

    private void receive(DatagramSocket socket, CountDownLatch sendingFinished) {
        byte[] buffer = new byte[1500];
        DatagramPacket packet = new DatagramPacket(buffer, buffer.length);
        int last = -1;

        while (true) {
            try {
                packet.setLength(buffer.length);
                socket.receive(packet);
                long now = System.nanoTime();

                if (packet.getLength() < 12) {
                    continue;       // not even the header fits: ignored
                }

                ByteBuffer bb = ByteBuffer.wrap(packet.getData(),
                        packet.getOffset(), packet.getLength());
                int sequence = bb.getInt();
                long sentAt = bb.getLong();

                if (sequence < 0 || sequence >= total) {
                    continue;       // sequence out of range: somebody else's packet
                }

                synchronized (received) {
                    if (!received.add(sequence)) {
                        duplicates++;
                        continue;
                    }
                }
                if (sequence < last) {
                    outOfOrder++;
                }
                last = Math.max(last, sequence);

                latenciesUs[sequence] = (now - sentAt) / 1000;
                receivedTotal++;

            } catch (SocketTimeoutException e) {
                // Two seconds with nothing. If the sending is over, we stop.
                if (sendingFinished.getCount() == 0) {
                    return;
                }
            } catch (IOException e) {
                return;     // socket closed: the end
            }
        }
    }

    private void report() {
        // Only the latencies of the packets that actually arrived.
        long[] valid = new long[receivedTotal];
        int j = 0;
        for (int i = 0; i < total && j < valid.length; i++) {
            if (latenciesUs[i] > 0) {
                valid[j++] = latenciesUs[i];
            }
        }
        valid = Arrays.copyOf(valid, j);

        if (valid.length == 0) {
            System.out.println("No reply was received.");
            return;
        }

        // Jitter: mean variation between CONSECUTIVE latencies. It is computed
        // BEFORE sorting, because it depends on the time order.
        double jitterSum = 0;
        for (int i = 1; i < valid.length; i++) {
            jitterSum += Math.abs(valid[i] - valid[i - 1]);
        }
        double jitterUs = valid.length > 1 ? jitterSum / (valid.length - 1) : 0;

        long sum = 0;
        for (long v : valid) {
            sum += v;
        }
        long[] sorted = valid.clone();
        Arrays.sort(sorted);

        double loss = 100.0 * (total - receivedTotal) / total;

        System.out.println();
        System.out.println("=== UDP PROBE: " + host + ":" + port + " ===");
        System.out.printf("%-24s %d bytes%n", "Datagram size", datagramSize);
        System.out.printf("%-24s %d/s%n", "Rate", perSecond);
        System.out.println();
        System.out.printf("%-24s %d%n", "Sent", total);
        System.out.printf("%-24s %d%n", "Received", receivedTotal);
        System.out.printf("%-24s %d (%.2f%%)%n", "Lost",
                total - receivedTotal, loss);
        System.out.printf("%-24s %d%n", "Duplicated", duplicates);
        System.out.printf("%-24s %d%n", "Out of order", outOfOrder);
        System.out.println();
        System.out.println("--- LATENCY (ms) ---");
        System.out.printf("%-24s %.3f%n", "Minimum", sorted[0] / 1000.0);
        System.out.printf("%-24s %.3f%n", "Mean",
                (sum / (double) valid.length) / 1000.0);
        System.out.printf("%-24s %.3f%n", "Median",
                sorted[sorted.length / 2] / 1000.0);
        System.out.printf("%-24s %.3f%n", "p95",
                sorted[(int) (sorted.length * 0.95)] / 1000.0);
        System.out.printf("%-24s %.3f%n", "Maximum",
                sorted[sorted.length - 1] / 1000.0);
        System.out.printf("%-24s %.3f%n", "Mean jitter", jitterUs / 1000.0);
    }

    public static void main(String[] args) throws Exception {
        String host = args.length > 0 ? args[0] : "localhost";
        int port = args.length > 1 ? Integer.parseInt(args[1]) : 9100;

        for (int size : new int[]{64, 512, 1400}) {
            new UdpProbe(host, port, 1000, size, 500).measure();
            Thread.sleep(500);
        }
    }
}

Typical output on localhost:

=== UDP PROBE: localhost:9100 ===
Datagram size            64 bytes
Rate                     500/s

Sent                     1000
Received                 1000
Lost                     0 (0.00%)
Duplicated               0
Out of order             0

--- LATENCY (ms) ---
Minimum                  0.041
Mean                     0.078
Median                   0.069
p95                      0.142
Maximum                  1.884
Mean jitter              0.031

Comments. Four things this exercise teaches.

Sending and receiving in different threads is what makes the measurement one of the network and not of your program. If you waited for each packet's reply before sending the next, you would be measuring a stop-and-wait protocol, and at 500 packets per second your own rate would be the dominant factor.

The rate control with next += intervalNs accumulates on the planned instant instead of sleeping a fixed interval. The difference matters: sleeping "20 ms" on each turn accumulates the processing time and the real rate ends up slower than the one requested. Computing the next absolute instant corrects the drift.

The jitter is computed before sorting and it is the metric most often forgotten. A link with a constant 50 ms latency is perfectly usable for a video call; one with a 20 ms mean but 40 of jitter is not, because the receiver cannot predict when the next fragment will arrive and has to add a large buffer, which in turn adds delay.

And about the three sizes: on localhost the differences between 64, 512 and 1400 bytes are minimal because there is no real network. On a real link you would see that 1400 bytes has more latency (more bits have to be serialised onto the wire) and that going up to 1500 sends the loss rate soaring through fragmentation. It is worth testing between two machines if you get the chance.

Solution 2

package com.nexussoftware.bibliotech.network;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.net.DatagramPacket;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.MulticastSocket;
import java.net.NetworkInterface;
import java.net.StandardSocketOptions;
import java.nio.charset.StandardCharsets;
import java.util.Enumeration;
import java.util.logging.Logger;

/**
 * Nexus Software's internal chat over multicast. With no central server:
 * everybody is equal, everybody publishes to the group and receives from it.
 */
public class BiblioTechChat implements AutoCloseable {

    private static final Logger LOG = Logger.getLogger(BiblioTechChat.class.getName());

    /** Range 239.x.x.x: administrative scope, private use. */
    private static final String GROUP = "239.10.10.20";
    private static final int PORT = 9099;
    private static final int MAX_DATAGRAM = 1400;
    private static final int MAX_MESSAGE = 500;

    private final String user;
    private final long startup = System.currentTimeMillis();

    private MulticastSocket socket;
    private InetSocketAddress group;
    private NetworkInterface networkInterface;
    private volatile boolean running = false;

    public BiblioTechChat(String user) {
        this.user = user;
    }

    public void start() throws IOException {
        socket = new MulticastSocket(PORT);
        socket.setTimeToLive(1);        // it does not leave the local network

        // FILTERING OF OUR OWN MESSAGES - the option chosen.
        //
        // There are two ways:
        //   (a) IP_MULTICAST_LOOP = false: the system does not hand our own
        //       sends back to us. It is clean and costs nothing, BUT it also
        //       stops TWO INSTANCES ON THE SAME MACHINE seeing each other,
        //       because the filter is per machine, not per socket.
        //   (b) Comparing the sender against the local addresses and filtering
        //       in the code. More work, but it allows testing with three
        //       instances on the same computer.
        //
        // We choose (b) because the exercise asks to be able to test with three
        // local instances, and besides it is the behaviour you want in a
        // chat: if I open two windows, I want to see both.
        socket.setOption(StandardSocketOptions.IP_MULTICAST_LOOP, true);

        group = new InetSocketAddress(InetAddress.getByName(GROUP), PORT);
        networkInterface = chooseInterface();

        // Modern API (Java 14+): with an explicit NetworkInterface.
        socket.joinGroup(group, networkInterface);
        running = true;

        LOG.info(() -> "Joined " + GROUP + ":" + PORT
                + " through " + networkInterface.getName());
    }

    private NetworkInterface chooseInterface() throws IOException {
        Enumeration<NetworkInterface> nis = NetworkInterface.getNetworkInterfaces();
        NetworkInterface fallback = null;
        while (nis.hasMoreElements()) {
            NetworkInterface ni = nis.nextElement();
            if (!ni.isUp() || !ni.supportsMulticast()) {
                continue;
            }
            if (!ni.isLoopback()) {
                return ni;      // we prefer a real interface
            }
            fallback = ni;      // loopback is useful for local testing
        }
        if (fallback == null) {
            throw new IOException("There is no interface with multicast");
        }
        return fallback;
    }

    /** Receive loop. It runs in its own thread. */
    private void receive() {
        byte[] buffer = new byte[MAX_DATAGRAM];
        DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

        while (running) {
            try {
                packet.setLength(buffer.length);        // classic mistake 1
                socket.receive(packet);

                // Filter (b): we discard what we sent ourselves.
                if (isOurs(packet.getAddress(), packet.getPort())) {
                    continue;
                }

                String message = new String(packet.getData(), packet.getOffset(),
                        packet.getLength(), StandardCharsets.UTF_8);   // classic mistake 2

                // EVERYTHING arriving through the group comes from a stranger.
                if (message.length() > MAX_MESSAGE) {
                    LOG.warning("Message too long, discarded");
                    continue;
                }
                System.out.println(sanitise(message));

            } catch (IOException e) {
                if (!running) {
                    return;     // orderly close
                }
                LOG.warning("Error receiving: " + e.getMessage());
            }
        }
    }

    /**
     * A packet is ours if it comes from a local address AND from our
     * source port. Comparing the address alone would not do: it would also
     * filter out the messages of the other instances on the same machine.
     */
    private boolean isOurs(InetAddress sender, int senderPort) {
        if (senderPort != socket.getLocalPort()) {
            return false;
        }
        try {
            Enumeration<InetAddress> own = networkInterface.getInetAddresses();
            while (own.hasMoreElements()) {
                if (own.nextElement().equals(sender)) {
                    return true;
                }
            }
        } catch (Exception e) {
            // If we cannot check it, we would rather show too much than too little.
            return false;
        }
        return false;
    }

    public void send(String text) throws IOException {
        long mark = (System.currentTimeMillis() - startup) / 1000;
        String message = String.format("[%3ds] %-14s %s", mark, user, text);

        byte[] data = message.getBytes(StandardCharsets.UTF_8);
        if (data.length > MAX_DATAGRAM) {
            System.out.println("(message too long, not sent)");
            return;
        }
        socket.send(new DatagramPacket(data, data.length,
                InetAddress.getByName(GROUP), PORT));
    }

    /**
     * Sanitises before printing to the console: a message from the group may
     * contain ANSI escape sequences that manipulate the terminal
     * of whoever reads it (clearing the screen, moving the cursor, colours).
     */
    private String sanitise(String text) {
        StringBuilder sb = new StringBuilder(text.length());
        for (int i = 0; i < text.length(); i++) {
            char c = text.charAt(i);
            sb.append(Character.isISOControl(c) && c != '\t' ? '?' : c);
        }
        return sb.toString();
    }

    @Override
    public void close() {
        if (!running) {
            return;
        }
        try {
            send("*** has left ***");
        } catch (IOException ignored) {
            // We are leaving anyway.
        }
        running = false;
        try {
            socket.leaveGroup(group, networkInterface);
        } catch (IOException e) {
            LOG.fine("Failure leaving the group: " + e.getMessage());
        }
        socket.close();     // unblocks the receive()
    }

    public static void main(String[] args) throws IOException {
        String user = args.length > 0 ? args[0] : "Anonymous";

        BiblioTechChat chat = new BiblioTechChat(user);
        chat.start();

        Thread receiver = new Thread(chat::receive, "chat-receiver");
        receiver.setDaemon(true);
        receiver.start();

        Runtime.getRuntime().addShutdownHook(new Thread(chat::close));

        chat.send("*** has joined ***");
        System.out.println("BiblioTech chat. Type /quit to finish.");

        try (BufferedReader keyboard = new BufferedReader(
                new InputStreamReader(System.in, StandardCharsets.UTF_8))) {
            String line;
            while ((line = keyboard.readLine()) != null) {
                if (line.equals("/quit")) {
                    break;
                }
                if (!line.isBlank()) {
                    chat.send(line);
                }
            }
        }
        chat.close();
    }
}

Test with three terminals:

java -cp classes com.nexussoftware.bibliotech.network.BiblioTechChat "Marta Ruiz"
java -cp classes com.nexussoftware.bibliotech.network.BiblioTechChat "Diego Alonso"
java -cp classes com.nexussoftware.bibliotech.network.BiblioTechChat "Nuria Vidal"
Marta's terminal:
BiblioTech chat. Type /quit to finish.
[  2s] Diego Alonso   *** has joined ***
[  5s] Nuria Vidal    *** has joined ***
Anyone got Effective Java?
[ 12s] Diego Alonso   I have it, returning it tomorrow
[ 18s] Nuria Vidal    *** has left ***

Comments. The interesting point is the filtering of our own messages, and the reason for choosing option (b) is explained in the code: IP_MULTICAST_LOOP = false is cleaner, but the filter is applied by the system per machine, not per socket, so three instances on the same computer would stop seeing each other and the exercise could not be tested. The manual comparison additionally requires comparing source address and port, not only the address: if you compared only the address, you would filter out the messages of your colleagues on the same machine, which is exactly what we wanted to avoid.

Notice too that the chat has no server. It is a genuine peer-to-peer system, the model presented in 09-01: every node publishes to the group and receives from the group. Nobody is special, nobody keeps anybody's state, and if one instance goes down the others do not even notice. In exchange, there is no history, no guaranteed delivery and no way of knowing who is connected other than the announcements — which can also be lost. That is the price of having no server.

And the sanitise before printing is not academic paranoia: a message containing \033[2J clears the screen of whoever receives it, and more elaborate sequences can do worse things on some terminals.

Solution 3

package com.nexussoftware.bibliotech.network;

import java.io.ByteArrayOutputStream;
import java.io.DataOutputStream;
import java.io.IOException;
import java.io.InputStream;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.InetAddress;
import java.net.SocketTimeoutException;
import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.concurrent.ThreadLocalRandom;
import java.util.logging.Logger;

/**
 * Reliable file transfer over UDP with stop and wait.
 *
 * Format of each data datagram:
 *   [int    ] sequence number
 *   [byte   ] 1 if it is the last block, 0 if not
 *   [int    ] length of the data
 *   [bytes  ] data
 *
 * Format of the acknowledgement:
 *   [int    ] sequence number acknowledged
 */
public class ReliableUdpSender {

    private static final Logger LOG = Logger.getLogger(ReliableUdpSender.class.getName());

    private static final int DATA_PER_BLOCK = 1024;
    private static final int HEADER = 4 + 1 + 4;        // int + byte + int
    private static final int MAX_ATTEMPTS = 5;
    private static final int INITIAL_WAIT_MS = 200;

    /** Percentage of datagrams discarded on purpose, for testing. */
    private final int simulatedLoss;

    private int retransmissions = 0;

    public ReliableUdpSender(int simulatedLoss) {
        this.simulatedLoss = simulatedLoss;
    }

    public void send(String host, int port, Path file) throws IOException {
        long size = Files.size(file);
        long start = System.currentTimeMillis();

        try (DatagramSocket socket = new DatagramSocket();
             InputStream input = Files.newInputStream(file)) {

            InetAddress destination = InetAddress.getByName(host);
            // connect() filters: we only accept acknowledgements from this destination.
            socket.connect(destination, port);

            byte[] block = new byte[DATA_PER_BLOCK];
            byte[] ackBuffer = new byte[16];
            DatagramPacket ack = new DatagramPacket(ackBuffer, ackBuffer.length);

            int sequence = 0;
            long sentBytes = 0;
            long checksum = 0;

            while (true) {
                int read = input.read(block);
                boolean last = false;

                if (read == -1) {
                    // End of file: we send an empty block marked
                    // as the last, so the receiver knows it has finished.
                    read = 0;
                    last = true;
                } else {
                    // We check whether anything is left without consuming it:
                    // available() is only indicative, so we mark the
                    // last one with a final empty block. Simpler and safer.
                    for (int i = 0; i < read; i++) {
                        checksum += block[i] & 0xFF;
                    }
                    sentBytes += read;
                }

                byte[] datagram = build(sequence, last, block, read);

                if (!sendWithAck(socket, destination, port,
                        datagram, sequence, ack, ackBuffer)) {
                    throw new IOException("The receiver does not acknowledge block "
                            + sequence + " after " + MAX_ATTEMPTS + " attempts");
                }

                if (last) {
                    break;
                }
                sequence++;
            }

            long ms = System.currentTimeMillis() - start;
            System.out.println();
            System.out.println("=== TRANSFER COMPLETED ===");
            System.out.printf("%-26s %s%n", "File", file.getFileName());
            System.out.printf("%-26s %d%n", "Bytes", sentBytes);
            System.out.printf("%-26s %d%n", "Blocks", sequence + 1);
            System.out.printf("%-26s %d%n", "Retransmissions", retransmissions);
            System.out.printf("%-26s %d%%%n", "Simulated loss", simulatedLoss);
            System.out.printf("%-26s %d ms%n", "Time", ms);
            System.out.printf("%-26s %d%n", "Checksum", checksum);
            System.out.printf("%-26s %.1f KB/s%n", "Speed",
                    ms == 0 ? 0 : sentBytes / 1024.0 / (ms / 1000.0));
        }
    }

    /** Builds the datagram with its binary header in big-endian. */
    private byte[] build(int sequence, boolean last, byte[] data, int length)
            throws IOException {
        ByteArrayOutputStream bytes = new ByteArrayOutputStream(HEADER + length);
        DataOutputStream output = new DataOutputStream(bytes);
        output.writeInt(sequence);
        output.writeBoolean(last);
        output.writeInt(length);
        output.write(data, 0, length);
        output.flush();
        return bytes.toByteArray();
    }

    /** Sends and waits for the acknowledgement, retrying with increasing backoff. */
    private boolean sendWithAck(DatagramSocket socket, InetAddress destination,
                                int port, byte[] datagram, int sequence,
                                DatagramPacket ack, byte[] ackBuffer)
            throws IOException {

        int wait = INITIAL_WAIT_MS;

        for (int attempt = 1; attempt <= MAX_ATTEMPTS; attempt++) {
            if (attempt > 1) {
                retransmissions++;
            }

            // --- SIMULATED LOSS ---
            // The datagram is discarded BEFORE sending it, to prove that
            // the retry mechanism works without needing a bad network.
            if (ThreadLocalRandom.current().nextInt(100) >= simulatedLoss) {
                socket.send(new DatagramPacket(datagram, datagram.length,
                        destination, port));
            } else {
                LOG.fine("Simulated loss of block " + sequence);
            }

            socket.setSoTimeout(wait);
            long limit = System.currentTimeMillis() + wait;

            while (System.currentTimeMillis() < limit) {
                try {
                    ack.setLength(ackBuffer.length);        // classic mistake 1
                    socket.receive(ack);

                    if (ack.getLength() < 4) {
                        continue;
                    }
                    int acknowledged = ByteBuffer.wrap(ack.getData(),
                            ack.getOffset(), ack.getLength()).getInt();

                    if (acknowledged == sequence) {
                        return true;
                    }
                    // Acknowledgement of an earlier block (it arrived late):
                    // discarded and we carry on waiting for ours.
                    LOG.fine("Stale acknowledgement: " + acknowledged
                            + ", we expected " + sequence);

                } catch (SocketTimeoutException e) {
                    break;      // deadline expired: next attempt
                }
            }
            wait *= 2;          // increasing backoff
        }
        return false;
    }

    public static void main(String[] args) throws IOException {
        Path file = Path.of(args.length > 0 ? args[0] : "catalog.csv");
        int loss = args.length > 1 ? Integer.parseInt(args[1]) : 20;
        new ReliableUdpSender(loss).send("localhost", 9101, file);
    }
}
package com.nexussoftware.bibliotech.network;

import java.io.IOException;
import java.io.OutputStream;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.nio.ByteBuffer;
import java.nio.file.Files;
import java.nio.file.Path;
import java.util.logging.Logger;

/** Receiver of the reliable transfer over UDP. */
public class ReliableUdpReceiver {

    private static final Logger LOG = Logger.getLogger(ReliableUdpReceiver.class.getName());

    private static final int DATA_PER_BLOCK = 1024;
    private static final int HEADER = 4 + 1 + 4;
    private static final int MAX_DATAGRAM = HEADER + DATA_PER_BLOCK;

    public void receive(int port, Path target) throws IOException {
        Files.createDirectories(target.getParent() == null
                ? Path.of(".") : target.getParent());

        try (DatagramSocket socket = new DatagramSocket(port);
             OutputStream output = Files.newOutputStream(target)) {

            System.out.println("Waiting for a file on UDP:" + port);
            socket.setSoTimeout(30_000);        // we give up if nobody sends

            byte[] buffer = new byte[MAX_DATAGRAM + 1];
            DatagramPacket packet = new DatagramPacket(buffer, buffer.length);

            int expected = 0;
            int duplicates = 0;
            long bytes = 0;
            long checksum = 0;
            long start = System.currentTimeMillis();

            while (true) {
                packet.setLength(buffer.length);        // classic mistake 1
                socket.receive(packet);

                if (packet.getLength() < HEADER
                        || packet.getLength() > MAX_DATAGRAM) {
                    LOG.warning("Datagram of invalid size discarded");
                    continue;
                }

                ByteBuffer bb = ByteBuffer.wrap(packet.getData(),
                        packet.getOffset(), packet.getLength());
                int sequence = bb.getInt();
                boolean last = bb.get() != 0;
                int length = bb.getInt();

                // VALIDATION: the length comes from the network. Without this,
                // a malicious sender causes an exception or something worse.
                if (length < 0 || length > DATA_PER_BLOCK
                        || length > bb.remaining()) {
                    LOG.warning("Invalid declared length: " + length);
                    continue;
                }

                if (sequence < expected) {
                    // DUPLICATE: our previous acknowledgement was lost and
                    // the sender resent. We must RE-ACKNOWLEDGE but NOT write
                    // the data again, or the file would come out corrupt.
                    duplicates++;
                    acknowledge(socket, packet, sequence);
                    continue;
                }
                if (sequence > expected) {
                    // With stop and wait this should not happen: it means
                    // we lost an intermediate block. We do not acknowledge, and the
                    // sender will resend the missing one.
                    LOG.warning("Block " + sequence + " out of order; "
                            + "we expected " + expected);
                    continue;
                }

                // Correct block, in order: it is written.
                byte[] data = new byte[length];
                bb.get(data);
                output.write(data);
                for (byte b : data) {
                    checksum += b & 0xFF;
                }
                bytes += length;

                acknowledge(socket, packet, sequence);
                expected++;

                if (last) {
                    output.flush();
                    long ms = System.currentTimeMillis() - start;
                    System.out.println();
                    System.out.println("=== RECEPTION COMPLETED ===");
                    System.out.printf("%-26s %s%n", "Saved to", target);
                    System.out.printf("%-26s %d%n", "Bytes", bytes);
                    System.out.printf("%-26s %d%n", "Blocks", expected);
                    System.out.printf("%-26s %d%n", "Duplicates detected", duplicates);
                    System.out.printf("%-26s %d ms%n", "Time", ms);
                    System.out.printf("%-26s %d%n", "Checksum",
                            checksum);
                    return;
                }
            }
        }
    }

    private void acknowledge(DatagramSocket socket, DatagramPacket original, int sequence)
            throws IOException {
        byte[] ack = ByteBuffer.allocate(4).putInt(sequence).array();
        // We reply TO THE SENDER, which comes in the original packet.
        socket.send(new DatagramPacket(ack, ack.length,
                original.getAddress(), original.getPort()));
    }

    public static void main(String[] args) throws IOException {
        new ReliableUdpReceiver().receive(9101, Path.of("received", "catalog.csv"));
    }
}

Test with 20 % simulated loss:

Terminal 1 (receiver):
Waiting for a file on UDP:9101

=== RECEPTION COMPLETED ===
Saved to                   received/catalog.csv
Bytes                      2847
Blocks                     4
Duplicates detected        1
Time                       834 ms
Checksum                   291476

Terminal 2 (sender):
=== TRANSFER COMPLETED ===
File                       catalog.csv
Bytes                      2847
Blocks                     4
Retransmissions            3
Simulated loss             20%
Time                       841 ms
Checksum                   291476

Comments. The checksums match: the transfer is correct despite the 20 % loss. Three points deserve attention.

Duplicate detection is essential and its logic is not obvious. A duplicate does not happen because the network duplicates packets: it happens because our acknowledgement was lost. The sender did not receive it, its deadline expired and it resent a block we had already written. If we wrote it again, the file would come out with repeated blocks. And notice that it has to be re-acknowledged: if we simply ignored it, the sender would keep resending until its attempts ran out and would abort a transfer that was actually going fine.

The increasing backoff dominates the time. 841 ms for 2847 bytes is a ridiculous speed, and it is not the network's fault: it is the 200, 400 and 800 ms waits of the retries. Stop and wait is the simplest reliability scheme and the slowest, because there is only one block in flight at a time. TCP uses a sliding window —many blocks in flight simultaneously, acknowledged cumulatively— and that is why it reaches speeds orders of magnitude higher.

And the reflection the exercise asked for. This pair of classes adds up to about 250 lines of carefully considered code: numbering, acknowledgements, time limits, retries with increasing backoff, duplicate detection, length validation and an integrity check. The equivalent with TCP is:

try (Socket socket = new Socket()) {
    socket.connect(new InetSocketAddress(host, port), 3000);
    Files.copy(file, socket.getOutputStream());
    socket.shutdownOutput();
}

Four lines, faster, more robust and without a single bug of its own. And what we have written still has no congestion control, no sliding window, no reordering, and no protection against a sender that swamps the receiver.

That is exactly the lesson of section 2: if you end up implementing reliability over UDP, you should almost always be using TCP. The legitimate exception —and QUIC's reason for existing— is when you need partial or tailored reliability, something TCP does not offer because it is all or nothing. But for transferring a file, TCP wins hands down.

Conclusion

You have learned the other transport, and with it BiblioTech has gained two capabilities TCP simply cannot provide.

You know what you lose with UDP —delivery, ordering, uniqueness, flow control, congestion control and any notion that the other end is still alive— and what you gain: immediate sending with no setup, an 8-byte header, message boundaries that make all the delimiter work of 09-02 unnecessary, a total absence of per-client state and, above all, the ability to talk to many at once. And you have the criterion for choosing, which is not speed but three questions: is stale data still worth anything?, do I need one to many? and is the operation idempotent?

You can handle the two classes and their asymmetry: DatagramPacket is the envelope —with two completely different uses, data plus destination for sending and an empty buffer for receiving— and DatagramSocket is the letterbox, one single one for every client, with no connections, no pool and no state. With the two warnings that define the protocol: send() almost never fails even if the destination is switched off, so its success means nothing; and connect() does not connect, it only installs a local filter that is useful as a defence against injections.

You have seen demonstrated the two classic mistakes, which together cause most UDP problems in Java: not calling setLength(buffer.length) before every receive(), which truncates every message to the length of the first one without a single warning; and using getData() without getOffset() and getLength(), which drags along remains of the previous message and produces corrupt data that looks legitimate.

You understand datagram size: the theoretical maximum of 65,507 bytes is of no use against the real MTU of 1500, because going beyond it forces fragmentation, and losing a single fragment loses the whole datagram — fragmentation multiplies the loss and many firewalls drop fragments outright. Hence the prudent figures: 512 bytes to be safe on any network, 1400 as the practical limit. And you know that a datagram that does not fit in your buffer is truncated silently, with the trick of asking for one byte more to detect it.

You have checked with your own measurements that loss, duplication and disorder have to be tolerated — with a 26 % loss on localhost, with no network in between, as soon as the sender outpaces the receiver: that is the absence of flow control. And you know how to build reliability by hand when it is needed, with its four pieces: a random identifier per request to match replies and prevent injections, setSoTimeout with retry and increasing backoff, an attempt limit, and idempotence of the operation — the decisive argument for LEND staying on TCP.

You know broadcast, which reaches the whole local network, does not cross routers and does not exist in IPv6; and multicast, which is its properly done version: groups you subscribe to voluntarily, the private 239.x.x.x range, TTL 1 so as not to leave the local network, and the modern API joinGroup(SocketAddress, NetworkInterface) instead of the deprecated one, because on any machine with Docker, VPN or Wi-Fi plus Ethernet the interface the system picks is not the one you want.

And BiblioTech has gained real pieces. DiscoveryResponder and ServerFinder implement BTDP/1: the workstations ask by broadcast "where is BiblioTech?" and the server replies by unicast with its address and its TCP port — the manual IP configuration on every desk has disappeared, and that was literally impossible with TCP. TelemetryPublisher sends metrics every few seconds with the ScheduledExecutorService of 08-05, never blocking the server and not caring whether the collector is down: you can switch it off with Ctrl+C and the server does not flinch. Plus UdpEchoServer, ReliableUdpRequest, MulticastNotices, TelemetryCollector and the exercise classes: UdpProbe with its jitter measurement, BiblioTechChat with no central server, and the reliable transfer pair that demonstrates —in 250 lines against 4— why reimplementing TCP over UDP is almost never a good idea.

And you have a clear verdict for BiblioTech, which sums the lesson up: UDP does not replace TCP, it complements it. Query, list and lend go over TCP because they need reliability, produce large replies and are not idempotent; discovery, telemetry and notices go over UDP because they need one to many, must not block and tolerate loss. A real system uses both.

So far you have worked at the transport level: bytes, sockets, datagrams, protocols you designed yourself. In the next lesson, URL and HttpURLConnection, you move up a level. You are going to stop inventing protocols and use the one that already rules the world: HTTP. You will see the anatomy of a URL and the URL and URI classes with their differences, and why encoding the parameters with URLEncoder is essential as soon as an accent or a space appears. You will see HTTP explained in depth —request, response, methods, status codes, headers— with a real exchange shown in plain text, and with the connection that ties it all together: that is exactly what travels over the socket of 09-02, only now the protocol was designed by somebody else and half the planet understands it. You will learn URL.openStream() for the simple case and HttpURLConnection for real control —methods, headers, mandatory time limits, status codes, and the classic mistake of not reading getErrorStream() when a 404 arrives—, and you will download a binary file to disk combining it with module 7. With an honest assessment ahead: HttpURLConnection is an old, verbose API full of traps, and that is why Java 11 brought a new one that you will see in 09-06 — but it is still alive in mountains of legacy code and you have to be able to read it. BiblioTech will start talking to the outside world: it will query a metadata service by ISBN and download a book's cover.

Java Programming Course

Module 1: Introduction to Java

Module 2: Control Flow

Module 3: Object-Oriented Programming

Module 4: Advanced Object-Oriented Programming

Module 5: Data Structures and Collections

Module 6: Exception Handling

Module 7: File Input/Output

Module 8: Multithreading and Concurrency

Module 9: Networking

Module 10: Advanced Topics

Module 11: Java Frameworks and Libraries

Module 12: Building Real-World Applications

© Copyright 2026. All rights reserved