When the previous lesson closed, BiblioTech was already correct: the catalogue supports two employees at once, the loan registry maintains its invariant and the counter loses no increments. But the code to get there was artisanal: locks that have to be released in a finally, acquisition orders that have to be respected by convention, and a thread created by hand for every task. The two hundred due-date notices still need two hundred threads, two hundred megabytes of stacks and one join() wait per thread.
This lesson changes the level of abstraction. java.util.concurrent —designed by Doug Lea and incorporated in Java 5— offers ready-made, tested parts for everything you were doing by hand in the previous lessons: thread pools that get reused, tasks that return results and propagate exceptions, periodic schedulers, and synchronisers that replace wait/notify with something you can reason about.
The mental shift is this: you stop thinking about threads and start thinking about tasks. You describe what has to be done; the executor decides where and when it happens. It is the same difference as between managing memory by hand and having a garbage collector.
By the end, BiblioTech's 200 notices will be sent with a pool of eight threads in three seconds, with a progress bar, with a concurrency limit on the shared resource and with clean cancellation.
Contents
- Why
java.util.concurrentreplaces manual management Executor,ExecutorServiceandExecutors- The
Executorsfactories and when to use each - The warning about unbounded queues
- A bespoke
ThreadPoolExecutor - Choosing the pool size
- The executor's lifecycle
CallableandFuture- Cancelling tasks
invokeAllandinvokeAnyScheduledExecutorServiceCountDownLatchCyclicBarrierSemaphoreExchangerand a comparison table of synchronisersForkJoinPooland divide and conquer- BiblioTech: 200 notices with a pool, progress and cancellation
- Common Mistakes and Tips
- Exercises
- Why
java.util.concurrent replaces manual management
java.util.concurrent replaces manual managementRemember the 08-02 code for sending notices:
// WHAT YOU USED TO DO: a thread per task, complete manual management.
Thread[] threads = new Thread[200];
for (int i = 0; i < 200; i++) {
threads[i] = new Thread(new NoticeTask(notices.get(i)), "bibliotech-notices-" + i);
threads[i].start();
}
for (Thread t : threads) {
t.join();
}
// And each task's result? volatile fields read by hand.
// And if one fails? An uncaught exception handler.
// And if I want to cancel everything? interrupt() on all 200, one by one.
// And if there are 200,000 notices? OutOfMemoryError.Six problems, all of them real:
| Problem | Consequence |
|---|---|
| A thread per task | ~1 MB of stack and tens of µs per thread; it does not scale |
| No limit on threads | With many tasks, OutOfMemoryError: unable to create native thread |
| Threads are not reused | You pay for creation over and over |
Runnable returns nothing |
volatile fields and implicit conventions |
| Exceptions never reach the caller | Silent failures (08-02) |
| Manual cancellation | Walking an array calling interrupt() |
java.util.concurrent solves all six. The same code becomes:
// WHAT YOU WILL DO: describe tasks, the executor places them.
ExecutorService executor = Executors.newFixedThreadPool(8);
List<Future<NoticeResult>> futures = new ArrayList<>();
for (Notice n : notices) {
futures.add(executor.submit(new NoticeTask(n))); // Callable: returns a value
}
for (Future<NoticeResult> f : futures) {
NoticeResult r = f.get(); // waits, and RETHROWS the exception if there was one
}
executor.shutdown();Eight threads instead of two hundred, typed results, exceptions that arrive, and cancellation with one call.
Executor, ExecutorService and Executors
Executor, ExecutorService and ExecutorsThree similar names worth not confusing:
| Name | What it is | Role |
|---|---|---|
Executor |
An interface with one method: void execute(Runnable) |
The minimal abstraction: "run this" |
ExecutorService |
An interface extending Executor |
Adds results (submit), shutdown and waiting |
Executors |
A utility class with static methods | A factory of pre-configured implementations |
Executor is deliberately minimalist, and that is its virtue: it separates what is run from how it is run.
// The complete interface. A single method.
public interface Executor {
void execute(Runnable command);
}
// Three valid implementations of the SAME interface:
Executor onTheSameThread = task -> task.run();
Executor aNewThread = task -> new Thread(task).start();
Executor aPool = Executors.newFixedThreadPool(8);
// The client code does not change:
onTheSameThread.execute(() -> System.out.println("hello"));
aPool.execute(() -> System.out.println("hello"));ExecutorService adds what is needed in practice:
public interface ExecutorService extends Executor {
// Submit tasks and get a Future
Future<?> submit(Runnable task);
<T> Future<T> submit(Callable<T> task);
<T> Future<T> submit(Runnable task, T result);
// Submit several
<T> List<Future<T>> invokeAll(Collection<? extends Callable<T>> tasks);
<T> T invokeAny(Collection<? extends Callable<T>> tasks);
// Shutdown
void shutdown();
List<Runnable> shutdownNow();
boolean isShutdown();
boolean isTerminated();
boolean awaitTermination(long time, TimeUnit unit);
}The key difference between execute and submit, and it is one of the traps of the topic:
// execute(Runnable): returns nothing. If the task throws an exception,
// it goes to the thread's uncaught exception handler (08-02) and
// is normally printed on the console.
executor.execute(() -> { throw new RuntimeException("visible failure"); });
// submit(...): returns a Future. If the task throws an exception,
// it is STORED in the Future and is NOT printed anywhere.
// If nobody calls get(), the failure vanishes WITHOUT A TRACE.
executor.submit(() -> { throw new RuntimeException("INVISIBLE failure"); });That second case is one of the most frequent causes of "my task does nothing and no error appears". The rule: if you use submit, call get() or check the result somehow.
- The
Executors factories and when to use each
Executors factories and when to use eachimport java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.ScheduledExecutorService;
// 1. FIXED pool: N threads, always the same ones, unbounded queue.
ExecutorService fixed = Executors.newFixedThreadPool(8);
// 2. ELASTIC pool: creates threads on demand, recycles them after 60 s.
ExecutorService elastic = Executors.newCachedThreadPool();
// 3. A SINGLE THREAD: guarantees execution order.
ExecutorService single = Executors.newSingleThreadExecutor();
// 4. SCHEDULED: delayed and periodic tasks.
ScheduledExecutorService scheduled = Executors.newScheduledThreadPool(2);
// 5. WORK STEALING: a ForkJoinPool with parallelism = cores.
ExecutorService stealing = Executors.newWorkStealingPool();| Factory | Threads | Queue | When to use it | Risk |
|---|---|---|---|---|
newFixedThreadPool(n) |
Exactly n |
Unbounded | Stable load, computation | The queue grows without limit |
newCachedThreadPool() |
0 … Integer.MAX_VALUE |
SynchronousQueue (no capacity) |
Many short I/O tasks | Creates threads without limit |
newSingleThreadExecutor() |
1 | Unbounded | Tasks that must be ordered | The queue grows without limit |
newScheduledThreadPool(n) |
n (grows if needed) |
Delay queue | Periodic work | One long task delays the others |
newWorkStealingPool() |
= cores | Per-thread queues | Divisible computation | No ordering guarantee |
newVirtualThreadPerTaskExecutor() |
One virtual thread per task (Java 21) | — | Thousands of I/O tasks | See 10-06 |
Use cases in BiblioTech:
// Sending notices: I/O, bounded and known load -> fixed pool.
ExecutorService notices = Executors.newFixedThreadPool(8);
// Writing the audit log: it MUST preserve ordering -> a single thread.
// Also, being a single thread, the file needs no synchronisation.
ExecutorService audit = Executors.newSingleThreadExecutor();
// Due-date notices every hour and a backup every night.
ScheduledExecutorService maintenance = Executors.newScheduledThreadPool(2);The newSingleThreadExecutor case deserves a comment: a single-thread executor is confinement (08-04, section 17) implemented as a service. Everything running in it is serialised by construction, so its state needs no lock at all. It is a very powerful and under-used technique.
- The warning about unbounded queues
The Executors factories are convenient and they have a serious trap you must know about before using them in production.
newFixedThreadPool and newSingleThreadExecutor use a LinkedBlockingQueue with no capacity limit.
If tasks arrive faster than the pool consumes them, the queue grows indefinitely. There is no mechanism to hold it back. The result, after minutes or hours of apparently normal operation, is an OutOfMemoryError.
// TIME BOMB
ExecutorService pool = Executors.newFixedThreadPool(4);
// Each task takes 100 ms; 1,000 arrive per second.
// The pool processes 40/s. 960 tasks pile up in memory every second.
// In 10 minutes: ~576,000 objects queued. OutOfMemoryError.
while (thereAreRequests()) {
pool.submit(new SlowTask(nextRequest()));
}newCachedThreadPool has the symmetric and worse problem: its queue is a SynchronousQueue, which stores nothing, so every task that arrives while all the threads are busy triggers the creation of a new thread, without limit (up to Integer.MAX_VALUE). With a burst of ten thousand slow tasks, the JVM tries to create ten thousand threads and dies with OutOfMemoryError: unable to create native thread.
The solution: a bounded queue and an explicit rejection policy, which is what is built in the next section.
Professional rule. The
Executorsfactories are fine for example code, internal tools and start-up tasks. For a service receiving external load, build your ownThreadPoolExecutorwith a bounded queue. It is the explicit recommendation of Google's style guides and of most industry handbooks.
- A bespoke
ThreadPoolExecutor
ThreadPoolExecutorAll the factories above return, underneath, a ThreadPoolExecutor. Building it directly gives you control over the six parameters that matter.
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class BiblioTechExecutor {
/**
* A ThreadFactory that names the threads. Without it, the pool's threads
* are called "pool-1-thread-1", which in a dump (08-03) says nothing.
*/
static class NamedThreadFactory implements ThreadFactory {
private final String prefix;
private final boolean daemon;
private final AtomicInteger counter = new AtomicInteger(1);
NamedThreadFactory(String prefix, boolean daemon) {
this.prefix = prefix;
this.daemon = daemon;
}
@Override
public Thread newThread(Runnable r) {
Thread t = new Thread(r, prefix + "-" + counter.getAndIncrement());
t.setDaemon(daemon);
// Safety net: any exception escaping a task submitted
// with execute() gets logged (06-07, 08-02).
t.setUncaughtExceptionHandler((thread, error) ->
System.err.println("[" + thread.getName() + "] uncaught failure: " + error));
return t;
}
}
public static ThreadPoolExecutor create() {
return new ThreadPoolExecutor(
4, // 1. core size
8, // 2. maximum size
60L, TimeUnit.SECONDS, // 3. lifetime of extra threads
new ArrayBlockingQueue<>(100), // 4. BOUNDED queue
new NamedThreadFactory("bibliotech-notices", false), // 5.
new ThreadPoolExecutor.CallerRunsPolicy() // 6.
);
}
}The six parameters, one by one:
1. corePoolSize — threads kept alive even when idle. The pool creates them as tasks arrive and does not destroy them (unless you use allowCoreThreadTimeOut(true)).
2. maximumPoolSize — the absolute cap on threads.
3. keepAliveTime — how long a thread above the core size survives while idle.
4. workQueue — where the tasks wait. The most important decision.
5. threadFactory — how threads are created: name, daemon status, priority, exception handler.
6. RejectedExecutionHandler — what to do when the queue is full and no more threads can be created.
The algorithm the pool follows for a new task —and this is where nearly everyone reasons wrongly:
flowchart TD
A["A task arrives"] --> B{"active threads<br/>fewer than corePoolSize?"}
B -->|Yes| C["Create a new thread<br/>and run it"]
B -->|No| D{"does it fit in the queue?"}
D -->|Yes| E["Enqueue<br/>WAITS its turn"]
D -->|No| F{"active threads<br/>fewer than maximumPoolSize?"}
F -->|Yes| G["Create an extra thread<br/>and run it"]
F -->|No| H["REJECT<br/>RejectedExecutionHandler"]
The counter-intuitive consequence: the pool prefers queuing over creating extra threads. With core=4, max=100 and an unbounded queue, never will more than 4 threads be created: the queue never fills up, so the step that creates extra threads is never reached. That is exactly what happens with newFixedThreadPool, and it explains why maximumPoolSize is only useful if the queue is bounded.
The four rejection policies:
| Policy | Behaviour | When to use it |
|---|---|---|
AbortPolicy (default) |
Throws RejectedExecutionException |
When losing a task is unacceptable and you want to know |
CallerRunsPolicy |
The submitting thread runs the task | Excellent: creates natural backpressure |
DiscardPolicy |
Discards silently | Almost never: silent failures are poison |
DiscardOldestPolicy |
Discards the oldest queued task | Data where recency matters more (metrics) |
CallerRunsPolicy deserves an explanation because it is the most useful and the least obvious: when the pool is saturated, the task is run by the very thread that called submit. That thread becomes busy and therefore stops producing new tasks for that time. The result is an automatic backpressure mechanism: the producer slows down to the consumer's pace, with no infinite queues and nothing discarded.
- Choosing the pool size
Picking up the 08-01 distinction between CPU-bound and I/O-bound tasks:
| Task type | Formula | BiblioTech example |
|---|---|---|
| CPU-bound | N or N + 1 |
Recalculating all the fines |
| I/O-bound | N × (1 + wait/compute) |
Sending 200 notices |
| Mixed | Measure, or split into two pools | Importing (read + validate) |
public class PoolSizing {
private static final int CORES = Runtime.getRuntime().availableProcessors();
/** Pure computation: more threads than cores only adds context switches. */
public static int forCompute() {
return CORES;
}
/**
* I/O: Brian Goetz's formula.
* threads = cores * targetUtilisation * (1 + wait / compute)
* With utilisation 1.0, 300 ms of waiting and 20 ms of computing on 8 cores:
* 8 * 1.0 * (1 + 15) = 128 threads.
* In practice you cap it at a sensible maximum: 128 threads are 128 MB
* of stacks, and the remote resource probably cannot take 128 requests.
*/
public static int forIo(double waitMs, double computeMs) {
double ratio = waitMs / computeMs;
int suggested = (int) Math.ceil(CORES * (1 + ratio));
return Math.min(suggested, 64); // sanity cap
}
public static void main(String[] args) {
System.out.println("Cores : " + CORES);
System.out.println("Compute pool : " + forCompute());
System.out.println("Notices pool : " + forIo(300, 20));
System.out.println("Import pool : " + forIo(50, 30));
}
}The most important advice about sizing: separate the pools by type of work. A single pool shared between computation and I/O tasks is the worst of both worlds: the I/O tasks occupy threads without using CPU, and the computation ones block the I/O ones.
// GOOD: three pools with different purposes and sizes, isolated
// from each other: saturating the notices pool does not affect the compute one.
private final ExecutorService computePool = Executors.newFixedThreadPool(CORES);
private final ExecutorService ioPool = Executors.newFixedThreadPool(32);
private final ScheduledExecutorService scheduledPool =
Executors.newScheduledThreadPool(2);This idea —bulkhead isolation— comes from the watertight compartments of a ship: if one floods, the rest stays afloat. It is revisited in 12-07.
- The executor's lifecycle
An ExecutorService has three states: running, shutting down and terminated.
| Method | What it does |
|---|---|
shutdown() |
Stops accepting new tasks; finishes the pending ones. Does not block |
shutdownNow() |
Stops accepting; interrupts the running ones; returns the unstarted ones |
isShutdown() |
Has either of the two been called? |
isTerminated() |
Have all the tasks finished yet? |
awaitTermination(t, u) |
Blocks until everything finishes or the deadline expires |
The critical mistake: if you do not shut the executor down, the JVM does not terminate. A pool's threads are non-daemon by default, so they stay alive waiting for work indefinitely (08-01, section 8). Your main finishes and the process just sits there.
The correct shutdown pattern, as recommended by the ExecutorService documentation:
import java.util.concurrent.ExecutorService;
import java.util.concurrent.TimeUnit;
public final class GracefulShutdown {
/**
* Two-phase shutdown:
* 1. shutdown() + bounded wait: let what is running finish.
* 2. If they do not finish, shutdownNow() (interruption) + short wait.
* 3. If they still do not finish, log it: there are tasks that do not
* respond to interruption, and that is a BUG in those tasks
* (the empty InterruptedException catch of 08-02).
*/
public static void shutdown(ExecutorService executor, long seconds) {
executor.shutdown(); // phase 1: no more accepted
try {
if (!executor.awaitTermination(seconds, TimeUnit.SECONDS)) {
executor.shutdownNow(); // phase 2: interrupt
if (!executor.awaitTermination(seconds, TimeUnit.SECONDS)) {
System.err.println("The executor has not terminated: "
+ "there are tasks ignoring the interruption");
}
}
} catch (InterruptedException e) {
// WE have been interrupted while waiting.
executor.shutdownNow();
Thread.currentThread().interrupt(); // restore the flag (08-02)
}
}
}An important detail about shutdownNow(): it interrupts, it does not kill. A task that swallows the InterruptedException carries on quite happily. The whole 08-02 protocol comes due here: an empty catch means your application cannot be shut down.
Java 19+:
ExecutorServiceisAutoCloseable. Since Java 19 it can be used withtry-with-resources(06-06), andclose()performs ashutdown()and waits for the tasks to finish:try (ExecutorService executor = Executors.newFixedThreadPool(8)) { for (Notice n : notices) { executor.submit(new NoticeTask(n)); } } // close(): shutdown() + indefinite waitIt is far cleaner, but it waits indefinitely: if a task never finishes, your
trynever exits. For code with deadlines, the two-phase pattern is still the right one.
Callable and Future
Callable and FutureIn 08-02 a problem was left open: Runnable.run() returns void and cannot throw checked exceptions. Callable solves it:
@FunctionalInterface
public interface Callable<V> {
V call() throws Exception; // returns V and CAN throw
}Runnable |
Callable<V> |
|
|---|---|---|
| Method | void run() |
V call() throws Exception |
| Returns a value | No | Yes |
| Checked exceptions | No | Yes |
| Submitted with | execute or submit |
submit |
Future<V> is the object representing "a result that will arrive". The <V> is simply the type of the result: a Future<Report> promises a Report (generics in depth in 10-01).
public interface Future<V> {
V get() throws InterruptedException, ExecutionException;
V get(long time, TimeUnit unit) throws ..., TimeoutException;
boolean cancel(boolean mayInterruptIfRunning);
boolean isCancelled();
boolean isDone();
}A complete example with BiblioTech:
package com.nexussoftware.bibliotech.persistence;
import java.nio.file.Path;
import java.util.concurrent.*;
public class ImportWithFuture {
public static void main(String[] args) {
ExecutorService executor = Executors.newFixedThreadPool(2);
try {
// Callable<Report>: returns a Report and can throw
// checked exceptions. Both impossible with Runnable.
Callable<Report> task = () -> {
CatalogImporter imp = new CatalogImporter();
return imp.importFrom(Path.of("data/inventory.csv")); // throws IOException
};
Future<Report> future = executor.submit(task);
// The main thread is NOT blocked: it can carry on working.
System.out.println("[main] import launched, serving the menu");
showMenu();
// isDone() does not block: it lets you poll without waiting.
while (!future.isDone()) {
System.out.println("[main] importing...");
TimeUnit.MILLISECONDS.sleep(500);
}
// get() with a DEADLINE: never use get() without one in production.
Report report = future.get(30, TimeUnit.SECONDS);
System.out.println("[main] imported: " + report.imported());
} catch (TimeoutException e) {
System.err.println("[main] the import exceeded the deadline");
} catch (ExecutionException e) {
// The task threw an exception. get() WRAPS it in
// ExecutionException; the original is in getCause().
// It is exactly the exception chaining of 06-03.
Throwable cause = e.getCause();
System.err.println("[main] the import failed: " + cause.getMessage());
if (cause instanceof java.io.IOException) {
System.err.println("[main] file problem; the previous catalogue is kept");
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
GracefulShutdown.shutdown(executor, 10);
}
}
static void showMenu() { /* ... */ }
}The three points to retain:
1. get() with no deadline blocks indefinitely. If the task hangs, your thread hangs with it. In production, always get(time, unit).
2. ExecutionException wraps the cause. The real exception is in getCause(). It is the chaining of 06-03, and you have to unwrap it to make decisions:
try {
Report r = future.get(30, TimeUnit.SECONDS);
} catch (ExecutionException e) {
Throwable cause = e.getCause();
// Recover the original type to apply the 06-07 policy
if (cause instanceof InvalidFormatException ife) {
logAndDegrade(ife);
} else if (cause instanceof java.io.IOException ioe) {
abortWithCode(ioe);
} else {
throw new BiblioTechException("Unexpected failure in the import", cause);
}
}3. The three exceptions from get() mean different things:
| Exception | Meaning | Reaction |
|---|---|---|
ExecutionException |
The task failed | Unwrap with getCause() and apply the policy |
TimeoutException |
The task is still running | Decide: wait longer or cancel(true) |
InterruptedException |
You have been interrupted | Restore the flag; the task is still alive |
Note the last row: a TimeoutException cancels nothing. The task keeps running. If you want it to stop, you have to call cancel(true) explicitly.
- Cancelling tasks
Future.cancel(boolean mayInterruptIfRunning) is the cancellation of 08-02 exposed as a method.
Future<Report> future = executor.submit(task);
// cancel(false): only stops it STARTING if it is still queued.
// If it is already running, it lets it finish.
future.cancel(false);
// cancel(true): in addition, it INTERRUPTS the thread running it.
// It is literally a thread.interrupt() on the pool's thread.
future.cancel(true);cancel(true) only works if the task cooperates. The whole 08-02 protocol applies just the same: check isInterrupted(), catch InterruptedException, do not swallow it. A task that ignores interruption is uncancellable, whether it is in a pool or not.
/**
* A cancellable task inside a pool. Identical to the 08-02 pattern:
* the Future's cancel(true) translates into interrupt() on this thread.
*/
public class CancellableNoticeTask implements Callable<NoticeResult> {
private final Notice notice;
public CancellableNoticeTask(Notice notice) { this.notice = notice; }
@Override
public NoticeResult call() throws Exception {
for (int attempt = 1; attempt <= 3; attempt++) {
// A cancellation point on every pass.
if (Thread.currentThread().isInterrupted()) {
throw new InterruptedException("notice cancelled before attempt " + attempt);
}
try {
send(notice); // may throw InterruptedException
return NoticeResult.success(notice);
} catch (SendFailedException e) {
if (attempt == 3) {
return NoticeResult.failure(notice, e.getMessage());
}
TimeUnit.MILLISECONDS.sleep(100L * attempt); // backoff
}
}
return NoticeResult.failure(notice, "retries exhausted");
}
private void send(Notice n) throws Exception { /* ... */ }
}Checking the state after cancelling:
Future<Report> f = executor.submit(task);
TimeUnit.SECONDS.sleep(2);
boolean cancelled = f.cancel(true);
System.out.println("cancel() returned : " + cancelled); // false if it had finished
System.out.println("isCancelled() : " + f.isCancelled());
System.out.println("isDone() : " + f.isDone()); // true: cancelled is also "done"
try {
f.get();
} catch (CancellationException e) {
// get() on a CANCELLED Future throws CancellationException,
// which is UNchecked. It is easy to forget to catch it.
System.out.println("confirmed: the task was cancelled");
}
invokeAll and invokeAny
invokeAll and invokeAnyTwo methods for submitting a collection of tasks at once.
invokeAll: runs them all and blocks until they all finish. It returns the list of Futures, all already completed.
List<Callable<NoticeResult>> tasks = new ArrayList<>();
for (Notice n : notices) {
tasks.add(new CancellableNoticeTask(n));
}
// Blocks until they ALL finish. The returned Futures
// are already complete: get() does not block.
List<Future<NoticeResult>> results = executor.invokeAll(tasks);
int successes = 0, failures = 0;
for (Future<NoticeResult> f : results) {
try {
if (f.get().isSuccessful()) successes++; else failures++;
} catch (ExecutionException e) {
failures++;
LOG.log(Level.WARNING, "notice failed", e.getCause());
}
}With a global deadline, which is what is recommended:
// A GLOBAL deadline for the whole set. Those that do not finish in time
// are CANCELLED automatically, and their get() will throw CancellationException.
List<Future<NoticeResult>> results =
executor.invokeAll(tasks, 30, TimeUnit.SECONDS);invokeAny: returns the result of the first one to finish successfully and cancels the rest. Useful when there are several ways of getting the same thing and you want the fastest.
// Three sources for the same catalogue; the first to answer will do.
List<Callable<Catalog>> sources = List.of(
() -> loadFromLocalCache(),
() -> loadFromMainFile(),
() -> loadFromBackup()
);
// Returns the first correct result; cancels the other two.
// If they ALL fail, it throws ExecutionException with the last cause.
Catalog c = executor.invokeAny(sources, 10, TimeUnit.SECONDS);invokeAll |
invokeAny |
|
|---|---|---|
| Waits for | All | The first success |
| Returns | List<Future<T>> |
T (the value, not a Future) |
| With the rest | Nothing | Cancels them |
| If one fails | Its Future throws in get() |
Ignored, unless they all fail |
| Use case | Batch work | Redundant sources, the fastest one |
ScheduledExecutorService
ScheduledExecutorServiceFor delayed or periodic work. It replaces Timer/TimerTask, which are from Java 1.3 and have two serious defects: a single thread for all the tasks, and one uncaught exception kills the whole timer so the other tasks are never run again.
import java.util.concurrent.*;
public class BiblioTechMaintenance {
private final ScheduledExecutorService scheduler =
Executors.newScheduledThreadPool(2, r -> {
Thread t = new Thread(r, "bibliotech-scheduler");
t.setDaemon(true); // maintenance: must not prevent shutdown
return t;
});
public void start() {
// 1. ONCE, in 5 seconds' time.
scheduler.schedule(
() -> System.out.println("[maintenance] start-up complete"),
5, TimeUnit.SECONDS);
// 2. PERIODIC at a fixed rate: every hour from the start instant.
scheduler.scheduleAtFixedRate(
this::sendDueNotices,
0, 1, TimeUnit.HOURS);
// 3. PERIODIC with a fixed delay: 24 h from when the previous one ENDS.
scheduler.scheduleWithFixedDelay(
this::backup,
1, 24, TimeUnit.HOURS);
}
/**
* GOLDEN RULE: a periodic task must catch EVERY exception.
* If it lets one escape, the JVM CANCELS the schedule and the task
* IS NEVER RUN AGAIN, silently, with no warning at all.
* It is the most treacherous failure of this API.
*/
private void sendDueNotices() {
try {
int sent = noticeService.sendDueNotices();
LOG.log(Level.INFO, "Due-date notices sent: {0}", sent);
} catch (Exception e) {
LOG.log(Level.SEVERE, "Failure sending due-date notices", e);
// Do NOT rethrow: if it escapes, the schedule ends forever.
}
}
private void backup() {
try {
backupManager.backup();
} catch (Exception e) {
LOG.log(Level.SEVERE, "Backup failure", e);
}
}
}scheduleAtFixedRate versus scheduleWithFixedDelay — the difference matters:
scheduleAtFixedRate(t, i, p, u) |
scheduleWithFixedDelay(t, i, p, u) |
|
|---|---|---|
| The period is measured from | The start of the previous run | The end of the previous run |
| If the task lasts longer than the period | The next ones are delayed and chained without overlapping | There is always a real p gap |
| Rhythm | Constant (it tries to keep the schedule) | Variable (depends on the duration) |
| Use for | Work on a schedule: hourly reports, metrics | Work whose duration varies: backups, cleanup |
A task lasting 3 s, period 5 s:
scheduleAtFixedRate: [---3s---]__2s__[---3s---]__2s__[---3s---]
0 3 5 8 10
starts every 5 s exactly
scheduleWithFixedDelay:[---3s---]____5s____[---3s---]____5s____[--
0 3 8 11 16
5 s AFTER finishingThe detail that ruins production systems: if a scheduleAtFixedRate task throws an uncaught exception, the schedule is cancelled silently. The task is never run again, there is no error in the log except the one you put there, and nobody notices until somebody asks why no notices have arrived for three weeks. Always wrap the body in a try/catch (Exception).
CountDownLatch
CountDownLatchA countdown latch: a counter that only goes down. The threads waiting in await() are released when it reaches zero. It is single-use: it cannot be reset.
CountDownLatch latch = new CountDownLatch(5); // 5 pending events
latch.countDown(); // subtracts 1 (never below 0)
latch.await(); // blocks until it reaches 0
latch.await(10, TimeUnit.SECONDS); // with a deadline; returns false if it expires
latch.getCount(); // current count (for showing progress)Use 1: waiting for N tasks to finish.
import java.util.concurrent.*;
public class AwaitNTasks {
public static void main(String[] args) throws InterruptedException {
final int TASKS = 200;
ExecutorService pool = Executors.newFixedThreadPool(8);
CountDownLatch finished = new CountDownLatch(TASKS);
for (int i = 0; i < TASKS; i++) {
final int n = i;
pool.execute(() -> {
try {
sendNotice(n);
} catch (Exception e) {
LOG.log(Level.WARNING, "notice " + n + " failed", e);
} finally {
// MANDATORY in finally: if a task fails and does not
// count down, main's await() NEVER returns.
finished.countDown();
}
});
}
// Progress while we wait, with no blind sleep polling.
while (!finished.await(500, TimeUnit.MILLISECONDS)) {
long pending = finished.getCount();
System.out.printf("\rProgress: %d/%d (%.0f%%)",
TASKS - pending, TASKS,
100.0 * (TASKS - pending) / TASKS);
}
System.out.println("\nAll the notices processed");
GracefulShutdown.shutdown(pool, 10);
}
}The countDown() ALWAYS goes in a finally. If a task fails before calling it, the counter never reaches zero and whoever is waiting blocks forever. It is the number-one CountDownLatch mistake.
Use 2: a starting gate — start N threads at exactly the same time. Very useful for concurrency tests, because it maximises overlap:
/**
* Two latches: one to start everybody at once, another to
* wait for everybody to finish. It is the skeleton of a properly
* built concurrency stress test: without the gate, the first threads
* would finish before the last ones started, and there would be no
* real overlap (the effect you saw in exercise 2 of 08-01).
*/
public static long stressTest(int threads, Runnable task) throws InterruptedException {
CountDownLatch startGate = new CountDownLatch(1); // the starting pistol
CountDownLatch finishLine = new CountDownLatch(threads); // the finish line
for (int i = 0; i < threads; i++) {
new Thread(() -> {
try {
startGate.await(); // everybody waits here
task.run();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
finishLine.countDown();
}
}, "stress-" + i).start();
}
long start = System.nanoTime();
startGate.countDown(); // go! the N start at once
finishLine.await();
return System.nanoTime() - start;
}
CyclicBarrier
CyclicBarrierA cyclic barrier: N threads wait for each other at a point; when the last one arrives, they all continue and the barrier resets. That last word is the difference from CountDownLatch.
import java.util.concurrent.*;
public class PhasedProcessing {
public static void main(String[] args) {
final int WORKERS = 4;
// The barrier action is run by the LAST thread to arrive,
// before releasing the others. Ideal for consolidating the phase.
CyclicBarrier barrier = new CyclicBarrier(WORKERS, () ->
System.out.println("--- phase completed by all 4, consolidating ---"));
for (int i = 0; i < WORKERS; i++) {
final int id = i;
new Thread(() -> {
try {
for (int phase = 1; phase <= 3; phase++) {
System.out.printf(" worker %d processes phase %d%n", id, phase);
TimeUnit.MILLISECONDS.sleep(100 + id * 50);
barrier.await(); // waits for the other three
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} catch (BrokenBarrierException e) {
// If a thread is interrupted or times out, the barrier BREAKS
// and ALL the others receive this exception. That is correct:
// without every participant, the phase makes no sense.
System.err.println("broken barrier: " + e);
}
}, "worker-" + i).start();
}
}
}CountDownLatch |
CyclicBarrier |
|
|---|---|---|
| Reusable | No (single use) | Yes (it resets) |
| Who waits | Some wait, others count | All the participants wait |
| The counter | Goes down with countDown() |
Goes up on reaching await() |
| Action on completion | None | Yes, an optional Runnable |
| If a participant fails | The rest keep waiting | It breaks for everybody |
| Typical case | Waiting for a batch to finish | Simulations and phased computations |
In BiblioTech, CountDownLatch is what you need nearly always: the 200 notices are independent and nobody has to wait for anybody halfway through. CyclicBarrier fits iterative phased computations, where each iteration needs all the previous ones to have finished.
Semaphore
SemaphoreA semaphore holds a number of permits. acquire() takes one (and blocks if there are none); release() gives one back. It is the tool for limiting concurrency on a resource.
package com.nexussoftware.bibliotech.service;
import java.util.concurrent.*;
/**
* Limits simultaneous exports to 3.
*
* Reason: each export opens the whole catalogue file, formats it in
* memory and writes it. Three at once saturate the disk and multiply
* memory usage; with twenty, the server crawls.
* The semaphore puts a hard cap without limiting the rest of the pool.
*/
public class ExportService {
private static final int MAX_CONCURRENT = 3;
// 3 permits. The 'true' enables fairness: those who have been
// waiting longest go first, which prevents starvation (08-04).
private final Semaphore permits = new Semaphore(MAX_CONCURRENT, true);
private final ExecutorService pool = Executors.newFixedThreadPool(16);
public Future<Path> export(String format) {
return pool.submit(() -> {
// Waits for a permit. If there are 3 exports in progress, this
// thread blocks here until one finishes.
permits.acquire();
try {
System.out.printf("[%s] exporting (%d permits free)%n",
Thread.currentThread().getName(),
permits.availablePermits());
return generateFile(format); // heavy work
} finally {
// MANDATORY in finally: a permit not returned is
// lost FOREVER. After three failures, the semaphore
// sits at 0 and nobody ever exports again.
permits.release();
}
});
}
/** A variant that does not wait: it rejects instead of queuing. */
public Future<Path> exportWithoutWaiting(String format) {
return pool.submit(() -> {
if (!permits.tryAcquire(5, TimeUnit.SECONDS)) {
throw new ServiceOverloadedException(
"There are " + MAX_CONCURRENT + " exports in progress; retry");
}
try {
return generateFile(format);
} finally {
permits.release();
}
});
}
private Path generateFile(String format) throws Exception {
TimeUnit.SECONDS.sleep(2);
return Path.of("reports/catalog." + format);
}
}Common uses of a semaphore:
| Use | How |
|---|---|
| Limiting concurrency on a resource | new Semaphore(n), acquire/release |
| Turning a collection into a bounded one | One permit per free slot |
| Simple mutual exclusion | new Semaphore(1) — but prefer a lock |
| Signalling between threads | One thread does release, another acquire |
The difference from a lock, which always gets asked: a lock is owned by the thread that took it and only that thread can release it; a semaphore has no owner, so one thread can acquire a permit and another release it. That makes it more flexible and also easier to misuse —hence the mandatory finally—.
Exchanger and a comparison table of synchronisers
Exchanger and a comparison table of synchronisersExchanger<V> lets two threads swap objects at a rendezvous point. Both call exchange(object) and each receives the other's.
Exchanger<List<Material>> exchange = new Exchanger<>();
// Reader thread: fills a buffer and swaps it for an empty one.
List<Material> myBuffer = new ArrayList<>();
// ... fill it ...
myBuffer = exchange.exchange(myBuffer); // receives the other's empty buffer
// Processor thread: processes a full buffer and returns an empty one.
List<Material> toProcess = exchange.exchange(new ArrayList<>());It is a narrow niche —the double-buffer pattern between exactly two threads— and in practice it is almost always better solved with a BlockingQueue (08-06). It is worth knowing in case it turns up.
Complete comparison table:
| Synchroniser | Participants | Reusable | What for |
|---|---|---|---|
CountDownLatch |
Some count, others wait | No | Waiting for N events to happen |
CyclicBarrier |
N, all equal | Yes | Synchronising phases between N threads |
Semaphore |
Anyone | Yes | Limiting concurrent accesses |
Exchanger |
Exactly 2 | Yes | Pairwise object exchange |
Phaser (Java 7) |
Variable, dynamic | Yes | A barrier with participants joining and leaving |
Phaser is a more flexible CyclicBarrier allowing participants to be registered and deregistered on the fly. It is powerful and uncommon; mention it and do not use it unless the need is obvious.
ForkJoinPool and divide and conquer
ForkJoinPool and divide and conquerForkJoinPool (Java 7) is designed for tasks that can be split recursively into independent subtasks: the divide and conquer model.
Its distinctive feature is work stealing: each thread has its own subtask queue, and when it runs out of work it steals from another thread's queue at the opposite end. This spreads the load automatically without a central coordinator and with very little contention.
flowchart TD
A["Calculate fines for 100,000 loans"] --> B["Chunk 1: 0-50,000"]
A --> C["Chunk 2: 50,000-100,000"]
B --> D["0-25,000"]
B --> E["25,000-50,000"]
C --> F["50,000-75,000"]
C --> G["75,000-100,000"]
D --> H["... down to the threshold<br/>direct computation"]
E --> H
F --> H
G --> H
H --> I["Combine results<br/>upwards"]
RecursiveTask<V> for tasks that return a value; RecursiveAction for those that do not.
package com.nexussoftware.bibliotech.service;
import java.util.List;
import java.util.concurrent.ForkJoinPool;
import java.util.concurrent.RecursiveTask;
/**
* Calculates the sum of fines for a list of loans by dividing
* the work recursively.
*
* RecursiveTask<Double>: the <Double> is the type of the result
* this task produces (10-01 for generics).
*/
public class ParallelFineCalculation extends RecursiveTask<Double> {
/**
* THRESHOLD: below this size the work is computed directly.
* Choosing it well is the most important thing in the whole pattern:
* - too low: the cost of creating and coordinating subtasks
* exceeds the computation, and it runs SLOWER than sequentially;
* - too high: there are not enough chunks for all the cores.
* Rule of thumb: between 100 and 10,000 elements, and MEASURE.
*/
private static final int THRESHOLD = 1_000;
private final List<Loan> loans;
private final int from;
private final int to;
private final FineCalculator calculator;
public ParallelFineCalculation(List<Loan> loans, int from, int to,
FineCalculator calculator) {
this.loans = loans;
this.from = from;
this.to = to;
this.calculator = calculator;
}
@Override
protected Double compute() {
int size = to - from;
// BASE CASE: a small chunk, direct sequential computation.
if (size <= THRESHOLD) {
double sum = 0;
for (int i = from; i < to; i++) {
sum += calculator.calculate(loans.get(i));
}
return sum;
}
// RECURSIVE CASE: split in half.
int middle = from + size / 2;
ParallelFineCalculation left =
new ParallelFineCalculation(loans, from, middle, calculator);
ParallelFineCalculation right =
new ParallelFineCalculation(loans, middle, to, calculator);
// CORRECT PATTERN: fork ONE and compute the other on THIS thread.
// Doing left.fork() + right.fork() + left.join() + right.join()
// wastes the current thread, which would just sit and wait.
left.fork(); // queued: another thread can steal it
double rightResult = right.compute(); // this thread works
double leftResult = left.join(); // collect the forked one
return leftResult + rightResult;
}
// --- Usage ---
public static double calculateAll(List<Loan> loans,
FineCalculator calculator) {
// The COMMON pool: shared by the whole JVM, with
// (cores - 1) threads. Parallel streams use it too (10-04).
ForkJoinPool common = ForkJoinPool.commonPool();
return common.invoke(new ParallelFineCalculation(
loans, 0, loans.size(), calculator));
}
}The common pool (ForkJoinPool.commonPool()) is an instance shared by the whole JVM, with cores - 1 threads, used by default for fork/join tasks and for CompletableFuture (08-07) and parallel streams (10-04).
Its great danger: since it is shared by the whole application, a task that blocks in it —a slow file read, a get() that waits— steals a thread from every other user of the pool. With cores - 1 threads, a handful of blocking tasks is enough to make it useless.
// FORBIDDEN in the common pool: blocking operations.
ForkJoinPool.commonPool().submit(() -> {
return Files.readAllLines(hugePath); // blocks a shared thread
});
// CORRECT: your own pool for blocking work.
ForkJoinPool ownPool = new ForkJoinPool(4);
try {
ownPool.invoke(new ParallelFineCalculation(...));
} finally {
ownPool.shutdown();
}A note on
parallelStream(). The parallel streams of the collections API use exactly thisForkJoinPool.commonPool()underneath, and turn the pattern above into a single line. They are not part of this module: they are explained in 10-04, along with the whole Streams API, where when they pay off and when they backfire is also discussed.
A note on virtual threads. All the pool-sizing arithmetic in this lesson —
Nthreads for CPU,N × (1 + wait/compute)for I/O, bounded queues, rejection policies— exists because a platform thread is expensive. Java 21's virtual threads cost nanoseconds and a few hundred bytes, and with them the recommended pattern for I/O tasks becomesExecutors.newVirtualThreadPerTaskExecutor(): one thread per task, no pool and nothing to size. For CPU tasks, on the other hand, bounded pools are still the right answer. They are explained in 10-06.
- BiblioTech: 200 notices with a pool, progress and cancellation
Everything together. It is case B of 08-01, genuinely solved.
package com.nexussoftware.bibliotech.service;
import com.nexussoftware.bibliotech.domain.Loan;
import java.util.ArrayList;
import java.util.List;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.logging.Level;
import java.util.logging.Logger;
/**
* Bulk sending of due-date notices.
*
* Properties:
* - A BOUNDED pool with named threads and a bounded queue.
* - A SEMAPHORE limiting simultaneous accesses to the notices file to 5.
* - A COUNTDOWNLATCH to show progress and wait for the end.
* - Clean cancellation through shutdownNow() -> cooperative interruption.
* - Aggregated result with atomic counters (details in 08-06).
*/
public class NoticeService implements AutoCloseable {
private static final Logger LOG = Logger.getLogger(NoticeService.class.getName());
private static final int THREADS = 8;
private static final int MAX_CONCURRENT_WRITES = 5;
private final ThreadPoolExecutor pool;
private final Semaphore fileAccess =
new Semaphore(MAX_CONCURRENT_WRITES, true);
public NoticeService() {
AtomicInteger n = new AtomicInteger(1);
this.pool = new ThreadPoolExecutor(
THREADS, THREADS,
0L, TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<>(500), // BOUNDED queue
r -> {
Thread t = new Thread(r, "bibliotech-notices-" + n.getAndIncrement());
t.setUncaughtExceptionHandler((h, e) ->
LOG.log(Level.SEVERE, "Uncaught failure in " + h.getName(), e));
return t;
},
new ThreadPoolExecutor.CallerRunsPolicy()); // backpressure
}
/** Aggregated result of the send. Immutable (08-04). */
public record SendSummary(int total, int sent, int failed,
int cancelled, long milliseconds) {
public double successRate() {
return total == 0 ? 0 : 100.0 * sent / total;
}
}
/**
* Sends all the notices in parallel.
*
* @param deadlineSeconds global deadline; on expiry the pending work is cancelled
*/
public SendSummary sendAll(List<Loan> overdue, long deadlineSeconds)
throws InterruptedException {
final int total = overdue.size();
final CountDownLatch finished = new CountDownLatch(total);
final AtomicInteger sent = new AtomicInteger();
final AtomicInteger failed = new AtomicInteger();
long start = System.nanoTime();
List<Future<?>> futures = new ArrayList<>(total);
// 1. Submit every task to the pool.
for (Loan l : overdue) {
futures.add(pool.submit(() -> {
try {
// A cancellation point before starting.
if (Thread.currentThread().isInterrupted()) {
return;
}
// The semaphore limits concurrent access to the file.
fileAccess.acquire();
try {
writeNotice(l);
sent.incrementAndGet();
} finally {
fileAccess.release(); // ALWAYS
}
} catch (InterruptedException e) {
Thread.currentThread().interrupt(); // restore (08-02)
} catch (Exception e) {
failed.incrementAndGet();
LOG.log(Level.WARNING, "Notice failed for " + l.id(), e);
} finally {
// ALWAYS, or the await() never returns.
finished.countDown();
}
}));
}
// 2. Progress while waiting, with a global deadline.
long limitNs = TimeUnit.SECONDS.toNanos(deadlineSeconds);
boolean completed = false;
while (!(completed = finished.await(300, TimeUnit.MILLISECONDS))) {
long done = total - finished.getCount();
System.out.printf("\r [%s] %d/%d notices (%.0f%%)",
bar(done, total), done, total, 100.0 * done / total);
if (System.nanoTime() - start > limitNs) {
System.out.println("\n Deadline expired: cancelling what is pending");
for (Future<?> f : futures) {
f.cancel(true); // interrupts or drops from the queue
}
break;
}
}
long ms = (System.nanoTime() - start) / 1_000_000;
int cancelled = total - sent.get() - failed.get();
System.out.printf("\r [%s] %d/%d notices (100%%)%n",
bar(total, total), total, total);
return new SendSummary(total, sent.get(), failed.get(), cancelled, ms);
}
private static String bar(long done, long total) {
int width = 30;
int filled = total == 0 ? 0 : (int) (width * done / total);
return "#".repeat(filled) + "-".repeat(width - filled);
}
/** Simulates writing the notice: 250 ms of I/O. */
private void writeNotice(Loan l) throws InterruptedException {
TimeUnit.MILLISECONDS.sleep(250);
if (l.id().hashCode() % 37 == 0) {
throw new IllegalStateException("employee has no contact address");
}
}
/** AutoCloseable: two-phase shutdown (06-06 + section 7). */
@Override
public void close() {
pool.shutdown();
try {
if (!pool.awaitTermination(30, TimeUnit.SECONDS)) {
pool.shutdownNow();
if (!pool.awaitTermination(10, TimeUnit.SECONDS)) {
LOG.severe("There are notice tasks ignoring the interruption");
}
}
} catch (InterruptedException e) {
pool.shutdownNow();
Thread.currentThread().interrupt();
}
}
}Usage:
package com.nexussoftware.bibliotech.presentation;
import java.util.List;
public class NoticeDemo {
public static void main(String[] args) throws InterruptedException {
List<Loan> overdue = generateOverdue(200);
System.out.println("=== SENDING DUE-DATE NOTICES ===");
System.out.println("Pending notices: " + overdue.size());
System.out.println("Estimated sequential time: "
+ (overdue.size() * 250 / 1000) + " s");
System.out.println();
// try-with-resources: close() shuts the pool down in an orderly way.
try (NoticeService service = new NoticeService()) {
NoticeService.SendSummary r = service.sendAll(overdue, 60);
System.out.println();
System.out.println("=== SUMMARY ===");
System.out.println("Total : " + r.total());
System.out.println("Sent : " + r.sent());
System.out.println("Failed : " + r.failed());
System.out.println("Cancelled : " + r.cancelled());
System.out.println("Time : " + r.milliseconds() + " ms");
System.out.printf ("Success : %.1f%%%n", r.successRate());
System.out.printf ("Speed-up : %.1fx over sequential sending%n",
(overdue.size() * 250.0) / r.milliseconds());
}
}
}Output:
=== SENDING DUE-DATE NOTICES ===
Pending notices: 200
Estimated sequential time: 50 s
[##############################] 200/200 notices (100%)
=== SUMMARY ===
Total : 200
Sent : 194
Failed : 6
Cancelled : 0
Time : 10247 ms
Success : 97.0%
Speed-up : 4.9x over sequential sendingAn honest analysis of the result, which is where the learning is:
- The speed-up is 4.9x and not 8x, even though the pool has 8 threads. The cause is the semaphore with 5 permits: only 5 notices can write at a time, so the real parallelism is limited by that 5, not by the 8 threads. 200 × 250 ms / 5 ≈ 10 s, exactly what was measured. The bottleneck is not the pool: it is the protected resource, and this is the norm, not the exception, in real systems.
- If you raise the semaphore to 8, the speed-up rises to ~8x and the time drops to 6.3 s. If you raise it to 20 without raising the threads, nothing changes: the limit becomes the pool. Sizing is about finding which resource is the scarce one.
- The 6 failures did not stop the send. Each task catches its own exceptions and calls
countDown()in thefinally: the degradation policy of 06-07, now in concurrency. - Eight threads, not two hundred. 8 MB of stacks instead of 200 MB, and eight thread creations instead of two hundred.
Common Mistakes and Tips
Mistake 1: not shutting the executor down. The pool's threads are non-daemon; the JVM does not terminate. It is the number-one cause of "my program does not finish".
Mistake 2: using newFixedThreadPool with external load. Unbounded queue → OutOfMemoryError after hours of apparent normality. Bounded queue and a rejection policy.
Mistake 3: using newCachedThreadPool with many slow tasks. It creates threads without limit until it takes down the JVM.
Mistake 4: submitting with submit and never calling get(). The task's exception is stored in the Future and appears nowhere at all. A total silent failure. Use execute if you do not care about the result, or check the Future.
Mistake 5: get() with no deadline in production. If the task hangs, the caller hangs. Always get(time, unit).
Mistake 6: not unwrapping ExecutionException. The real exception is in getCause(). Logging the ExecutionException as it comes hides the useful information.
Mistake 7: letting a scheduleAtFixedRate task throw an exception. The schedule is cancelled forever, silently. Wrap the body in try/catch (Exception).
Mistake 8: countDown() outside the finally. If the task fails first, whoever is waiting blocks indefinitely.
Mistake 9: a semaphore's release() outside the finally. Lost permits are never recovered; after a few failures the semaphore sits at zero and nobody ever gets through again.
Mistake 10: believing maximumPoolSize is reached with an unbounded queue. The pool queues before creating extra threads: with an unbounded queue it never goes above corePoolSize.
Mistake 11: blocking in ForkJoinPool.commonPool(). It is shared by the whole JVM and has cores - 1 threads. A few blocking tasks make it useless for everybody.
Mistake 12: using a single pool for computation and for I/O. They get in each other's way. Separate pools per type of work.
Tip 1: name the threads with a ThreadFactory. pool-1-thread-3 says nothing in a dump; bibliotech-notices-3 does. It is the 08-02 tip applied to pools.
Tip 2: use CallerRunsPolicy when you cannot lose tasks. It creates automatic backpressure: the producer slows itself down.
Tip 3: measure before sizing. The formulas are starting points. The real size depends on your load, your hardware and your external dependencies.
Tip 4: separate pools by type of work (bulkheads). Isolate the failures: do not let notice saturation stop a report being generated.
Tip 5: a single-thread executor is free confinement. Everything running in it is serialised, so its state needs no locks. Very useful for file writes and ordered logs.
Tip 6: remember that cancel(true) is interrupt(). Without cooperative tasks (08-02), cancellation does not work, whether you are in a pool or not.
Exercises
Exercise 1: Pool comparison
Write PoolComparison which runs the same load —300 tasks each sleeping 100 ms, simulating I/O— with five strategies: (a) one thread per task by hand, (b) newFixedThreadPool(4), (c) newFixedThreadPool(32), (d) newCachedThreadPool(), and (e) a bespoke ThreadPoolExecutor with 16 threads, a queue of 50 and CallerRunsPolicy. For each one measure the total time with System.nanoTime() and count how many distinct threads ran tasks (using a synchronised Set of thread names). Print a table and explain the results.
Exercise 2: Import with Callable, deadline and cancellation
Rewrite BiblioTech's catalogue import as a Callable<Report> run in an ExecutorService. The program must:
- Launch the import and carry on showing a simulated menu.
- Show the progress while
!future.isDone(). - Apply a 5-second deadline with
get(5, TimeUnit.SECONDS). - On the deadline expiring, call
cancel(true)and verify withisCancelled(). - Distinguish the three endings in the
catch:ExecutionException(unwrapping the cause),TimeoutExceptionandInterruptedException. - Shut the executor down with the two-phase pattern.
Exercise 3: Reading room with a semaphore and latches
Model BiblioTech's reading room: it holds 4 employees at a time and there are 20 employees wanting to get in. Write ReadingRoom with:
- A
Semaphore(4, true)controlling the occupancy, withenter()/leave()and thereleasein afinally. - A "starting gate"
CountDownLatchmaking the 20 threads try to enter at exactly the same time, and another "finish line" one to wait for everybody to have finished. - An atomic counter of the instantaneous occupancy, and a check that fails loudly if it ever exceeds 4.
- A
ScheduledExecutorServiceprinting the current occupancy every 200 ms for the duration of the simulation, and shutting down at the end.
Each employee stays in the room for a random time between 200 and 600 ms.
Solutions
Solution to Exercise 1
import java.util.Collections;
import java.util.HashSet;
import java.util.Set;
import java.util.concurrent.*;
public class PoolComparison {
static final int TASKS = 300;
static final long DURATION_MS = 100;
/** A simulated I/O task that records which thread ran it. */
static Runnable task(Set<String> threadsUsed) {
return () -> {
threadsUsed.add(Thread.currentThread().getName());
try {
TimeUnit.MILLISECONDS.sleep(DURATION_MS);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
};
}
record Result(String strategy, long ms, int threads) { }
/** (a) One thread per task, managed by hand. */
static Result threadPerTask() throws InterruptedException {
Set<String> threads = Collections.synchronizedSet(new HashSet<>());
Thread[] ts = new Thread[TASKS];
long start = System.nanoTime();
for (int i = 0; i < TASKS; i++) {
ts[i] = new Thread(task(threads), "manual-" + i);
ts[i].start();
}
for (Thread t : ts) t.join();
return new Result("thread per task",
(System.nanoTime() - start) / 1_000_000, threads.size());
}
/** (b)-(e) Any ExecutorService. */
static Result withExecutor(String name, ExecutorService ex)
throws InterruptedException {
Set<String> threads = Collections.synchronizedSet(new HashSet<>());
long start = System.nanoTime();
for (int i = 0; i < TASKS; i++) {
ex.execute(task(threads));
}
ex.shutdown();
ex.awaitTermination(2, TimeUnit.MINUTES);
return new Result(name,
(System.nanoTime() - start) / 1_000_000, threads.size());
}
public static void main(String[] args) throws InterruptedException {
System.out.printf("%d tasks of %d ms of simulated I/O%n", TASKS, DURATION_MS);
System.out.printf("Sequential would be: %d ms%n%n", TASKS * DURATION_MS);
Result[] results = {
threadPerTask(),
withExecutor("fixed(4)", Executors.newFixedThreadPool(4)),
withExecutor("fixed(32)", Executors.newFixedThreadPool(32)),
withExecutor("cached", Executors.newCachedThreadPool()),
withExecutor("bespoke (16, queue 50, CallerRuns)",
new ThreadPoolExecutor(16, 16, 0L, TimeUnit.MILLISECONDS,
new ArrayBlockingQueue<>(50),
new ThreadPoolExecutor.CallerRunsPolicy()))
};
System.out.printf("%-38s | %8s | %8s%n", "Strategy", "ms", "threads");
System.out.println("---------------------------------------|----------|---------");
for (Result r : results) {
System.out.printf("%-38s | %8d | %8d%n", r.strategy(), r.ms(), r.threads());
}
}
}Indicative output (8-core machine):
300 tasks of 100 ms of simulated I/O
Sequential would be: 30000 ms
Strategy | ms | threads
---------------------------------------|----------|---------
thread per task | 147 | 300
fixed(4) | 7621 | 4
fixed(32) | 1024 | 32
cached | 163 | 300
bespoke (16, queue 50, CallerRuns) | 1938 | 17Analysis, row by row:
- One thread per task (147 ms) is the fastest. With 300 purely waiting tasks, having 300 threads waiting at once is optimal in time. But that is 300 MB of reserved stacks; with 30,000 tasks,
OutOfMemoryError. Fast and unscalable. fixed(4)(7.6 s) is the slowest: only 4 tasks at a time, 300/4 = 75 rounds × 100 ms. Sizing an I/O pool as if it were a CPU pool is the classic mistake.fixed(32)(1.0 s) is a good balance: 32 threads, 32 MB, and nearly 30x speed-up.cached(163 ms) created 300 threads, just like the manual version: itsSynchronousQueuestores nothing, so every task arriving while all are busy triggers a new thread. Fast here, and a bomb under a heavier load.- The bespoke pool (1.9 s) used 17 threads: the pool's 16 plus
main's thread. That extra thread isCallerRunsPolicyin action: when the queue of 50 filled up,mainran tasks itself. That is backpressure working —the producer slowed itself down— and it is why memory never grew.
Solution to Exercise 2
package com.nexussoftware.bibliotech.persistence;
import java.nio.file.Path;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
import java.util.logging.Level;
import java.util.logging.Logger;
public class ImportWithDeadline {
private static final Logger LOG = Logger.getLogger(ImportWithDeadline.class.getName());
/** Immutable report of the result. */
public record Report(int imported, int discarded, long ms) { }
/**
* The import as a Callable<Report>: it returns a value and can
* throw checked exceptions. It is cancellable: it checks for
* interruption on every line (08-02).
*/
static class ImportTask implements Callable<Report> {
private final Path file;
private final AtomicInteger progress = new AtomicInteger();
ImportTask(Path file) { this.file = file; }
int progress() { return progress.get(); }
@Override
public Report call() throws Exception {
long start = System.nanoTime();
int imported = 0, discarded = 0;
for (int line = 0; line < 50_000; line++) {
// CANCELLATION POINT: the Future's cancel(true) turns
// into interrupt() on this thread.
if (Thread.currentThread().isInterrupted()) {
throw new InterruptedException(
"import cancelled at line " + line);
}
if (line % 97 == 0) discarded++; else imported++;
progress.incrementAndGet();
if (line % 1000 == 0) {
TimeUnit.MILLISECONDS.sleep(30); // simulates heavy validation
}
}
return new Report(imported, discarded,
(System.nanoTime() - start) / 1_000_000);
}
}
public static void main(String[] args) {
ExecutorService executor = Executors.newSingleThreadExecutor(
r -> new Thread(r, "bibliotech-importer"));
ImportTask task = new ImportTask(Path.of("data/inventory.csv"));
Future<Report> future = executor.submit(task);
try {
// 1-2. The main thread stays alive: it serves the menu and paints progress.
System.out.println("[main] import launched; the menu is still active");
while (!future.isDone()) {
System.out.printf("\r[main] progress: %d lines", task.progress());
TimeUnit.MILLISECONDS.sleep(300);
}
// 3. A 5-second deadline.
Report report = future.get(5, TimeUnit.SECONDS);
System.out.printf("%n[main] COMPLETED: %d imported, %d discarded, %d ms%n",
report.imported(), report.discarded(), report.ms());
} catch (TimeoutException e) {
// 4. The deadline does NOT cancel on its own: you have to ask.
System.out.printf("%n[main] deadline expired after %d lines; cancelling%n",
task.progress());
boolean requested = future.cancel(true);
System.out.println("[main] cancel() returned : " + requested);
System.out.println("[main] isCancelled() : " + future.isCancelled());
System.out.println("[main] the catalogue keeps its previous content");
} catch (ExecutionException e) {
// 5. The task failed: the real cause is in getCause() (06-03).
Throwable cause = e.getCause();
if (cause instanceof InterruptedException) {
System.out.println("\n[main] the import stopped because of cancellation");
} else {
LOG.log(Level.SEVERE, "The import failed", cause);
System.out.println("\n[main] failure: " + cause.getMessage());
}
} catch (InterruptedException e) {
// WE have been interrupted; the task is still alive.
System.out.println("\n[main] wait interrupted");
future.cancel(true);
Thread.currentThread().interrupt();
} finally {
// 6. Two-phase shutdown.
executor.shutdown();
try {
if (!executor.awaitTermination(5, TimeUnit.SECONDS)) {
executor.shutdownNow();
if (!executor.awaitTermination(5, TimeUnit.SECONDS)) {
LOG.severe("The importer ignores the interruption");
}
}
} catch (InterruptedException e) {
executor.shutdownNow();
Thread.currentThread().interrupt();
}
System.out.println("[main] executor shut down; the JVM can terminate");
}
}
}Output (when the deadline expires):
[main] import launched; the menu is still active
[main] progress: 41893 lines
[main] deadline expired after 42017 lines; cancelling
[main] cancel() returned : true
[main] isCancelled() : true
[main] the catalogue keeps its previous content
[main] executor shut down; the JVM can terminateThree points:
TimeoutExceptioncancels nothing on its own. The task was still happily running;cancel(true)had to be called. This is the most frequent expectation mistake withFuture.cancel(true)worked because the task cooperates. Without theisInterrupted()check in the loop, the import would have run to the end and the executor could not have been shut down.- The two-phase shutdown at the end guarantees the JVM can terminate. Without it, the
bibliotech-importerthread would still be alive.
Solution to Exercise 3
package com.nexussoftware.bibliotech.service;
import java.util.concurrent.*;
import java.util.concurrent.atomic.AtomicInteger;
public class ReadingRoom {
private static final int CAPACITY = 4;
private static final int EMPLOYEES = 20;
// A FAIR semaphore: those who have waited longest get in first,
// which prevents starvation (08-04, section 12).
private final Semaphore seats = new Semaphore(CAPACITY, true);
// Instantaneous occupancy, to check the invariant.
private final AtomicInteger inside = new AtomicInteger();
private final AtomicInteger maxObserved = new AtomicInteger();
private volatile boolean invariantBroken = false;
/** An employee enters, stays a while and leaves. */
public void visit(String employee) throws InterruptedException {
seats.acquire(); // blocks if 4 are already inside
try {
int current = inside.incrementAndGet();
// Recording the maximum observed, with a CAS loop (08-06).
maxObserved.updateAndGet(m -> Math.max(m, current));
if (current > CAPACITY) {
invariantBroken = true;
System.err.println("!!! INVARIANT BROKEN: " + current + " inside !!!");
}
System.out.printf(" [+] %-12s enters (inside=%d, waiting=%d)%n",
employee, current, seats.getQueueLength());
TimeUnit.MILLISECONDS.sleep(
ThreadLocalRandom.current().nextInt(200, 601));
System.out.printf(" [-] %-12s leaves (inside=%d)%n",
employee, inside.get() - 1);
} finally {
inside.decrementAndGet();
// MANDATORY in finally: a lost permit never comes back,
// and after four losses the room would be closed forever.
seats.release();
}
}
public static void main(String[] args) throws InterruptedException {
ReadingRoom room = new ReadingRoom();
// Two latches: the "starting pistol" and the "finish line".
CountDownLatch startGate = new CountDownLatch(1);
CountDownLatch finishLine = new CountDownLatch(EMPLOYEES);
// Periodic occupancy monitor.
ScheduledExecutorService monitor = Executors.newSingleThreadScheduledExecutor(
r -> {
Thread t = new Thread(r, "bibliotech-occupancy-monitor");
t.setDaemon(true);
return t;
});
monitor.scheduleAtFixedRate(() -> {
try {
System.out.printf(" [monitor] occupancy=%d/%d queue=%d%n",
room.inside.get(), CAPACITY, room.seats.getQueueLength());
} catch (Exception e) {
// An uncaught exception WOULD CANCEL the schedule
// forever and silently.
e.printStackTrace();
}
}, 200, 200, TimeUnit.MILLISECONDS);
// 20 threads that wait for the gate and start at once.
for (int i = 1; i <= EMPLOYEES; i++) {
final String name = "employee-" + String.format("%02d", i);
new Thread(() -> {
try {
startGate.await(); // everybody waits here
room.visit(name);
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
} finally {
finishLine.countDown(); // ALWAYS
}
}, name).start();
}
System.out.println("=== READING ROOM (capacity " + CAPACITY + ") ===");
TimeUnit.MILLISECONDS.sleep(200); // let everybody reach await()
long start = System.nanoTime();
startGate.countDown(); // all at once!
finishLine.await();
long ms = (System.nanoTime() - start) / 1_000_000;
monitor.shutdown();
monitor.awaitTermination(2, TimeUnit.SECONDS);
System.out.println();
System.out.println("=== SUMMARY ===");
System.out.println("Employees served : " + EMPLOYEES);
System.out.println("Maximum occupancy : " + room.maxObserved.get()
+ " (limit " + CAPACITY + ")");
System.out.println("Invariant respected : " + !room.invariantBroken);
System.out.println("Total time : " + ms + " ms");
System.out.println("Inside at the end : " + room.inside.get());
System.out.println("Permits available : " + room.seats.availablePermits()
+ " (must be " + CAPACITY + ")");
}
}Output (fragment):
=== READING ROOM (capacity 4) ===
[+] employee-03 enters (inside=1, waiting=16)
[+] employee-07 enters (inside=2, waiting=15)
[+] employee-01 enters (inside=3, waiting=14)
[+] employee-12 enters (inside=4, waiting=13)
[monitor] occupancy=4/4 queue=16
[-] employee-03 leaves (inside=3)
[+] employee-05 enters (inside=4, waiting=15)
...
=== SUMMARY ===
Employees served : 20
Maximum occupancy : 4 (limit 4)
Invariant respected : true
Total time : 2143 ms
Inside at the end : 0
Permits available : 4 (must be 4)Four observations:
- The maximum occupancy observed is exactly 4, never 5, on any run. The semaphore honours its contract.
- The permits available at the end are back to 4. That is the check that no
release()was lost. If you remove thefinallyand provoke an exception inside, you will see that number drop and the room end up closed. - The starting gate with
CountDownLatchmaximises contention. Without it, the first employees would have entered and left before the last ones started, and there would have been no queue. It is the correct technique for a concurrency test. - The 2.1 seconds total add up: 20 employees × ~400 ms average / 4 at a time ≈ 2 s. The occupancy, not the number of threads, determines the time — the same lesson as the notices semaphore in section 17.
Conclusion
You have changed level of abstraction: you no longer manage threads, you describe tasks.
You know why java.util.concurrent replaces manual management: a thread per task does not scale, there is no limit stopping ten thousand being created, threads are not reused, Runnable returns nothing, exceptions never reach the caller and cancelling means walking an array. All six problems disappear with an executor.
You know the trinity Executor / ExecutorService / Executors and the difference between execute —the exception is visible— and submit —the exception is stored in the Future and vanishes if nobody calls get()—, which is one of the costliest traps of the topic. You know what each Executors factory does, and above all you know the warning about unbounded queues: newFixedThreadPool piles tasks up until OutOfMemoryError, and newCachedThreadPool creates threads without limit to the same end by another route. That is why you know how to build a bespoke ThreadPoolExecutor, with its six parameters, its ThreadFactory that names the threads, its bounded queue and its rejection policy —with CallerRunsPolicy as the most useful, because it turns saturation into automatic backpressure—. And you understand the pool's counter-intuitive algorithm: it queues before creating extra threads, so maximumPoolSize is worthless if the queue is unbounded.
You know how to size: N for computation, N × (1 + wait/compute) for I/O, and —the advice that pays most— separate pools per type of work, the bulkhead isolation that stops one part's saturation sinking the rest. And you know how to shut down an executor with the two-phase pattern —shutdown, wait, shutdownNow, wait, log—, remembering that if you do not shut it down the JVM does not terminate, and that shutdownNow() interrupts but does not kill: a task that swallows the InterruptedException stays alive, and the whole 08-02 protocol comes due here.
Callable and Future solve what was left open in 08-02: a task that returns a value and propagates checked exceptions. With the three get() exceptions clearly told apart —ExecutionException which wraps the real cause in getCause(), TimeoutException which cancels nothing and leaves the task running, and InterruptedException which affects you and not the task—, the rule of never using get() without a deadline in production, and cancel(true) which is not magic but an interrupt() on the pool's thread. Plus invokeAll for batches with a global deadline and invokeAny to keep the fastest source.
You know ScheduledExecutorService and the difference between scheduleAtFixedRate —period from the start, constant rhythm— and scheduleWithFixedDelay —period from the end, a guaranteed real gap—, with the warning that ruins systems: an uncaught exception in a periodic task cancels the schedule forever, silently.
And you have the synchronisers: CountDownLatch, single-use, for waiting on N events and for starting N threads at once, with the countDown() always in a finally; CyclicBarrier, reusable, with its barrier action and its breakage for everybody if one participant fails; Semaphore for putting a hard cap on concurrency over a resource, with the release() always in a finally because a lost permit never returns; and Exchanger and Phaser recognised even though they are rarely needed. Plus ForkJoinPool with RecursiveTask, its work stealing, the correct fork() + compute() + join() pattern, the critical importance of the threshold, and the warning about the common pool: it is shared by the whole JVM and blocking it harms everybody.
BiblioTech sends its 200 notices in ten seconds with eight threads, with a progress bar through CountDownLatch, with a semaphore limiting simultaneous file accesses to five, with a global deadline and clean cancellation, with every individual failure logged without stopping the batch, and with AutoCloseable so that try-with-resources shuts the pool down. And the analysis of the result teaches the most valuable lesson of the section: the speed-up was 4.9x and not 8x because the bottleneck was not the pool but the five-permit semaphore. Sizing well is about finding which resource is the scarce one — and it is almost never the one you think.
But a crack remains. NoticeService uses AtomicInteger for its counters without them ever having been explained, and the Catalog you made safe in 08-04 is still based on a HashMap protected by locks: every read pays for a lock(), even though reading gets in nobody's way. And the module 5 ReservationQueue is still an ArrayDeque that two threads would corrupt, with the producer-consumer pattern promised back then and still not implemented.
In the next lesson, Concurrent Collections and Atomic Variables, that crack is closed. You will see a real demonstration of a HashMap corrupted by two threads —with the infinite loop that can pin a core at 100% forever— and the ConcurrentModificationException of the 05-02 fail-fast; the three generations of solution, from Collections.synchronizedMap with its trap —iterating still requires manual synchronisation and compound operations are still not atomic— to ConcurrentHashMap with its atomic compound operations putIfAbsent, computeIfAbsent, compute and merge, which are the correct way of solving check-then-act; CopyOnWriteArrayList for event listeners; the BlockingQueues in all their variants and, at last, the complete implementation of the producer-consumer pattern promised in module 5, applied to the reservation queue and with a poison pill for shutdown; and the atomic variables with the compare-and-swap operation explained from the inside, its retry loop, the ABA problem and LongAdder for counters under heavy contention. By the end you will have the definitive decision table: when synchronized, when a Lock, when an atomic, when a concurrent collection and when simply an immutable object.
Java Programming Course
Module 1: Introduction to Java
- Introduction to Java
- Setting Up the Development Environment
- Basic Syntax and Structure
- Variables and Data Types
- Operators
- Console Input and Output
- Your First Complete Program: BiblioTech
Module 2: Control Flow
- Conditional Statements
- Loops
- Switch Statements
- Break and Continue
- Debugging and Execution Traces
- Project: The BiblioTech Interactive Menu
Module 3: Object-Oriented Programming
- Introduction to OOP
- Classes and Objects
- Methods
- Constructors
- Inheritance
- Polymorphism
- Encapsulation
- Abstraction
- The Object Class: equals, hashCode and toString
Module 4: Advanced Object-Oriented Programming
- Interfaces
- Abstract Classes
- Inner Classes
- Anonymous Classes
- Lambda Expressions
- Functional Interfaces and Method References
- Enums and Records
Module 5: Data Structures and Collections
- Arrays
- The Collections Framework
- ArrayList
- LinkedList
- HashMap
- HashSet
- Queue and Deque
- Stack
- Sorting and Searching Collections
Module 6: Exception Handling
- Introduction to Exceptions
- The Try-Catch Block
- Throw and Throws
- Custom Exceptions
- The Finally Block
- Try-with-resources and AutoCloseable
- Error Handling Strategies and Logging
Module 7: File Input/Output
- Reading Files
- Writing Files
- File Streams
- BufferedReader and BufferedWriter
- Serialization
- The NIO.2 API: Path and Files
- Interchange Formats: CSV and Properties
Module 8: Multithreading and Concurrency
- Introduction to Multithreading
- Creating Threads
- Thread Lifecycle
- Synchronization
- Concurrency Utilities
- Concurrent Collections and Atomic Variables
- Asynchronous Tasks with CompletableFuture
Module 9: Networking
- Introduction to Networking
- Sockets
- ServerSocket
- DatagramSocket and DatagramPacket
- URL and HttpURLConnection
- The Modern HTTP Client
Module 10: Advanced Topics
- Generics
- Annotations
- Reflection
- Java 8 Features: Streams and Optional
- Dates and Times with java.time
- Java 9 and Beyond
- Memory, Garbage Collection and Performance
Module 11: Java Frameworks and Libraries
- Introduction to Java Frameworks
- Spring Framework
- Hibernate
- JUnit
- Maven
- Advanced Testing with Mockito
- Essential Ecosystem Libraries
