JEP 543: Structured Concurrency (Java 28)

Simplify concurrent programming by introducing an API for structured concurrency, which treats groups of related tasks running in different threads as single units of work. This streamlines error handling and cancellation, improves reliability, and enhances observability.

Structured Concurrency was first delivered as an incubator module via JEP 428 (JDK 19) and then JEP 437 (JDK 20). It became a preview API via JEP 453 (JDK 21), with the fork method changed to return a Subtask rather than a Future. It re-previewed in that form via JEP 462 (JDK 22), JEP 480 (JDK 23), and JEP 499 (JDK 24). It re-previewed again via JEP 505 (JDK 25) with several API changes; most notably, the public constructors of StructuredTaskScope were replaced with static factory methods. It re-previewed again via JEP 525 (JDK 26) with a few minor changes, and again via JEP 533 (JDK 27) with an update to add a type parameter for the exception thrown by the join() method.

We here propose to finalize Structured Concurrency in JDK 28, without further change.

Promote a style of concurrent programming that can eliminate common risks arising from cancellation and shutdown, such as thread leaks and cancellation delays.

Improve the observability of concurrent code.

It is not a goal to replace any of the concurrency constructs in the java.util.concurrent package, such as ExecutorService or Future.

It is not a goal to create the definitive structured concurrency API for all Java programs. Other structured concurrency constructs can be defined by third-party libraries or in future JDK releases.

It is not a goal to define a means of sharing streams of data among threads (i.e., channels). We might propose to do so in the future.

It is not a goal to replace the existing thread interruption mechanism with a new thread cancellation mechanism. We might propose to do so in the future.

We manage complexity in programs by breaking tasks down into multiple subtasks. In ordinary single-threaded code, the subtasks execute sequentially. However, if the subtasks are sufficiently independent of each other, and if there are sufficient hardware resources, then the overall task can be made to run faster, i.e., with lower latency, by executing the subtasks concurrently. For example, a task that composes the results of multiple I/O operations will run faster if each I/O operation executes concurrently in its own thread. Virtual threads (JEP 444) make it cost-effective to dedicate a thread to every such I/O operation, but managing the huge number of threads that can result remains a challenge.

The java.util.concurrent.ExecutorService API, introduced in Java 5, can execute subtasks concurrently.

For example, here is a method, handle(), that represents a task in a server application. It handles an incoming request by submitting two subtasks to an ExecutorService. One subtask executes the method findUser(), while the other executes the method fetchOrder(). The ExecutorService immediately returns a Future for each subtask, and executes the subtasks concurrently according to its scheduling policy. The handle() method awaits the subtasks' results via blocking calls to their futures' get() methods, so the task is said to join its subtasks.

Response handle() throws ExecutionException, InterruptedException { Future<String> user = executor.submit(() -> findUser()); Future<Integer> order = executor.submit(() -> fetchOrder()); String theUser = user.get(); // Join findUser int theOrder = order.get(); // Join fetchOrder return new Response(theUser, theOrder); }

Because the subtasks execute concurrently, each subtask can succeed or fail independently. (Failure, in this context, means to throw an exception.) Often, a task such as handle() should fail if any of its subtasks fail. Understanding the lifetimes of the threads can be surprisingly complicated when failure occurs:

If findUser() throws an exception then handle() will throw an exception when calling user.get(), but fetchOrder() will continue to run in its own thread. This is a thread leak which, at best, wastes resources; at worst, the fetchOrder() thread will interfere with other tasks.

If the thread executing handle() is interrupted, the interruption will not propagate to the subtasks. Both the findUser() and fetchOrder() threads will leak, continuing to run even after handle() has failed.

If findUser() takes a long time to execute, but fetchOrder() fails in the meantime, then handle() will wait unnecessarily for findUser() by blocking on user.get() rather than cancelling it. Only after findUser() completes and user.get() returns will order.get() throw an exception, causing handle() to fail.

In each case, the problem is that our program is logically structured with task-subtask relationships, but these relationships exist only in our minds.

This not only creates more room for error, but it makes diagnosing and troubleshooting such errors more difficult. Observability tools such as thread dumps, for example, will show handle(), findUser(), and fetchOrder() on the call stacks of unrelated threads, with no hint of the task-subtask relationship.

We might attempt to do better by explicitly cancelling other subtasks when an error occurs, for example by wrapping tasks with try-finally and calling the cancel(boolean) methods of the futures of the other tasks in the catch block for the failing task. We would also need to use the ExecutorService inside a try-with-resources statement, as shown in the examples in JEP 444, because Future does not offer a way to wait for a task that has been cancelled. But all this keeping track of the inter-task relationships can be tricky to get right, and it often makes the logical intent of the code harder to discern.

This need to manually coordinate lifetimes is due to the fact that ExecutorService and Future allow unrestricted patterns of concurrency. There are no constraints upon, or ordering of, any of the threads involved. One thread can create an ExecutorService, a second thread can submit work to it, and the threads which execute the work have no relationship to either the first or second thread. Moreover, after a thread has submitted work, a completely different thread can await the results of execution. Any code with a reference to a Future can join it, i.e., await its result by calling get() — even code in a thread other than the one which obtained the Future. In effect, a subtask started by one task does not have to return to the task that submitted it. It could return to any of a number of tasks — or possibly none.

Because ExecutorService and Future allow for such unstructured use, they do not enforce or even track relationships among tasks and subtasks, even though such relationships are common and useful. Accordingly, even when subtasks are submitted and joined in the same task, the failure of one subtask cannot automatically cause the cancellation of another: In the above handle() method, the failure of fetchOrder() cannot automatically cause the cancellation of findUser(). The future for fetchOrder() is unrelated to the future for findUser(), and neither is related to the thread that will ultimately join it via its get() method.

Rather than manage such cancellation manually, we should aim to reliably automate it.

In contrast to the freewheeling assortment of threads under ExecutorService, the execution of single-threaded code always enforces a hierarchy of tasks and subtasks. The body block {...} of a method corresponds to a task, and the methods invoked within the block correspond to subtasks. An invoked method must either return to, or throw an exception to, the method that invoked it. An invoked method cannot outlive the method that invoked it, nor can it return to or throw an exception to a different method. Thus all subtasks finish before the task, each subtask is a child of its parent, and the lifetime of each subtask relative to the others and to the task is governed by the syntactic block structure of the code.

For example, in this single-threaded version of handle(), the task-subtask relationship is apparent from the syntactic structure:

Response handle() throws IOException { String theUser = findUser(); int theOrder = fetchOrder(); return new Response(theUser, theOrder); }

We do not start the fetchOrder() subtask until the findUser() subtask has completed, whether successfully or unsuccessfully. If findUser() fails then we do not start fetchOrder() at all, and the handle() task fails implicitly. The fact that a subtask can return only to its parent is significant: It implies that the parent task can implicitly treat the failure of one subtask as a trigger to cancel other unfinished subtasks and then fail itself.

In single-threaded code, the task-subtask hierarchy is reified in the call stack at run time. We thus get the corresponding parent-child relationships, which govern error propagation, for free. When observing a single thread, the hierarchical relationship is obvious: findUser(), and later fetchOrder(), appear subordinate to handle(). This makes it easy to answer the question, "what is handle() working on now?"

Concurrent programming would be easier, more reliable, and more observable if the parent-child relationships between tasks and their subtasks were evident from the syntactic structure of the code and also reified at run time — just as for single-threaded code. The syntactic structure would delineate the lifetimes of subtasks and enable a runtime representation of the inter-thread hierarchy, analogous to the intra-thread call stack. That representation would enable error propagation and cancellation as well as meaningful observation of the concurrent program.

(The Java Platform already has an API for imposing structure on concurrent tasks, namely java.util.concurrent.ForkJoinPool, which is the execution engine behind parallel streams. However, that API is designed for compute-intensive tasks rather than tasks which involve I/O.)

Structured concurrency is an approach to concurrent programming that preserves the natural relationship between tasks and subtasks, which leads to more readable, maintainable, and reliable concurrent code. The term "structured concurrency" was coined by Martin Sústrik and popularized by Nathaniel J. Smith. Ideas from other languages, such as Erlang's hierarchical supervisors, inform the design of error handling in structured concurrency.

Structured concurrency derives from a simple principle: If a task splits into concurrent subtasks then they all return to the same place, namely the task's code block.

In structured concurrency, subtasks work on behalf of a task. The task awaits the subtasks' results and monitors them for failures. As with structured programming techniques for code in a single thread, the power of structured concurrency for multiple threads comes from two ideas: The flow of execution through a block of code has well-defined entry and exit points, and the lifetimes of operations are nested in a way that mirrors their syntactic nesting in the code.

Because the entry and exit points of a block of code are well defined, the lifetime of a concurrent subtask is confined to the syntactic block of its parent task. Because the lifetimes of sibling subtasks are nested within that of their parent task, they can be reasoned about and managed as a unit. Because the lifetime of the parent task is, in turn, nested within that of its parent, the runtime can reify the hierarchy of tasks into a tree that is the concurrent counterpart of the call stack of a single thread. This allows policies, such as deadlines, to be applied to entire sub-trees of tasks, and allows observability tools to present subtasks as subordinate to their parent tasks.

Structured concurrency is a great match for virtual threads, which are lightweight threads implemented by the JDK. Virtual threads share operating-system threads, allowing for enormous numbers of virtual threads. In addition