Multicore TypeScript, Inspired by Rust's Rayon

Multicore TypeScript, inspired by Rust's Rayon. Wrap an array in par() and its map, filter and reduce run on every core when GeaStack compiles your program, with the same result Node gives. In our benchmarks it runs 11% faster than Rust + Rayon.

GeaStack ×20

1.00× Rust + Rayon ×20

1.11× C++ threads ×20

1.18×

Run time at 20 threads relative to GeaStack, averaged over nine workloads (geometric mean). Lower is better.

A desktop CPU today has a dozen or more cores, and a JavaScript program uses one of them. The language has no shared-memory threads. The way to use a second core is a Worker, which runs in its own heap: every object it needs is copied in by message, and every result is copied back out. SharedArrayBuffer shares raw bytes, not objects, so an array of records or a tree of class instances can't be shared at all. For most work, the copying costs more than the second core saves, and most JavaScript stays on one thread.

Rust solved the same problem from the other side. With Rayon, turning a loop parallel is often a one-word change, iter() to par_iter(), and the compiler rejects the change if two threads could race on the same data. We wanted that for TypeScript: ordinary code over ordinary objects, spread across every core, with the compiler checking that it is safe.

GeaStack compiles TypeScript to native code, where threads can share one heap. That makes it possible, and @geastack/parallel is the library that does it:

npm install @geastack/parallel

import { par, range, sorted } from '@geastack/parallel'

const lengths = par(words).map((word) => word.length)

const primeCount = range(2, 10_000_000).mapReduce( (n) => (isPrime(n) ? 1 : 0), (a, b) => a + b, 0 )

const ranked = sorted(players, (a, b) => b.score - a.score)

Each call splits its input across every CPU core. The callbacks are ordinary closures and can read any variable in scope.

@geastack/parallel is plain TypeScript. Under Node it runs one element at a time, so you test and debug it as usual. Compiled with GeaStack, the same file runs on every core and returns exactly the Node result, down to the last bit of a floating-point sum.

Each operation cuts its input into at most 1024 pieces whose boundaries depend only on the input's length, processes each piece in order and combines the pieces in order. That is the loop Node runs. Threads only decide which pieces run where. So map and filter keep input order, findIndex returns the lowest matching index, a floating-point reduce adds in the same order on one thread or 20, and when several callbacks throw, you get the exception a plain loop would have hit first.

Rust rules out data races with ownership types. TypeScript has nothing like them, so the GeaStack compiler checks each callback passed to the library, and every function it calls, and refuses one whose result could depend on which thread ran first:

let total = 0 par(values).map((x) => { total += x })

parallel region assigns total, a variable that existed before the parallel region (first at main.ts:7:3)

Writing to an array or object that existed before the call, console.log, Math.random() and Date.now() are refused the same way, each with the source position of the offending line. The fix is almost always to return the value instead of storing it: par(values).reduce((a, b) => a + b, 0). Reading and allocating are always allowed, so callbacks can build strings, arrays, class instances and whole trees and return them.

We wrote nine workloads in TypeScript against the library, then ported each one to Rust on Rayon and to C++ on a std::thread pool. On a 20-thread AMD Ryzen 9 9900X3D desktop, running Ubuntu under WSL2, GeaStack was 11% faster than Rayon and 18% faster than the C++ pool, averaged over the nine workloads. It was 31× faster than Node running the same TypeScript, and 30× faster than scriptc 0.3.6. Those figures compare each runtime's best of 7 runs; compared by median run instead, GeaStack was 11% faster than Rayon, 16% faster than the C++ pool, 31× faster than Node and 29× faster than scriptc.

The ports keep the TypeScript's algorithm and data layout, except where that would make the Rust unidiomatic. For sort, Rayon uses its own par_sort_by instead of the library's sample sort, and for strings it writes into presized strings instead of concatenating. Rayon wins strings, 7.91 ms against GeaStack's 8.02 ms, and GeaStack wins sort, 23.2 ms against 24.3 ms.

GeaStack beat Rayon on four of the nine workloads, tied on five and lost on none, counting anything within 2% as a tie. The biggest lead is on binary-trees: 71.2 ms against Rayon's 126 ms. Going from one thread to 20 made GeaStack 12.89× faster, Rayon 12.50× and the C++ pool 12.09×. On an 8-thread Intel Xeon E3-1231 v3 server running Linux, the same release was 4% faster than Rayon (5% faster by median), 9% faster than the C++ pool, 9.0× faster than Node and 8.3× faster than scriptc.

Milliseconds, best of 7 runs, with the median run in parentheses; every run's output matched Node's. The workloads, ports, harness and raw results are in geastack/benchmarks.

The library is written in TypeScript on top of two native primitives. tasks(count, body) runs body once per index, and tabulate(length, body) returns the array of body's results, which is what map uses. Under Node both are for loops. When compiled, GeaStack's runtime hands the work to a persistent thread pool, and tabulate sizes its result once so every thread writes its share straight into it.

GeaStack's reference counts are not atomic, and they stay that way in parallel code. Inside a parallel region each thread owns the memory it allocates and counts those objects normally. Count changes on anything else are deferred and applied when the threads join. The library's own tests build every fixture under ThreadSanitizer and AddressSanitizer and compare each output with Node's.

Callbacks can't share mutable state. There is no concurrent map, atomic counter or lock: a callback that needs to contribute to a total returns its part, and reduce or mapReduce combines the parts. Concurrent containers along the lines of Rust's DashMap are possible, but they need a write protocol the compiler can check, so they will come as a separate design rather than a loosening of this one.

The library is a fixed set of operations: map, filter, flatMap, every, some, findIndex, reduce and mapReduce over arrays and integer ranges, plus join and sorted. Rayon's work-stealing scheduler can split a computation into an arbitrary tree of tasks. Here, a parallel operation started inside another one runs inline on its thread, so nesting join or par calls doesn't add parallelism.

Under Node, the library always runs sequentially. That build is for testing and debugging; the speed comes from compiling with GeaStack.

Add the package to a GeaStack project and build as usual. gea build loads the library's compiler plugin from your dependencies:

npm install @geastack/parallel gea build

GEA_PARALLEL_THREADS sets the thread count of a compiled program; it defaults to the number of hardware threads. The README covers the full API, and the design notes describe the compiler check and the runtime in detail.

The library is open source at github.com/geastack/parallel.