Exploring Apple's On-Device Model: Twenty Billion Parameters in Flash

Julien Simon

Oct 25, 2026

1

Share

macOS 27 put a language model on every Mac that can run Apple Intelligence, and any script can call it: type fm respond in a terminal and it answers.[1] So what is it? How is it built, who built it, and what is it good for?

Apple's answers fit in three sentences. The model is Core Advanced, the larger of Apple's two on-device models, with 20 billion parameters of which 1 to 4 billion work at a time. Apple says it is "custom-built in collaboration with Google." It powers Siri's new voices and dictation, and the apps that call it.[2] Everything else you have to work out.

I can't read Apple's weights. Apple's license bars decompiling, disassembling and reverse engineering its software, and I won't test that line: I opened no model file, decompiled or disassembled nothing, dumped no memory, attached no debugger and left the Mac's system protection on.[3] I watched the Mac from the outside instead, with tools macOS ships (a file-read trace that gives file names, positions and sizes but never contents, the system log, the memory and power counters, the lists of names Apple's frameworks export) and with Apple's public commands: what the model reads from disk, how much memory it holds, how much power it draws, how it counts text and how it answers. I did that on a MacBook Pro with an M3 Max and 48 GB, beside open models that run on the same Mac. Where Apple gives no reason for a choice, I give mine and label it "Assumption." The scripts and the data are in a public repository, where each finding is a numbered item with its hypothesis or question, experiment, results and conclusion.[4]

Apple made a 20-billion-parameter model cost a Mac about what a 4-billion-parameter one does. It keeps the model in flash storage, wakes a few experts for each prompt, and runs them on the Neural Engine. Two seconds after a long request, Apple's model and its safety models held 2.0 GiB of memory; an open model of the same class, built the usual way, held about 15 GiB.[5] On the same 20 math problems, it used about a third of the energy of the best open build I ran.[6] You pay in math, where it finished last of four, and in control: you can't pin it or tune it.[7] And the one part of Google you can measure from outside is the tokenizer.[8]

A model card is the page a lab publishes to say what a model is. Apple has not published one for Core Advanced, so here is the one I could build from outside. Each line says where it comes from: Apple's own words, my measurements on this Mac, my inferences from them, or third parties. The repository holds each line's data.[9]

How Apple's on-device model runs on a Mac

An app never touches the model. It calls Apple's Swift framework, the fm command or fm serve; a system service, modelmanagerd, then opens sessions with two processes that run as a system user.[10] The safety process holds two 300-million-parameter models, a safety classifier and a guard for images, with their own, smaller tokenizer; by their names, they screen the request. The log also names a text sanitizer, a deny-list of output tokens and an output safety configuration. The main process runs Core Advanced: a tokenizer, a small adapter for the use case, an image encoder, the dense block that picks experts, the experts, and a small draft model, which by its name guesses a few tokens ahead for the main model to check. 636 MiB of model pages stay loaded on the Neural Engine; the rest stays in a 4.95 GB file in flash, and a cache brings in the experts a prompt picks.[10] A request the device cannot handle goes to Private Cloud Compute, Apple's server models, and an app must choose that model itself.[11]

Two things stand out. First, Apple dropped adapters for developers in OS 27 but kept them for itself: the bundle the public API loads is the base model plus an adapter named for it, and Apple's catalog lists a draft model for 135 use cases.[10] Assumption: Apple tunes each of its own features with a small adapter it ships and retrains with each OS, and a third-party adapter, trained on last year's base model, would break at every update. Second, the safety models are a separate process with a separate tokenizer. Apple can update them without touching the main model, and they cost memory, 0.7 GiB of the 2.0.

Apple names the jobs. On its research page, Core Advanced is "natively multimodal, enabling helpful features like expressive voices and higher-accuracy dictation," and Siri's new voices are "powered by" it.[2] For apps, Apple's developer guide lists what the on-device model is for: summarizing, extracting entities, understanding, refining and classifying text, creative writing, tags and game dialog. The same guide lists three "capabilities to avoid": basic math, code and logical reasoning. Apple's 2025 report said its earlier on-device model "is not designed to be a chatbot for general world knowledge."[12]

Then Apple names the limit. "Traditional large language models," its page says, "require all weights to reside in active memory (DRAM), creating a massive footprint that limits scalability on consumer hardware."[13] A Mac's memory is shared by the CPU, the GPU and every app you have open. The model sits in it all day, beside your browser and your editor, and on a laptop it runs on a battery. Apple's storage page sets its larger tier for Macs with an M3 or later and at least 12 GB, and MacRumors says that is what Core Advanced requires.[14]

So the job is a model good enough for dozens of short tasks, light enough in memory to stay loaded next to your apps, and frugal enough with power to run voice and dictation all day. Three of the choices below are new this year: experts kept in flash, experts chosen per prompt, and a model that runs at more than one size. Apple made the other four before: the Neural Engine, heavy compression, separate safety models and a short task list.[15]

Keep the model in flash, choose experts per prompt

A mixture-of-experts model splits each layer into many small networks, the experts, and runs only a few for each token. The usual design picks new experts for every token, so every expert must sit in memory. Apple's page says why that does not work here: "NAND-to-DRAM bandwidth is too slow to swap weights token by token, as standard MoE models require." So Core Advanced "makes routing decisions per prompt." A small dense block picks a fixed set of experts when it reads your prompt and reselects them from time to time as it writes, with "a high percentage of always-active 'shared experts'" to keep the traffic down.[13]

This is Apple's own line of research. In December 2023, Apple researchers showed how to run models "that exceed the available DRAM capacity by storing the model parameters in flash memory," and warned that flash is fast for large reads, not for "smaller, random reads."[16] In January 2025 they described Instruction-Following Pruning, which fixes the parameters for a task once and so "significantly reduces the weight loading cost."[17] Apple says Core Advanced is built on it.[13] And on October 5th, 2026, six Apple researchers posted a paper on routing for models on devices, where the workload "becomes an I/O-bound one," with experts "reloaded every S inference steps."[18]

The Mac shows the design at work. While the model answers, it reads one 4.95 GB file in blocks of exactly 294,912 bytes. On the first prompts after loading, it reads about 1% of the file a fifth of a second in; later prompts, with experts already in memory, read less. Then it reads a few blocks at a steady rhythm: every 32 tokens in a code answer written at 58 tokens a second, every 32 in an essay at 33 tokens a second, every 31 in a geology answer at 45.[19] The rhythm is counted in tokens, not seconds. Assumption: the model reselects every 32 tokens; on a long prompt the gaps were irregular, which fits reselections that find their experts already in memory and read nothing.[19] Assumption: the blocks are experts; the file name, Apple's description and the byte count all point there, and other shapes give the same count.[20]

Now compare an open model of the same class, built for per-token routing. Google's Gemma 4 26B A4B stores 25.2 billion parameters and runs 3.8 billion for each token, 8 of its 128 experts plus a shared one.[21] At 4 bits on the same Mac, two seconds after the same long request, its server held about 15 GiB. Apple's two processes held 2.0 GiB.[5] That is the case for Apple's design in one comparison: the same idea, a big model with a small part at work, costs seven times the memory when experts change at every token. About half of that is bit width, 4 bits against about 2; the rest is that per-token routing makes the server hold every expert. Google aims that model at "consumer GPUs and workstations," not phones.[21]

Google's answer for small devices is different again. Gemma 3n keeps large per-layer embedding tables out of accelerator memory, so only "the core transformer weights" must sit there, and its larger model contains the smaller one; the small Gemma 4 models carry the same per-layer embeddings.[22] Assumption: Google builds for many makers' phones and boards, so it splits the model by kind of parameter, which any device can do; Apple builds for one family of chips and one storage system it controls, so it splits the model by expert and leans on fast flash. (My Gemma memory figures do not show Google's design at work: by my reading of its 8.1 GiB, the server kept all of Gemma 4 E4B's 8 billion parameters in memory.[22])

Apple also lets the model run at more than one size. Core Advanced "uses a predetermined number of active parameters tailored to each specific use case," from 1 to 4 billion, so weights are "loaded incrementally across requests of varying difficulty."[13] Apple's only published results for Core Advanced, for voice and dictation, were measured "At its 1-billion-parameter activation size."[23] Assumption: speech, which runs all the time, uses the smallest size, and text tasks use more. Google described a similar switch for Gemma 3n and says it is "not part of today’s launched implementations."[22]

Assumption, on the 32-token period: Apple's October paper found that "a segment length of 16 outperforms a segment length of 32" on every benchmark it ran, so Apple ships 32 to halve the flash reads and accepts a small loss in quality.[18]

Apple's 2025 report says "we compressed the on-device model to 2 bits-per-weight," and its new page says the new models were compressed with the same kind of training, without a bit width.[24] What the Mac shows fits 2 bits. The Neural Engine's log counts 636 MiB of model pages that stay loaded, at most 2.7 billion weights at 2 bits, which fits 1 to 4 billion active. One 294,912-byte block is exactly the size of one expert of the dimensions in Apple's October paper at 2 bits.[25] Assumption: the bit width serves the flash design too, because every bit saved is a byte not read at each reselection.

Apple's 2024 report already described its on-device model "running on Apple Neural Engine."[26] On this Mac, the Neural Engine drew 4.2 watts while the model read a long prompt and 3.8 while it wrote, and the GPU stayed at its idle clock.[27] Apple gives no reason specific to the chip; its 2025 report credits its optimizations as a whole with "reducing power consumption."[24]

I measured what that buys. The same 20 grade-school math problems went to Apple's model and to three open models on the GPU, Google's Gemma 3 4B and Gemma 4 E4B and Alibaba's Qwen3 4B, while macOS's power meter sampled the chip twice a second. Apple's model used about a third of the energy of the best open build for the same 20 problems, and the least per problem solved