Mammoth-1 and Elephant-1: frontier decision models

On this page

Background

Method

2.1 Formulation

2.2 Readout

2.3 Architecture

2.4 Data

2.4.1 Data sources

2.4.2 Data processing

2.4.3 Looping on data processing

2.4.4 Data evolution

2.5 Uncertainty-aware data construction

2.6 Training

Results

Future directions

Summary

Appendix

A1. Scores by each Jev Decision Index benchmark

Checkpoint · OCT 2026

Mammoth-1 and Elephant-1: frontier decision models

We provide the technical report for training the world's best decision models.

Decisions, decisions, decisions. Training a frontier decision model is about data, data, data. Today we release the two best decision models in the world: elephant-1 (9B) and mammoth-1 (27B).

The decision model format was introduced by TypeSafe in September 2026 with their release of Jev and the System One API (named after Daniel Kahneman’s description of how the brain uses an intuitive system to make fast decisions). Many other decision models have since been trained and hosted. These models are often compared on two benchmarks:

Jev Decision Index: composed of 37 public benchmarks, and an undisclosed amount of private tests.

Artificial Analysis Decision Index: knowledge, reasoning, tool calling and safety evaluations.

mammoth-1 is the best decision model in the world, ranking #1 of 208 models on the Jev Decision Index 0.4.

elephant-1 is the best 9B decision model in the world on the Jev Decision Index (beating Jev 1.14) and tied with the previous best decision model d1-ultra on the Artificial Analysis Decision Index. It beats models such as Microsoft-Decision-1, Perplexity Decider v1.1 27B and Jev.

Comparative performance of mammoth-1 and elephant-1 on the Jev Decision Index and our replication of the Artificial Analysis Decision Index. mammoth-1 leads the Jev Decision Index at 71.24; elephant-1 is the top 9B model. On the Artificial Analysis Decision Index, mammoth-1 scores 65.9 and elephant-1 65.2, against the official 65 score for d1-ultra.

In this work we detail how we post-trained Qwen3.5-9B and Qwen3.5-27B to achieve this performance, namely with a) block-bidirectional attention mask for full attention layers (that improves vs causal attention when we fix the data) and b) a highly scalable data recipe.

A decision model takes some state and a set of questions, and returns a probability distribution over each question’s answers. The state is a JSON value (such as a support ticket or a document). Images and video are then referenced inline. Typically, each question has instructions of three types:

Choice: 1-255 options that have an optional description (e.g. “which team handles this?”). This is the canonical decision model output (Figure 2), where a probability is returned per option (all summing to 1) and a confidence is given. Typically, the top probability is rescaled such that 0 is a uniform guess and 1 is absolute certainty.

Yes/no: descriptions of what counts as true and false, before one number P(true) is returned.

Score: a scale of 2-10 levels (e.g. “how upset is the customer?”, 0 = calm to 4 = furious). A probability per level (summing to 1) is then returned, along with the expected level (the probability-weighted average) and a confidence which is 1 when all probability sits on one level, 0 when it’s uniformly distributed.

Schematic of a decision model. The input and output of a decision model when faced with a choice question. Here the state (a support ticket) and the question about the ticket (where to route it) go through one forward pass of the elephant-1 decision model and a probability for each option is returned.

Normal language models are of course able to make the exact same decisions, but are very inefficient when doing so, needing to decode into text and have the answer parsed from the generated outputs. In addition, such text, unless explicitly prompted, provides no reliable measure of its own certainty.

In contrast, decision models a) do not need to parse given the answer is one of the given options; b) are cheap because nothing is generated; c) easily thresholdable, meaning an option can be chosen or escalated only when that option is likely enough; and finally d) we can train them to be uncertainty calibrated.

As described, a single decision model request is a state sss and a set of questions. A question qqq has instructions and an ordered option set Oq=(o1,…,ok)O_q = (o_1, \dots, o_k)Oq​=(o1​,…,ok​). For each question, the model then returns a probability distribution p∈Δk−1p \in \Delta^{k-1}p∈Δk−1 over the options.

Note, we train and evaluate calibration on ppp alone.

Each request is one shared prefix xPx_PxP​ that contains the system prompt and the state, followed by one suffix xSx_SxS​ per question, containing its instructions and its options labeled with letters ℓ1,…,ℓk\ell_1, \dots, \ell_kℓ1​,…,ℓk​. Each suffix then ends at an answer position t∗t^\astt∗. The answer is a softmax over the head logits of the option letters at that position:

Here ht∗h_{t^\ast}ht∗​ is the final hidden state at t∗t^\astt∗ and WUW_UWU​ the LM head. We process the prefix once and reuse its cache. This means each additional question costs only one forward pass over the suffix alone.

The base Qwen3.5-27B is a hybrid model. Within each of its sixteen repeating units there are three Gated DeltaNet (GDN) layers (recurrent and causal), followed by one full-attention layer (Figure 3a). We modify the attention mask of only the sixteen full-attention layers (Figure 3b).

To describe how we do this modification, let b(t)=0b(t) = 0b(t)=0 for tokens in the prefix xPx_PxP​ and b(t)=1b(t) = 1b(t)=1 for tokens in the suffix xSx_SxS​. In the full-attention layers, token iii may attend to token jjj if and only if

This contrasts to the causal mask of the base model, which allows only j≤ij \le ij≤i. Each token therefore attends to its entire block and to all earlier blocks. This has two consequences:

A question is encoded with knowledge of its options.

The prefix never attends to a suffix, so its cache is identical for every question in the request and can be shared between them.

This generalizes the prefix-LM mask (Raffel et al., 2020) to two blocks. We note that Perplexity’s Decider v1.1 applies a similar change.

Architecture of mammoth-1. (a) State and question are read by Qwen3.5-27B, which repeats three Gated DeltaNet (GDN) layers per full-attention layer in each of its sixteen blocks. The answer is the softmax over the option letters’ logits at the answer slot. (b) In the full-attention layers we replace the causal mask (left) with a block-bidirectional one (right). This means that every token sees its whole block and the blocks before it. We keep the GDN layers causal. elephant-1 uses the same architecture, scaled down: Qwen3.5-9B with the same unit repeated eight times (32 layers, against mammoth-1’s 64).

To compare the effect of this attention variant, we varied the attention mask and fixed the data we trained on. We used an earlier variant of the data mixture (Mix 1 - see below) used to train the final elephant-1 model for both attention versions, and then evaluated on a public 10k sample of the Jev Decision Index (we note this is a single run).

Across other experiments, we similarly find that bidirectional attention outperforms the causal variant.

We achieved dramatic improvements in decision performance by scaling the data mixture across experiments and runs.

A core part of our process at Base Labs is what we refer to colloquially as “plugging the holes in the Swiss cheese”: we train a model, identify weak points and failure modes, agentically generate synthetic data with guided recipes to target those failure modes, and then repeat. Of course, this relies on having broad and reliable evaluations, and also relies on not simply copying evaluation data but rather addressing the underlying problem, and both of these take significant nuance and experience to get right. We plan to release and discuss our system for doing this in the future.

Our data mixture for training both elephant-1 and mammoth-1 consisted of 6.21M examples and was the eighth mix we attempted. With each, we evaluated on the public split of the Jev Decision Index for the 9B model. Growing this mixture saw our score improve from 45.5 to 66.0 on this benchmark. After scaling, simply repairing the data gave a further 7.9 points by Mix 7, and elephant-1 (Mix 8) scores 72.2 on the public board.

In our iteration, we used the 9B backbone (elephant-1) for experimenting with data mixtures and augmentation, before using the final dataset (Mix 8) to train mammoth-1. All the experiments and iterations herein therefore use elephant-1. We did extensive data deduplication and ensured zero contamination between our training sets and the indexes/benchmarks we evaluated on.

Our final data mixture consists of examples in four categories, as well as targeted repairs within them (Figure 4):

Public decision data (0.59M). This split includes commercially licensed public datasets that we converted into decision data. In total, 93 tasks from 61 public datasets, reasoning probes (code execution, mathematics, chess, logic) and 25 community decision datasets. A third of these datasets have soft labels from human voters. Here we also included our first synthetic tasks. This means that we included answers that came from a deterministic program or a known outcome rather than a model (for example, suppose the state says that 1% of support tickets are fraud, 90% of fraud tickets mention a refund, and 10% of non-fraud tickets mention a refund. The question asks how likely a ticket that mentions a refund is to be fraud, which we use a program to compute exactly).

Core decision skills (3.62M). Based on the discussions of the field (Liquid AI and Jev Decision Index), we split decision making into five skills: knowledge and reasoning, language, tools and automation, retrieval and classification, and arts and human taste. For each category, we attempted to understand what made these questions ‘hard’ (e.g. how question type, options, case structure interacted) and then synthetically generated new examples.

Safety and abstention (1.00M). Our earlier experiments revealed that decision models were terrible at knowing when they should not make a decision. To address this gap, we generated data that included safety classification (when a request is harmful), tool-call refusal (when no tools work for a request), knowledge abstention, and harder multiple choice questions (where wrong answers are worse than not knowing).

Judging and expert domains (1.00M). Decision models are – naturally – used in contexts where they judge the outputs of other models. We therefore generated and added data for the decision model to learn to grade against rubrics, compare between answers, grade answers we can verify easily (e.g. math) and judge answers in expert domains such as law, finance and health.

Targeted repairs. We included abstention training, which complements abstention data, that we detail in Section 2.5.

These data splits were added sequentially as we came to understand how to train a ‘good’ general decision model (Figure 4). We first added public decision data (Mix 1 to 3), then decision skills at Mix 4, then safety and abstention, and judging and expert domains, at Mix 6. The final data mixture is:

Training rows of each data mixture. Both elephant-1 and mammoth-1 are trained on Mix 8.

Data that we generate for the above then passes through one pipeline (Figure 5). Note that all public data keeps the original labels and joins into the mix at the final stage. In brief, this pipeline first generates new examples using e.g. DeepSeek V4-Pro or GLM-5.3. To account for the fact that no single check will catch all corrupted or bad examples, we stack several checks together. These include:

Label. We first label the correct/intended answer independently using either a panel of five judge models or deterministically (as above, in the