Strands Decider: Why Not an Encoder?

Here, we see that e1b wins handily at lower prompt lengths, but scales worse as the prompt grows. This is consistent with the great median performance. Whether this matters depends on the workload mix: both JF100 and JevBench trend small, to e1b’s benefit. The scaling difference here comes down to the compute costs of the two models: e1b has 26 full attention layers with the associated quadratic compute cost, while 75% of v19’s layers are Gated DeltaNet with linear compute cost. These layers have a higher fixed cost, but better asymptotic cost, leading to the higher latency floor2.

Finally, let’s look at one interesting sub-plot, going back to e1a (our unmasked 2.6B encoder-only candidate). On the eval sets, it beats v19 on seven of nine tasks, and is very slightly behind on two. But it does significantly worse at JevBench (153 vs 168), and a little worse at JF100 (158 vs 162). This suggests that the encoder generalizes worse, at least without the masked approach e1b takes. Is being able to attend to the question when reading the state actually a weakness?