What Are Looped Transformers? A Technical Deep Dive
A looped transformer applies the same block of layers to a token more than once. Instead of stacking L distinct layers and running each exactly once, you keep a small stack of shared layers and run it in a loop: four re…
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
A looped transformer applies the same block of layers to a token more than once. Instead of stacking L distinct layers and running each exactly once, you keep a small stack of shared layers and run it in a loop: four real layers applied thirty-two times give you 132 layers of computation out of the parameters of eight. The idea goes back to the Universal Transformer in 2018, and the modern name for it is recurrent depth.
What makes it interesting is that the loop count is not baked into the checkpoint. In an ordinary transformer, depth is a property of the weight file — you ship eighty layers, and every prompt gets eighty layers. In a looped model the number of passes, r, is an argument you pass at call time, so the same weights can be told to think harder about a hard question than an easy one. And the extra thinking happens inside the hidden state, between emitted tokens, rather than in text anyone can read.
Which is why, on 2 September 2026, this became an argument about safety. The Information reported that OpenAI's Astra uses recurrent depth — the same weights applied several times over — and that this makes some of its internal reasoning harder to read. Astra shipped as a preview the next day, went generally available on the fourth, and by then the argument had sorted itself into two camps that were not really talking to each other.
One camp read it as a redline crossed. Ryan Greenblatt: Astra "fundamentally doesn't have a fully monitorable chain of thought in the same way as past models. Very very bad news." Steven Adler, until recently on OpenAI's safety staff, called it a violated redline. The other camp read it as nothing at all. @nrehiew_: a looped transformer has "zero impact, because it is simply spending more compute per token." Sebastian Raschka called it a tiny architectural tweak and noted, correctly, that reusing layers does not by itself suppress a visible chain of thought.
Then Jakub Pachocki posted, which is unusual for a chief scientist mid-news-cycle, and said something better than either camp: the depth of Astra's computation graph is within a factor of two of GPT-4 — and, separately, that chain-of-thought monitoring "is fragile and unfortunately trending in a negative direction, for reasons not contingent on architecture changes."
The architecture claim rests on one anonymous source. No technical report, no code, no patent, nobody outside OpenAI in a position to check it. So I stopped reading the argument and read the architecture, which — unlike Astra — is documented in detail, mostly by people who published long before any of this was a story. Thirteen papers later, what a loop actually buys turns out to be narrower than the alarmed reading requires and stranger than the deflationary one admits.
This post is the mechanical version: what the forward pass does line by line, how the test-time dial is manufactured during training, what the loop costs in FLOPs and KV cache with the arithmetic shown, what the literature establishes, and whether this is the thing that pushes models further. Every mechanism is drawn out as a diagram.
No notation in this section. Just what changes, why it helps, and what it costs. Every symbol in the post is collected in an appendix at the end.
A model is a stack of layers. Your question goes in the bottom, passes up through each layer in turn, and an answer comes out the top. Each layer is a separate pile of numbers — the "weights" — sitting in a file on disk.
An ordinary model has, say, eight layers and uses each one exactly once. A looped model has four layers and runs through those same four repeatedly. Watch which one finishes first.
An ordinary modelEight different layers. Each used once.the answer comes out hereyour question goes in here8layersdone —now idleA looped modelFour layers. Used again and again.Each climb hands its rough working down tothe next climb. Your question is fed in againalongside it, so the next pass has both.What loops is the working, not the question.the answer comes out after the last passrough working4 layersyour question, every passstores 8 layers · takes 8 passesStorage and thinking are one number.stores 4 layers · takes as many as you askHalf the storage. You choose the thinking.
That is the trick. Storage and thinking used to be the same number. Looping separates them into two dials.
Two honest notes before going on. The looped stack takes longer, not less time — you can see it still working after the other has stopped, and that is the correct picture: four passes over four layers is sixteen layers of work, and you wait for all sixteen. And this is not a free win. Four layers run four times are not as good as sixteen real layers — shared layers are worth measurably less than purpose-built ones, and section 6 puts a number on exactly how much less. What you get is most of the benefit from a quarter of the file.
(Real looped models keep a layer or two outside the loop at each end. Section 3 has the exact shape.)
The obvious objection: if you run the same four layers on the same question again, why doesn't it just produce the same answer again?
Because the second pass does not start over. It starts from where the first pass got to. The model carries a scratchpad of partial work — that chip in the diagram above — and each pass reads the previous pass's working, plus your original question, and improves on it. It is closer to re-reading a difficult paragraph than to asking the same question twice.
Until now there were two ways to make a model smarter, and both are expensive.
Make it bigger. More layers, more weights, a bigger file. That costs memory on every machine that ever runs it, forever.
Make it write out its reasoning. This is what "thinking" models do — they work through the problem in words before answering. That is not a trick; it is the model buying itself more steps, out loud. But the words fill up the space available for your actual document, and you pay for every one of them.
Looping buys the same extra steps silently. It adds no weights and writes nothing down.
you payyou getthe file on disk never moveswhat you pay keeps climbingwhat you get flattens outhigher is more ·shape only, notmeasured valuesEarly on the amber line runs ahead — you get more than you pay for. By the right-hand end the indigo has overtaken it. That crossing is the point of diminishing returns.
Watch the flat green line: the file never gets bigger. That is the claim in one image — you buy more thinking without buying a bigger model. And that is worth real money: a small file that reasons like a big one fits on cheaper hardware, or fits more copies on the same machine. You convert a cost you pay always into a cost you pay only on the questions that need it.
Three things that image does not show, and you should not leave without them.
It is not free memory. The weights stay the same size, but while the model is answering it keeps a working scratchpad of everything each pass computed, and that scratchpad grows with every pass. At the highest settings it is larger than the model itself. Section 6 has the arithmetic.
The dial has to be built in. You cannot take an ordinary model and ask it to run its layers twice. It has to be trained from the start to accept any number of passes, and section 5 is about how that is done.
The returns flatten sooner than you would like. Watch the amber line go flat. For the best-documented model, more looping stops helping somewhere between eight and thirty-two passes depending on the question — and that is a lightly trained model. The one team that trained a looped model really hard found the useful range collapsed to two passes. The better the model is to begin with, the fewer extra passes it has any use for. That is the strongest argument against the whole idea and section 9 takes it seriously.
This is the part that surprised researchers, and three separate teams have now found it independently.
Extra passes help enormously with problems you have to work through — arithmetic, multi-step logic, following a chain of reasoning. They help far less with problems that just require knowing a fact. Some, but nothing like as much.
Extra passes let a model think harder about what it already knows. They do not teach it anything new. For that you still need a bigger model.
So the practical summary, before any machinery: a looped model gets you much of the reasoning of a far larger model out of a far smaller file, by spending time rather than by getting bigger — and it is worth doing exactly when your problem needs working out rather than looking up.
A looped model is not a smaller model. It is a model that thinks longer instead of being bigger.
Everything below is how that is built and what it costs. If you only wanted the idea, you have it.
Start with why anyone wants depth, because that is what the whole architecture is answering.
A transformer with a fixed number of layers is a parallel object. Every token's computation runs through the same bounded chain of operations, and that chain does not get longer when the problem gets harder. Li, Liu, Zhou and Ma made this precise at ICLR 2024: constant-depth, constant-precision transformers are confined to AC0\mathrm{AC}^0AC0, a proper subset of TC0\mathrm{TC}^0TC0. Both names denote what a circuit of fixed depth can compute: AC0\mathrm{AC}^0AC0 allows unlimited AND and OR gates, TC0\mathrm{TC}^0TC0 adds majority gates and is strictly larger. That is a small class. It does not contain the kinds of problems that are inherently serial — composing a permutation group step by step, iterated squaring, evaluating a circuit whose gates depend on earlier gates.
The same paper gives the escape hatch, and it is why chain-of-thought works at all. Let the model emit T intermediate tokens and read them back, and with O(log n) embedding width it can solve anything computable by a boolean circuit of size T. Chain-of-thought is not a prompting trick that happens to help. It is a depth mechanism — the context window used as a tape, every emitted token one more step of serial computation. Merrill and Sabharwal supply the other half: transformers of depth Θ(log n) express regular-language recognition (state tracking) and graph connectivity (multi-step reasoning), neither of which fixed-depth transformers can express under standard complexity conjectures.
So there are exactly two ways to buy serial depth. Spend tokens, and the computation happens in the context window where you can read it. Spend iterations, and it happens in the hidden state where you cannot.
Two ways to buy serial depthSame complexity-theoretic fix. Different currency — and only one of them is written down.Chain of thought · token spacet1t2t3t4t5each emitted token is one more step of serial computationthe context window is being used as a tapelegiblecosts: context window, latency, output tokensT steps ≈ circuits of size TLi, Liu, Zhou & Ma · ICLR 2024 · arXiv 2402.12875Recurrent depth · latent spacet1s4 = R(e, s3)s3 = R(e, s2)s2 = R(e, s1)s1 = R(e, s0)× rone weight setthe same computation, inside the hidden state, between tokensnot legiblecosts: FLOPs, wall-clock, KV cacheno context cost · no CoT data neededGeiping et al. · arXiv 2502.05171
Recurrent depth takes the second route, and everything downstream — the parameter savings, the KV cache problem, the monitorability argument — falls out of that one choice.
Chain-of-thought buys serial depth in token space. Recurrent depth buys it in latent space. They are two implementations of the same complexity-theoretic fix.
One caveat to carry through the rest of this. The complexity results above are asymptotic — they need depth that grows with the input. A looped model run at a fixed loop count r is still a constant-depth transformer, exactly as expressive as the unrolled l_P + r·l_R + l_C-layer network it is equivalent to. Looping does not change the complexity class unless
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。