What Claude Code's multi-agent code review costs and finds

Claude Code’s multi-agent /code-review max caught 23 of 27 serious bugs on hard pull requests, against 21 for a single agent, but cost about $21 a review instead of 11 cents and took 56 minutes instead of 40 seconds.

Anthropic’s Claude Code team has written down how it reviews code: several agents look for bugs at once, “then a separate group votes on each finding, and it only gets posted if enough of them agree it’s real.” Anthropic’s Deputy CISO, Jason Clinton, calls narrow review agents “much more effective than one mega-prompt or super security agent.” Those posts are about Anthropic’s managed service and its internal process, and neither gives a count of the bugs the agents find or miss per pull request, how much else they say, how long a review takes or what it costs. The local /code-review command that ships with Claude Code, which in the team’s words “runs a lighter version locally”, has the same shape at its max setting and is the one anyone can run, so I measured it.

I ran Claude Code’s local /code-review command, not Anthropic’s managed Code Review service1, with one agent and with its multi-agent fan-out, scored both on 20 pull requests from a public benchmark5, and put them against Quickchat Review, the reviewer I built4. The benchmark’s own score put the three leading reviews within 0.8 points of each other, so I also had two AI judges compare the reviews head to head10.

With its agents on Sonnet 5.5, the fan-out does catch bugs. It does not catch all of them, it takes most of an hour, it adds twice as many claims that the answer key does not list, and it cost $20.95 per finished review.8 Quickchat Review works on the same idea, many narrow AI reviewers and a check before anything is posted. On Opus 5.5 the judges’ ratings do not separate it from the fan-out, and it got there for $4.61 and 8.2 minutes per pull request, under a quarter of the fan-out’s recorded price per finished review and about a seventh of its wall-clock time under our concurrency and rate limits.

The product. Claude Code 2.1.3031 and its local /code-review command, a skill bundled with the CLI that works on any plan and bills to your usage or API key; Anthropic’s managed Code Review service is a separate product I did not test.

How it ran. Headless, in a locked-down Docker sandbox with no GitHub token and no GitHub writes2:

claude -p '/code-review high <base>...<head>' --output-format json

In Claude Code’s own transcripts, high and max select two different review recipes, and the documentation describes neither3. At high it starts one agent: high effort → 8 inline angles → dedup (no verify) → ≤10 findings. At max it fans out: max effort → 5+5 angles × 8 candidates → 1-vote verify → sweep → ≤15 findings, about ten finder agents, a verifier agent per candidate, then a sweep. To make max affordable on twenty PRs I forced every subagent onto Sonnet 5.5, capped each review at $30 and gave it 90 minutes4; none of that is a shipped default, and neither are the rest of my settings2. The two recipes differ in more than their number of agents, so this compares the two settings Claude Code ships, not parallelism alone3.

The three review shapes compared in this post, each as its tool intends it.

The benchmark. Martian’s Code Review Bench: 20 of its 50 PRs from Sentry, Grafana, Cal.com, Discourse and Keycloak, scored by Martian’s own pipeline with one LLM judge, GPT-6.1 Sol, for every reviewer7. Its pipeline splits each review into claims and matches them against the key; a claim that matches nothing is an unmatched claim7. Twenty and not fifty because the full set of max reviews would have cost about $1,000 and run for about 16 hours5. The 20 are the PRs where Quickchat Review on two other models had done worst, a selection expected to favour every other reviewer; Claude Code high on Opus 5.5 and Quickchat Review on Sonnet 5.5 also ran on all 505.

Costs and times. Claude Code’s dollar figures are the estimates it records for each session, apart from one recount labelled where it appears, and Quickchat Review’s are computed from token usage, both at list prices; every time is wall-clock with several reviews running at once8 9. max has three cost figures, each labelled where it appears: $20.95 per finished review, $19.90 over all 20 PRs in the charts, and about $440 actually spent8.

Both runs reviewed the same 20 PRs with Sonnet 5.5 doing the work4. The fan-out matched two more of the 27 serious answer-key issues and the same number of issues overall; two issues is about the scoring noise on this benchmark, so I cannot say whether that difference would repeat7. Match counts are totals over the 20 PRs and the other rows are means; the max column’s time and cost rest on its 19 finished reviews, and its match and claim counts cover all 20, with the timed-out review counted as zero and five reviews scored from reports recovered from their transcripts9.

The fan-out still missed 4 of the 27 serious issues. It produced twice as many unmatched claims, and its recorded subtotal was 188 times one agent’s, $398.03 against $2.128.

On the same 20 hard PRs with Sonnet 5.5, Claude Code max matched 2 more serious answer-key issues than high and the same number overall. The chart’s $19.90 a review spreads the recorded subtotal over all 20 PRs; per finished review it is $20.95.

The unforced configuration, on one small PR. On browser-use #857, a 97-line change with five defects in our answer key6, /code-review high on Opus 5.5 found all five in 70 seconds for 32 cents. max with Opus 5.5 on every agent4 started 9 finder agents and 14 verifiers, made 608 API calls and had spent $40.64, by our recount of those calls8, when our 30-minute limit stopped it at 31 minutes. It printed nothing, because Claude Code prints the review only when it finishes.

Cumulative cost of the all-Opus 5.5 max run on browser-use #857 until the 30-minute limit stopped it with nothing printed, with the high run on Opus 5.5 as the dot near the origin.

The model running the recipe decides how much of it runs. Same max prompt, same PR, a different model forced onto the review agent and every child: Opus 5.5 launched 9 finders and 14 verifiers, Sonnet 5.5 launched 5 finders and no verifiers, and Haiku 5.5 launched no finders and 5 verifiers6. The Sonnet run cost $13.31 and took 32 minutes, the Haiku run $0.53 and 13.5 minutes, and both found all five defects. Each is a single run, and the Sonnet and Haiku runs had a $15 budget cap the agents could see, while the Opus run had none6.

Opus 5.5 launched nearly all of the max recipe and printed nothing before the limit, while Sonnet 5.5 and Haiku 5.5 each skipped part of it and still found all five defects.

Agents cut work when they can see the budget. Five of the 19 finished max reviews launched no verifier at all. The agents can see a running budget line in their context, and four of the five named it as the reason, one in these words: “Per-candidate verifier agents were skipped because of budget.”3 No review reached the $30 cap, so an uncapped max might run more verifiers and cost more3.

Cost did not track PR size. Finished reviews cost $15.17 to $26.09 each, with only a weak association with changed lines (r = 0.18)8.

Claude Code max with Sonnet 5.5 agents spent $15 to $26 on every finished hard PR, close to the $30 cap it could see, while high on Sonnet 5.5 stayed between six and sixteen cents.

The verifier checks that a bug is real, not that the PR introduced it. In the all-Opus max run on browser-use #857 the verifiers assessed 18 candidate bugs, marked 16 confirmed and 2 plausible, and refuted none; two of the confirmed bugs existed before the PR6.

Several reviews exited successfully without a usable report. Five of the twenty max reviews returned a closing note instead of the review, for example: “The last background agents have finished. The report above stands, with no findings to add or remove.” Those five PRs delivered 2 of their 63 findings. One review ran into the 90-minute limit without a report, and 25 of 339 agents died on rate-limit errors from our API key, 23 of them inside 11 reviews that still reported success, so a check of the exit status alone would have missed all of this9.

The answer key did rank the reviews: max scored 30.0 on the benchmark’s F2, well below the leaders. It did not separate the leaders: Quickchat Review on Sonnet 5.5 at 43.9, Claude Code high on Opus 5.5 at 43.2 and Claude Code high on Sonnet 5.5 at 43.1, three very different setups within 0.8 points. Martian’s key holds about 3.7 entries per PR, the reviews made 8 to 26 claims per PR, and the key cannot tell a true claim it lacks from a wrong one7.

So I added a second instrument: a judge that reads two reviews of the same PR side by side, with the diff, the touched files and the answer key, and says which one a maintainer would rather receive, with a verdict each for recall, precision, finding quality and writing10. Two judges from different providers, GPT-6.1 Sol and Claude Opus 5.5, rated 298 matchups each with identical prompts10, and the results are on the Elo scale, over the reviews that exist11.

The judges rated the fan-out highest, though not clearly above Quickchat Review on Opus 5.5. The five main configurations on the same 20 PRs:

Match counts are totals over the 20 PRs, unmatched claims are means over all 20, and time and cost are means, for max over its 19 finished reviews8. Rows on the same model compare complete configurations, which differ in more than the harness; Quickchat Review also ran six quality passes on GPT-6 Luna, under 2% of its cost4.

The review the answer key scores last, Claude Code max with Sonnet 5.5 agents, is the one the judges rate highest.

The judges credited max with findings the answer key does not list: over their 194 recall verdicts on its pairs it won 162, tied 22 and lost 10. It also came last on writing, with 14 wins against 179 losses11. Against high on Opus 5.5 it rates +148 (95% interval +64 to +241), about a 70% expected win rate under the fitted model, at 61 times the recorded cost; that margin comes mostly from the Opus judge, and the GPT judge on its own does not separate the two11. It rates 38 points above Quickchat Review on Opus 5.5, with a 95% interval from −33 to +135: the interval includes zero, so these data do not say which of the two is better11.

Among the five main setups, the judges rate Claude Code max with Sonnet 5.5 agents best at finding problems and worst at precision and at writing them up.

The two judges agreed on precision 76% of the time and on recall 70%, but on the overall winner only 56%, and each judge’s highest-rated configuration ran on its own model family12. GPT favoured lean reviews and Opus thorough ones, so the overall ranking rests on how much a missed bug weighs against a wasted comment; the recall, precision and writing verdicts are where the two judges mostly agree.

The two judges named the same overall winner in only 56% of matchups, but agreed far more often on recall, precision and writing.

For an inexpensive first pass, I would start with /code-review high on Opus 5.5. It reviewed every one of the 50 benchmark PRs in 42 to 151 seconds for about 36 cents and matched 51 of the 66 serious answer-key issues, so it also missed 155. On the twenty hard PRs the judges rated it 120 points above the same command on Sonnet 5.5 (1518 against 1398), a gap whose interval only just excludes zero11, though the Sonnet 5.5 version costs about a third as much. Run it on your branch before you open a PR, which is the use its documentation describes. It gives no severity, no verdict and no clean split between bugs the PR introduced and bugs that were already there, and its prompt asks for at most ten findings unless you pass --max-findings, a request rather than an enforced limit3.

I would not run max on every PR. Anthropic’s model-configuration documentation says the max effort level “may show diminishing returns