返回资讯中心
外部精选
软件工程
#开发者实时流

Droid 1.0: Building a Frontier Harness

A new engine powering Factory's enterprise agent, built for complex projects and everyday development.

Hacker News 最新gmays2026/11/5 · 今天30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

A new engine powering Factory's enterprise agent, built for complex projects and everyday development.

Over the past year, we have worked with hundreds of thousands of developers across some of the largest enterprises in the world. Working side by side with these teams, we've built and hardened a frontier agent harness that directly meets the demands of enterprise development: high-quality results on complex tasks, efficient use of resources, and strong safety controls.

Today, we introduce Droid 1.0: more accurate, efficient, and faster than ever before, powered by new design principles and a programmatic tool-calling engine.

Droid is our agent harness. An agent harness is everything around the model: the prompts, the tools, how tool results come back, and the loop that runs it all. Agents now work for hours or days across hundreds of turns, and every turn re-sends instructions, tools, and history.

Today's update to Droid is the largest one yet. On the same tasks, Droid outperforms the native harnesses built for their models up to 20% more efficiently. It's also designed for what comes next: when frontier models can plan whole workflows as code, Droid lets them run hundreds of tool calls in a single model request.

Factory is model-independent by design. We believe the best agent harness is one built for every model, so our users can choose the best model on the market or let Droid choose the most efficient mix using our auto model router, Factory Router.

Building a harness optimized for many models makes the harness better. When you tune against a single model, it's hard to separate real progress from adaptations to that model's quirks. Working across models ensures the strongest improvements generalize and hold up beyond the model that inspired them.

That's the idea behind our internal Agent Factory. Agents study benchmark failures alongside the friction our users hit in real sessions, propose changes, and test each one on every model family it affects. Our engineers review the evidence and decide what ships.

The result is the most efficient enterprise-ready coding agent tuned for every frontier model, whether open- or closed-weight.

The rest of this post covers the head-to-head results, the design principles behind the harness, how we built our internal Agent Factory, and the core changes that drove the most impact.

On the same tasks using the same models, Droid scores higher at a lower cost than every other harness we tested.

GPT-6.1 Sol

$0.70$0.90$1.10$1.3044%48%52%56%60%COST PER PASSED TASK · CHEAPER →PASS RATE · HIGHER →HIGHEST QUALITYDroid$0.79 · 57.6%OpenCode51.0%Codex54.0%Devin45.0%

Cursor isn’t shown: it has no access to OpenAI models.

HIGHEST QUALITYLOWEST COSTDroid$4.73 · 65.2%Cursor62.1%OpenCode62.1%Devin59.6%Claude Code57.6%

Terminal-Bench 4. With GPT-6.1 Sol, Droid scores 3.5 points higher than Codex at 12% lower cost, and 6.6 to 12.6 points higher than OpenCode and Devin. With Claude Opus 5.5, Droid scores 3.0 to 7.6 points higher than Cursor, OpenCode, Devin, and Claude Code, at up to 24% lower cost. Methodology

This gap matters more as agents run longer. Over hundreds of steps, a better agent makes fewer mistakes for later steps to build on, and a cheaper one lets you run more agents on the same budget. Droid tunes its prompts, tool descriptions, and tool-calling mode for each model it supports, so every supported model gets these gains, including when you bring your own API keys.

The largest change behind these results was letting the model call tools from a program.

In a standard agent loop, the model calls tools directly and every tool output is appended to its context. Programmatic tool calling (PTC) takes a different approach to long-horizon work: the agent writes a program that invokes the tools.

With PTC, the agent can reason up front, write the code for a tool-calling workflow, and then execute the entire program end to end in a single invocation. For example, it could write a workflow to read hundreds of files and find the few that mention a topic, triage dozens of Linear tickets, or run a tricky database migration in pieces following a deterministic process.

CodeAct (Wang et al., 2024) showed that acting through code beats JSON and text tool calls across 17 models. Cloudflare and Anthropic brought the idea to production in late 2025, and Codex brought it to coding agents this year.

We made PTC the default in Droid 1.0 for three reasons. First, it keeps the agent's context window clean, because all it receives is the output of its program. Ordinary tool outputs (raw files, search results, logs) stay inside the program as variables. Second, it cuts the number of model requests, because one program runs many calls, which makes complex tasks faster and cheaper. Third, it makes the agent more reliable: checks and loops run deterministically in code, eliminating the chance that a model could be lazy and drop a case despite originally planning it.

Droid implements PTC as a Script tool. The model writes JavaScript that runs in a sandbox with no file, network, or system access of its own.

A Script can make hundreds of tool calls in one model request, so the instructions and history are sent once rather than re-sent every few calls. Printed output is capped in size, and the full results are logged where the model can search them later. Calls that don't depend on each other run concurrently, so a batch takes as long as its slowest call.

Direct tool calls: 10 model calls and 15 tool calls, done in 56.8 s. Script tool: 5 model calls and the same 15 tool calls, done in 41.6 s, 15.2 s sooner.

Time saved

One common session pattern, fixing a failing unit test with GPT-6.1 Sol at high reasoning, replayed both ways on one clock. With direct tool calls, the model is called again after every one or two tool calls, 10 times in all. With the Script tool, each model call writes one program that runs its tool calls and returns one result, so the same 15 tool calls take 5 model calls. Time spent in model calls falls from 42.3 s to 28.0 s, and the task finishes in 41.6 s instead of 56.8 s. Tool durations are measured. The direct run places the same calls one or two per model call, with 0.15 s between a tool result and the next model call. Each tool call has its own row, as wide as the call took. Hover or tap a bar for its exact timing.

Because these calls are code, one result can influence the next call. For instance, a program can search, read every match, and filter the results in one pass. With direct calls, each step would cost a model request.

Compared to direct tool calling, Script mode cuts Droid's cost by 15% on Claude Opus 5.5. Most Scripts are small, and the per-task savings concentrate in the 12.5% of them that make six or more tool calls.

Two Scripts from that tail, taken from our own sessions (simplified, with names and identifiers changed), demonstrate the effect. For each, a Script and direct calls make the same tool calls. The difference is how many model requests they take: direct calls fit a few into each request and wait for the model between steps, while a Script runs them all in one program. Direct-call figures are estimates, assuming 3 to 10 calls per request.

The Linear cleanup moved 151 issues between queues and verified each move, 312 tool calls in all. With direct calls, every request re-sends the instructions and the growing history, so even at 10 calls per request the task would take 36 requests and read ~3.1M input tokens. The Script took 2 requests and ~61,000 tokens: one request to write the program, and one to read what it printed.

The CI sweep shows the other half of the savings: what the model reads. The Script checked 5,384 desktop jobs across 547 runs, 550 tool calls, 60 at a time, in 2 requests instead of an estimated 59 to 187. The model saw only a five-number summary, including 41 failed jobs. Under 0.1% of the tool results reached its context. The sweep ran in the background for about 8 minutes while the model worked on another task.

The same 312 tool calls, two ways. Direct calls, modeled at 10 calls per request: 36 model requests, and every raw result lands in the model's context. One Script: 2 model requests, one to write the program and one to read the 6 lines it printed; the raw results stay in the program.

Direct calls fit 10 calls in each request, and the model reads every result before the next batch.

Task: Move 151 approved issues to the backlog, and verify that only the status changed on each one.

Context0 raw results

nothing yet

One Script · calls run inside the program

running

Request 1 · the model writes the Script

Programruns outside the model

0nested calls

raw results stay here

only what the program prints (text())

Contextwhat the model reads

nothing printed yet

Hide contextShow the Script

The Script, written in request 1text() prints to the model

1const remaining = new Set(readReport('<report>').notAttempted) // 151 issues

2

3// both queues, read in parallel

4const queues = await Promise.all([listAll('Triage'), listAll('<queue 2>')])

5const targets = queues.flat().filter((i) => remaining.has(i.id))

6

7const verified = [], failures = []

8for (const batch of chunks(targets, 3)) { // 3 issues at a time

9 await Promise.allSettled(batch.map((i) => setStatus(i, 'Backlog')))

10 const after = await Promise.allSettled(batch.map((i) => getIssue(i)))

11 after.forEach((r, n) =>

12 (onlyStatusChanged(batch[n], r.value) ? verified : failures).push(batch[n].id))

13 if (failures.length) break // stop on the first unexpected change

14 if (verified.length % 30 === 0) text(${verified.length} of 151 verified)

15}

16

17writeResult('<result file>', { verified, failures })

18text({ verified: verified.length, failures })

A real Script from Factory’s own sessions, the Linear cleanup, simplified, with names changed. Both sides run on one illustrative clock. The direct side is a modeled estimate at 10 calls per request: each batch waits for the model to read the previous batch’s results. At 3 calls per request it would take 107 requests. With about 28,000 tokens of instructions re-sent on every request, the direct side reads ~3.1M input tokens; the Script reads 61,000. Result sizes are estimated from typical issue sizes.

How it improves quality

A cleaner context. Raw files, search results, and logs stay in the program, so the model reasons over less noise.

Loops and workflows run deterministically. The Linear cleanup compared 15 fields before and after every write, kept exact counts, and was written to stop at the first issue that changed in any other way.

Planning happens up front. The model works out the whole sequence in its reasoning, and the program it writes is the plan. Stronger models plan further ahead, so their Scripts are more complex and do more in a single request.

Frontier models already get the most out of programmatic tool calling, and we expect the gains to grow as the frontier does.

Script mode is on by default for GPT-6.1 Sol, GPT-6.1 Terra, GPT-6 Astra, GPT-6 Sol, GPT-5.6 Sol, Claude Fable 5.1, Claude Fable 5, Claude Opus 5.5, Claude Opus 5, and Kimi K3.

Programmatic tool calling is simple to add and easy to get wrong. These are the choices that mattered most.

Script mode, and the per-model choice of whether to use it, came out of the same loop that produces most of Droid's changes. New models and snapshots arrive every few weeks. No team can keep up by tuning a harness for each one by hand.

Next loop

01

Bring in

New models are added to Droid and checked.

02

Evaluate

Every model runs the same tasks.

03

Read traces

Agents find what failed runs share.

04

Propose

A draft change with a stated hypothesis.

05

Test

A controlled experiment on every affected model.

An engineer merges

The harness every model runs on, and the

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →