返回资讯中心
外部精选
软件工程
#开发者实时流

AI review optimized for recall (catches more defects than Codex or Claude)

A self-hosted AI code reviewer for GitHub pull requests: many narrow reviewers, one verified review.

Hacker News 最新piotrgrudzien2026/10/27 · 今天30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

A self-hosted AI code reviewer for GitHub pull requests: many narrow reviewers, one verified review.

A single model asked to "review this PR" reads the diff once, from one angle, and stops when it has found something worth saying. QuickReview fans each change out to a dozen or so narrow, read-only lens passes that each look for one kind of problem and never see each other's output. It merges their findings so one root cause gets one comment, and an adversarial validator re-derives every candidate from the code before one review is posted.

On 185 real PRs with 520 defects each verified in the code, QuickReview on GPT-6.1 Sol finds 97% of the high-severity defects, 93% of the medium and 84% of the low; at each severity the best of the six other reviewers we ran finds 90%, 82% and 52%. On the same model it also finds more than the model maker's own reviewer at every severity: against Claude Code's /code-review on Claude Opus 5.5 and against Codex CLI on GPT-6.1 Sol. Its last changes were mined from its misses on 145 of the 185 PRs; on the 20 kept out, it finds 83% of all defects and the next best 72%. That recall costs more: about $2 and 17.5 minutes per PR, where the other tools take $0.07–0.53 and 0.7–2.3 minutes (Benchmarks).

Around the review sits a small, deterministic TypeScript daemon with no LLM in its decision path. It takes GitHub App webhooks, runs an authoritative reconcile poll, keeps one sticky pi session per PR, and posts as your GitHub App through one gated mutation path, on your infrastructure, under your keys.

Status: beta. The reviewer has run continuously since mid-August 2026 on a busy commercial repository. The packaging for other teams is young: expect rough edges in setup. dry_run = true is the shipped default, so a fresh install never posts anything.

Plan the lenses by rule. Plain code, not a model, reads the changed files and the PR text and plans the passes, each with the reason it was planned. The PR's session may add a lens or raise its reasoning tier, with a stated reason; it may never drop one.

Fan out isolated lenses. Each pass is a leaf agent with read-only tools, the diff and a worktree at the exact head. None can see another pass, spawn agents, or run your code. pass-baseline and pass-tests run on every code change, pass-sweep reads every changed file line by line, and the rest are picked by path, content and size: behaviour delta, code quality, intent against the PR text, backward compatibility, security, data safety, concurrency, bounds, rollout and more.

Merge duplicates. One root cause becomes one comment anchored at its cause, with the other sites in its body (docs/duplicate-merge.md).

Validate. pass-validate re-reads the code for every candidate and keeps, demotes, doubts or kills it from a closed vocabulary. A plausible but wrong finding costs more than a missed nit.

Post one review. Severities map to tags by fixed rules (any surviving blocker means REQUEST_CHANGES), the review posts through the gate, and every candidate, posted or not, lands in a per-finding reasoning record.

The lens catalogue, with each lens's trigger and severity cap, is §3.3 of the QuickReview report.

Recall is the headline: a review that misses the defect has failed, whatever else it does. Precision, cost and time are shown beside it, because that is where QuickReview pays. Unless a chart or table names another build or model, its results are the frozen build of 2026-10-04 on gpt-6.1-sol, shipped defaults, one review per PR. The third-party tools we ran used the best configuration we found for each (except Claude Code, which ran /code-review at its default effort and was not varied), the OpenAI-based ones on the same model, under the same judge on the same PRs.

AACR-Bench, verified defects. AACR-Bench's own reference comments are mostly not real problems, so we built a key of 520 defects, each verified in the code and graded by its consequence: high would block a merge (a user-visible bug on a normal path, a crash, data loss, a security hole, a broken build), medium should be fixed before merge, low is real but fine as a follow-up. The chart at the top is this key. QuickReview misses 2.5 of the 90 high-severity defects, 10.5 of the 145 medium and 46 of the 285 low (a partly found defect counts as half); the best other reviewer at each severity misses 9, 26 and 136.5. Over the panel and Codex CLI the high-severity lead holds under a paired bootstrap (+8.3 points [+1.9, +14.6] over the panel). The price: 6.9 comments per PR, 42% of them true but minor.

Same model, two reviewers. With the model held fixed, QuickReview still finds more than the model maker's own reviewer on each of the three models where we ran both, at 11 to 20 times the cost ($3.24 against $0.16 per PR on Opus). This contrasts two tools, not a measured design effect: they also differ in budget and inputs, Claude Code ran /code-review at its default effort, the Opus and GPT-6 Sol rows are September builds of QuickReview, and the 40 Opus PRs are all PRs QuickReview was tuned on.

Martian, 50 PRs. Martian scores against about three "golden" comments per PR and counts every other comment as a false positive. It extracts about 12.8 issues per PR from QuickReview's reviews, so QuickReview has the highest recall and the lowest precision. We audited the goldens: of the 86 that describe a real defect the PR introduces, it finds 75 (87%), the most of the seven reviewers judged alike.

Qodo PR-Review-Bench, 100 PRs. 580 injected issues: 309 functional bugs and 271 violations of rules written in the repo. QuickReview finds the most functional bugs of the thirteen tools (52.1%) and the second-most issues overall, at the lowest precision; it is fourth on F1.

Judged side by side. A pairwise judge reads two reviews of the same PR blind and picks the better one. GPT-6.1 Sol and Claude Opus 5.5 both judged, and a Bradley-Terry fit over their 596 verdicts gives Elo; alone, the two judges pick the same winner only 56% of the time, which is why they are pooled. On the 20 Martian PRs where QuickReview on GPT-6.1 Sol or DeepSeek did worst, QuickReview on Opus 5.5 (six of its lenses on GPT-6 Luna) beat Claude Code's /code-review at high effort on Opus 5.5 in 29 of 40 verdicts, at more than ten times its cost (about $4 per PR). Only Claude Code's max mode (Opus 5.5 directing Sonnet 5.5 agents, about $20 per PR) ranks higher, by a margin inside the intervals.

Other models. QuickReview's models are configurable. On AACR-Bench it holds most of its recall on cheap open models, and it falls apart on a weak one:

The best third-party reviewer at each severity finds 81 of 90 high (Codex CLI), 119 of 145 medium and 148.5 of 285 low (both the pi-review panel). The open-model rows ran a later build (mid-October, with cost changes made after the freeze) and a later judge; DeepSeek ran at low reasoning effort on 143 of its PRs, and GLM's review was missing a lens pass on 7. The GPT-6 Luna and Opus rows ran a September build; on the same 40 PRs as Opus, the frozen GPT-6.1 Sol build finds 33 of 34 high, 28 of 31 medium and 44 of 61 low. Read the table as directional.

AACR-Bench (185 PRs, 10 languages, 520 verified defects; a partly found defect counts ½)

Martian (50 PRs, Core; one gpt-6.1-sol judge for every row, the vendor rows re-judged from their published comments)

Qodo PR-Review-Bench (100 PRs; one gpt-6.1-sol judge under Qodo's stated matching rule; five more vendors are in the paper)

Held-out PRs. The last tuning round was mined from misses on part of each benchmark. On the PRs kept out of it, the frozen build finds 83% of AACR-Bench's defects (20 PRs), ties qodo-extended-v2 on Martian's real goldens (43 of 52, 30 PRs), and on Qodo (31 PRs) reaches 60.3% recall and F1 53.0, level with qodo-exhaustive's 51.3.

Nobody on your team is actually reviewing the volume of code your team now produces. The author asks their agent to write the code, the reviewer asks their agent to review it, the author replies "address the comments" — two humans spending a day as slow message brokers between two AIs. The review was already AI on both sides. The dead time was ours.

QuickReview moves human judgment up a level, to "is this something we need and want?", while the machine checks what is checkable. The philosophy, in short, is in MANIFESTO.md.

Optional, and each degrades cleanly when unconfigured: an operator TUI, a Slack concierge, a local companion session, opt-in escalation notes, and the OpenViking-backed reviewer memory.

You need Docker with Compose v2, a model login, and either gh signed in to your own GitHub account or a GitHub account that can create an App on the repository to review. On Windows, run everything inside WSL2.

There is a second route that needs no App: [github] auth = "user" runs the reviewer on your own GitHub account. Use it on one box where gh is already signed in. It is polling-only, with no webhooks and no tunnel. Every review it posts is a COMMENT that states its verdict, because an automated verdict must never count as your approval and GitHub refuses either on your own PR. Your token reaches the daemon only as the 0600 file <secrets dir>/github_token, which qcr github-token sync refreshes from gh. It never comes from the environment or a command line. On the App route no personal access token is used.

once: install qcr, the host wrapper the image writes (docs/host-wrapper.md);

the images are public on ghcr.io

docker run --rm --user "$(id -u):$(id -g)" -v ~/.local/bin:/out
ghcr.io/quickchatai/quickreview:<version> install

qcr # the home: sign in, connect GitHub, choose a repository

A bare qcr opens the home: QuickReview's own pi, with the plugin loaded and a panel of your repository's open pull requests. On a first run the panel is a short setup checklist, and anything already set up is found and skipped: sign in to OpenAI with pi's device code (device-code login must be on in ChatGPT → Settings → Security), check that gh is signed in, and choose a repository from your recently pushed ones or paste any owner/name or URL. alt+n runs the next step. The home then lists the pull requests, "To review" and "Reviewed". qcr tui opens the admin TUI, and only it starts the companion. Through the Docker wrapper the home runs in a container of its own: its login is kept in the qcr-login volume, its config and sessions in the qcr-state volume, and its gh uses the token the wrapper finds on this machine (GITHUB_TOKEN, GH_TOKEN or gh auth token). The always-on daemon, a GitHub App, webhooks and a server over ssh are the setup wizard's steps: docs/onboarding.md.

This computer, in Docker (docker/README.md): the pick when the machine is itself an always-on server, such as a cloud VM; on a laptop it reviews only while the machine is awake.

A server over ssh, recommended when your own computer sleeps. Setup rides the Ansible deploy (deploy/ansible/README.md); it needs passwordless sudo there and ansible-playbook on your machine.

Polling is the default: the daemon checks GitHub every few minutes and exposes nothing, at the cost of trigger latency up to one poll interval and quiet author-reply routing and [feedback] loop. Webhooks need an HTTPS URL: a Cloudflare quick tunnel, ngrok or your own named Cloudflare tunnel, on the App route only. The trade-offs are in docker/README.md.

dry_run = true stays on until you flip it, through the wizard's go-live step or by hand (docs/ownership.md); a dry-run install is a complete one whose reviews are files you read. Slack is set up separately, any time, with npm run slack:setup from a clone (the image does not carry it yet).

The npm package is the whole app, the same code the image runs, as the qcr command on Node 26 or later (npm, pnpm and bun all work; an older Node is not blocked, but only 26+ is supported):

npm i -g @quickchatai/quickreview

qcr login # pi's own /login, for the review's models qcr review # review this branch against its base, with no

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →