
I have an issue with what agets affirm, do you have too?
Independent verification for code written by AI agents.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Independent verification for code written by AI agents.
Elliot checks what your agent claims, and what your README promises, against the code. Every finding comes with proof.
Quick start · What it found · How it works · Commands · Configuration · FAQ
npx -y github:EmanueleCell4/elliot-mcp init
Restart your agent, then type:
/elliot-audit
A minute or three later the report is in elliot-report/<date>-<commit>/ at the root of the repo: report.html, report.md and claims.json. There is nothing to open, no dashboard and no account.
Requirements: Node 20 or newer, git, and Claude Code, Cursor or Codex.
Coding agents say "Done ✅" when the work isn't done. They delete failing tests, skip checks, describe security that doesn't exist, and write READMEs that promise what the code never does. CI stays green because tests check the code the agent wrote, not the promises the product makes.
I found this out the hard way. My own legal-tech app had a security page promising role-based access enforced in the database. The data layer bypassed it by design, and every permission check was a function that ignored its input. The agent had told me it was done.
The usual fix is to ask a second AI to review the first one. That gives you another opinion from the same kind of system that made the mistake. In the first automatic audit I ran, the agent doing the checking cited line numbers that don't exist in 4 of 17 submissions.
Elliot works the other way around: the agent proposes, Elliot verifies. Every quote must really be in the file, at that line, at that commit. Every "this doesn't exist" is searched by Elliot itself. A fresh reviewer then tries to prove each finding wrong before it reaches the report.
Elliot 0.3 ran unattended inside Claude Code on 9 popular open-source tools from the AI coding-agent ecosystem (October 2026). A separate reviewer that had not seen the audits then checked every finding against the code.
What it still misses: on the same 9 repos it found 6 of the 10 serious issues I had found by hand. The most common miss is a missing authorization check (for example, a download endpoint that doesn't check who is asking). Elliot doesn't look for those yet; see the roadmap.
If you wrote code in the same session, the protocol tells the agent to run the audit in a fresh subagent, so the auditor is not the author.
The full report is in examples/lex-ai-report.html. It was produced by an earlier version, before the independent review step existed.
Set-up: init
npx -y github:EmanueleCell4/elliot-mcp init # every agent it detects npx -y github:EmanueleCell4/elliot-mcp init --claude # only Claude Code npx -y github:EmanueleCell4/elliot-mcp init --cursor # only Cursor npx -y github:EmanueleCell4/elliot-mcp init --codex # only Codex npx -y github:EmanueleCell4/elliot-mcp init --all # all three
init is idempotent: running it twice changes nothing, and it never overwrites a command file you edited. It writes:
Run it in your agent from the repository root. In Claude Code and Codex, anything you type after the command is passed to the agent as extra instructions, for example /elliot-audit focus on the security page.
The same server also checks individual claims while the agent works. It ships instructions telling the agent to use them before it says "done".
These checks answer VERIFIED, NOT_VERIFIED or INCONCLUSIVE. Inconclusive means Elliot could not confirm, for example because of a GitHub rate limit. It never guesses.
Tool Used by What it does
elliot_audit_start the agent Pins the commit, lists the files that make claims, runs the risk sweep, returns the protocol.
elliot_record_claim the agent Records one claim with a verdict. Checks the source quote, every evidence quote and every absence search.
elliot_record_behavior the agent Records risky code behaviour the docs never mention, with checked code quotes.
elliot_review_packet the agent Returns a self-contained prompt for a fresh reviewer subagent.
elliot_record_review the reviewer Confirms, downgrades or retracts a finding. A retraction needs a code quote.
elliot_audit_finish the agent Refuses to finish while risk-sweep items or reviews are missing, then writes the report.
elliot_snapshot the agent Fingerprints the working tree before a task.
verify_changes the agent Compares the tree with a snapshot, or with HEAD.
verify_url the agent Checks status and expected text at a URL.
verify_github the agent Checks pr_merged, checks_passing or commit_exists.
verify_npm the agent Checks a package version on the registry.
elliot_report you Summarises recent receipts.
Verdicts in an audit
Verdict Meaning
VERIFIED The code implements the claim. A small gap that misleads nobody is kept as a note.
PARTIAL Implemented, with a gap that would make a user who relies on it do something wrong.
FALSE The code contradicts the claim, or the feature, command, flag or option does not exist.
UNVERIFIABLE Depends on a deployment, a contract or a service outside the repo.
UNDISCLOSED Risky behaviour in the code that the docs never mention.
Manual installation
Use this if you'd rather not run init.
claude mcp add elliot -- npx -y github:EmanueleCell4/elliot-mcp
Codex (~/.codex/config.toml)
[mcp_servers.elliot] command = "npx" args = ["-y", "github:EmanueleCell4/elliot-mcp"]
Cursor (.cursor/mcp.json) and any other MCP client
{ "mcpServers": { "elliot": { "command": "npx", "args": ["-y", "github:EmanueleCell4/elliot-mcp"] } } }
Then add the command yourself, or ask your agent to "run an Elliot claim audit of this repository". For agents that support skills, also add skills/elliot/SKILL.md.
All settings are optional environment variables.
your-repo/ ├── elliot-report/ │ └── 2026-10-22-dd84474/ │ ├── report.html # open in a browser │ ├── report.md # paste into an issue or PR │ └── claims.json # everything, for scripts and CI └── .elliot/ ├── receipts.jsonl # one tamper-evident receipt per check └── audits/ # audit state
Add .elliot/ to .gitignore. Commit elliot-report/ if you want the report to live with the code.
{ "elliot": "0.3.0", "id": "rcpt_3f9a1c2b7d10", "check": "claim", "claim": "Role-based access is enforced at the database layer through row-level security.", "target": "lex-ai@dd84474", "verdict": "FALSE", "reason": "All data access uses the service-role client, which bypasses RLS.", "evidence": { "source": { "file": "app/security/page.tsx", "line_start": 14 }, "evidence": [ { "file": "lib/supabase/index.ts", "line_start": 10 } ] }, "checked_at": "2026-10-21T19:42:11.000Z", "checker": "local", "digest": "sha256:…" }
The digest makes a receipt tamper-evident: change a field and the digest no longer matches. A local receipt is still produced on your own machine, so it proves nothing to a third party. Independently signed receipts are on the roadmap.
Isn't this just an LLM judging an LLM? No. The agent proposes verdicts, but every piece of evidence is checked deterministically: quotes against files at a pinned commit, absences with Elliot's own git grep. A second agent that never saw the audit reviews every finding, and a retraction has to be backed by a code quote that Elliot also checks.
Does it send my code anywhere? Elliot itself makes no network calls during an audit. It reads files and runs git locally. Your agent reads the code as it normally does, with the model you already use. The verify_url, verify_github and verify_npm checks call those services only when the agent uses them.
Does it change my code? No. It writes only .elliot/ and elliot-report/, and init writes the config files listed above.
What does it cost? Nothing. Elliot is MIT-licensed and the audit runs on the agent and model you already pay for. An audit of a medium-sized repo took 1 to 3 minutes in our tests.
Can it audit private repos? Yes. It runs where your code is. Only verify_github needs GITHUB_TOKEN for private repos.
Can I trust a PASS? Treat a clean report as "nothing found among the claims checked", not as a certificate. Elliot is a static reading of the code. It does not run your app yet, and it does not look for missing authorization checks yet.
I found a false positive. Please open an issue with the report's claims.json entry. False positives are the number Elliot is built to keep at zero.
File, URL, GitHub and npm checks with local receipts
/elliot-audit: claim audit with checked citations, report in elliot-report/
Risk sweep, undisclosed behaviours, materiality test and independent review
Authorization checks: unauthenticated handlers, missing ownership checks
Verify the agent's own "done" at the end of every task, by running tests and the app
Hooks that run Elliot automatically when the agent stops (Claude Code, Codex)
A GitHub Action that audits the claims touched by each pull request
Independently signed receipts with a public verification link
Development
git clone https://github.com/EmanueleCell4/elliot-mcp && cd elliot-mcp npm install npm test # 19 end-to-end tests: spawns the server over stdio and drives it with the MCP client
The server is plain Node (ES modules) with two dependencies: the MCP SDK and zod. The main pieces:
Issues, false positives and stories of agents telling you they were done are very welcome.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。