返回资讯中心
外部精选
软件工程
#开发者实时流

Emet 推出 AI 代码核查工具:不评分,直接在你的数据上重算答案

来源标题:Emet: Code checking AI. Not another model grading the answer

Emet 发布一款只读验证工具,在 AI 智能体给出回答后,用同样的查询在自己的数据上重跑一遍,并用代码重新推导每个数字,逐句核对结论。它读取智能体导出的完整工具调用记录,输出准确率报告并通过 Slack 告警,强调并非再让一个模型给答案打分。

Hacker News 最新fuse_raf2026/11/17 · 今天12 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Emet shows you what your agent did, and whether what it said is true — it asks your data the same question and re-derives every number in code, with the math.

Read the integration guide →

QUERY AUDIT / Q2 ROAS

YOUR SYSTEMEMET / READ-ONLY VERIFICATIONHTTPSAFTER THE ANSWER IS SENT01Your agent answers your user

02The finished turn is exported, after the answer is sent

03Emet re-runs the query on your data (read-only)

04Every number re-derived in code

05Every sentence checked against the result

06Accuracy Report + Slack alert

01

Your systemYour agent answers your user

The finished turn is exported, after the answer is sent

Emet / read-only verificationEmet re-runs the query on your data (read-only)

Every sentence checked against the result

Emet reads the finished turn your agent exports: every tool call, the arguments it was called with, and the raw result that came back.

{ "span_id": "tool.call_1", "kind": "tool", "tool_name": "get_meta_insights", "payload": { "arguments": { "metric": "roas", "date_start": "2026-06-16", "date_end": "2026-06-22", "attribution": "7d_click_1d_view", "time_zone": "America/New_York", "currency": "USD" }, "result": { "roas": 3.46 } } }

Next step →

01 / 05What your agent did.

Emet reads the finished turn your agent exports: every tool call, the arguments it was called with, and the raw result that came back.

A perfectly grounded answer built on a wrong query is still wrong. Emet checks the question your agent asked of the data, not just the number it got back.

Emet holds its own read-only access to your data and asks your warehouse the same question, correctly scoped. It never asks the AI what the data said.

Every factual claim gets a verdict and a reason. Code decides the numbers. Then the wording of each claim is checked on its own by an independent model from a different AI provider than the one that wrote the answer — it never scores the answer as a whole, and it can't overrule the math.

Your user gets the answer at full speed. Moments later the Accuracy Report lands — and if a number was wrong, the owner gets the fix in Slack before it reaches a deck.

{ "span_id": "tool.call_1", "kind": "tool", "tool_name": "get_meta_insights", "payload": { "arguments": { "metric": "roas", "date_start": "2026-06-16", "date_end": "2026-06-22", "attribution": "7d_click_1d_view", "time_zone": "America/New_York", "currency": "USD" }, "result": { "roas": 3.46 } } }

The AI said strongest quarter yet. Your data said down 50%.

ROAS for one week reported as the quarter

Last-click on a data-driven account; a date range off by one in the account's time zone; spend summed across currencies

"Campaigns were ~26.8% of email volume." Actual: 98.7%.

A period-over-period change computed against the wrong prior period

"Email was over 96% of revenue in August and YTD." True for August only.

"The $13,867 gap is due to a 5-day attribution window." Nothing in the data says why.

Examples from a one-week client case study at a multi-brand retailer.

Interpretive claims — opinions and recommendations — are deliberately not scored.

Eval tools check faithfulness to context. Emet checks truth against the source.

LLM evaluation tools: The context the model saw, or a test set you maintain

Emet: Your live source data, re-queried with its own read-only access

LLM evaluation tools: An LLM judge, or a scorer you write yourself

Emet: Deterministic re-derivation built in, per data source; an independent model checks the wording

LLM evaluation tools: Only if you write that check

Emet: Built in: dates, windows, time zones, currency, attribution

LLM evaluation tools: A score per response or per test

Emet: A verdict and a reason per claim, with the math

Keep your tracing. Langfuse, Arize Phoenix, LangSmith and Logfire keep working, and Emet runs alongside them.

At onboarding we confirm every data source is connected and Emet can query each one, so it knows exactly what it's checking against.

ROAS, pipeline, margin — as your business defines them, not a generic default.

Fiscal quarters, attribution windows, time zones and currencies, mapped once.

#emet-alerts

EmetAPP9:00 AM

CORRECTED Meta Ads ROAS — Q2 Paid Media analysis

AI said → Emet re-derived: 3.46x 2.43xFirst seen: Just now

claim"Q2 ROAS was 3.46x"re-derivedroas_q2 = 90,110.63 ÷ 37,084.09 = 2.43xwhythe AI queried Jun 16–22 instead of all of Q2 (Apr 1 – Jun 30)

Open Accuracy ReportAcknowledgeAssign to…

Example Slack alert · actions shown for reference

Slack — one alert per issue, not per report

Emet Observability Dashboard — every report, plus accuracy, usage and cost by team and model

Read-only, scoped by you, revocable at once

Never in the request path — exports run after your user has the answer

Stores the trace, the claims, the verdicts and the query results behind them — never a copy of your tables.

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →