返回资讯中心
外部精选
软件工程
#开发者实时流

Show HN: Tracegate – local CI that only skips tests it has proven it can skip

Local CI that runs only what a change can affect, and earns the right to skip the rest.

Hacker News 最新davidtufts2026/10/31 · 17 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Local CI that runs only what a change can affect, and earns the right to skip the rest.

Tracegate records what each test actually touches: the functions it executed, the modules it loaded and the repo files it read. It writes that per-file footprint into a ledger. A test whose footprint nothing has changed since its last green run can reuse that verdict. It only gets that right after a measured evidence period:

It cannot turn a red into a green: a failing file's footprint is dropped, a moved tree is refused, and footprints older than 72 hours are never reused.

Tracegate was built and runs in production inside a private TypeScript monorepo of about 2,700 backend and 1,000 frontend test files. It sits in a local merge gate that batches pull requests into merge trains and bisects a red train automatically.

Phase 0 is done (ROADMAP.md §4): the deterministic core is extracted here as a standalone tool for Vitest + TypeScript.

Phase 0.5 is done: the broker. One queue for all of a machine's CI work: gates, merge trains, background pre-gating, measurements, deploy builds, and every AI agent session's test runs. Agents reach it over MCP. See Broker and docs/design/broker.md.

Phase 0.6 is done: adaptive capacity. The broker samples CPU, memory, memory pressure, swap and heat, and grants each job only the workers and memory the machine can take right now. Every default derives from the host, so one config serves a laptop and a 64-core runner. See Adaptive capacity.

Phase 1a is done: the Red Reader. Every red run leaves a packet that an AI agent session in the repository (Claude Code, for example) reads over MCP. The agent records an escalate-only opinion that never changes a verdict. See Red Reader.

Phase 1b is done: escalation. An escalated real opinion on a MISS raises the runner's automatic flake to hole, so the lane suspends. The stricter verdict wins, and a person's later triage has the last word. See Red Reader.

Phase 2 is done, in shadow: the Conductor. It ranks the files a run must execute fail-first and scores that order on every red run, without changing the set. It also advises how to compose a merge train: which PRs to gate alone, and which pairs have disjoint reach. See Conductor.

Phase 3 is done: dependency changes. A lockfile change used to run the whole lane. Now a test whose footprint loaded no changed package keeps its verdict, once the rule has met its own bar. An oracle for the remaining cases runs in shadow only. See Dependency changes.

Phase 0.5c is done: a make jobserver per lease (lease --jobserver), so make, Ninja and cargo children stay within the grant.

On npm as tracegate-ci (see CHANGELOG.md):

Most AI-in-CI products try to fix or skip. They publish no failure rates and have a documented habit of silencing tests. Tracegate CI goes the other way. The first model in the gate may only make it stricter: it reads every red the gate wants to write off as a flake and may escalate it to a real bug, never the reverse. Model decisions are cached by input hash, so a gate is replayable. They run in shadow before they decide anything, and the model gets no tools and no credentials.

Node.js ≥ 22, git.

Vitest 4.1 to 5.x, or Jest 30.

The recorder hooks the vm seam the runner evaluates modules through: Vitest's vm.runInThisContext, Jest's vm.compileFunction.

CI runs the end-to-end against each supported Vitest version (npm run test:vitest-matrix) and on a Jest copy of the sample (npm run test:e2e-jest).

A runner that moves the seam degrades to safe. tracegate doctor --canary tells you when that has happened.

TypeScript 5 or 6 installed in the target repository (used to parse source files).

TypeScript 7 is the native compiler, and its npm package has no JS parser API. A repository on 7 keeps it, and installs 6 beside it under the name Tracegate looks for first: npm i -D tracegate-typescript@npm:typescript@6. tracegate doctor says which one it parses with.

npm 10.9 can crash installing Vitest 4 (Cannot read properties of null (reading 'edgesOut')). That is an npm bug, not Tracegate's; --legacy-peer-deps avoids it.

Quick start

npm install --save-dev tracegate-ci npx tracegate init # proposes one lane per Vitest project; prints, changes nothing npx tracegate init --write --mcp npx tracegate doctor --canary # Node, Vitest, typescript, recorder, ledger, and one recorded file npx tracegate run # shadow at first: everything runs; status shows the countdown npx tracegate savings # what reuse would have saved so far, from the first run on

init --write does three things:

It never overwrites an existing config. The sections below are the same steps done by hand.

/plugin marketplace add davidscotttufts/tracegate-ci /plugin install tracegate@tracegate

The plugin bundles:

  1. Install and wire the recorder

npm install --save-dev tracegate-ci # or: npm install --save-dev /path/to/tracegate-ci

Add the recorder as the first setup file, only when a run asks for it:

// vitest.config.ts export default defineConfig({ test: { setupFiles: [...(process.env.TRACEGATE_RECORD ? ['tracegate-ci/vitest-setup'] : []), /* yours */], }, });

Unset, nothing is loaded and nothing changes.

A lane is one Vitest project. tracegate.config.json at the repository root:

{ "lanes": { "api": { "dir": "packages/api", "src": "src", "globals": ["test/setup/"] }, "web": { "dir": "packages/web", "src": "src", "entryBar": { "days": 1, "earlyAudit": 0.2 } } }, "entryBar": { "days": 14, "observations": 20 }, "audit": 0.05 }

Key Default Meaning

lanes.<l>.dir . The Vitest project directory, relative to the repo root.

lanes.<l>.src src Source directory inside dir. Only modules here are tracked per function; any other loaded module is tracked whole.

lanes.<l>.globals see below Extra inputs (relative to dir) whose change runs the whole lane. * matches within a path segment; a trailing / matches a directory.

lanes.<l>.rootGlobals, rootGlobals see below The same, relative to the repo root.

lanes.<l>.entryBar — Per-lane override of the bar. A shorter days is an early entry; earlyAudit is the audit rate until the default bar's days would have been met.

lanes.<l>.lockfile auto The lockfile hashed into every footprint (default: the first of package-lock.json, npm-shrinkwrap.json, pnpm-lock.yaml, yarn.lock in dir, then at the root).

lanes.<l>.replayExclude [] Test-file patterns replay must not list (e.g. an e2e folder Vitest does not run).

entryBar { "days": 14, "observations": 20 } Enforce only after this many observations over this many days, with no un-triaged miss.

audit 0.05 Share of reusable verdicts re-run anyway, uniformly at random.

missSuspendDays 14 How long a miss suspends enforcement.

maxAgeHours 72 Footprints older than this are never reused (TRACEGATE_MAX_AGE_HOURS).

fullRunMaxHours 24 A lane whose last complete green full run is older runs whole. 0 disables.

keepPerTest 6 Footprints kept per test file.

ingestBudgetSeconds 300 Ingest stops recording after this long (TRACEGATE_INGEST_BUDGET_S).

ledger <git common dir>/tracegate Ledger directory, relative to the root (TRACEGATE_LEDGER).

sync — Default --remote for tracegate sync.

Global inputs that are always included: in each lane dir package.json, lockfiles, vitest.config., vitest.workspace., vitest.setup., vite.config., tsconfig*.json; at the root package.json, lockfiles, .nvmrc, .node-version, .tool-versions, tracegate.config.json. Configured globals add to these. Add your setup-file directory (as test/setup/ above): setup files are loaded by every test.

Without a config file there is one lane, default, at the root with src/.

The one-command form records, reports and ingests:

npx tracegate run --lane api -- --maxWorkers=2 # args after -- go to vitest run

It does, in order: pick the mode (TRACEGATE / TRACEGATE_<LANE>, default auto) → select → vitest run with the recorder, the selection (under enforce) and a JSON reporter → report → re-run each MISS alone (a pass is triaged flake) → ingest (only if HEAD did not move) → record a complete full run if the whole lane ran green. The exit code is Vitest's. If selection fails, the lane runs whole and records nothing.

The same as three steps, for a CI script of your own:

RUN=$(mktemp -d) npx tracegate select --lane api --run "$RUN" # writes $RUN/selected.txt, predict.json HEAD0=$(git rev-parse HEAD) ( cd packages/api && TRACEGATE_RECORD="$RUN" TRACEGATE_LANE=api npx vitest run
--reporter=default --reporter=json --outputFile.json="$RUN/results.json"
$( [ "$(npx tracegate mode --lane api)" = enforce ] && cat "$RUN/selected.txt" ) ) npx tracegate report --lane api --run "$RUN" # logs misses to the ledger npx tracegate ingest --lane api --run "$RUN" --expect-head "$HEAD0"

(Under enforce with an empty selected.txt, skip Vitest: it would otherwise run everything.)

npx tracegate status # ledger, entry bar and mode per lane npx tracegate savings [--lane api] [--json] # reusable file runs and test time so far npx tracegate report-card [--json] [--out docs/evidence/card.md] # miss rates with 95% intervals npx tracegate mode --lane api # off | shadow | enforce npx tracegate triage --lane api --file packages/api/test/x.test.ts --verdict flake|hole [--head <sha12>] npx tracegate replay --lane api --head <commit> --at <iso time> [--drop <regex>] [--ignore-lock] [--out r.json] npx tracegate full-done --lane api # record a complete, green whole-lane run npx tracegate sync --remote user@host:/abs/path/.git/tracegate

triage: hole = a real footprint gap (fix the recorder); flake = the test fails on its own. Only flake lifts the suspension. Passing when re-run alone is not proof of a flake: read the failure. A wrong argument whose assertion depends on timing passes alone.

savings reads the shadow log, so it has numbers from the first recorded run, long before the entry bar lets a lane skip anything. A file that was reusable and then failed (a MISS) never counts as a saving. The projection keeps the audit sample.

report-card publishes Tracegate's own error rates per lane, from the append-only logs: misses per checked verdict and per run with 95% Wilson intervals, each miss's triage, the longest a hole could have gone undetected (time since the last full run before it), audit cost, and the Red Reader and Conductor scores. See docs/evidence/.

Why not Vitest recordDependencies? docs/compare/vitest-record-dependencies.md runs both on the same commits. In short: Tracegate selects by function, sees files read at run time, and audits its own skips; neither sees what a child process reads.

replay re-decides a past run offline, as of its clock and with only the footprints recorded before it, optionally with causes removed. It answers "what would this run have executed if this change had not been in it?". It is read-only on the ledger.

The ledger

$(git rev-parse --git-common-dir)/tracegate/, shared by every worktree of the clone: fp/ and blobs/ are a cache (deleting them is always safe and only costs predictions); shadow-.tsv, misses-.tsv, triage-.tsv, reasons-.tsv, full-*.tsv are append-only evidence, one file per host. Never delete the logs: the entry bar reads them.

When several things share one machine (gates, a background daemon, measurements, deploy builds, and AI coding agents each running tests), let one process hand out capacity:

A gate: 4 of the machine's tokens, about 15 minutes.

tracegate lease --class gate --tokens 4 --est 900 --label "gate api" -- npm run ci

A measurement that needs the machine to itself. It reserves the next window and

starts by itself; nobody has to say "go".

tracegate lease --class measure --exclusive -- ./bench.sh

Background work that yields when anything important needs the room (exits 75).

tracegate lease --class daemon --tokens 2 --requeue -- tracegate run --lane api

tracegate queue # who holds what, who waits, and the reservation tracegate explain j42 # why j42 waits, and when it is reserved to start

The first lease s

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →