
Self Improvement Harness based on pi. pi-harness-rsi
A pi extension for harness self-improvement (RSI, after Harness² / arXiv:2610.14224)
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
A pi extension for harness self-improvement (RSI, after Harness² / arXiv:2610.14224) with enforced context economy: every artifact that may enter an LLM context is budgeted by one choke point, and nothing is ever silently truncated.
Design: budgets, the editable surface, the safety model and a full example workflow are documented below — and enforced in code (src/context.ts, src/guard.ts, src/ledger.ts) with an executable test suite, so every invariant here is executable rather than aspirational. The loop: snapshot → probe ×k → propose INV/DOM ×2k → probe ×2k → compose ×k → probe ×k → select → adopt → reflect.
pi -e /path/to/pi-harness-rsi/index.ts # one-off pi install /path/to/pi-harness-rsi # registers in settings
Commands
One command, verb first. The verb may be omitted: then the whole line is a start task.
/rsi <verb> [flags] <args>
Verb What it does
(none) State view — the in-flight run if there is one, otherwise the latest ledger and the exact command to type next. Never calls a model.
start <task> Full loop: probe → propose → probe → compose → probe → select → adopt → reflect. Flags: --k <n> --domain <name> --mode parallel|sequential --yes --force
probe <task> Reference executions + digests only (no editor, no edits, no adoption). Flags: --k <n> --domain <name>
status [runId] The in-flight run, else the latest ledger: identity R=4k, P=3k, USD, fallbacks, diversity, budget usage.
watch [--once] [--lines <n>] Live view of the in-flight run: stage, elapsed, USD, latest notes — polls until the run closes. --once prints a single snapshot instead.
stop Interrupt the in-flight run before its normal end (see Interruption below).
adopt [label] [--run <id>] [--yes] [--force] Human gate: apply a composed surface, snapshot the library, reflect.
revert <domain>/<slug> [--force] Restore the previous adopted surface for one library entry, under the same guards as adoption. Must resolve, else it errors.
library [domain] Domains, entries, INDEX.md, EXPERIENCE.md.
help The table above, in the terminal.
/rsi with no arguments is the entry point: it prints where things stand and what to type next.
The three commands you actually type — everything else is for when you want to look:
/rsi probe fix the flaky session test # 1. what does the harness do today? (no edits) /rsi status # read the digests, then... /rsi start --domain acme run the verifier after every edit # 2. learn the habit (asks before adopting) /rsi # 3. what next? one exact command
Every command that changes state ends by printing exactly one command to type next, so the workflow is discoverable from the terminal (/rsi alone) and from the end of any run.
Argument rules (they are what keep the surface predictable):
Adoption and reflection also maintain the library's health: a new lesson whose cue matches an older one supersedes it (the old entry stays on disk for audit, but stops entering missions), and every rendered lesson carries its adopted: date so the editor can weigh staleness.
Headless (pi --print "/rsi …") mirrors every report to stdout; adoption stays gated — without --yes a headless run ends as outcome: pending and prints the /rsi adopt … command to run next. There is no input loop to keep responsive, so headless runs await the loop instead of detaching it: the process exits when the run has actually finished, not while it is still starting.
A running ledger whose owner pid (recorded in the ledger, same host) is no longer alive is reported as orphaned — /rsi status says so and /rsi watch does not present it as in flight, so a crashed or killed run cannot masquerade as a live one.
Only one run can be in flight per project: start, probe and adopt refuse while another run is active (in this session or in another process — see the run lock); watch inspects it and stop interrupts it.
Prerequisites: pi ≥ 1.3, bun, the extension loaded (see Load it), and a verifier you trust — the whole loop is evidence-driven, so point rsi.verifier at the command that tells you whether the project is healthy:
// ~/.config/pi/settings.json (or the project's .pi/settings.json) { "rsi": { "verifier": "bun test", "maxUsd": 2 } }
First run — look before you leap. Start with a probe: it executes the same task k times against the current surface in a throwaway worktree and writes digests (errors, tool timeline, rework, verifier output). No editor, no edits:
/rsi probe fix the flaky session test
Read /rsi status. The digests under .pi/harness/runs/<id>/digests/ tell you where the harness bleeds: repeated identical calls, failing checks, paths never read. That is the input the improvement loop will act on.
Then run the loop. The task is not “the change you want” — it is the capability you want the harness to acquire (it is what the reference executions measure):
/rsi start --domain acme-api run the verifier before every edit and re-read files after editing
What happens, stage by stage:
Iterate. Repeat /rsi start with a new task in the same domain; each run feeds on the previous library (INDEX.md + top lessons enter the propose mission, still budget-capped). Check or undo with:
/rsi status # latest run + the exact next command /rsi library acme-api # domains, INDEX.md, EXPERIENCE.md /rsi adopt final0 --run <id> # apply a candidate you left pending /rsi revert acme-api/<slug> # restore the previous adopted surface
Headless / scripted:
pi --print "/rsi probe --domain acme-api $TASK" # evidence only, silent adoption impossible pi --print "/rsi start --domain acme-api --yes $TASK" # explicit opt-out of the gate
Cost & knobs: a k=1 run is 4 solver probes + 3 editor sessions (7 child-pi calls, each capped by stageTimeoutMs); with small models that is typically a few tens of cents, and the run aborts fail-closed at maxUsd (default $2). --k 2 doubles both sides of the identity (R=8, P=6) and buys a real selection stage. PI_RSI_FAKE=1 exercises the entire loop with zero model calls.
Scenario: acme-api (TypeScript). The agent keeps editing without running the tests and re-reads nothing, so it repeats the same failing check three times per session. You want the harness to learn the habit, not a one-off reminder.
$ cd ~/code/acme-api && pi
▸ /rsi probe add a rate-limit header to the login endpoint … 1 reference run, digest in .pi/harness/runs/20261018-121500-add-a-rate-limit/digests/ref0.md
▸ /rsi status run 20261018-121500-add-a-rate-limit task: add a rate-limit header to the login endpoint ledger: R 1 rollouts (probe-only run — R=4k/P=3k identity does not apply) usd: 0.0412 / max 2 (setting) outcome: probed notes:
- ref0: # digest: ref0 (model=…, 96s, turns=11) — 1 deliverable file(s), verifier=yes
▸ cat .pi/harness/runs/20261018-121500-add-a-rate-limit/digests/ref0.md
digest: ref0 (model=…, 96s, turns=11)
tools: bash×4 (2 err) edit×2 read×5 errors:
- bash exit=1 @step=3 window: "FAIL src/auth.test.ts …" deliverable: src/auth.ts | git-diff: +41 −12 verifier: bun test exit=1 first_divergence: @step=3 hypothesis: bash failed first — the rest of the run reacts to it rework: 2 consecutive identical grep calls
→ the digest shows the real defect: 2 identical greps, 2 failing checks, no re-read.
▸ /rsi start --domain acme-api run the verifier after every edit and re-read files before the next edit rsi: profile=small k=1 mode=parallel fake=0 maxUsd=2 confirmAdopt=1 [rsi] final0: # digest: final0 (model=…, 88s, turns=9) — diff 2 file(s) changed rsi: adopt final0? adopt final0: added 2, removed 0, modified 1, always-on Δ +312 chars, surface Δ +1 104 chars
- .pi/skills/acme-api-notes/SKILL.md
- .pi/skills/acme-api-notes/scripts/check.sh ~ AGENTS.md The live tree is updated and a library snapshot is written. ✔ Yes rsi: adopted final0 · USD 0.0873 · R 4/4 P 3/3 identity OK /rsi library acme-api
▸ git status M AGENTS.md ?? .pi/skills/acme-api-notes/ # review and commit these like any other change ?? .pi/harness/library/ # your learned experience — version it
(runs/ and worktrees/ are self-ignored)
▸ /rsi library acme-api --- INDEX.md ---
- add-a-rate-limit-header — checks failing after an edit → re-read the edited file and run the verifier before the next edit (2026-10-18)
Next week, /rsi start --domain acme-api <new task> loads that lesson into the propose mission automatically (cue match), and the new reference digests sit next to the old ones — so whether rework actually went down is measurable, not vibes. If a run makes things worse:
▸ /rsi revert acme-api/add-a-rate-limit-header rsi: restored previous surface for acme-api/add-a-rate-limit-header (AGENTS.md)
(The pre-adoption surface comes back and the skill files it did not have are removed again — previous/ in the library is what makes this possible.)
Everything is auditable: each run’s missions, digests, session evidence, candidate surfaces and the ledger (R=4k, P=3k, USD, fallbacks, budgets) live under .pi/harness/runs/<id>/ — nothing happened that the ledger cannot account for.
AGENTS.md (always-on, char-budgeted per profile)
.pi/prompts/<name>.md
.pi/skills/<name>/{SKILL.md, scripts/**}
Because scripts/** is executable capability mounted into every probe sandbox, candidates are script-scanned (fetch-and-execute, sudo, credential paths, reverse shells, env exfil — fail-closed like the tripwires), and the adoption gate shows the content of every added/modified script (bounded excerpt
Never editable: tests/**, /*.test.ts, .github/, package.json, lockfiles (rsi.forbiddenGlobs), plus grader-gaming tripwire phrases (pass_rate, delete the failing test, …) scanned in every candidate.
The extension manages most of this itself: on every run it writes .pi/harness/.gitignore containing worktrees/, runs/ and run.lock (idempotent, it never touches .gitignore at the repo root or .git/info/exclude), so the ephemeral artifacts below never show up in git status.
Commit these — they are the harness's durable state, and without them a fresh clone starts from zero:
Never commit — regenerated and/or self-contained:
If you prefer a single rule at the repo root instead of the generated file:
.pi/harness/runs/ .pi/harness/worktrees/ .pi/harness/run.lock
(Do not ignore .pi/harness/ or .pi/harness/library/ wholesale — that would silently drop the learned experience you are supposed to version).
Artifact Budget
AGENTS.md 6 000 chars
trajectory digest 6 000 chars, fixed section shape, drops lowest-priority sections whole
propose mission 16 000 chars
compose mission 24 000 chars
select mission / reflect mission 16 000 / 8 000 chars
<notes> 600 charsskill description / skill body warn 300 / 4 000 chars
context.assemble() is the single choke point: mandatory parts raise (BudgetError) instead of truncating; optional parts are dropped whole with a
dropped-over-budget notice so the editor gets a pointer, never a slice.
Profile large doubles these (set rsi.profile).
{ "rsi": { "profile": "small", // or "large" (doubles every budget below) "k": 1, // references per domain index (R=4k, P=3k) "mode": "parallel", // or "sequential" "editorModel": null, // child-pi model overrides (null = inherit the session's model) "solverModel": null, "thinking": null, // child-pi --thinking level, when set "tools": "read,bash,edit,write,grep,find,ls", // solver tool allowlist "editorTools": "read,grep,find,ls", // editor is answer-only "verifier": null, // e.g. "bun test" — evidence line, never a score "confirmAdopt": true, // keep the P6 human gate on "requireClean": true, // refuse to start on a dirty tree (unless --force) "maxUsd": 2, "maxParallel": 2, "libraryDir": ".pi/harness/library", // absolute or repo-relative "worktreeRoot": ".pi/harness/worktrees", "keepRuns": 20, // run dirs kept under .pi/harness/runs (pending adoptions are never pruned) "stageTimeoutMs": 600000, "budgets": {}, // per-key overrides on the active profile "forbiddenGlobs": [] // replaces defaults when non-empty } }
Environment overrides: P
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。