
Tokenade is now Open Source
Cut your AI coding agent's token bill — locally, automatically, on every prompt.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Cut your AI coding agent's token bill — locally, automatically, on every prompt.
Install · What it does · Benchmark · Website
Tokenade is a single Rust binary that sits between your coding agent (Claude Code, Codex, Cursor, Copilot CLI, OpenCode, Kilo Code, Qwen Code…) and the model. It strips what the model never needed to see — 2,000-line build logs, whole files when one function mattered, verbose MCP payloads, re-reads of content already in context — before it is billed, and keeps every byte it folds recoverable on demand.
Tokenade ranks #1 on THOL, the Token-Harness Optimizer Leaderboard — real end-to-end agent runs, every optimizer measured the same way: 39% cheaper on long sessions. Method and full results: https://tokenade.net/en/thol.
Command compaction — test runs, builds, logs, git, kubectl, docker, package managers… noisy output is folded to what matters (failures, numbers, decisive lines) before the agent reads it.
Read deduplication — content the agent already has comes back as a short reference, or as a diff when it changed.
Code intelligence — map, query, skeleton, impact and hybrid semantic search, so the agent stops reading whole files to find one thing.
MCP optimization — MCP servers are wrapped: responses compacted in transit, tool manifests trimmed to their callable skeleton.
Web — tokenade web / tokenade search return compact Markdown and result lists instead of raw HTML.
Context and output — prompt-cache-aware compaction of stale context, and a lean-output style note that drops filler without dropping facts.
Secret redaction — credentials are scrubbed from what Tokenade logs, caches and folds.
Nothing is lost — every fold carries a hash; tokenade expand-ref gives the exact bytes back.
Install
npm (prebuilt binaries)
npm install -g @tokenade/cli tokenade install # detects your coding agents and wires Tokenade in tokenade healthcheck # optional: verify the wiring
The binary for your platform ships as an optional per-platform npm package; if that is skipped, postinstall downloads the archive from this repository's GitHub Release and checks it against the release's Ed25519-signed manifest (SHA-256). Linux (x64 glibc/musl, arm64), macOS (x64, arm64), Windows (x64, arm64). Node ≥ 18.
Prebuilt archives are also attached to every GitHub Release.
Requires a recent stable Rust toolchain (rustup) and a C/C++ compiler (tree-sitter grammars and SQLite are compiled from source).
git clone https://github.com/pi-infected/tokenade cd tokenade cargo install --locked --path crates/tokenade-cli # or use a script below tokenade install
Per-OS scripts in scripts/build/ build the same targets as the release CI and print where the binary is:
The semantic-search embedding model is downloaded on first use (Hugging Face, with a signed GitHub Release asset as fallback) and cached; set TOKENADE_MODEL_DIR to a directory that already holds it for offline machines.
Full integration (hooks: command compaction, read dedup, web, MCP, code search, lean output, context optimization): Claude Code, OpenCode, Kilo Code, Copilot CLI, Hermes, Cline, T3 Code.
Most features: OpenClaw and Pi (no MCP wrapping); Qwen Code (no web replacement); Codex CLI and the Codex app, Cursor, Grok (no web replacement or read dedup); Droid (no web, MCP wrapping or read dedup).
Partial: Aider (run commands through tokenade wrap …), Antigravity and Claude Desktop chat (MCP optimization only). Claude Desktop's Code tab runs Claude Code and gets full integration.
tokenade install --only <agent> limits the install to one agent; tokenade uninstall removes everything it added.
Your savings ledger and dashboard (tokenade gain, tokenade dashboard) stay on your machine. The only network calls are:
Turn auto-update off with TOKENADE_NO_AUTO_UPDATE=1 (or tokenade upgrade --off); tokenade upgrade updates on demand. TOKENADE_NO_WEBSEARCH=1 / TOKENADE_NO_WEBFETCH=1 keep your agent's own web tools.
1.x installs update themselves to 2.0 (or run npm i -g @tokenade/cli@latest). On its first run, 2.0 deletes the leftover 1.x licence, quota and usage-report files under ~/.tokenade once and keeps your savings history. tokenade login, activate and report no longer exist. Run tokenade install again if your agent's hooks look stale.
tokenade wrap 'cargo test' # run a noisy command, get the compacted output some-cmd | tokenade read - # compact any blob; format auto-detected tokenade map # codebase structure tokenade query parse_config # locate a symbol tokenade skeleton src/lib.rs # signatures only tokenade semantic "where is auth" # search by meaning tokenade expand-ref <hash> # get folded bytes back tokenade dashboard # what you saved: graphical local report tokenade gain # same totals, in the terminal tokenade help # everything else
Contributing
Bug reports, compactor fixtures and pull requests are welcome — see CONTRIBUTING.md and the Code of Conduct. Security issues: see SECURITY.md.
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in this project by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。