
Wiretuple:用位置元组无损减少 LLM 结构化输出的 token 消耗
来源标题:Show HN: Wiretuple – Cut LLM structured-output tokens with positional tuples
Wiretuple 是一个用 Go 编写的 HTTP 反向代理,可将 LLM 结构化输出和函数调用的输出 token 削减 35%–60%,且不损失信息。它把重复的 JSON 键名改写为位置元组,响应在下游再还原为完整 JSON,对调用方保持不变。项目无需外部依赖,单二进制即可运行,兼容 OpenAI 和 Anthropic 客户端。
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Cut 35–60 % of the output tokens of LLM Structured Outputs and Function Calling — losslessly. wiretuple is a single-binary HTTP reverse proxy written in Go with zero external dependencies: your app keeps calling the same API, and repeated JSON keys become positional tuples on the wire.
client ──► wiretuple ──► upstream API │ request: schema {"items":[{"id","total","currency"}]} rewritten to prefixItems + "Tuple: [id, total, currency]" │ response: [["a1", 12.5, "EUR"], ["a2", 3, "USD"]] rehydrated to [{"id":"a1","total":12.5,"currency":"EUR"}, ...] │ └─ incremental SSE rehydration, same bytes downstream
Why
A schema that asks for 40 line items with 7 fields makes the model emit the same seven key names 40 times. Those keys are pure overhead: the receiver already knows them from the schema. wiretuple moves the key list into one legend and sends values as positions.
go build -trimpath -ldflags="-s -w" -o bin/wiretuple ./cmd/wiretuple
No go.mod requirements, no cgo, no runtime data files.
OpenAI-compatible client
export OPENAI_BASE_URL=http://127.0.0.1:8787/v1 ./bin/wiretuple -upstream https://api.openai.com
Anthropic-compatible client
export ANTHROPIC_BASE_URL=http://127.0.0.1:8787 ./bin/wiretuple -upstream https://api.anthropic.com
Flags:
Provider Endpoint Structured output Function calling
OpenAI /v1/chat/completions response_format.json_schema.schema tools[].function.parameters
Anthropic /v1/messages tool schema in input_schema tools[].input_schema
Both stream: false (recomputed Content-Length) and stream: true (SSE, rehydrating only the JSON fragments) are covered. Anything that is not JSON (for example [DONE] or a non-JSON event) is forwarded byte for byte.
Small payloads are not worth a legend. The gate estimates saved output tokens against the fixed legend cost and keeps the request byte-identical when net savings are below -break-even:
net_savings_% = (tokens_before - tokens_after - legend_tokens) / tokens_before * 100
A single flat object usually stays untouched; a repeated object array with long keys activates.
./bin/wiretuple -upstream https://api.openai.com -shadow audit.db
Traffic passes through untouched. Each request/response pair is simulated and appended to a zero-dependency SQLite 3 file, readable with any SQLite client:
SELECT count(*), avg(net_savings_pct) FROM shadow_records;
The console prints a summary on shutdown: rows, activated, activation_rate, tokens_before, tokens_after, output_savings, net_savings.
Deterministic scenarios (e-commerce invoice, dev-tool pull request batch, MCP tool definitions) with structural equality asserted after rehydration:
$ make bench
| Scenario | tokens_before | tokens_after | output_savings_% | net_savings_% | p99_overhead_ms | equality |
|---|---|---|---|---|---|---|
| E-commerce invoice line items | 2041 | 801 | 60.8% | 59.2% | 0.095 | OK |
| Dev-tool pull request batch | 1602 | 889 | 44.5% | 43.0% | 0.078 | OK |
| MCP tool invocations batch | 1617 | 711 | 56.0% | 53.7% | 0.084 | OK |
Token counts use a deterministic size model (one token per four UTF-8 bytes), so the table is reproducible without an API key. Equality is checked with reflect.DeepEqual on decoded values plus legend key order.
internal/compiler — lossless SchemaNode, deterministic key order (required first, then lexicographic), tuple schemas with prefixItems and items: false (Draft 2020-12, accepted by structured-output APIs that reject additionalItems), nested tuples, and the rehydration tree.
internal/gate — break-even estimator with an injectable token model.
internal/rehydrator — byte-level state machine; handles UTF-8 split across chunks, escaped quotes/backslashes, brackets and commas inside strings, nested tuples, and degrades to raw output on shape violations without breaking the stream.
internal/proxy — reverse proxy, provider detection, SSE codec, bounded bodies, upstream-only forwarding, cancellation.
internal/shadow — passive audit plus a pure-Go SQLite 3 writer.
Development
make test # go test ./... make race # go test -race ./... make bench # savings table + equality checks make build # bin/wiretuple, bin/wiretuple-bench make vet
License
MIT © 2026 Dimoncelli.
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。