Laplace's Scissors：直接编辑 LLM 权重的工具

Rotate an LLM’s output matrix to compute edits while preserving protected answers.

Rust · Triton · CUDA / HIP · Readout rotation

Quick start · Commands · Implementation and tests · Math and design

Laplace’s Scissors computes rotation edits for an LLM’s original output matrix. probe constructs numerical inputs from model weights and traces directions with weak output discrimination, cancellation across layers, and the associated weight coordinates. diagnose uses cases with expected answer strings to find plane or joint rotations that repair failed answers while protecting successful ones. Each objective and protection condition is checked with actual GPU outputs. Scaling analysis applies the same edit policy across checkpoints to measure deviations from a reference curve and the amount recovered.

flowchart LR A[Original checkpoint] --> B[Observe hidden states] B --> C[Protected readout rotation] C --> D[GPU validation] D --> E[Accepted proposal] E --> F[Generate or export]

Loading

Commands

Purpose Command Main outputs

Fetch a model and inspect its structure fetch, inspect Pinned revision, file hashes, execution graph

Trace weak discrimination directions in numerical inputs probe Layer cancellation, coordinate contributions, protected rotation

Edit outputs using expected answers diagnose Rotation candidates checked against actual outputs, accepted proposal

Generate with the original or edited model generate Greedy answer

Save an edit as a checkpoint export Edited output matrix and model files

Analyze deviations across model scales scale Loss curves, recovered deviation, weight contributions

Quick start

Environment

Prepare a Rust toolchain, a Python environment that can import Triton 3.8, and a CUDA or HIP driver. Python provides the Triton compilation bridge. Rust handles model loading, tokenization, execution, edit computation, and result storage.

export LAPLACE_CACHE="${XDG_CACHE_HOME:-$HOME/.cache}/laplaces-scissors" export LAPLACE_PYTHON=python3 cargo build --release

.cargo/config.toml places Cargo build artifacts outside the project. LAPLACE_CACHE stores models, kernels, and run results. Select the backend and device index with LAPLACE_BACKEND=auto|cuda|hip and LAPLACE_DEVICE. Set LAPLACE_DRIVER_LIBRARY for the driver library path, or ROCM_PATH / HIP_PATH for the ROCm SDK. The HIP backend also checks the selected Python environment for the rocm-sdk-core package. Device names and execution targets come from the driver.

cargo run --release -- fetch Qwen/Qwen3-0.6B cargo run --release -- inspect Qwen/Qwen3-0.6B --out "$LAPLACE_CACHE/graph.json" cargo run --release -- cases --count 12 --seed 7 --out "$LAPLACE_CACHE/cases.jsonl"

Model inputs accept organization/model, organization/model@revision, a Hugging Face model URL, or a local safetensors model directory. Hub inputs are pinned to a commit SHA, with SHA256 recorded for each file. Use --revision to select a revision and HF_TOKEN for authentication.

cargo run --release -- generate Qwen/Qwen3-0.6B
--prompt 'What is 2 + 2? Reply with the number only.'

Case format

Write custom cases as JSONL. expected is the zero-based index of the correct entry in choices. The choices specify the expected answer; the model generates an answer string for prompt. See the example below and the example JSONL.

{"id":"sum","prompt":"What is 2 + 2? Reply with the number only.","choices":["3","4","5"],"expected":1,"oracle":{"sum":4}}

The original chat template builds the prompt, whose token prefix stays fixed. The expected answer is encoded separately and followed by EOS to form the target token sequence. The answer body must survive a tokenizer round trip exactly. When the original generation ends with EOS and exactly matches the expected string, every prefix of that generation is protected. For failed cases, each target answer token and EOS must score above every competing token in the vocabulary.

Numerical probes · Answer repair · Checkpoint export · Scaling analysis

probe takes model weights and an execution graph. Seeded selections of embedding rows form numerical prefixes, a reference vector, and an orthonormal basis. Two vectors with the same norm as the reference vector are injected directly at the decoder input. Responses are compared with Jeffreys divergence, the symmetric KL divergence between output distributions. A weak direction has low output discrimination between equal-sized injections under the recorded numerical input distribution. Model acquisition for this command loads safetensors and configuration files.

cargo run --release -- probe Qwen/Qwen3-0.6B
--directions 12 --prefix-length 3 --seed 7 --probe-angle 0.05
--max-angle-degrees 3 --coordinates 12 --intervention-fraction 0.1
--out "$LAPLACE_CACHE/runs/probe"

cargo run --release -- probe Qwen/Qwen3-0.6B
--inputs "$LAPLACE_CACHE/runs/probe/inputs.json"
--proposal "$LAPLACE_CACHE/runs/probe/proposal.json"
--out "$LAPLACE_CACHE/runs/probe-replay"

--directions sets the maximum dimension of the search basis constructed from weights. The actual dimension follows the hidden width and the rank of embedding differences. --prefix-length sets the number of numerical vectors before the injection position. --probe-angle is the spherical angle used to construct the two inputs, in radians. The minimum eigenvector of a small response Gram matrix proposes a direction; measurements through the actual decoder compare its divergence with that of the existing basis directions.

Residual branches from the selected pair of runs provide signed layer contributions. The branch with the largest negative contribution supplies --coordinates weight coordinates and attention-head totals. --intervention-fraction controls diagnostic interventions that change each selected coordinate’s magnitude in both directions. Each intervention reruns from the fixed numerical prefix. Signed contributions and measured intervention effects are recorded separately. --coordinates 0 performs layer tracing; --intervention-fraction 0 performs divergence tracing and protected rotation.

The edit is a readout rotation that protects the original output matrix’s row Gram matrix, the midpoint of the two hidden states, and reference and control states. --max-angle-degrees, --max-edit-norm, and --tolerance set bounds on angle, weight change, and protected-output error. Among three candidates within the angle bound, the program saves an edit that increases measured divergence and satisfies the protection tolerance. Candidates apply actual GPU rotation and readout to observed hidden states. --inputs loads saved inputs; protected_hidden can hold additional protected vectors from the same model. Adding --proposal reruns the saved input pair through the full decoder.

Probe outputs are manifest.json, inputs.json, trace.json, transport.json, report.json, status.json, and an accepted proposal.json. Replay also records replay-frames.json. Results include input and model hashes, actual hardware and precision, divergence, layers and weight coordinates, preservation errors, numerical residuals, and execution counts by stage. Proposal version 3 stores the divergence objective and injection-input hash. Reapplication verifies weight and execution-configuration hashes.

cargo run --release -- diagnose Qwen/Qwen3-0.6B
--cases "$LAPLACE_CACHE/cases.jsonl"
--max-planes 8 --max-angle-degrees 120 --margin 0.05
--max-new-tokens 64 --context 2048 --time-budget 20m
--out "$LAPLACE_CACHE/runs/rotation"

cargo run --release -- generate Qwen/Qwen3-0.6B
--prompt 'What is 2 + 2? Reply with the number only.'
--proposal "$LAPLACE_CACHE/runs/rotation/proposal.json"

--max-planes sets the number of rotation planes to investigate. --max-angle-degrees bounds the rotation angle, and --margin sets the minimum logit gap between each target token and every competitor. --tolerance bounds each protected-output change divided by 1 + |original logit|; its default is 0.00002. The inequalities include a numerical allowance derived from this tolerance and the observed logit magnitude. For each plane, merged forbidden intervals yield the angle with the smallest weight change. Candidates are checked with actual GPU readout in order of weight change.

After checking plane candidates, a joint response computation adjusts multiple targets together. --joint-steps sets the number of adjustment steps and defaults to 12; setting it to 0 runs the plane candidates. --joint-rank sets the maximum joint rotation basis dimension and defaults to 64. --max-angle-degrees also bounds the joint rotation’s largest principal angle. --max-edit-norm bounds the Frobenius norm of the output-weight change. Acceptance requires the full-vocabulary margin, protected-output tolerance, target answer and EOS, and preservation of originally successful generation paths.

Joint search computes and shares the output basis and cost Gram matrix once. When twice the response count is smaller than the rotation dimension, it solves the Cayley system in that smaller space. Candidate scores use the compressed output basis on the GPU. After search, a rotation completion minimizes weight change while preserving observations. The original GPU readout checks that the completion stays within angle and cost bounds and preserves every prefix’s token choice, satisfied margins, and protected-output tolerance. completion in candidates.json records costs before and after completion and whether it was applied. computation in report.json records compressed-output and original-validation call counts and timings by computation stage.

--max-new-tokens bounds generated tokens, including EOS. --context must accommodate the prompt plus that generation allowance. Omitting --cases generates symbolic problems using --count and --seed. Diagnosis uses the original template with thinking disabled. generate --thinking selects the thinking format.

The time budget starts with model loading after file acquisition and covers computation and attribution. The output directory contains manifest.json, status.json, baseline.json, candidates.json, report.json, and accepted proposal.json and attribution.json files. Reports record the model revision, input token paths, weight dtypes, hardware, measured validation values, and square-root and readout call counts. Status values are complete for completion, partial for budget expiration, and failed for execution errors.

A proposal contains either orthonormal plane vectors and a half-angle coordinate, or an orthonormal joint basis and a small orthogonal matrix. At runtime, the final hidden vector is rotated before multiplication by the original output matrix. In models with shared embeddings, rotation applies at readout calls. Reapplying an answer-repair proposal checks hashes for weights, configuration, and tokenizer files, plus the output-operation contract. generate --proposal reads proposal versions 1, 2, and 3.

--attribution-rows sets the number of output rows to report and defaults to 6. For each row, results include high-contribution coordinates, the loss after reverting that row’s edit, the total contribution sum, and integration error. Contributions follow the specified straight-line readout intervention path and retain negative values. Setting the option to 0 saves the generation proposal and validation results.

export saves a new checkpoint with the proposal applied to the output matrix. It preserves original tensors and writes the edited lm_head.weight in FP32. It unties embedding and readout weights and saves configuration, execution graph, and shard index files. The output matrix uses CPU F64 block multiplication; original-model loading and contract checks use the shared model runtime. Run the exported model using its directory path.

cargo run --release -- export Qwen/Qwen3-1.7B
--proposal "$LAPLACE_CACHE/edits/proposal.json"
--out "$LAPLACE_CACHE/exports/qwen3-1.7b-edited" cargo run --release