Show HN: I removed every float from GPT-2 for identical neural compression

Hard shell. No floats. Same taco on every machine.

Taco is a lossless compressor that uses a language model (GPT-2). The model predicts the next piece of the data, and a range coder stores only what the model did not predict. The engine of Taco, HardShell, runs the model with integer arithmetic only. Thus every CPU and GPU calculates the same predictions, bit for bit. A file that one machine compresses ("folds") decompresses ("unfolds") on every other machine.

taco fold notes.txt # makes notes.txt.taco taco unfold notes.txt.taco # restores notes.txt, byte for byte

Real ./try output of the 0.1.0 release on macOS (CPU), Linux (CUDA) and Windows (CUDA). Long lines are shortened; the full output is in docs/img/try-output/.

Alpha: a proof of concept (version 0.1.0, 2026-10-19). Taco is an experiment. It is slow, and it is not a replacement for zstd or xz. Keep a second copy of data that you fold. Read Limits before you use it.

How this code was written. Most of the code, the tests and the documents in this repository were written by Claude (an AI model by Anthropic) with Claude Code, under the direction of the author, to get to results faster. The author made the design decisions. Trust the code only as far as the tests show: conformance vectors, the cross-machine check, fuzzing and the comparison of every fast path with the reference path.

Proof: the same bytes on every machine

Try it

What Taco is, and what it is not

Results

What is guaranteed

Hardware

Limits

How it works

Build from source

Usage

Roadmap

Development

Licence

Proof: the same bytes on every machine

This is the point of Taco. Speed and ratio come second.

The release program of version 0.1.0 folded the demo sample (demo/samples/sample.txt, 3,672 bytes, --profile text) and unfolded it again on 7 backends: 3 operating systems, 2 CPU architectures and 4 kinds of processors. Each fold ran on the backend in the row, and each unfold restored the sample byte for byte (measured, 2026-10-19):

Thus a file that any of these machines folds unfolds on all of the others. To add a row, run ./try on your machine (Try it) and compare the hash. More checks of the same property, with more inputs, are in Same bits on different machines.

There are two ways. Both give the same .taco bytes.

Open the Releases page and download taco-demo-0.1.0.zip (about 120 MB). It has the programs for macOS (Apple Silicon), Linux (x86-64) and Windows (x86-64), the model file gpt2.hsm, and a one-command demo. If you want only the program for one system, download taco-0.1.0-<system>.tar.gz (or .zip for Windows); it does not include the model file.

Unpack the zip and open a terminal in the unpacked folder (taco-demo-0.1.0).

Run ./try (macOS, Linux) or try.cmd (Windows).

The demo folds and unfolds a short English text (3,672 bytes), checks it, and prints the SHA-256 of the .taco file. It takes about 10 s to 1 minute. The end of the output:

PASS: the file came back byte for byte. Original: 3672 bytes. Folded: 648 bytes. SHA-256 of the .taco file: 2fab1ef7a1c70cd4e39ddfd88a8d66389ae07a403111db131326ae425460652e

This hash is the same on every machine and every backend. Run the demo on a second computer and compare. To fold your own files, use ./taco fold notes.txt and ./taco unfold notes.txt.taco in the same folder (Windows: taco.cmd). Details and troubleshooting: demo/README.md.

You need: rustup (it installs Rust 1.99.0 from rust-toolchain.toml), uv for the Python tools, a C compiler (for zstd), about 1.5 GB of disk and 8 GB of memory. A GPU is optional.

git clone https://github.com/tac0mania/taco.git && cd taco uv run --project tools python -m baseline.fetch # GPT-2 weights; the tool checks each SHA-256 uv run --project tools python -m convert.gpt2 # makes tools/data/out/gpt2.hsm (about 1 minute) shasum -a 256 tools/data/out/gpt2.hsm # must be aaa03eb96e9ad795...b51001ba5c2 cargo build --release --locked -p taco-cli export TACO_MODEL="$PWD/tools/data/out/gpt2.hsm" ./target/release/taco fold demo/samples/sample.txt --profile text -o /tmp/sample.taco shasum -a 256 /tmp/sample.taco # must be the hash in section A ./target/release/taco unfold /tmp/sample.taco -o /tmp/sample.txt cmp demo/samples/sample.txt /tmp/sample.txt && echo "restored byte for byte"

The converter calculates with exact fractions, so your gpt2.hsm is the same file, byte for byte, as the one in the release. If its hash is different, do not use it.

Your .taco file must have the hash of section A. Then your build and the release program are compatible: each one unfolds the files of the other.

scripts/release/build.sh <target> makes the release archive itself. It builds with fixed paths and times, so two builds of one commit on one machine gave the same binary, byte for byte (measured on Linux). BUILDINFO in each archive records the commit and the SHA-256 of the program.

Caution

Taco is slow. With a GPU, it folds text at about 85 to 129 KB/s and unfolds at about 26 to 31 KB/s. With the CPU only, it folds at about 0.3 to 2 KB/s and unfolds at about 0.1 to 0.5 KB/s (measured). Start with a file of a few KB.

Taco is a compressor whose output does not depend on the machine. Its parts are:

Taco is not the first or the best compressor with a language model:

What Taco adds is the combination: an inspectable integer specification, a repeatable model conversion, several independent implementations, and tests that show they agree.

All numbers are measured unless they are labelled "published" or "est.". Details are in docs/research/. The sizes do not include the model file (127 MB), which both machines must have.

File Original Taco xz -9e zstd -19

English book (32 KiB) 32,768 8,432 12,400 12,463

HTML page 24,603 4,759 7,652 7,713

English (4 KiB) 4,096 1,237 1,684 1,599

Server log (16 KiB) 16,384 3,129 3,164 3,139

Excel file (binary) 8,192 8,285 (stored raw) 1,672 2,097

Against another compressor with a language model, on the whole alice29.txt (152,089 bytes):

ts_zip is 44% smaller. It uses a larger and newer model (RWKV, 169M parameters; GPT-2 small has 124M), and Taco's profile text restarts the model context at each 16 KiB block.

The 16 Loghub log files (4,458,875 bytes in total), the defaults of the profile logs (blocks of 64 KiB, context overlap 512), whole files for the other compressors (docs/research/2026-10-19-overlap-cli.md, docs/research/2026-10-19-logs-profile.md):

Speed of the profile logs with the GPU mixer (RTX 3060 at 95 W, CUDA, commit 1493af3):

What can make logs smaller (not in this version): the skip rule, 4.8 to 7.3% smaller (est., simulation, ADR 0017); and a model fine-tuned on the logs of known systems, 53.2% smaller on those systems but 4.6% larger on other systems (measured, frozen; Experiments).

Model Bits per byte (16 text files, 42,256 bytes)

GPT-2 small, float 1.867

GPT-2 small, integer (gpt2.hsm) 1.871 (+0.2%)

The +0.2% is a total. Single files change from −9.0% to +6.6%. The conversion saturates large activations, which changes the predictions; against a float model with the same saturation, the integer model is 2.3% larger (docs/research/2026-10-18-integer-gpt2.md §6).

What Speed

Engine, decode over a full 1,024-token window 20,472 tokens/s

taco fold, English text 85 to 129 KB/s

taco unfold, English text 26 to 31 KB/s

taco fold, CPU only about 0.3 to 2 KB/s

taco unfold, CPU only about 0.1 to 0.5 KB/s

For comparison, zstd -3 compresses at hundreds of MB/s and xz -9e at about 0.7 MB/s. ts_zip states "up to 1 MB/s" on an RTX 4090 (published, a different GPU).

This test checks that the memory does not grow with the file (ADR 0018). It used 200,000,000 bytes of random words and the small test model, on the devbox CPU.

Both unfolds restored the input byte for byte. Its times say nothing about Taco's speed: the test model has random weights, so every block was stored with zstd or raw, and unfold ran no model step. The real speed is in the table above. The largest files folded with GPT-2 so far are 2.4 to 2.7 MB (measured).

Check Result

Mac (ARM) folds, Linux (x86) unfolds, and the reverse (scripts/xcheck, 12 inputs) 12 of 12 restored each way; 12 of 12 .taco files identical

Native Windows x86 against the Mac (scripts/wcheck) 9 of 9 .taco files identical, 9 of 9 restored

CPU, CUDA and wgpu (Metal, GL) the same bytes, 14 of 14 GPU cases

CI on each push: Linux x86-64 and Linux ARM64 the same 50 conformance hashes

Fuzzing: damaged containers, coder streams and model files millions of cases, 0 failures (docs/security.md)

What is guaranteed

No floating-point arithmetic in fold or unfold. The engine, the coder, the mixer and the GPU kernels use integers only. scripts/check-no-floats.sh checks this on each push.

The same result on every machine, when each operation is fully specified. Integers alone are not sufficient. Taco makes these conditions rules (CLAUDE.md §1):

Then a compiler, a CPU type or a GPU driver has no freedom to change a result. Tests check these conditions. Tests cannot prove that the code obeys them everywhere.

The model file is the same on every machine. The Mac and Linux make the same gpt2.hsm, byte for byte. Its SHA-256 is aaa03eb96e9ad7957432168c8ac6b4a99ed55d57e637e4786a664b51001ba5c2.

A wrong result is never silent. Each .taco file stores the SHA-256 of the original data, and taco unfold checks it.

The checksum finds an error. It cannot repair one. If taco unfold fails, try again with --backend cpu or on a different machine. If the error happened during taco fold (a bug or a hardware fault on that machine), no decoder can restore the file. Keep the original until you have unfolded the .taco file one time.

These problems can still occur, as in every program: a bug in our code, a bug in a GPU driver (Taco runs a self-test before it uses a GPU, and uses the next backend if the test fails), or a hardware fault. In each case the SHA-256 check finds the error.

Hardware Backend Result

Apple M2 (macOS) CPU (NEON), GPU (wgpu on Metal) bit-exact

AMD Ryzen 7 5800H (Linux, native Windows) CPU (AVX2) bit-exact

NVIDIA RTX 3060 Laptop CUDA bit-exact

NVIDIA RTX 3060 Laptop wgpu on OpenGL (Linux), DX12 with DXC (Windows) bit-exact

AMD Radeon iGPU (Ryzen 7 5800H), native Windows wgpu on DX12 with DXC bit-exact

NVIDIA RTX 3060, AMD Radeon iGPU, native Windows wgpu on Vulkan bit-exact (4 op tests; the full model tests did not run yet)

GitHub Actions: Linux x86-64, Linux ARM64 CPU the same 50 conformance hashes

Not tested yet: Intel GPUs; AMD GPUs on Linux; AMD discrete GPUs; macOS on Intel; big-endian and 32-bit CPUs; phones and browsers.

Limit Details

Slow See Speed. A GPU is strongly recommended.

Small files unfold slowly One block decodes one token after the other: about 10 to 13 s for a 16 KiB block on the RTX 3060 (est. from measured times).

English only GPT-2 is an English model. On German and French, Taco is larger than PPMd; on Chinese, larger than gzip (measured).

Binary data Photos, video, archives and other binary files are stored with zstd or raw. Use zstd for them.

Logs The profile logs is smaller than xz, but paq8px is smaller still (measured).

The model file 127 MB. Both machines must have the same file. The release does not download it yet.

Untrusted model files A model file can state a very large KV cache (finding F6, open). Load only the published gpt2.hsm.

Wide models Model-file version 2 runs on the CPU and with CUDA, not on wgpu yet. The default gpt2.hsm is version 1 and runs everywhere.

Format Container version 2 is frozen: later versions of Taco read the files of this version.

Release 0.1.0-alpha.1 Its programs fold the default profile auto on the CPU (use --profile text for the GPU), and a crafted .taco file can use about 11 GB in GPU unfold (F2; use --backend cpu for untrusted files). Both are fixed on main; the GPU logs speeds above are from main.

How it works

A symbol that the model expects with probability p costs about −log₂(p) bits. For each position, the mo