Break the 64 GB RAM Wall: Run 284B DeepSeek-V4-Flash (< 59 GiB RAM) & 30B MoEs (up to 111 tok/s) at 256K Context on Pure CPU — Zero GPUs Required.

⚡ Quickstart · 🧩 Supported Models · 🖥️ Supported Chips & Platforms · 🏆 Leaderboard · 📄 Preprint PDF (paper/) · 📐 Architecture · ❓ FAQ

Real-time Frontier & Agentic MoE inference on the CPU workstation you already own — 100% in RAM, zero NVMe swap:

TL;DR — The 3 Paradigm-Breaking Pillars of llama-modchip:

🐧 Linux & 🍎 macOS (1-Command Automated Setup & Chat)

git clone https://github.com/synchopate/llama-modchip.git cd llama-modchip ./modchip chat

⚡ Zero-Compile Direct Run (Pre-Compiled Linux x86_64 Binary + Hugging Face .gguf)

⚠️ Important Compatibility Note (GGML_TYPE_Q2_0 = 40): Our Fractal-64 models use custom block-64 BF16-scale Gray-coded tensors (GGML_TYPE_Q2_0, type ID 40) and custom deepseek4 / Stockfish-PVS kernels. Opening them in unpatched llama.cpp, Ollama, or LM Studio will fail with unknown tensor type 40. Use either ./modchip above or our standalone single-file pre-compiled binary (llama-server-modchip-linux-x86_64, zero .so dependencies) bundled directly inside the Hugging Face repositories and GitHub Release v2.4.0:

1. For 30B Nemotron-3.5-A3B (14.35 GiB — runs in 16–32 GB RAM at 25–60.4 tok/s):

hf download Synchopate/NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64-CPU-GGUF
llama-server-modchip-linux-x86_64 NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64.gguf --local-dir . chmod +x llama-server-modchip-linux-x86_64 ./llama-server-modchip-linux-x86_64 -m NVIDIA-Nemotron-3.5-30B-A3B-Fractal-64.gguf -c 262144 -t 16 --port 8080

2. For 284B DeepSeek-V4-Flash (56.56 GiB — fits in 64 GB RAM at 9.1–14.7 tok/s):

hf download Synchopate/DeepSeek-V4-Flash-0731-Fractal-64GB-CPU-GGUF --local-dir ./dsv4 chmod +x ./dsv4/llama-server-modchip-linux-x86_64 ./dsv4/llama-server-modchip-linux-x86_64 -m ./dsv4/DeepSeek-V4-Flash-0731-Fractal-00001-of-00003.gguf -c 32768 -t 16 -fa on --port 8080

🪟 Windows 10/11 (1-Click Hybrid GPU + CPU MoE Pack)

Open windows/ and double-click windows/AVVIA_MODCHIP.bat (or run ./modchip start in Git Bash). It automatically detects your CPU cores, free RAM, and GPU VRAM (nvidia-smi), downloads official llama.cpp CUDA 12.4 / Vulkan binaries, computes the optimal --n-cpu-moe split, and opens the Web Chat UI (http://127.0.0.1:8080).

curl -sL https://raw.githubusercontent.com/synchopate/llama-modchip/main/scripts/check_cpu_readiness.py | python3

👉 Paste your terminal scorecard in a 📊 1-Click GitHub Benchmark Issue or GitHub Discussions to be added to the Official Community CPU Leaderboard!

Each model family has a dedicated deep-dive optimization guide with exact mathematical proofs, operator-level latency breakdowns, and quality gate benchmarks:

Full microarchitectural details and tuning flags for every supported processor and GPU hybrid are documented in docs/HARDWARE_OPTIMIZATIONS.md:

1. DeepSeek-V4-Flash 284B (13.17B Active) on a 64 GB Zen 5 CPU (AMD Ryzen 9 9955HX, 0 GPUs)

Full operator-level profiling and mathematical proofs in docs/DEEPSEEK_V4_OPTIMIZATIONS.md:

Full ablation tables and 100/100 Quality Gate report in docs/NEMOTRON_OPTIMIZATIONS.md:

Run ./modchip bench (or windows\BENCHMARK_PC.bat on Windows) and ➕ Submit Your Scorecard (1-Click GitHub Issue)!

llama-modchip/ ├── modchip # Unified CLI (setup | start | chat | bench | eval | quantize | web | claude | stop) ├── install.sh # 1-Command Automated Installer (Linux) ├── README.md / ARCHITECTURE.md / FAQ.md # Clean Overview, Full Mathematical Paper & Engineering FAQ ├── RELEASE_NOTES.md # v2.4.0 Release Notes ├── docs/ │ ├── NEMOTRON_OPTIMIZATIONS.md # Deep-dive on Nemotron-3.5-30B-A3B & Nemotron-H-8B optimizations │ ├── DEEPSEEK_V4_OPTIMIZATIONS.md # Deep-dive on DeepSeek-V4-Flash 284B profiling, SVD & L2-quadrature MoE │ └── HARDWARE_OPTIMIZATIONS.md # Chip-by-chip optimizations (Zen 5, Threadripper, Vulkan, CUDA, Metal, ARM64) ├── patches/ │ ├── deepseek-v4-flash-fractal-256k.patch # Unified v2.4.0 patch (DeepSeek-V4-Flash 284B + Nemotron-3.5 30B, Linux/macOS/Windows) │ └── zen5-nemotron-256k-optimizations.patch # Standalone ggml/llama core patch for Nemotron-3.5 ├── windows/ # 1-Click Windows 10/11 Hybrid GPU+CPU MoE Pack (AVVIA_MODCHIP.bat, modchip_win.ps1) ├── apple-silicon/ # Apple M4 Pro (24 GB) Metal setup & 128K benchmarks ├── fedora/ # Fedora 44 / AMD Ryzen 7 8745HS (GEEKOM A8) all-in-one script ├── speculative/ # RouteCrack + Stockfish-PVS v2 (rc_stockfish.h, ngram-map.cpp, 96+ techniques) ├── benchmarks/ # eval_quality_and_tools.py (100-pt Quality Gate) & results/*.json ├── gateway/sse_gateway.py # Agentic SSE Proxy + Compact Tool Table + Dark-Mode Web Chat HUD (:8787) └── scripts/ # build_fractal_moe_gguf.sh, transcode_dsv4.{c,py}, chat.py, llama_modchip.py

🏢 Enterprise & Hardware Sponsorships

llama-modchip demonstrates that modern multi-core CPUs and compact Mini-PCs with 64–128 GB of DDR5 RAM can run 284B Frontier MoEs (DeepSeek-V4-Flash) and 30B Hybrid MoEs (Nemotron-3.5-30B-A3B) locally at full 256K context—eliminating $30,000+ multi-GPU hardware requirements for air-gapped enterprise code intelligence, security auditing, and autonomous agent fleets.

We actively partner with hardware manufacturers, cloud silicon teams, and AI research labs to build and validate architecture-specific kernel patches:

📖 Nemotron Deep-Dive (docs/NEMOTRON_OPTIMIZATIONS.md) · 🐋 DeepSeek-V4 284B Deep-Dive (docs/DEEPSEEK_V4_OPTIMIZATIONS.md) · 🖥️ Hardware & Chip Matrix (docs/HARDWARE_OPTIMIZATIONS.md) · 📄 Full Paper (ARCHITECTURE.md) · ❓ FAQ (FAQ.md) · 🚀 Marketing Launch Kit (docs/marketing-launch-kit/)