From the creator of Redis; run LLM locally with ds4

DwarfStar 4 is a narrow C inference engine for high-memory Mac, CUDA and ROCm machines. It supports DeepSeek V4 and V4.1 Flash, GLM 5.x and Qwen3.8 Flash Next, with text and vision models, local APIs, a CLI and a native agent in one stack.

SUPPORTED: DEEPSEEK V4 / V4.1 + GLM 5.x + QWEN3.8 · MIT LICENSE · C / METAL / CUDA / ROCM · QWEN ON 64GB

$ ./ds4 DwarfStar 4 · DeepSeek V4 Flash · ds4f-q2 local backend ready · model loaded

/read src/kvcache.c 1,412 lines loaded into context why can a prefix survive a server restart? The KV cache is keyed by the SHA1 of the rendered prompt prefix and persisted to disk, so a matching prefix is reloaded instead of recomputed.

PRINCIPLE · LOCAL MODEL STACK

DeepSeek V4 Flash is a large mixture-of-experts model. The usual path is remote serving; ds4 starts from the opposite constraint.

Asymmetric quantization targets the routed experts while preserving critical paths. The model becomes practical on high-memory machines.

The local engine exposes a CLI, HTTP APIs and a native agent, all sharing the same model state and cache.

Not a generic GGUF runner. ds4 follows a small, opportunistic set of model families and validates each supported layout end to end.

Compress the routed experts, keep critical shared paths precise. That is how the supported routed-MoE builds fit their target machines.

Save long prefixes to SSD and resume by prompt hash. Restarts do not have to mean full re-prefill.

Use ./ds4 for chat, ./ds4-server for local APIs and ./ds4-agent for persistent coding sessions.

Project GGUFs, a self-contained engine and agent-facing interfaces, checked against official model outputs.

MODEL DeepSeek V4 / V4.1 GLM 5.x · Qwen3.8 supported GGUF layouts only asymmetric 2-bit + imatrix load ENGINE ds4 engine written in C metal · cuda · rocm parallel · batch · speculate KV cache RAM ⇄ SSD · survives restarts serve ./ds4 interactive CLI ./ds4-server OpenAI + Anthropic API ./ds4-agent native coding agent personal → distributed memory classes

RUNTIME MAP · SIMPLIFIED. SEE ARCHITECTURE NOTES FOR THE FULL DRAWING.

Download the project GGUF, build for your backend, then start the CLI or server. Generic GGUF files are not the target.

Pick your platform and memory: get a conservative starting path and understand which execution modes apply.

V4 Flash Q2 is the baseline. At 128 GB, GLM 5.3 Q2 and Qwen Q4 also fit; V4.1 Q2 streams from SSD.

./download_model.sh ds4f-q2 && make

REF · M5 MAX 128GB · 32K CTX: 34.4 T/S GEN · 557 T/S PREFILL

Estimates from the ds4 benchmark table. Full guide in Hardware and Installation.

Reference rows from upstream. Read prefill and generation separately, especially for long-context agent workloads.

ds4-server speaks OpenAI and Anthropic-style APIs, so local coding agents can connect to your own machine with a base URL.

Start with the quickstart, check the hardware matrix, then connect your editor, agent or API client to the local server.