Like 4

Text Generation

Transformers

Safetensors

qwen3_5_text

qwen3.5

risc-v

rv32im

vector-unit

npu

compiled-weights

gated-deltanet

qwenputer

conversational

License: apache-2.0

Model card Files Files and versions xet

Community

Deploy

Copy to bucket

Use this model Instructions to use divagr1925/QwenPU with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers How to use divagr1925/QwenPU with Transformers:

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-generation", model="divagr1925/QwenPU") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)

pip install -U transformers accelerate

Load model directly

from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("divagr1925/QwenPU") model = AutoModelForCausalLM.from_pretrained("divagr1925/QwenPU", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks

Google Colab

Kaggle

Local Apps Settings

vLLM How to use divagr1925/QwenPU with vLLM:

Install vLLM from pip:

pip install vllm

Start the vLLM server:

vllm serve "divagr1925/QwenPU"

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "divagr1925/QwenPU", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker

docker model run hf.co/divagr1925/QwenPU

SGLang How to use divagr1925/QwenPU with SGLang:

Install SGLang from pip:

pip install sglang

Start the SGLang server:

python3 -m sglang.launch_server
--model-path "divagr1925/QwenPU"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "divagr1925/QwenPU", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker images

docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "divagr1925/QwenPU"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "divagr1925/QwenPU", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Docker Model Runner How to use divagr1925/QwenPU with Docker Model Runner:

docker model run hf.co/divagr1925/QwenPU

A language model inside the language model

The vector unit

Checks

Quickstart

Model details

Citation

License

Code on GitHub · Qwenputer · Base model: Qwen/Qwen3.5-4B · Apache-2.0

QwenPU is Qwen3.5-4B with a 32-bit RISC-V CPU and a vector unit compiled into its weights. The vector unit runs a whole neural network inside the model's forward passes: Andrej Karpathy's stories260K, the 260K-parameter Llama from llama2.c, runs as a program on it and writes its stories.

It's part of Qwenputer: the same CPU, compiled into transformer layers in front of Qwen's own, every weight set by a compiler, with Qwen3.5's Gated DeltaNet state as its memory. Qwenputer-npu gave that CPU dot-product instructions. QwenPU gives it a vector unit of its own: separate memories in DeltaNet heads, 64-lane matrix rows a token, and whole vector results streamed through the token stream.

Work in progress. QwenPU is exact, but not yet small: the checkpoint is 26.7 GB, so it needs a 40 GB or larger GPU to load whole. The story demo and the checks run the computer's layers on their own and fit a 24 GB GPU. Trimming the hidden size, and with it the checkpoint, is next.

A language model inside the language model

stories260K is a C program here (guests/stories260k/stories_vx.c). Its int4 weights sit in the vector unit's weight memory, loaded from the program, and every matrix row, residual add, softmax and attention read is a vector op. None of it is compiled into the checkpoint, so any model that fits runs the same way.

Every build writes the same story, token for token:

Once upon a time, there was a little girl named Lily. She loved to play outside in the park. One day, she saw a big, red ball. She wanted to play with it, but it was too high

The vector unit

Op Does

VMAT int8 · int4 matrix rows, 64 lanes a token, times the row scale; or the argmax

VQ quantizes a vector register into int8 activations or a K / V cache row

VEW residual add, SiLU-gated product, RoPE

VMATR attention scores over the K cache; probabilities times the V cache

VEXP exp-table softmax numerators

VRED sum or sum of squares of the activations

VGET, VPUT move values between the CPU and the unit

Its memories are DeltaNet heads: 16 vector registers of 64 words, 384 int8 activations, 4,152 row-chunks of int4 weights, and K / V caches for 32 positions. An op's record starts with config tokens carrying its operands, then streams its results, and each token is one exact step of the compiled layers. SiLU and exp are fixed-point function units, generic like the CPU's adders.

Checks

Every vector op runs random test programs on this checkpoint's own chip layers, teacher-forced against a reference model of the unit, along with the first 30,000 steps of the story program: zero mismatches.

The circuit's executor writes the whole story free-running on a GPU, checked against the reference at every one of its 3,184,920 steps: exact. It runs at about 3,300 trace tokens a second on an L4, so a story token takes about 16 seconds, against 41 for Qwenputer-npu.

Quickstart

git clone https://github.com/divagr18/QwenPU && cd QwenPU && uv sync --extra hf uv run python guests/stories260k/storyrun.py guests/stories260k/stories_vx.elf --vx hf download divagr1925/QwenPU --local-dir qwenpu uv run python -m chipformer.rv.vxcheck qwenpu guests/stories260k/stories_vx.elf

storyrun writes the story on the computer's executor; vxcheck runs the vector unit on this checkpoint's layers (--chip-only loads only those on the GPU, about 13 GB).

Model details

Base Qwen/Qwen3.5-4B, Qwen's own weights unchanged

Architecture stock Qwen3_5ForCausalLM (text only); config-only changes

Computer 13 compiled layers before Qwen's 32: RV32IM + Zba + bignum unit + the vector unit

Hidden size 5120 (Qwen's 2560 plus the computer's dims)

Vocabulary Qwen's, plus 394,164 trace tokens

Size 26.7 GB in bf16, mostly zero padding

Precision exact in bf16

Citation

@software{agrawal2026qwenputer, author = {Agrawal, Divyansh}, title = {{Qwenputer}: a computer compiled into the weights of a language model}, year = {2026}, url = {https://github.com/divagr18/qwenputer}, license = {MIT} }

License

Apache-2.0. This is a modified version of Qwen/Qwen3.5-4B: the config is widened and computer layers are added, but Qwen's own weights are unchanged. stories260K and llama2.c are by Andrej Karpathy (MIT).

","unk_token":null,"pad_token":"<|endoftext|>","chat_template":"{%- set image_count = namespace(value=0) %}\n{%- set video_count = namespace(value=0) %}\n{%- macro render_content(content, do_vision_count, is_system_content=false) %}\n {%- if content is string %}\n {{- content }}\n {%- elif content is iterable and content is not mapping %}\n {%- for item in content %}\n {%- if 'image' in item or 'image_url' in item or item.type == 'image' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain images.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set image_count.value = image_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Picture ' ~ image_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|image_pad|><|vision_end|>' }}\n {%- elif 'video' in item or item.type == 'video' %}\n {%- if is_system_content %}\n {{- raise_exception('System message cannot contain videos.') }}\n {%- endif %}\n {%- if do_vision_count %}\n {%- set video_count.value = video_count.value + 1 %}\n {%- endif %}\n {%- if add_vision_id %}\n {{- 'Video ' ~ video_count.value ~ ': ' }}\n {%- endif %}\n {{- '<|vision_start|><|video_pad|><|vision_end|>' }}\n {%- elif 'text' in item %}\n {{- item.text }}\n {%- else %}\n {{- raise_exception('Unexpected item type in content.') }}\n {%- endif %}\n {%- endfor %}\n {%- elif content is none or content is undefined %}\n {{- '' }}\n {%- else %}\n {{- raise_exception('Unexpected content type.') }}\n {%- endif %}\n{%- endmacro %}\n{%- if not messages %}\n {{- raise_exception('No messages provided.') }}\n{%- endif %}\n{%- if tools and tools is iterable and tools is not mapping %}\n {{- '<|im_start|>system\n' }}\n {{- "# Tools\n\nYou have access to the following functions:\n\n<tools>" }}\n {%- for tool in tools %}\n {{- "\n" }}\n {{- tool | tojson }}\n {%- endfor %}\n {{- "\n</tools>" }}\n {{- '\n\nIf you choose to call a function ONLY reply in the following format with NO suffix:\n\n<tool_call>\n<function=example_function_name>\n<parameter=example_parameter_1>\nvalue_1\n</parameter>\n<parameter=example_parameter_2>\nThis is the value for the second parameter\nthat can span\nmultiple lines\n</parameter>\n</function>\n</tool_call>\n\n<IMPORTANT>\nReminder:\n- Function calls MUST follow the specified format: an inner <function=...></function> block must be nested within <tool_call></tool_call> XML tags\n- Required parameters MUST be specified\n- You may provide optional reasoning for your function call in natural language BEFORE the function call, but NOT after\n- If there is no function call available, answer the question like normal with your current knowledge and do not tell the user about function calls\n</IMPORTANT>' }}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {%- if content %}\n {{- '\n\n' + content }}\n {%- endif %}\n {%- endif %}\n {{- '<|im_end|>\n' }}\n{%- else %}\n {%- if messages[0].role == 'system' %}\n {%- set content = render_content(messages[0].content, false, true)|trim %}\n {{- '<|im_start|>system\n' + content + '<|im_end|>\n' }}\n {%- endif %}\n{%- endif %}\n{%- set ns = namespace(multi_step_tool=true, last_query_index=messages|length - 1) %}\n{%- for message in messages[::-1] %}\n {%- set index = (messages|length - 1) - loop.index0 %}\n {%- if ns.multi_step_tool and message.role == "user" %}\n {%- set content = render_content(message.content, false)|trim %}\n {%- if not(content.startswith('<tool_response>') and content.endswith('</tool_response>')) %}\n {%- set ns.multi_step_tool = false %}\n {%- set ns.last_query_index = index %}\n {%- endif %}\n {%- endif %}\n{%- endfor %}\n{%- if ns.multi_step_tool %}\n {{- raise_exception('No user query found in messages.') }}\n{%- endif %}\n{%- for message in messages %}\n {%- set content = render_content(message.content, true)|trim %}\n {%- if message.role == "system" %}\n {%- if not loop.first %}\n {{- raise_exception('System message must be at the beginning.') }}\n {%- endif %}\n {%- elif message.role == "user" %}\n {{- '<|im_start|>' + message.role + '\n' + content + '<|im_end|>' + '\n' }}\n {%- elif message.role == "assistant" %}\n {%- set reasoning_content = '' %}\n {%- if message.reasoning_content is string %}\n {%- set reasoning_content =