
Nvidia-Nemotron-Labs-Dory1.0-4B-NVFP4
Use this model Instructions to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。
Like 1
Follow
NVIDIA 71.5k
Text Generation
Transformers
Safetensors
PyTorch
nvidia/nemotron-post-training-v3
nvidia/nemotron-pre-training-datasets
English
dory
nvidia
nemotron-labs
conversational
8-bit precision
modelopt
License: openmdw-1.1
Model card Files Files and versions xet
Community 6
Deploy
Copy to bucket new
Use this model Instructions to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
Libraries
Transformers How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with Transformers:
Use a pipeline as a high-level helper
from transformers import pipeline
pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)
pip install -U transformers accelerate
Load model directly
from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4", device_map="auto")
Notebooks
Google Colab
Kaggle
Local Apps Settings
vLLM How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with vLLM:
Install vLLM from pip:
pip install vllm
Start the vLLM server:
vllm serve "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"
Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{
"model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'
Use Docker
docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4
SGLang How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with SGLang:
Install SGLang from pip:
pip install sglang
Start the SGLang server:
python3 -m sglang.launch_server
--model-path "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"
--host 0.0.0.0
--port 30000
Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{
"model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'
Use Docker images
docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"
--host 0.0.0.0
--port 30000
Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{
"model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4",
"messages": [
{
"role": "user",
"content": "What is the capital of France?"
}
]
}'
Docker Model Runner How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with Docker Model Runner:
docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4
Quick Start Guide</strong>","children":[{"id":"vllm","label":"<strong>vLLM</strong>","children":[],"isValid":true,"title":"vLLM"},{"id":"llamacpp","label":"<strong>llama.cpp</strong>","children":[],"isValid":true,"title":"llama.cpp"},{"id":"api-client","label":"<strong>API Client</strong>","children":[],"isValid":true,"title":"API Client"}],"isValid":true,"title":"Quick Start Guide"},{"id":"deployment-geography-global","label":"Deployment Geography: Global","children":[],"isValid":false,"title":"Deployment Geography: Global"},{"id":"model-architecture","label":"Model Architecture","children":[],"isValid":true,"title":"Model Architecture"},{"id":"model-design","label":"Model Design","children":[],"isValid":true,"title":"Model Design"},{"id":"training-methodology","label":"Training Methodology","children":[],"isValid":true,"title":"Training Methodology"},{"id":"input","label":"Input","children":[],"isValid":true,"title":"Input"},{"id":"output","label":"Output","children":[],"isValid":true,"title":"Output"},{"id":"software-integration","label":"Software Integration","children":[],"isValid":true,"title":"Software Integration"},{"id":"model-versions","label":"Model Version(s)","children":[],"isValid":true,"title":"Model Version(s)"},{"id":"training-testing-and-evaluation-datasets","label":"Training, Testing, and Evaluation Datasets","children":[],"isValid":false,"title":"Training, Testing, and Evaluation Datasets"}],"isValid":true,"title":"NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"},{"id":"training","label":"Training","children":[{"id":"testing-datasets","label":"Testing Datasets:","children":[],"isValid":true,"title":"Testing Datasets:"},{"id":"evaluation-datasets","label":"Evaluation Datasets:","children":[],"isValid":true,"title":"Evaluation Datasets:"},{"id":"inference","label":"Inference","children":[{"id":"quantized-checkpoints","label":"Quantized Checkpoints","children":[],"isValid":true,"title":"Quantized Checkpoints"},{"id":"dgx-spark-inference-performance","label":"DGX Spark Inference Performance","children":[],"isValid":true,"title":"DGX Spark Inference Performance"}],"isValid":true,"title":"Inference"},{"id":"ethical-considerations","label":"Ethical Considerations","children":[],"isValid":true,"title":"Ethical Considerations"}],"isValid":true,"title":"Training"}]}"> NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 Model Summary
Highlights
License/Terms of Use
Benchmarks
Quick Start Guide vLLM
llama.cpp
API Client
Deployment Geography: Global
Model Architecture
Model Design
Training Methodology
Input
Output
Software Integration
Model Version(s)
Training, Testing, and Evaluation Datasets
Training Testing Datasets:
Evaluation Datasets:
Inference Quantized Checkpoints
DGX Spark Inference Performance
Ethical Considerations
NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4
Model Summary
Total Parameters 4B
Architecture Looped (latent-recurrent) normalized-GPT with interleaved sliding-window attention (SWA) and full-attention layers
Precision NVFP4
Context Length Up to 256K tokens
Supported Languages English
Best For General-purpose reasoning on a small footprint
License OpenMDW License Agreement, version 1.1
Release Date October 16, 2026
NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 is a looped (latent-recurrent) large language model (LLM) trained by NVIDIA. The model employs a normalized-GPT architecture with interleaved hybrid sliding window attention (SWA) and full attention layers. By repeatedly applying a shared block of layers, Dory1.0-4B can spend more compute per token allowing it to scale intelligence. This model is ready for commercial or non-commercial use.
Highlights
Elastic Inference Compute: A single checkpoint serves different inference compute budgets. By changing how many times the shared block of layers is applied, you can trade inference compute for accuracy to suit each task, with no retraining and no model swap.
Token Efficiency: With looping, the model needs fewer generated tokens to solve a task. Extra computation per token goes into thinking in latent space rather than into longer outputs.
Memory-Efficient Looping: Standard looping shares parameters across loops but still stores a separate KV cache for each one, so memory grows with every added loop. Dory supports last-loop KV caching, which keeps a single cache per recurrent attention layer. When enabled, Dory carries no KV-cache overhead relative to an equivalent non-looped model, regardless of the inference loop count.
Native NVFP4 Pre-training: Built on a normalized-GPT (nGPT) backbone, Dory is pre-trained natively in NVFP4, NVIDIA's 4-bit floating-point format, before continuing in BF16.
Multiple Formats: The model is available in BF16, MXFP8, NVFP4 and Q5_K_M formats.
License/Terms of Use
Governing Download Terms: Use of this model is governed by the OpenMDW-1.1 model license.
Benchmarks
Task Dory1.0-4B
Qwen3.5-4B Qwen3.5-9B Gemma 4 26B A4B Qwen3.6-35B-A3B
1 loop 2 loops 4 loops
Coding & Agentic
SciCode 27.3 34.94 35.11 25.82 28.21 40.32 37.24
SWE-bench Verified 50.3 59.5 61.8 54.7 61.0 57.40 73.4†
Terminal-Bench 2.1 12.5 18.1 22.2 15 24 35 43
τ³-bench (Banking) 11.6 12.89 14.69 5.03 7.22 13.66 9.54
GDPval-AA-V2 (Elo) 815 946 967 424 546 751.6 987
Reasoning
GPQA Diamond (no tools) 69.19 72.41 72.16 77.71 80.93 80.43 85.35
HLE (text-only, no tools) 8.25 10.89 10.43 11.31 15.43 17.79 19.18
Instruction Following
IFBench 74.29 77.94 79 53.83 59 77.89 57.44
Long Context
AA-LCR 63 71 73 64.7 68.4 64.6 68.5
General Knowledge
AA-Omniscience (accuracy) 10.03 10.08 10.03 15.02 18.05 21.13 19.95
Accuracy numbers measured by NVIDIA under a consistent harness (NeMo Gym / Nemo Evaluator SDK). Terminal-Bench 2.1 uses the Terminus-2 harness for all models. † Reported scores.
AA v4.3.2 scores are internal estimates computed under a consistent evaluation harness and may deviate from official Artificial Analysis (AA) scores.
(1) AA-LCR: We use the corrected answer keys (from AA-LCR v1.1) but keep the older judge (Qwen3-235B-A22B) instead of GPT-5.6 Luna.
(2) GDPval-AA v2: Elo ratings of selected reference models (out of 12): DeepSeek-V4-Pro (1223), Qwen3.6-35B-A3B (992), Qwen3.5-122B-A10B (925), gpt-oss-120b (745).
(3) Repeats: Number of repeated runs per benchmark: AA-Briefcase-Lite (1), GDPval-AA v2 (1), AutomationBench-AA (1), Terminal-Bench 4.0 (1), SciCode (8), Humanity's Last Exam (1), GDP.pdf text-only (5), AA-Omniscience (10), AA-LCR (16).
Looping is token-efficient. As the loop count increases, the model solves the same tasks with fewer generated tokens.
Quick Start Guide
vLLM
Note: Dory support in vLLM is under review in vllm-project/vllm#60149. Until it is included in a vLLM release, install vLLM from that PR:
VLLM_USE_PRECOMPILED=1 pip install "git+https://github.com/vllm-project/vllm.git@refs/pull/60149/head"
VLLM_USE_PRECOMPILED=1 reuses compatible prebuilt vLLM kernels, avoiding local CUDA compilation. This section will be updated once a vLLM release supports Dory.
Launch the Server
vllm serve --model nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4
--hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "per_loop"}'
--max-model-len 262144
--max-num-seqs 128
--enable-prefix-caching
--async-scheduling
--reasoning-parser nemotron_v3
--tool-call-parser qwen3_coder
--enable-auto-tool-choice
Validated context: 256K tokens. Lower --max-model-len if you are memory-constrained or want more KV-cache headroom at higher concurrency.
Choosing the Loop Count
We recommend two loops as the default starting point for a good balance between accuracy and compute. Most of the accuracy gain from looping comes from the second loop, with smaller gains beyond that (see the loop-count comparison).
Use one loop when minimizing compute per token is the priority. Consider four loops when additional accuracy matters more, particularly for agentic tasks such as Terminal-Bench and SWE-bench (see benchmark results). Because higher loop counts can also reduce the number of generated tokens, evaluate the end-to-end latency and cost on your workload when choosing a setting.
To use a different loop count, update n_recurrent_loops in --hf-overrides in the command above.
Reducing the KV Cache Memory Overhead
The command above uses per-loop caching, which stores a separate KV cache for each recurrent loop, so KV cache memory grows with the loop count. Last-loop KV caching instead keeps a singl
正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。