返回资讯中心
外部精选
软件工程
#开发者实时流

Nvidia-Nemotron-Labs-Dory1.0-4B-NVFP4

Use this model Instructions to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Hacker News 最新fprog2026/10/17 · 2026/10/1730 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Like 1

Follow

NVIDIA 71.5k

Text Generation

Transformers

Safetensors

PyTorch

nvidia/nemotron-post-training-v3

nvidia/nemotron-pre-training-datasets

English

dory

nvidia

nemotron-labs

conversational

8-bit precision

modelopt

License: openmdw-1.1

Model card Files Files and versions xet

Community 6

Deploy

Copy to bucket new

Use this model Instructions to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

Transformers How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with Transformers:

Use a pipeline as a high-level helper

from transformers import pipeline

pipe = pipeline("text-generation", model="nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)

pip install -U transformers accelerate

Load model directly

from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained("nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4", device_map="auto")

Notebooks

Google Colab

Kaggle

Local Apps Settings

vLLM How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with vLLM:

Install vLLM from pip:

pip install vllm

Start the vLLM server:

vllm serve "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:8000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker

docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4

SGLang How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with SGLang:

Install SGLang from pip:

pip install sglang

Start the SGLang server:

python3 -m sglang.launch_server
--model-path "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Use Docker images

docker run --gpus all
--shm-size 32g
-p 30000:30000
-v ~/.cache/huggingface:/root/.cache/huggingface
--env "HF_TOKEN=<secret>"
--ipc=host
lmsysorg/sglang:latest
python3 -m sglang.launch_server
--model-path "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"
--host 0.0.0.0
--port 30000

Call the server using curl (OpenAI-compatible API):

curl -X POST "http://localhost:30000/v1/chat/completions"
-H "Content-Type: application/json"
--data '{ "model": "nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'

Docker Model Runner How to use nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 with Docker Model Runner:

docker model run hf.co/nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4

Quick Start Guide</strong>","children":[{"id":"vllm","label":"<strong>vLLM</strong>","children":[],"isValid":true,"title":"vLLM"},{"id":"llamacpp","label":"<strong>llama.cpp</strong>","children":[],"isValid":true,"title":"llama.cpp"},{"id":"api-client","label":"<strong>API Client</strong>","children":[],"isValid":true,"title":"API Client"}],"isValid":true,"title":"Quick Start Guide"},{"id":"deployment-geography-global","label":"Deployment Geography: Global","children":[],"isValid":false,"title":"Deployment Geography: Global"},{"id":"model-architecture","label":"Model Architecture","children":[],"isValid":true,"title":"Model Architecture"},{"id":"model-design","label":"Model Design","children":[],"isValid":true,"title":"Model Design"},{"id":"training-methodology","label":"Training Methodology","children":[],"isValid":true,"title":"Training Methodology"},{"id":"input","label":"Input","children":[],"isValid":true,"title":"Input"},{"id":"output","label":"Output","children":[],"isValid":true,"title":"Output"},{"id":"software-integration","label":"Software Integration","children":[],"isValid":true,"title":"Software Integration"},{"id":"model-versions","label":"Model Version(s)","children":[],"isValid":true,"title":"Model Version(s)"},{"id":"training-testing-and-evaluation-datasets","label":"Training, Testing, and Evaluation Datasets","children":[],"isValid":false,"title":"Training, Testing, and Evaluation Datasets"}],"isValid":true,"title":"NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4"},{"id":"training","label":"Training","children":[{"id":"testing-datasets","label":"Testing Datasets:","children":[],"isValid":true,"title":"Testing Datasets:"},{"id":"evaluation-datasets","label":"Evaluation Datasets:","children":[],"isValid":true,"title":"Evaluation Datasets:"},{"id":"inference","label":"Inference","children":[{"id":"quantized-checkpoints","label":"Quantized Checkpoints","children":[],"isValid":true,"title":"Quantized Checkpoints"},{"id":"dgx-spark-inference-performance","label":"DGX Spark Inference Performance","children":[],"isValid":true,"title":"DGX Spark Inference Performance"}],"isValid":true,"title":"Inference"},{"id":"ethical-considerations","label":"Ethical Considerations","children":[],"isValid":true,"title":"Ethical Considerations"}],"isValid":true,"title":"Training"}]}"> NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 Model Summary

Highlights

License/Terms of Use

Benchmarks

Quick Start Guide vLLM

llama.cpp

API Client

Deployment Geography: Global

Model Architecture

Model Design

Training Methodology

Input

Output

Software Integration

Model Version(s)

Training, Testing, and Evaluation Datasets

Training Testing Datasets:

Evaluation Datasets:

Inference Quantized Checkpoints

DGX Spark Inference Performance

Ethical Considerations

NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4

Model Summary

Total Parameters 4B

Architecture Looped (latent-recurrent) normalized-GPT with interleaved sliding-window attention (SWA) and full-attention layers

Precision NVFP4

Context Length Up to 256K tokens

Supported Languages English

Best For General-purpose reasoning on a small footprint

License OpenMDW License Agreement, version 1.1

Release Date October 16, 2026

NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4 is a looped (latent-recurrent) large language model (LLM) trained by NVIDIA. The model employs a normalized-GPT architecture with interleaved hybrid sliding window attention (SWA) and full attention layers. By repeatedly applying a shared block of layers, Dory1.0-4B can spend more compute per token allowing it to scale intelligence. This model is ready for commercial or non-commercial use.

Highlights

Elastic Inference Compute: A single checkpoint serves different inference compute budgets. By changing how many times the shared block of layers is applied, you can trade inference compute for accuracy to suit each task, with no retraining and no model swap.

Token Efficiency: With looping, the model needs fewer generated tokens to solve a task. Extra computation per token goes into thinking in latent space rather than into longer outputs.

Memory-Efficient Looping: Standard looping shares parameters across loops but still stores a separate KV cache for each one, so memory grows with every added loop. Dory supports last-loop KV caching, which keeps a single cache per recurrent attention layer. When enabled, Dory carries no KV-cache overhead relative to an equivalent non-looped model, regardless of the inference loop count.

Native NVFP4 Pre-training: Built on a normalized-GPT (nGPT) backbone, Dory is pre-trained natively in NVFP4, NVIDIA's 4-bit floating-point format, before continuing in BF16.

Multiple Formats: The model is available in BF16, MXFP8, NVFP4 and Q5_K_M formats.

License/Terms of Use

Governing Download Terms: Use of this model is governed by the OpenMDW-1.1 model license.

Benchmarks

Task Dory1.0-4B

Qwen3.5-4B Qwen3.5-9B Gemma 4 26B A4B Qwen3.6-35B-A3B

1 loop 2 loops 4 loops

Coding & Agentic

SciCode 27.3 34.94 35.11 25.82 28.21 40.32 37.24

SWE-bench Verified 50.3 59.5 61.8 54.7 61.0 57.40 73.4†

Terminal-Bench 2.1 12.5 18.1 22.2 15 24 35 43

τ³-bench (Banking) 11.6 12.89 14.69 5.03 7.22 13.66 9.54

GDPval-AA-V2 (Elo) 815 946 967 424 546 751.6 987

Reasoning

GPQA Diamond (no tools) 69.19 72.41 72.16 77.71 80.93 80.43 85.35

HLE (text-only, no tools) 8.25 10.89 10.43 11.31 15.43 17.79 19.18

Instruction Following

IFBench 74.29 77.94 79 53.83 59 77.89 57.44

Long Context

AA-LCR 63 71 73 64.7 68.4 64.6 68.5

General Knowledge

AA-Omniscience (accuracy) 10.03 10.08 10.03 15.02 18.05 21.13 19.95

Accuracy numbers measured by NVIDIA under a consistent harness (NeMo Gym / Nemo Evaluator SDK). Terminal-Bench 2.1 uses the Terminus-2 harness for all models. † Reported scores.

AA v4.3.2 scores are internal estimates computed under a consistent evaluation harness and may deviate from official Artificial Analysis (AA) scores.

(1) AA-LCR: We use the corrected answer keys (from AA-LCR v1.1) but keep the older judge (Qwen3-235B-A22B) instead of GPT-5.6 Luna.

(2) GDPval-AA v2: Elo ratings of selected reference models (out of 12): DeepSeek-V4-Pro (1223), Qwen3.6-35B-A3B (992), Qwen3.5-122B-A10B (925), gpt-oss-120b (745).

(3) Repeats: Number of repeated runs per benchmark: AA-Briefcase-Lite (1), GDPval-AA v2 (1), AutomationBench-AA (1), Terminal-Bench 4.0 (1), SciCode (8), Humanity's Last Exam (1), GDP.pdf text-only (5), AA-Omniscience (10), AA-LCR (16).

Looping is token-efficient. As the loop count increases, the model solves the same tasks with fewer generated tokens.

Quick Start Guide

vLLM

Note: Dory support in vLLM is under review in vllm-project/vllm#60149. Until it is included in a vLLM release, install vLLM from that PR:

VLLM_USE_PRECOMPILED=1 pip install "git+https://github.com/vllm-project/vllm.git@refs/pull/60149/head"

VLLM_USE_PRECOMPILED=1 reuses compatible prebuilt vLLM kernels, avoiding local CUDA compilation. This section will be updated once a vLLM release supports Dory.

Launch the Server

vllm serve --model nvidia/NVIDIA-Nemotron-Labs-Dory1.0-4B-NVFP4
--hf-overrides '{"n_recurrent_loops": 2, "recurrent_kv_cache_mode": "per_loop"}'
--max-model-len 262144
--max-num-seqs 128
--enable-prefix-caching
--async-scheduling
--reasoning-parser nemotron_v3
--tool-call-parser qwen3_coder
--enable-auto-tool-choice

Validated context: 256K tokens. Lower --max-model-len if you are memory-constrained or want more KV-cache headroom at higher concurrency.

Choosing the Loop Count

We recommend two loops as the default starting point for a good balance between accuracy and compute. Most of the accuracy gain from looping comes from the second loop, with smaller gains beyond that (see the loop-count comparison).

Use one loop when minimizing compute per token is the priority. Consider four loops when additional accuracy matters more, particularly for agentic tasks such as Terminal-Bench and SWE-bench (see benchmark results). Because higher loop counts can also reduce the number of generated tokens, evaluate the end-to-end latency and cost on your workload when choosing a setting.

To use a different loop count, update n_recurrent_loops in --hf-overrides in the command above.

Reducing the KV Cache Memory Overhead

The command above uses per-loop caching, which stores a separate KV cache for each recurrent loop, so KV cache memory grows with the loop count. Last-loop KV caching instead keeps a singl

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →