Like 42

Follow

NVIDIA 73.2k

Automatic Speech Recognition

NeMo

Safetensors

PyTorch

nvidia/Granary-v2

40 languages

nemo_speechlm

nvidia

nemotron-3.5

speech

asr

speech-translation

speaker-diarization

audio

transcription

speech-to-text

multilingual

timestamps

long-form-audio

language-identification

code-switching

audio-question-answering

instruction-following

hf-asr-leaderboard

arxiv: 2310.09424

License: openmdw-1.1

Model card Files Files and versions xet

Community 4

Deploy

Copy to bucket

Use this model Instructions to use nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

NeMo How to use nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B with NeMo:

import nemo.collections.asr as nemo_asr asr_model = nemo_asr.models.ASRModel.from_pretrained("nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B")

transcriptions = asr_model.transcribe(["file.wav"])

Inference

Inference Providers

Notebooks

Google Colab

Kaggle

NVIDIA-Nemotron-3.5-Transcribe-30B-A4B Description

Model Summary

Results at a Glance

Efficiency

Quick Start

Prompt Guide

License / Terms of Use

Deployment Geography

Use Case

Release Date

Model Overview References

Model Architecture

Language Support Matrix

Inputs

Outputs

Software Integration

Model Version(s)

Checkpoint Contents

Training, Testing, and Evaluation Datasets Training Dataset

Public Datasets

Private Datasets

Data Crawling and Scraping

Internally Generated Data, Including Prompt-Collected Data

Self-Sourced Synthetic Data

Testing Dataset

Evaluation Dataset

Benchmark Results

Inference

Ethical Considerations

Contributing

Inference Setup and Prompts vLLM

Evaluation Prompts

Bias

Explainability

Privacy

Safety & Security

NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

Nemotron 3.5 Transcribe is an open-weight, offline speech and audio model for transcription in 40 languages, with support for recordings up to one hour and a 65,536-token context. Its hybrid mixture-of-experts architecture activates approximately 4.3B parameters, including the audio encoder, and it is released in BF16, FP8, and NVFP4 precision.

Get started ↓ · Explore the tasks ↓ · See the evidence ↓ · NeMo Speech ↗

Model Overview

Description

NVIDIA Nemotron-3.5-Transcribe is an offline speech and audio model that processes recordings, together with text instructions and optional context, into text responses.

NVIDIA developed this Speech-Augmented Language Model (SALM) for multilingual transcription, speech translation, contextual transcription, speaker-attributed transcription, rich transcription, timestamp prediction, language identification, and audio question answering. The training mixture also includes a small amount of text instruction following and PII tagging/redaction tasks.

This model is ready for commercial or non-commercial use.

Model Summary

Total Parameters 34B (4.3B active), including the audio encoder

Architecture Mamba-2–Transformer hybrid Mixture of Experts language model with a Parallel Expert Encoder (Transformer ASR encoder and speaker encoder)

Precision BF16 (full-precision reference weights); FP8 and NVFP4 variants are available

Context Length 65,536 tokens; recordings up to 1 hour

Inputs / Outputs Audio and text / text, with optional word timestamps

Languages 40; see the language support matrix

Tasks Transcription in verbatim or written form, contextual transcription, speaker-attributed transcription, rich transcription, speech translation, language identification, and audio question answering

Deployment 1× H100 80 GB; see the hardware compatibility table

Speculative Decoding Native Multi-Token Prediction (MTP); optional DFlash2 draft model

Reasoning Mode Configurable through the chat template (enable_thinking=True/False)

Recommended Sampling Transcription, translation, and language identification: temperature 0, reasoning off; audio understanding: temperature 0.6, top_p 0.95, reasoning on

Best For Highest accuracy, customization, and fine-tuning

License OpenMDW-1.1

Results at a Glance

Results captured on October 12, 2026. Error rates are percentages. ↓ Lower is better. Complete results are listed under Benchmark Results.

With the optimized vLLM implementation in the Nemotron Transcribe NIM, one RTX PRO 6000 Blackwell Server Edition GPU transcribes up to 2,238 seconds of audio per second (NVFP4 with DFlash2 speculative decoding) and returns the transcript of a 30-second recording in 0.13 seconds. These serving optimizations are not yet part of the open-source release; we aim to upstream them to NeMo Speech and vLLM at a later date. The vLLM integration shipped in NeMo Speech on the release day is intended for model development and is expected to be roughly two times slower.

Quick Start

The example below serves this checkpoint on one H100 80 GB with vLLM 0.32.0 and the latest NeMo Speech main branch. Install both and download the checkpoint as described in Environment Setup.

export VLLM_PLUGINS=nemo_speechlm export VLLM_MAX_AUDIO_DECODE_DURATION_S=3600 export VLLM_MAX_AUDIO_CLIP_FILESIZE_MB=180

Word timestamps require the V1 model runner.

export VLLM_USE_V2_MODEL_RUNNER=0 vllm serve ./NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
--served-model-name nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
--trust-remote-code --dtype bfloat16 --gpu-memory-utilization 0.94
--max-model-len 65536 --max-num-batched-tokens 4096 --max-num-seqs 192
--enable-chunked-prefill --no-enable-prefix-caching
--use-replayssm --mamba-backend flashinfer --mamba-cache-mode none
--mm-processor-cache-gb 0 --no-async-scheduling
--limit-mm-per-prompt '{"audio":{"count":6,"length":120000000}}'
--speculative-config '{"method":"mtp","num_speculative_tokens":8}'
--reasoning-parser nemotron_v3
--trust-request-mm-kwargs
--middleware nemo.collections.speechlm2.vllm.salm.ctc_serving.ctc_timestamp_middleware

Once the server has loaded the model, transcribe a recording:

import base64 from openai import OpenAI

client = OpenAI(base_url="http://127.0.0.1:8000/v1", api_key="EMPTY") audio = base64.b64encode(open("audio.wav", "rb").read()).decode() response = client.chat.completions.create( model="nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B", messages=[{"role": "user", "content": [ {"type": "text", "text": "Transcribe the following audio."}, {"type": "audio_url", "audio_url": {"url": f"data:audio/wav;base64,{audio}"}}, ]}], temperature=0, extra_body={"chat_template_kwargs": {"enable_thinking": False}, "skip_special_tokens": False}, ) print(response.choices[0].message.content)

See Inference Setup and Prompts for other configurations, reasoning, word timestamps, and task-specific prompts.

Prompt Guide

Choose the transcription style with the instruction in the user message; keep the system prompt empty for transcription and translation.

This is an offline model. Internal audio chunking does not make it a real-time streaming ASR service. Language and task support vary; check the language support matrix. See Prompt Recommendations for more tasks and All Evaluated Prompts for the exact benchmark prompts.

License / Terms of Use

Use of this model is governed by the OpenMDW License Agreement, version 1.1 (OpenMDW-1.1). Users are responsible for complying with this license and with any third-party licenses or notices applicable to components included in the distributed model materials.

Use Case

Intended users are developers and researchers building speech interfaces, transcription and translation tools, meeting and media processing, customer-service analytics, and audio-understanding applications. Text context can supply task instructions and vocabulary hints. Speaker labels are intended to distinguish speakers within a recording; they do not establish real-world identity.

Release Date

Build.Nvidia.com 10/28/2026

Github 10/28/2026 via NVIDIA NeMo Speech

Hugging Face 10/28/2026 via NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

NGC 10/28/2026 via NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

References

Model page: NVIDIA-Nemotron-3.5-Transcribe-30B-A4B

Technical report: Nemotron 3.5 Transcribe Technical Report

Base language model: NVIDIA Nemotron-3.5 Lightning 30B-A3B BF16

Quick-start and inference documentation: see Inference quick start guide.

Model Architecture

Architecture Type: Other - hybrid Mamba-Transformer mixture-of-experts (MoE) language model with a Transformer audio encoder

Network Architecture: Nemotron 3.5 Lightning hybrid Mamba-Transformer MoE with Transformer ASR encoder and Sortformer speaker encoder

Design: Audio perception module consisting of transformer ASR encoder and Nemotron-3-Diarization speaker encoder, a connectionist temporal classification (CTC) timestamp alignment head, and NVIDIA Nemotron-3.5 Lightning 30B-A3B language backbone. The perception configuration uses a 1,280-dimensional identity connector followed by a learned projection to 2,688 dimensions. Audio features are combined with the text prompt for text generation. The model enables joint multi-token prediction (MTP), with two trained next-token prediction depths using a repeated layer and retaining the existing head. Reusing the repeated layer during inference supports up to eight speculative tokens. A dedicated DFlash2 draft model, a block-diffusion drafter based on DFlash, is also provided as an option for speculative decoding.

Base model: nvidia/NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16; 600M parameter pretrained ASR transformer; and approximately 99M parameter Nemotron-3-Diarization speaker encoder.

Parameter count: The language backbone has approximately 3.5B active parameters. Together with the approximately 800M-parameter joint audio encoder, the model has approximately 4.3B active parameters.

Precision: Weights are released in bfloat16 (BF16), with FP8 and NVFP4 quantized variants.

Language Support Matrix

✓ Supported

— Not specified

ASR means automatic speech recognition; code-switching means using more than one language within a recording. Translation support covers both directions between English and the listed language.

Regional variants listed in parentheses are supported as part of their language.

The matrix covers transcription and translation. Audio understanding, reasoning, and spoken instruction following were evaluated only in English and may not work reliably in other languages.

Inputs

Property Draft specification

Types Audio and text.

Formats Audio: .wav, .flac, .mp3, .ogg (any format that can be decoded to PCM). Text: String (plain text or chat messages).

Parameters Audio: one-dimensional waveform per processed channel. Text: one-dimensional token sequence.

Sampling and preprocessing Recipe audio sampling rate: 16 kHz. Use the exported model's preprocessor and nemotron3p5 prompt format. Encoder feature stride is configured as approximately 80 ms per audio token equivalent.

Limits The model is trained with recordings up to 1 hour long, and its maximum context size in training was 65536 tokens. It is not guaranteed to generalize beyond these limits.

Audio chunking The encoder uses 30-second post-feature chunks. This is an internal processing choice, not a 30-second recording limit or a streaming latency claim.

Language coverage See the 40-language support matrix.

Other Properties Related to Input: The model processes 16kHz single-channel audio decoded to a float32 PCM waveform. The maximum supported audio length is up to 1 hour.

Outputs

Property Specification

Types Text.

Formats String.

Parameters One-dimensional (1D) token sequence.

Content Depending on the prompt: transcripts, translations, speaker/event tags, language labels