NVIDIA Nemotron 3.5 Embed, Rerank, and Parse: Advancing Agents with World-Class Retrieval Models

Back to Articles

a]:hidden">

NVIDIA Nemotron 3.5 Embed, Rerank, and Parse: Advancing Agents with World-Class Retrieval Models

Enterprise + Article Published October 21, 2026

Upvote 1

Yauhen Babakhin ybabakhin Follow

nvidia

Ryan Chesler emelryan Follow

nvidia

Jiarui Cai jiaruic Follow

nvidia

Sean Sodha ssodha-nv Follow

nvidia

Isabel Hulseman ihulseman0220 Follow

nvidia

:last-child]:mb-0"> Nemotron 3.5 Embed and Rerank Leading Multimodal Dense Retrieval on ViDoRe v3

Reranking the Evidence for Agent Tasks

Scaling Multimodal Retrieval with NVFP4

High-Performance Serving with NVIDIA NIM

Adapting Retrieval to Domain-Specific Tasks

How We Built the Embed and Rerank Models

Nemotron 3.5 Parse Leading Accuracy with Faster Document Processing

How We Built Nemotron 3.5 Parse

Early Enterprise Partner Evaluations

Getting Started Acknowledgments and main contributors

An agent’s answers depend on the evidence it can retrieve. When that evidence is incomplete or irrelevant, the agent will repeat searches and inspect additional documents, consuming more inference tokens before completing the task. Better retrieval can help agents reach grounded answers with fewer wasted tokens. At enterprise scale, that waste leads to massive inefficiencies. Agents need the right retrieval models and systems to reach insights faster.

Today, we are expanding the Nemotron Retriever collection with new open, commercially available models for embedding, reranking, and document extraction. The new models include Nemotron 3.5 Embed 8B BF16, 1B BF16, 1B NVFP4, Nemotron 3.5 Rerank 8B BF16, and Nemotron 3.5 Parse at 631M parameters.

Together, these state-of-the-art models bring multimodal retrieval, relevance ranking, and document processing to enterprise agent workflows.

Better Retrieval for More Efficient Agents

Retrieval quality affects both the answers an agent produces and the LLM inference cost. Relevant evidence can help the agent answer a question or better decide its next step, and reranking improves the quality of evidence that enters its context.

Consider an agent asked to identify products at risk of stockouts next quarter and explain why. It must combine demand forecasts in charts, inventory levels in tables, and supplier lead times in reports, then reason about whether replenishment will arrive before inventory runs out. Completing the task requires finding the relevant evidence across documents and using it together in the analysis.

To understand how retrieval affects the agent’s ability to combine evidence and complete a task, we evaluated Nemotron 3.5 Embed and Rerank across ViDoRe v3, BRIGHT, and BrowseComp-Plus. ViDoRe v3 uses combined text and image inputs, while BRIGHT and BrowseComp-Plus use text-only inputs. In this pipeline, an agent queries the corpus using a search tool, inspects the evidence, and refines its next search attempt to find the most relevant information to a given query. The experiments below vary how the search tool is implemented: embedding only, embedding plus reranking, and with different models for each. The LLM powering the agent, tools, task set, corpus, and execution policy remain fixed while the retrieval configuration changes. We measure the agent's ability to find the relevant information for each task and total agent inference tokens per task, including input and output tokens across all turns.

Figure 1. End-to-end task success versus total agent inference tokens per task across retrieval configurations.

Nemotron 3.5 Embed and Rerank

Behind these agent successes are Nemotron 3.5 Embed and Nemotron 3.5 Rerank, which retrieve evidence across text and images and prioritize it for the agent’s context. Nemotron 3.5 Embed builds on the previous generation’s Nemotron 3 Embed text embedding models to retrieve evidence from document images, including tables, charts, infographics, datasheets, and schematics. This allows applications to search information conveyed by visual content alongside document text.

The four models provide different options for retrieval and second-stage ranking:

Table 1. Nemotron 3.5 Embed and Nemotron 3.5 Rerank model capabilities and use cases across various sizes and variants.

Leading Multimodal Dense Retrieval on ViDoRe v3

On ViDoRe v3.1, Nemotron 3.5 Embed 8B BF16 achieves 61.17 nDCG@10, ranking first among dense retrieval models. This is an 8.94-point improvement over the 52.23 score achieved by the Llama Nemotron Embed VL 1B model. The 1B model brings most of this visual retrieval quality into a smaller model. Distilled from the 8B teacher, Nemotron 3.5 Embed 1B BF16 scores 58.38 nDCG@10, retaining 95.4% of the teacher’s score.

Figure 2. Nemotron 3.5 Embed 8B BF16 ranked first among dense retrieval models on the ViDoRe v3.1 leaderboard as of 10/20/2026.

We also compare the models with Qwen3-VL-Embedding, EmbeddingGemma-2, and the previous-generation NVIDIA model across ViDoRe v3, Korean ViDoRe and MIRACL Vision. The chart below reports retrieval accuracy for image-only and combined image-and-text inputs.

Figure 3. Text and visual retrieval performance of Nemotron 3.5 Embed compared with prior-generation models and selected baselines.

The multilingual results show how the models retrieve evidence across different language settings. On MIRACL Vision, the 8B model achieves 72.6 nDCG@10, exceeding Qwen3-VL-Embedding-8B by 4.3 points. We also evaluate the models on KoViDoRe; both Nemotron 3.5 Embed 8B and 1B outperform the Qwen baselines when using combined image and text inputs.

Reranking the Evidence for Agent Tasks

Embedding models make large collections searchable by representing content as vectors. After retrieval returns a candidate set, Nemotron 3.5 Rerank 8B BF16 evaluates each candidate against the query to refine the ordering across text and image inputs. By distinguishing broadly related content from evidence that directly answers the question, the reranker helps agents use their context budget more effectively and focus on information needed to complete the task. For agentic retrieval systems, surfacing the strongest evidence earlier can reduce distracting context and help agents produce more grounded, efficient responses.

In collaboration with the MTEB team, we are launching the ViDoRe v3.1 reranking leaderboard to benchmark rerankers on document retrieval across image-only, text-only, and combined image-and-text inputs. Nemotron 3.5 Rerank 8B BF16 achieves 60.49 nDCG@10 with combined image-and-text inputs compared with Qwen-3-VL-Reranker-8B’s 59.17.

Figure 4. Nemotron-3.5-Rerank-8B-BF16 ranked first on ViDoRe v3.1 Reranking leaderboard as of 10/20/2026.

Figure 5. Text and visual reranker performance of Nemotron 3.5 Rerank 8B compared with Qwen 3 VL Reranker 8B. Both models reranked the top 50 candidates retrieved by the Nemotron 3.5 Embed 1B model.

Scaling Multimodal Retrieval with NVFP4

Retrieval quality is only part of the equation for production agentic systems. As document collections and query volumes grow, teams must balance accuracy, throughput, and infrastructure requirements.

Nemotron 3.5 Embed 1B NVFP4 provides an optimized deployment option for high-throughput multimodal retrieval. On NVIDIA GB200, it delivers 27.1% higher throughput compared with the 1B BF16 model, increasing from 101.3 to 128.8 documents per second while retaining 99.85% of its retrieval accuracy on ViDoRe v3 with combined image and text inputs. As Figure 6 shows, the NVFP4 model offers a strong accuracy-throughput tradeoff for teams scaling retrieval across production workloads.

This deployment flexibility complements domain-specific customization: teams can select the data type of model weights that fits their infrastructure, then adapt the embedding and reranking models to their own terminology, documents, and retrieval tasks.

Figure 6. ViDoRe V3 retrieval accuracy versus serving efficiency, comparing Nemotron 3.5 Embed with selected smaller open embedding baselines.

High-Performance Serving with NVIDIA NIM

NVIDIA NIM provides a high-performance deployment path for Nemotron 3.5 Embed. In testing on an NVIDIA RTX PRO 6000 Blackwell Server Edition, NIM delivered 6,031 query requests per second—approximately 2.9× the throughput of a comparable vLLM configuration at 2,099 requests per second—while maintaining comparable passage-processing throughput. This optimized serving stack helps enterprises scale retrieval workloads with production-ready deployment and support.

Figure 7. Nemotron 3.5 Embed NIM serving performance compared with the vLLM checkpoint on NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs.

Adapting Retrieval to Domain-Specific Tasks

Enterprise agents often search documents containing specialized terminology and company-specific information that general-purpose retrieval models may not fully capture. Fine-tuning allows teams to adapt Nemotron 3.5 Embed and Rerank using their own documents, queries, and relevance labels to their specific domains and use cases.

The Nemotron multimodal embedding fine-tuning recipe combines synthetic data generation, hard-negative mining, fine-tuning with NVIDIA NeMo AutoModel, and evaluation in one workflow. Teams can generate queries and relevance labels from their own documents to build training data for their retrieval tasks.

We evaluated the recipe on 8,202 chart and infographic images using 8,591 synthetic queries generated with Qwen3.8-Flash-Next, focusing on enumerative and open-ended queries with multiple relevant images. Fine-tuning the Nemotron-3.5-Embed-1B-BF16 on 6,872 queries for one epoch increased nDCG@10 from 37.34 to 44.12, a 18% relative gain over the base model, on 1,719 held-out synthetic queries. Using the same synthetic queries set, fine-tuning Nemotron-3.5-Embed-8B-BF16 for epoch increased nDCG@10 from 41.44 to 42.75, a 3% relative gain.

How We Built the Embed and Rerank Models

Nemotron 3.5 Embed 8B builds on Ministral-3-8B-Instruct, with its language component initialized from Nemotron-3-Embed-8B-BF16. The model is trained on image and text corpora using contrastive learning to produce representations for multimodal retrieval.

The Embed 1B models are distilled from Nemotron 3.5 Embed 8B, with their language component initialized from Nemotron-3-Embed-1B-BF16. Distillation combines cosine and mean squared error (MSE) losses to align the student’s embeddings with the teacher’s.

The language component remains frozen throughout embedding-model training. Text-only embeddings are therefore expected to remain unchanged from the corresponding Nemotron 3 Embed models, allowing existing text embeddings and vector indexes to be reused.

Nemotron 3.5 Rerank uses the same backbone architecture as Nemotron-3.5-Embed-8B in a cross-encoder configuration. While the embedding model encodes queries and documents separately and scores their similarity, the reranker jointly encodes each query–document pair. A final multilayer perceptron (MLP) scoring head produces a scalar relevance logit.

All models are trained on a mixture of publicly available and synthetic multilingual, multimodal data, and their final released weights are produced through checkpoint ensembling. Further details on the training data are available in each model’s Hugging Face model card.

Nemotron 3.5 Parse

Enterprise knowledge often arrives in PDFs whose embedded text captures only part of what is on the page. Extracting that text can lose reading order, separate table values from their headers, or omit information conveyed in charts. Scanned documents and image-based PDFs may contain no usable text layer at all. Before this information can support search, retrieval, or automated document workflows, the pixels need to become structured content.

Nemotron 3.5 Parse converts document images into machine-readable text while preserving the structure that gives the content meaning. It identifie