DualChem Benchmark

Products

Solutions

Research

Resources

Log in

Get Started

Get Started

← BlogPolicy

Introducing DualChem

By Madhu Sehwag·October 28, 2026·4 min read

Copy Link

Most existing AI benchmarks evaluate general capability, such as reasoning accuracy, coding performance, or factual recall, all of which do not accurately capture a model’s behavior in ambiguous, high-stakes environments. DualChem was designed to measure the latter: determining whether a model is capable of providing the same answer to support either legitimate scientific research or harmful activity.

For government agencies tasked with assessing increasingly capable frontier models, it is important to be able to better understand whether these systems can recognize harmful intent. DualChem is one of the first benchmarks designed to evaluate whether large language models provide assistance in such dual-use chemistry scenarios, and where current safeguards related to this ability may fall short, which in the real world would have to be balanced against not inhibiting beneficial uses.

Scale’s findings are captured in our recent research paper, “DualChem: Measuring LLM Uplift and Dual-Use Risk Awareness in Chemistry ”.

This effort represents the first benchmark resulting from a Collaborative Research and Development Agreement signed with the NIST Center for Advancing Innovation and Standards for Super Intelligence (CAISSI) in February 2025, combining Scale's evaluation expertise with input from CAISSI and U.S. Government subject-matter experts, who informed the scope and question taxonomy underpinning the benchmark.

Chemistry is one of the clearest testbeds for understanding dual-use risk in frontier AI systems. The same chemistry guidance that helps a graduate student troubleshoot an experiment could also lower the barrier for harmful misuse. Existing safety benchmarks often focus on explicit harmful requests, but in the real world, the same question may have both legitimate and harmful applications depending on context.

The DualChem benchmark consists of 600 expert-curated questions (reviewed by PhD-level expert evaluators before inclusion) spanning multiple laboratory chemistry domains, including molecular design, synthesis procedures, purification, validation, and information management. The questions were developed by chemistry experts, most with advanced degrees, to reflect realistic dual-use ambiguity where the same request could support either legitimate research or harmful misuse.

Rather than relying on synthetic or explicitly adversarial prompts, which can fail to capture how real chemistry questions are asked, the benchmark focuses on realistic chemistry questions with ambiguous dual-use applications.

We evaluated 30+ state-of-the-art frontier models on the benchmark; you can see how they performed here.

DualChem contributes something to the field that has largely remained unexplored: a rigorous, expert-validated framework for measuring a specific and consequential failure mode, one that doesn’t show up in standard capability or refusal benchmarks.

The findings also suggest that refusal-based safety mechanisms alone may be insufficient for handling complex dual-use scenarios. Independent assessments can help policymakers and procurement teams better understand how models behave in ambiguous dual-use scenarios and where safeguards may fall short.

This research highlights the importance of collaboration between industry, researchers, and government institutions in developing shared evaluation methodologies for frontier AI systems.

It is critical that public and private partnerships can enable evaluation frameworks that support institutional oversight, including adversarial stress-testing, robustness analysis under distribution shifts, and measurement approaches aligned with evaluators’ needs.

Scale Labs will continue partnering with governments and national AI safety institutes to advance research on frontier AI evaluation and deployment. The goal is to help institutions better understand how models behave across different contexts and what safeguards, testing, and evaluation mechanisms support reliable deployment.

Scale's team will match your project to the right experts, fast.

Reliable AI for the world’s most important decisions

Copyright © 2026 Scale AI, Inc. All rights reserved