LabRobotBench v0.1: Benchmarking wet lab robotics

Gamow Labs builds frontier AI systems to interpret the human genome. We’re building an agent-operated lab to accelerate that work.

Our in-silico simulation mirrors the command interfaces of our physical robots, allowing us to develop and evaluate agents that can run experiments and recover from equipment failures.

When a robot encounters an error, an experiment can stall until a person steps in. Agents that can diagnose the problem and safely resume work could keep experiments running overnight, reduce downtime, and let scientists run more experiments at once. This is what we’re building at Gamow Labs: accelerating human progress by using agents to increase experimental throughput.

To address this challenge, we present LabRobotBench. Its evaluations test how frontier AI models reason about physical space and complex biological workflows to troubleshoot and carry out wet-lab experiments.

Results · Cell assay recovery · Assays and cold stocks · Dose-response recovery · Partner with us

How much of the required physical workflow does each agent recover?

A robot stopped halfway through an experiment involving four trays of cells. Three could still be tested. The agent needed to finish preparing them, measure the results, and put them away. The fourth was already spoiled and needed to be set aside.

Each tray was at a different step when the robot stopped. The agent had to work out which steps were already done, then finish washing the cells, add a chemical used to measure cell activity, and run the measuring instrument. It also needed to save the results and return each tray with its lid. Repeating a chemical treatment could ruin a sample.

A tray can be safely back on its shelf while the experiment is still unfinished. The important question is whether the cells received the right treatment and were measured in time.

Watch Opus put the yellow-labeled tray back on the shelf. Its cells have been washed, but the agent never adds the chemical needed for the test or measures the result. The tray is put away; the experiment is left unfinished.

The agent had several jobs competing for its attention: finish two experiments, set aside one spoiled sample, return two trays of supplies to cold storage, and leave a comparison sample untouched. Some of these jobs had only a few minutes left.

One experiment needed its next chemical addition finished within 200 seconds. The cold supplies could spend no more than 240 seconds outside storage. Because the same robot moved all the trays, the agent had to choose what to handle first.

In a lab, helping one sample can mean leaving another waiting. An agent needs to keep track of both deadlines and choose an order that protects the materials.

GPT gets both trays of cold supplies back in time but fails to finish one experiment. Opus finishes both experiments, but leaves both trays of supplies out too long.

Watch Gemini carry the red-labeled tray to the cold-storage machine. By the time the tray is stored inside, it has spent about 422 seconds outside—well past the 240-second limit. The robot moves it successfully, but too late to satisfy the task.

This experiment tests how cells respond to different amounts of a chemical. The agent needed to finish testing two trays of cells and set aside a third that was already spoiled. One tray was close to its measurement deadline.

The agent was expected to finish an experiment within a fixed timeframe. After the cells were treated with the test chemical, it had 30 minutes to complete a measurement of their response. It needed to check which steps were already done, finish preparing the samples, and leave enough time for the measuring instrument to run.

Timing requirements like this are common in live biology experiments, where a sample’s response can change after treatment. Taking measurements at the required time helps scientists compare results across samples. The agent therefore needs to plan around the experiment’s timing, as well as operate the equipment.

Watch GPT take the tray out of the measuring instrument. The machine has finished, but its measurement ended 42.5 seconds after the deadline. The robot carries on normally, and the agent later reports the test as completed.

At Gamow Labs, we work at the frontier of AI and wet-lab biology to better understand rare diseases. We’re building agents that can operate robotic laboratories and help scientists uncover how genetic changes cause disease.

If you’re developing frontier models, building laboratory automation, or looking to expand your experimental capabilities, we’d love to work with you.