Can AI agents do senior expert work? What the evidence says

Studies now have experts grade AI work products against their own. They also show where AI breaks: open-ended judgment, choices of method, and the gap between how productive people feel and how productive they are.

Leaders are being asked to rely on analysis that an AI system produced, or helped produce: a forecast, a cost estimate, a policy brief, a clinical risk score. The fair question is whether that work is as good as a qualified expert’s. The honest answer from the published evidence is “on some tasks, under some conditions, and you have to test which”.

Here is what five recent studies show, and what they do not.

The most direct test so far is GDPval [1]. Its authors built tasks from real work products across 44 occupations in 9 sectors, and had experts grade AI output against human output in blind, head-to-head comparisons. The graders averaged 14 years of experience.

That design is a real step forward from quiz-style benchmarks. The limits matter just as much. Each task was one-shot, with all the context supplied up front. The authors note that real work involves building that context, going back and forth with colleagues and changing course, and they plan to add more interaction later. A senior expert’s value often lies in exactly those parts.

A field experiment with 758 consultants at Boston Consulting Group gave some of them access to a large language model [2]. On tasks inside what the authors call the “jagged technological frontier”, consultants with AI completed 12.2% more tasks, worked 25.1% faster and produced higher-quality work.

On a task chosen to sit outside that frontier, consultants using AI were markedly less likely to produce a correct solution than those working without it, about 19 points lower in the original working paper [3].

This study is about AI as an assistant, not autonomous agents. Its lesson carries over. The same tool can raise quality on one task and lower it on the next, and the boundary is not visible from the output.

A randomized study of 16 experienced open-source developers working on 246 real issues found that tasks took 19% longer when developers could use early-2025 AI tools (95% confidence interval: 2% to 39% longer) [4]. Before starting, the developers had forecast a 24% speed-up.

The authors now mark these results as out of date, and their follow-up with newer tools is hard to interpret because some developers refused to work without AI. The finding that lasts is about measurement, not speed. People’s sense of how much AI helps them is not evidence of how much it helps. Only a comparison can tell you.

A 2026 study gave the same financial dataset and the same six hypotheses to 150 autonomous AI coding agents [5]. Their answers varied widely, and the biggest source of disagreement was which measure each agent chose to test the hypothesis with. Human research teams show the same pattern, and the study was designed to replicate an earlier one on human teams.

Two further results matter for anyone relying on AI analysis. Written critiques from other AI reviewers barely reduced the spread. Showing the agents highly rated examples cut it sharply, by 80% to 99% within groups of similar measures, but the authors read that as imitation rather than better understanding. Agreement among AI outputs is not the same as being right.

A benchmark of AI agents on intensive-care prediction tasks, using hospital records, notes and chest X-rays, found that single-agent systems outperformed naive multi-agent systems and were better calibrated [6]. A “committee” of agents can sound more rigorous than one agent. In this test it was not.

Our free Evidence-Gated AI Checklist turns these into eight questions you can score in five minutes.

Most of these studies are recent preprints that have not yet been peer-reviewed. The consultant experiment [2] is published in a peer-reviewed journal, and the clinical benchmark [6] was accepted at a peer-reviewed conference. Findings in this field age quickly, and the systems tested are already being replaced. The questions they raise do not age as fast.