A new benchmark finds that even the best AI agents fail most real-world molecular design tasks.
Researchers built MolDesignBench, a set of roughly 2,000 molecule generation and optimization problems designed to mimic how chemists actually work - with implicit requirements buried in prose, explicit property and functional-group constraints, and some specifications that are simply impossible to satisfy. Agents had to work through 17 specialized chemistry tools to produce answers, not just guess a plausible-looking structure. Across a range of frontier large language models, the top performer solved only about 43 percent of tasks correctly. The researchers' failure analysis pinned most errors on two things: agents missing implicit constraints buried in the design narrative, and agents failing to recognize when a request was infeasible in the first place.
Most existing chemistry benchmarks test LLMs on narrow, explicitly stated, and always-solvable problems - an easier task than what a chemist actually faces. MolDesignBench's low scores suggest that claims about AI doing real chemistry work rest on benchmarks that don't resemble a real lab, and that agents can't yet be trusted to flag a dead end instead of confidently returning a molecule that only looks like it fits.
A tool that can't tell you a request is impossible is arguably more dangerous in a lab than one that's simply slow, and a 43 percent hit rate is a long way from replacing a bench chemist.