An AI agent that reproduces physics papers from scratch has started finding flaws the original peer reviewers did not catch.
Researchers deployed a single Claude Opus configuration to autonomously read, reproduce, and evaluate 111 open-access computational physics papers written with the Quantum ESPRESSO simulation software. The agent ran a read-plan-compute-compare loop and, without being prompted to critique, raised substantive methodological concerns on roughly 42% of the papers. Critically, 96.6% of those concerns only emerged after the agent had actually run the calculations; a reading-only pass surfaced concerns on just 1.8% of papers. The team then pointed a fresh agent at a single Nature Communications paper on 2D-material transistor simulation, and it produced a 14-concern physics inventory and a complete six-page comment challenging the paper's headline result.
Peer review has long relied on experts reading and reasoning about papers rather than independently running the underlying computations. Execution-driven scrutiny appears to surface problems that reading alone misses, including two methodological attacks absent from a published 21-reviewer peer review. If this scales across disciplines, it could change what scientific validation actually means.
None of this means the AI is always right; the paper does not claim it is. It does, however, raise an uncomfortable question about what peer review has been checking all along.