AI/ ai · research · machine-learning · automation

AI System Runs Its Own Science, Then Critiques Its Work

DiscoPER, a new LLM-powered research framework, autonomously generates and tests hypotheses on open datasets — then audits its own findings to hunt for gaps.

An AI framework called DiscoPER can conduct open-ended scientific research without being told what to look for — and then turn around and critique its own conclusions.

Built by researchers and described in a new arXiv preprint, DiscoPER uses a large language model to generate hypotheses, write and run code against datasets, and subject every claimed discovery to statistical testing before accepting it. The twist is a "second-order" reasoning layer: periodically, the system reviews its own accumulated findings as if they were raw data, looking for patterns it missed, confounding variables, and blind spots. It then redirects its own search toward those gaps. The system can also pull in images and other non-tabular sources, not just structured metadata.

Most autonomous research systems either work within a narrow, predefined search space or need a human to hand them a research question first — which rather defeats the purpose. DiscoPER's self-audit loop is the meaningful advance here: a system that not only searches but explicitly models what it has not yet searched is closer to how a working scientist thinks than anything that just runs hypotheses in a flat queue.

Tested on iNatDisco, a new ecological benchmark with ground-truth patterns drawn from peer-reviewed literature, DiscoPER recovered 8 of 9 known patterns with a 72.7% hypothesis support rate, beating both classical causal-discovery methods and standard LLM-guided approaches. The benchmark is new, and the researchers built it themselves, so independent replication on outside datasets would go a long way toward confirming those numbers.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →