Researchers have found a way to industrialize scientific fraud using nothing more than a tampered dataset and a public repository.
The attack, which the authors call indirect data poisoning, works by uploading a corrupted dataset to an open repository. When autonomous AI research agents retrieve that data and process it, they can produce and spread false scientific conclusions without any direct access to the agent, no injected prompts, and no fabricated papers — just misleading metadata and manipulated numbers. Across 450 experimental runs covering five high-stakes topics — including hiring discrimination and autonomous vehicle safety — the researchers tested three widely used AI systems and found the attack succeeded in 49.56% of cases. Agents caught the manipulation only 6% of the time.
This matters because AI agents are increasingly doing real scientific work: pulling data, running analyses, drafting findings. If bad actors can reliably corrupt that pipeline through open datasets alone, the industrial-scale scientific fraud that once required a coordinated lobby or a corrupt think tank could be replicated by a single motivated actor. The barrier to manufacturing scientific doubt just dropped.
The researchers tested two mitigations: a "scientist persona" prompt that tuned agent skepticism, and a structured data provenance audit with five checks. The persona helped but still left 16.67% of runs producing poisoned conclusions; the provenance audit cut the success rate to zero — though auditing every dataset at scale is its own unsolved engineering problem.