The research underpinning frontier AI governance decisions may not be as solid as regulators think.
A paper published on arXiv draws on interviews with 16 expert practitioners across biosecurity, cybersecurity, education, and labor to examine a core tool in AI oversight: human uplift studies. These are randomized controlled trials — or similar designs — that measure whether AI access meaningfully improves human performance on sensitive tasks, like synthesizing pathogens or finding software vulnerabilities. The findings are increasingly used to decide whether a model is safe enough to release. The problem, the authors argue, is that standard causal inference methods strain badly when applied to large language models. AI systems evolve too fast, user skill levels shift, and real-world testing environments leak — all of which undermine the internal, external, and construct validity that make RCT evidence trustworthy elsewhere.
This matters because the governance stakes are high. Labs and regulators cite uplift study results when drawing lines around model capabilities. If the methodology is shaky, so is the confidence behind deployment decisions. The paper is one of the more direct challenges yet to the assumption that AI risk evaluation can simply borrow its rigor from clinical trials or social science.
The authors offer a mapping of challenges to proposed solutions, but the honest takeaway is narrower: we are making consequential decisions about dangerous capabilities using evidence that experts in the field already know is hard to interpret correctly. That is not a reason to abandon uplift studies — it is a reason to stop treating their outputs as cleaner than they are.