A team at Allen AI built a benchmark to ask a blunt question: can AI systems predict what science will discover next?
The researchers released PreScience, a dataset built from 98,000 recent AI research papers plus 502,000 companion papers covering author histories and citation links. They defined seven forecasting tasks — including predicting collaborators, estimating citation counts, selecting prior work, and guessing which research threads will combine into future papers. They tested everything from basic heuristics to frontier language models and agentic systems, and they built a new evaluation metric, LACER, because existing tools for comparing generated text agreed poorly with human judgment.
The closing finding is the one that sticks: when task models were composed to generate a synthetic 12-month corpus of research, the output was measurably less diverse and less novel than actual human-authored papers from the same period. That gap matters because it draws a clear line between summarizing existing knowledge and genuinely anticipating what comes next — a distinction that AI boosters tend to blur.
The benchmark arrives as several labs are pitching AI as an autonomous scientific collaborator. PreScience gives skeptics a concrete measuring stick: not vibes, not cherry-picked demos, but a structured test across multiple forecasting dimensions. The dataset and code are public on Hugging Face and GitHub, which at least means the claims are checkable.