A new peer-review pipeline doesn't just read your paper's claims - it runs your code to check if they're true.
FactReview is an audit tool that pulls specific, checkable claims out of a manuscript, then checks each one against related work and citations. When a paper ships code, FactReview goes further: it executes the released artifacts, with a fixed budget for fixing broken runs, to see whether the results actually reproduce. Tested on 26 papers with 354 human-verified claims, the system recovered claims with 84.3% F1 accuracy. In head-to-head scoring against a plain LLM reviewer using the same backend model, FactReview rated 4.72 out of 5 overall, a 0.74-point gain.
Pull out the execution step and 17.0% of claim verdicts flip - more than any other evidence source the system checks, which suggests a lot of peer-review disputes hinge on whether code actually runs, not just whether prose reads convincingly. In a study with human reviewers, using FactReview cut mean review time by 58% while pushing coverage of benchmark claims from 87% to 99%.
It still leaves accept-or-reject calls to humans, which is the honest part: catching a claim that doesn't run isn't the same as judging whether the work matters.