A research paper introduces "EvalSafetyGap" — a framework for understanding why AI safety and capability measurements keep improving on paper while the actual properties being measured remain hard to confirm.
The paper combines a broad literature survey with a structured audit of ten AI models, drawing on eight evidence streams covering work from 2018 to 2026. The central claim is that evaluation-side and alignment-side metrics share a common flaw: both are proxies, and both buckle under optimization pressure — a dynamic the authors frame using Goodhart's Law. They introduce two additional analytical tools, an "Instability Decomposition" and an "Alignment Trilemma," to generate testable comparisons between where measurement goes wrong on the capability side versus the safety side.
The audit finding worth sitting with: in the ten-model sample, the relationship between capability scores and sustained adversarial robustness was statistically indeterminate (Pearson r = +0.232, p = 0.520). In plain terms, more capable models were not reliably more robust to adversarial attacks. The apparent gap between open and closed models on safety also turned out to be mostly a governance and disclosure story, not a behavioral one — and it shifted meaningfully depending on how a single borderline model was classified.
The paper does not produce a league table or rank the models it audits, which is the honest call given how inconsistent the underlying protocols are. What it does offer is a shared vocabulary for people trying to make AI evaluation less gameable — at a moment when every major lab is both building benchmarks and topping them.