A new benchmark finally puts a number on how much extra thinking reasoning models waste after they already have the right answer.
The paper introduces Step-wise Truncation Probing, a method that repeatedly samples answers from truncated points along a model's reasoning trace to find the earliest point where accuracy peaks. Everything the model generates after that point counts as waste: the researchers track this as an Overthinking Ratio, and the resulting drop in final accuracy as Accuracy Degradation. They then build a unified framework comparing three ways to cut reasoning short, dynamic routing, fixed token budgets, and dynamic early exit, scoring each against a token-accuracy Pareto frontier. The resulting benchmark, OTBench, ran across six models and four benchmark groups, generating 70.5 billion reasoning tokens in the process.
That token count is the real story. The paper frames reasoning compute directly against accuracy, and every token spent after peak accuracy is overhead with no payoff. A standardized way to quantify that waste matters more than another accuracy leaderboard, because it is the first step toward knowing when a model should stop reasoning with itself.
The paper measures the problem with real rigor. It does not solve it: the termination strategies it compares are theoretical ceilings, not shipped features, so the token bill stays exactly where it was until someone builds this into an actual product.