Sampling a reasoning model more times past a certain point makes it more confidently wrong, not more correct.
Researchers studying test-time scaling — the technique of running a model on the same problem many times and picking the best answer — found that the gains plateau far earlier than the compute bills suggest. Coverage, the probability that at least one correct answer appears somewhere in all those samples, does keep climbing. But a deployed model has to commit to one answer without knowing which attempt is right. That selection process hits a ceiling within a few dozen draws, which the paper calls the "modal ceiling." Beyond that, extra samples add cost and can nudge the model toward a wrong answer it has grown more confident in — the same trap humans fall into when they overthink.
The finding matters because test-time compute has become one of the main levers AI labs pull to squeeze more performance out of existing models without retraining them. If the practical benefit of that lever maxes out at a small number of samples, the scaling story gets complicated. The paper introduces a single metric — the effective number of samples — that lets practitioners read the cutoff from any existing sampling run, without additional benchmarks.
The underlying constraint is not generation but recognition: the model's ability to identify which answer is correct, not its ability to produce one at all. That is a different kind of problem than more compute can solve, and it suggests that the next gains in reasoning will require better selection strategies, not longer sampling runs — a finding that sits awkwardly beside the current industry enthusiasm for throwing more inference budget at hard questions.