A new benchmark designed to stress-test AI reasoning finds that the type of task matters far more than how many parameters a model has.
Researchers introduced the Complexity Ceiling Benchmark (CCB), which tests how language models perform as the number of required sequential steps grows from 5 to 50. The benchmark holds the content of each task fixed and only increases its depth, across three distinct problem types: grounded spatial state-tracking, abstract symbolic pointer manipulation, and transitive relational inference. Across 6,000 trials on five frontier and open-weight models, the results split sharply by domain. The two spatial and symbolic task types held up well — top models retained above 0.92 accuracy even at 50 steps. The transitive inference tasks were a different story: every model collapsed by step five, with the best model's 50-percent success horizon at roughly 4.7 steps.
The finding cuts against a common assumption in AI scaling discourse — that more capable or larger models will eventually reason their way through harder multi-step problems. Here, parameter count predicted almost nothing. What mattered was where in the reasoning chain errors first appeared, a variable the researchers call k*, which outperformed model size as a predictor of accuracy within a domain. The benchmark also found that 14.5 percent of correct final answers were reached through flawed intermediate steps — meaning models sometimes stumble into right answers for wrong reasons.
Forcibly requiring verbose step-by-step output made no difference to the ceiling, which is a quiet rebuke to the chain-of-thought school of thought. Scaling compute or prompting strategies may not close a gap that appears structural to the task type itself.