OpenAI's o3-mini beats its predecessor on a hard math benchmark without rambling longer to get there.
Researchers systematically compared reasoning chain lengths across o1-mini and o3-mini variants on the Omni-MATH benchmark. The headline finding: o3-mini (m) achieves superior accuracy without requiring longer reasoning chains than o1-mini. The paper also surfaces a counterintuitive pattern — across all tested models and compute settings, accuracy generally declines as reasoning chains grow longer, even after controlling for question difficulty. Newer models show a smaller drop, suggesting they're spending test-time compute more wisely rather than just more abundantly.
This matters because the dominant assumption in scaling circles has been that more thinking tokens equal better answers. If that relationship is weaker than believed, it reshapes how labs should evaluate and price reasoning models — and raises questions about whether throwing more compute at a problem is ever the right call past a certain point.
The paper also notes that o3-mini (h) — the high-compute variant — ekes out only a marginal accuracy gain over o3-mini (m) while burning substantially more reasoning tokens on every problem, including ones the medium variant already solves correctly. That's not efficiency; that's overhead dressed up as rigor.