AI still can't reliably do math — but it's getting harder to say exactly why.
A survey paper published on arXiv maps the full arc of AI mathematical reasoning, from the rule-based word-problem solvers of the early 2010s to today's large language models, neuro-symbolic theorem provers, and multi-agent proof systems. The authors organize the field into four buckets: informal reasoning over text and diagrams, formal proof in verified environments, open-ended mathematical discovery, and the training and inference techniques — chain-of-thought prompting, process reward models, reinforcement learning with verifiable rewards — that increasingly tie generation to verification. They also catalog the benchmarks used to measure progress across grade-school arithmetic, competition math, geometry, and expert-level problems.
The survey matters because it arrives at a moment when benchmark scores are outpacing real understanding. The authors document benchmark saturation and contamination, flag reporting mismatches between pass@1 and verifier-assisted pass@k results, and call out failure modes that don't show up in leaderboards: models that break under minor problem rephrasing, reward hacking that games evaluation metrics, and multimodal grounding failures when diagrams are involved. These aren't edge cases — they're structural.
The honest read here is that AI math reasoning is impressive in narrow, well-tested corridors and brittle almost everywhere else. The survey's call for verified-discovery workflows and better formalization infrastructure is sensible, but those are long-horizon research bets, not near-term product features. Until evaluation catches up to capability claims, the leaderboard numbers are best treated as marketing.