A new benchmark called ArgGYM scores AI models on reasoning that can change as new evidence comes in, not just fixed math or code problems.
Researchers built ArgGYM to test "defeasible reasoning" - the kind where a conclusion holds until it doesn't, then gets revised, and sometimes reinstated once more. It splits that skill into twelve tasks and checks every answer against a symbolic argumentation engine rather than a human grader or a similarity score. The frozen benchmark includes 1,440 verified test cases across fifteen difficulty tiers, built using two ways of ranking competing arguments and two ways of ranking argument sets. Across it, frontier and open-weight models show sharply different profiles: most can recover pieces of a correct structured answer, but completion of the full task drops off as problems add more steps and interlocking dependencies.
Most recent gains in AI reasoning come from domains like math and code, where there is one clean answer to check against. ArgGYM targets a messier, more common kind of reasoning - weighing provisional claims, counter-evidence, and reversals - that looks more like contract disputes or policy debates than a geometry proof. That the gap between "catches fragments" and "solves the whole thing" widens as problems get more complex suggests current training rewards pattern-matching more than tracking an argument from start to finish.
It's a useful corrective to benchmark results that treat reasoning as a single score: a model can look sharp on a leaderboard and still lose the thread the moment the evidence starts shifting.