AI/ ai · benchmarks · long-context · llm-evaluation

New arXiv Study Finds AI Models Falter on Complex Corpus Tasks

A new arXiv paper (2609.29245, posted Sept 25) shows long-context AI models degrade sharply on quadratic-complexity corpus tasks that most benchmarks ignore.

Long-context AI benchmarks have been measuring the wrong kind of hard.

A paper posted to arXiv on September 25, 2026 (arXiv:2609.29245) introduces a formal way to size up how brutal a task actually gets as a document collection grows. The researchers call it Corpus Task Complexity, or CTC, and split tasks into two buckets: low-CTC jobs like simple retrieval, where difficulty scales linearly with corpus size, and high-CTC jobs like flagging every contradictory claim across a literature, where difficulty scales quadratically or worse. Most existing evaluations, the paper argues, have only ever tested the easy bucket. To fix that, the authors built CTC-Bench, a 22-task suite that adds 10 new high-CTC tasks, and are releasing the code and data alongside it.

The gap shows up exactly where it counts: efficient attention tricks like block-sparse and hybrid attention, often pitched as free scaling for long-context models, matched full attention on low-CTC tasks but degraded sharply once tasks got quadratically harder. That undercuts the industry pitch that cheaper attention mechanisms are a lossless shortcut for feeding models bigger documents. It also means benchmark leaderboards built mostly on retrieval-style tasks have likely been overstating how much long-context reasoning current models can actually do.

Full attention still wins on the hard stuff, but it does not scale, which makes "more context, more capability" sound less like a spec sheet and more like a marketing line.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →