A new benchmark says today's AI document parsers can read a single page just fine, but they get lost trying to map out the document around it.
Researchers built LongDocBench from 85 real-world financial reports, textbooks, and academic papers totaling 2,582 pages, with some documents running up to 105 pages. It tests two things most document-parsing benchmarks skip: rebuilding a document's full table-of-contents hierarchy across pages, and tracing the links between tables or figures and their captions, notes, and sources. The dataset includes 3,937 human-verified heading nodes with an average nesting depth of 3.55 (max depth 9) and 3,258 contextual relationships tied to 2,680 table and figure objects. When the researchers fed human-verified TOC structures and relationship data into long-document question-answering tests, reasoning accuracy improved, and combining both types of structure helped more than either alone.
Most document-AI benchmarks measure whether a parser can read one page correctly - text order, tables, formulas. LongDocBench targets the layer above that: whether a system understands how a hundred-page report is actually organized, which matters for anyone building tools to summarize or query long filings, textbooks, or papers. The paper's own tests found current parsers handle page-level recognition well but still fall short on both structure-recovery tasks.
That gap suggests the polished demos of AI reading long PDFs are running ahead of the underlying plumbing.