AI document search has a picking problem, not a finding problem.
A new paper argues that in long-document visual question answering, covering PDFs that run to hundreds of pages and mix text, tables, charts, and figures, the weak link isn't retrieval, it's reranking. On the MMLongBench-Doc benchmark, the retriever BGE-M3 pulls in the right page 86 percent of the time in its top 20 results, but its final top-5 picks score only 0.254 on F1, and even ColPali, a retriever built to read page images, only manages 0.332. The researchers built Trident, a two-part fix: one module rewrites each candidate page into a structured record, with a visual caption, section path, and entity tags, before an LLM reranks it, and a second module prompts the underlying vision-language model to review answers from topical, entity, and structural angles before writing its final synthesis. Reranking with those structured records beat every baseline tested, including a system called PageIndex, while handing an LLM the same raw snippets barely changed the results at all.
That gap matters for anyone building search tools over contracts, filings, or technical manuals, since most of those systems live or die on the reranking step, typically the least scrutinized part of the pipeline. If evidence selection, not search coverage, is the real bottleneck, teams pouring resources into better retrieval models may be optimizing the wrong stage of the pipeline.
The headline gain, up to 6.6 points of accuracy on open-ended questions, comes from the authors' own benchmark and two LLM judges rather than human graders, so treat it as promising, not proven.