Researchers have built a retrieval-augmented generation system that gives AI agents a map of a document's structure instead of just scattered text chunks.
The system, called RIT-RAG, builds a tree for each document in a corpus from its table of contents or sitemap before any query arrives. When a question comes in, it retrieves a broad set of relevant chunks, then uses their positions in those trees to carve out smaller, manageable sub-trees that can span multiple documents. An LLM agent then walks those sub-trees, reading the nodes that look promising and rewriting its query if the first attempt comes up short. The approach was tested on financial, scientific, and customer-support benchmarks, plus a new 2.84-million-page technical-documentation benchmark the researchers built called EntQABench, where it beat the strongest existing baseline by 6.8 to 11.4 accuracy points across three different language models.
Most agentic RAG systems hand a model disconnected chunks of text and hope it can tell real evidence from text that merely sounds relevant. Structure-aware tools like PageIndex fix that for a single document but fall over once a corpus is too large to fit in context, because they have to commit to one document before reading anything. RIT-RAG's trick is letting retrieval point to a neighborhood and letting the agent decide what to actually read inside it, a more modest and more scalable division of labor.
It is still a benchmark win, not a production deployment, but the gap it closes - between search that finds text and search that understands documents - is the one that keeps RAG systems confidently wrong.