Researchers say the way most AI systems search long documents is fundamentally broken — and they have a replacement.
A team publishing on arXiv introduced HMARS, a hierarchical multi-agent memory system designed for long-context reasoning. Instead of splitting a document into chunks and grabbing the top-K most similar ones, HMARS assigns specialized sub-agents to bounded memory regions, mid-agents to coordinate context across those regions, and a frontier model to do final reasoning over whatever evidence gets surfaced. The paper also introduces two new benchmarks built specifically to test evidence breadth and context-dependent relevance — areas where standard retrieval-augmented generation tends to quietly fail.
The distinction matters because most retrieval systems discard potentially relevant content before reasoning even starts. If a document's relevance only becomes clear in light of other passages, top-K chunk retrieval will miss it. HMARS, the researchers argue, retrieves more of the right evidence rather than just reordering what it already found — a meaningful difference when the task involves multi-turn conversations or dense research documents.
The benchmark results show HMARS outperforming retrieval, reranking, full-context, graph-based, and agentic baselines — a wide field. That said, academic benchmarks designed by the same team proposing the system deserve some scrutiny before anyone treats this as a settled race.