A new memory architecture for AI agents promises to handle long, multi-turn conversations without the usual performance tax.
Researchers behind Akashic propose replacing the common approach of replaying full conversation histories with a system called MemAttention. Instead of feeding every prior exchange back into a model on each request, Akashic breaks context into bounded chunks and tracks semantic relationships across them. A hardware-software co-design layer then co-locates chunks likely to be retrieved together, cutting fragmentation and I/O overhead. Tested across four workloads and three model sizes, the system improved task accuracy by up to 10.2 percentage points, throughput by up to 1.21x, and sustainable request rate by up to 1.88x compared to prior memory baselines.
The problem it targets is real and growing. As AI agents stretch across longer sessions, tool calls, and cross-session workflows, the cost of re-ingesting full context histories compounds fast - and burying relevant evidence under irrelevant history quietly degrades output quality in ways users may not immediately notice. Akashic's chunked approach is a structural answer to what has largely been handled by brute-force context scaling.
The results are from a preprint, so independent replication is pending - but the throughput gains, if they hold, would matter more to infrastructure teams than to end users buying chatbot subscriptions.