A new arXiv paper argues that the semantic index behind search and question-answering systems does not need to be rebuilt every time documents change - it can be patched instead.
Most retrieval-augmented systems recompute meaning at query time or rebuild their underlying matrix decomposition whenever the corpus changes, on the assumption that incremental updates would be too costly to track accurately. The researchers tested that assumption on a synthetic corpus grown from 3,000 to 9,000 documents across 50 update events. Incremental low-rank updates came out 33.7 times cheaper per update than full recomputation, and 23.8 times cheaper in total, while still matching the fully rebuilt version almost exactly (principal-angle drift below 1e-11 degrees, recall@10 of 1.0). A separate technique let them adapt to a new embedding model by re-embedding only about 10 percent of the corpus, recovering 0.95 mean cosine similarity to a full re-embedding.
This matters because maintenance cost, not accuracy, is usually the reason production systems avoid persistent semantic indexes and fall back on re-deriving meaning at query time. If incremental updates really do scale with the amount of change rather than corpus size, that changes the cost calculus for anyone running retrieval-augmented generation at scale, especially as corpora grow into the millions of documents where full re-decomposition becomes genuinely painful.
The results are on a controlled synthetic pilot, not a production-scale corpus with messy real-world documents, so the multiples are best read as a proof of concept rather than a guarantee. Still, the core claim - that you can compile meaning once and patch it, rather than re-deriving it every time - is a reasonable rebuttal to how a lot of RAG infrastructure is built today.