A new audit of AI systems designed to forget on demand finds the model is innocent — the database is the problem.
Limited Memory Language Models, or LMLMs, store factual knowledge in an external database rather than inside model weights. The design promise: delete a fact from the database, and the model forgets it — no retraining needed. Researchers built a causal auditing framework to test that promise, running 12,228 alias-closure deletions across thirteen databases and six prompt formulations. They held the model fixed and varied the database state, hunting for wherever deleted facts could still slip through.
The model's own weights were not the culprit. Parametric leakage — the model returning a deleted answer from memory alone — was near zero across every test variant and every prompt style. The residual that survived deletion lived in the retrieval graph: nearby database entries reconstructed the deleted fact through what the researchers call retrieval artifacts. That residual ranged from 0.7% on a standard database to 13.6% on the most adversarial topology, and rewording the question did not meaningfully reduce it.
For anyone building AI systems subject to right-to-erasure requirements, that spread matters. A 13.6% leakage rate through retrieval is not a compliant deletion. The researchers conclude that the unlearning boundary in this class of systems is set by the database administrator, not the model developer — which is either reassuring or alarming, depending on who your database administrator is.
Regulators drafting AI unlearning standards should probably read this paper before finalizing what "deleted" is required to mean.