A new training method stores past knowledge in a compressed memory bank so a neural network can pick up old skills without relearning them from scratch.
The technique, called Neural Subspace Reallocation (NSR), treats the lightweight adapter modules commonly used to fine-tune large models - known as LoRA adapters - as retrievable memory units rather than throwaway task-specific patches. When a model starts a new task, NSR compresses previous adapters using a standard matrix decomposition method, parks them in a structure called a TaskKnowledgeBank, and retrieves the most relevant ones to warm-start training. A distillation step prevents the model from forgetting what came before. The team reports that on standard continual-learning benchmarks, NSR cuts the time to recover a recurring task by 10x and produces about 9x less backward interference than approaches that ignore history entirely.
The more interesting finding is what the paper isolates as the actual driver of those gains. The researchers ran a controlled experiment holding the memory bank constant while swapping in different policies for deciding which adapters to activate - including a reinforcement-learning controller. A simple similarity-based retrieval rule matched or beat the learned policy. That matters because the field has spent considerable effort designing smart scheduling algorithms; this result suggests the memory structure itself is doing the real work.
The compressed bank costs roughly 0.29 MB per task, keeping the total footprint manageable even as task counts grow - a practical constraint that most continual-learning papers gloss over. Whether this holds outside controlled benchmarks, where task boundaries are rarely clean, is the question that lab results can never quite answer.