A new theoretical paper argues that neural network forgetting is a built-in selection mechanism, not a bug.
Researchers propose what they call Repeated Reinforcement with Persistent Forgetting, or RPF, a mathematical framework for how neural networks decide what to keep and what to drop during training. The model works through three layers of increasing complexity: a simple case showing which patterns survive repeated exposure, a shared-parameter case showing that forgetting filters out weak signals while preserving strongly-supported ones, and an approximation showing the whole process behaves like automatic data compression. The team backs the theory with experiments on scalar memories and a small nonlinear network, testing how changing the timing of reinforcement and forgetting shifts what gets retained.
Most explanations for why neural networks generalize well point to architecture choices or sheer scale. This paper suggests forgetting itself is a dial researchers can turn independently, a mechanistic account of why models seem to compress what they learn down to the parts that recur most often. That reframes "catastrophic forgetting," usually discussed as a problem to solve, as a knob that might be tuned on purpose.
It's an elegant piece of theory, but the experiments are limited to toy setups (scalar memories and one small network), so whether this framework holds up in anything resembling a modern large language model remains an open and much harder question.