Researchers have a new fix for a stubborn AI problem: teaching a language model something new often makes it worse at what it already knew.
The method, called SVC, targets a specific piece of a model's internals called singular-vector channels, which are mathematical components that each represent one input-output transformation inside the model. Before fine-tuning on new data, SVC scores every channel on two things: how much it would help the model learn the new domain, and how much updating it would erase prior knowledge, the latter estimated using a general-purpose dataset as a proxy. It then uses a few statistical filtering steps, including a method called Otsu thresholding, to decide which channels get updated and which stay frozen. Tested across four LLM families and eight downstream tasks, SVC held onto pretrained capabilities better than existing parameter-efficient fine-tuning baselines while still performing well on the new tasks.
This matters because most current fine-tuning shortcuts, like LoRA, pick which parameters to adjust somewhat arbitrarily, without a principled way to estimate the forgetting cost in advance. SVC's contribution is a scoring system that treats stability and plasticity as something you can actually measure per-component, rather than a blunt tradeoff you accept after the fact.
It will not end catastrophic forgetting on its own, but it is a more surgical approach to a problem that has quietly limited how much companies can keep customizing the same base model before it starts losing its edge.