A research team has a new way to find the parameters inside large language models that matter most — including the ones that can quietly break everything.
The paper introduces Weight-Adjusted Gradients (WAG), a method that multiplies a model's weights by its first-order gradient information to estimate which parameters carry the most influence over model behavior. Prior importance metrics looked at weights or gradients in isolation; WAG looks at their interaction. The researchers found that a tiny fraction of parameters, when modified, can trigger dramatic performance collapse — a failure mode that existing tools missed entirely. They tested WAG across several models and applied it to mixture-of-expert routing, targeted unlearning, mixed-precision quantization, and layer selection for knowledge editing.
The practical stakes here are real. As labs compress, fine-tune, and edit models at scale, knowing which parameters are load-bearing — and which are tripwires — changes how safely those operations can be done. A unified diagnostic that flags both matters more than two separate signals that each miss half the picture.
Interpretability research tends to promise more than it delivers, but WAG's applications look concrete rather than theoretical. Whether it survives contact with the largest frontier models remains the open question.