A new paper describes a fix for a blind spot in how Mixture-of-Experts AI models forget data on request.
Machine unlearning is the process of surgically removing specific knowledge from a trained model, useful for scrubbing copyrighted text, private data, or dangerous know-how without retraining from scratch. Mixture-of-Experts models complicate this because each layer routes tokens to only a handful of specialized sub-networks, or experts, instead of running everything through the whole model. Researchers found that data marked for forgetting tends to activate a narrow set of experts heavily, while the data a model is supposed to retain barely touches those same experts. That mismatch means standard unlearning methods under-correct the exact experts most responsible for the unwanted knowledge. The proposed fix, called TRACE, identifies those forget-critical experts ahead of time and reweights retain-data training so each expert gets a more balanced signal. On two benchmarks, WMDP and MUSE-BOOKS, TRACE beat the best existing baseline by 9 percent on a utility measure while forgetting the target data just as thoroughly.
This matters because Mixture-of-Experts is no longer a niche architecture. It underpins models from several major labs, and most unlearning research to date has assumed a dense, uniform network. If the routing mismatch here is real and general, plenty of published unlearning results on MoE models may have been quietly overstating how well they actually erase data while preserving performance.
Still, a 9 percent relative gain on two academic benchmarks is a lab result, not a deployment guarantee, and compliance teams betting on unlearning to satisfy a legal takedown request should read the fine print before celebrating.