A new tool speeds up one of AI research's more tedious chores: proving that a model actually forgot what it was told to forget.
Researchers released SUPREME, an open-source framework that runs machine unlearning experiments across multiple GPUs instead of one. Machine unlearning tries to erase a specific slice of training data from a model without retraining it from scratch, but checking whether it worked requires running the full train-unlearn-evaluate cycle many times with different random seeds. Existing image classification unlearning tools, as far as the researchers know, only support a single GPU, which caps how many seeds a lab can practically test in reasonable time. SUPREME adds a registry system for plugging in new unlearning methods, metrics, and models, and the team demonstrated it on the Pins Face Recognition dataset with ResNet18 and ViT models across ten seeds of full-class and random-sample unlearning.
Unlearning matters because of rules like the 'right to be forgotten', which ask companies to remove someone's data from a trained model, not just a database. Unlearning research is also plagued by results that don't replicate, partly because running enough seeds to trust a result is expensive on a single GPU. A tool that parallelizes the process across accelerators makes it cheaper to tell a genuinely reliable unlearning method from one that just got lucky on one seed.
It's infrastructure, not a breakthrough, and the only public demo so far is on a face-recognition dataset with two architectures - so treat ten seeds on one benchmark as a starting point, not proof the tool scales to messier, real-world models.