A dead-simple data selection trick consistently outperforms the algorithms researchers have been complicating for years.
A new paper from arXiv introduces few-medoids, a coreset selection method for few-shot knowledge distillation. The idea: for each class in a dataset, pick the samples closest to that class's average image. No clustering runs, no greedy graph traversal — just find the center and grab what's nearest. The researchers tested it across four image datasets and three teacher-student model pairs, mixing convolutional and transformer architectures. In every case, few-medoids beat both random selection and established methods like herding and k-center Greedy.
The result matters because coreset selection — choosing a small, representative slice of a massive dataset to train on — is a persistent bottleneck in efficient AI training. The field has produced increasingly elaborate selection strategies, yet most fail to clear even a random baseline. Few-medoids clearing that bar and beating the fancier methods suggests the field may have been optimizing in the wrong direction.
The authors release their code publicly, which makes this a practical drop-in replacement rather than just a research curiosity. That said, the experiments cover image classification only — whether the same centroid logic holds for text, audio, or multimodal data is an open question the paper doesn't answer.