You may not need millions of training examples to build a capable reasoning model — you might need 800 carefully chosen ones.
Researchers have proposed a data-efficient distillation framework, dubbed DED, that trains smaller "student" models to reason by learning from larger "teacher" models — without the usual requirement to scale data into the hundreds of thousands of examples. The key findings: benchmark scores alone are a poor guide for picking a teacher model, bigger distillation datasets tend to hurt performance on problems outside the training domain, and exposing the student model to diverse reasoning paths produces more robust skills than simply feeding it harder or longer examples. Tested on standard mathematical benchmarks (AIME 2024 and 2025, MATH-500) and code generation (LiveCodeBench), DED reached state-of-the-art results using just 0.8k curated examples.
The scaling laws that govern large language model performance have become something close to received wisdom in AI research — more compute, more data, better results. DED pushes back on at least part of that story, suggesting that data curation quality can substitute for quantity in the distillation step specifically. That matters because distillation is one of the main paths smaller labs and enterprises use to build capable models without frontier-scale resources.
The caveat worth watching: these results are on narrow reasoning benchmarks, not general capability. A model that aces AIME problems on 800 examples is an interesting proof of concept; whether the same principle holds across broader, messier real-world tasks is a different question entirely.