A new dataset wants to prove that small AI models don't need big compute to get smart about science and math.
Researchers have released QVAC Genesis III, a 191.43-billion-token synthetic dataset covering 19 STEM domains at multiple difficulty levels. The corpus is built with a dual-generation approach: a weak, edge-scale 'student' model is fed problems, its wrong answers get converted into corrective explanations, and its right answers get expanded into reasoning that compares every possible option. The team also built an LLM-as-a-parser evaluation system to grade free-form answers for both correctness and validity. In tests with 1.7-billion-parameter models trained from scratch, models trained on QVAC Genesis III beat those trained on the open-source Cosmopedia-v2 corpus and the public Cosmo-1B model on ARC, GPQA Diamond, and MMLU STEM benchmarks, with gains up to 28.57% on ARC-E and 21.35% on ARC-C, plus a 99.45% valid-answer rate.
This matters because training-data quality, not just model size, has become the real bottleneck for edge and on-device AI. Big labs train massive models on private data troves, while open developers building small, STEM-focused models for phones and embedded devices haven't had an equivalent resource. QVAC Genesis III is a bet that smarter, denser training data can substitute for the token budgets small models don't have.
The benchmark wins are real but narrow: they're measured against one open corpus and one small reference model, not the field at large, so it's worth waiting to see how this data holds up outside a 1.7B-parameter sandbox.