A new study shows AI models can build other, smaller AI models on the fly, no training required.
The researchers tested the idea on ARC-1D, a simplified one-dimensional version of the Abstraction and Reasoning Corpus, a benchmark built to probe pattern-based reasoning. They built tiny specialist models, each handling one specific transformation, then used a hypernetwork, a network that generates the weights for another network, to produce those specialists' parameters directly from a handful of example transformations. The resulting weights formed an organized, structured space rather than a random scatter. The specialists showed partial ability to combine transformations and, in some cases, could handle transformations the hypernetwork had never been trained on.
Normally, specializing a model for a narrow task means retraining or fine-tuning it, which costs time and compute. This result suggests specialization could instead be compiled instantly from a few examples, with the hypernetwork acting like a compiler that writes small bespoke programs rather than running one giant general model for everything. The team also found that removing explicit labels for which task was which improved generalization to new transformations, hinting the system was learning the underlying pattern rather than memorizing categories.
ARC-1D is a toy benchmark built for exactly this kind of test, so whether the trick scales past tidy one-dimensional puzzles remains an open question.