A new hypernetwork can now write a custom set of weights for a 284-billion-parameter model, pushing a technique previously tested only on far smaller systems into genuinely large-model territory.
The system, called Internalizer, converts a document directly into a LoRA adapter for DeepSeek v4 Flash, a frozen 284-billion-parameter model, using nothing but a three-word instruction in the context window. Prior hypernetworks that map context to parameters had only been demonstrated on base models up to 14 billion parameters, so this is roughly a 20-fold jump in scale, even though the paper itself describes it as two orders of magnitude, a bigger claim than the actual ratio supports. Most of the hypernetwork's weights live in a shared, model-agnostic trunk, with only thin entry and exit layers swapped in per target model, which let the researchers train cheaply on small models before porting the approach up to DeepSeek v4 Flash. On unseen documents up to 4096 tokens, the generated adapters hit 84.9% top-1 and 97.8% top-5 teacher-forced accuracy, compared with 63.4% and 83.5% for the base model given the same bare instruction.
Once trained, a single forward pass turns any document into an adapter that can be served on its own for speed, or alongside the source document for extra accuracy. That matters for retrieval-heavy and long-context applications, where keeping every document in the window at inference time is expensive; this points toward baking documents into weights instead.
Call it real progress on a genuine bottleneck; just don't let the paper's own arithmetic oversell it, since 20x is already a solid jump without rounding up to 100x.