Researchers say you can stop calling a giant language model API every time you need to rank search results or fix broken JSON.
A paper out of arXiv introduces Program-as-Weights (PAW), a system that compiles natural-language task descriptions into lightweight neural adapters using a 4-billion-parameter compiler model. Those adapters then run on a frozen 0.6-billion-parameter interpreter — a Qwen3 variant — and match the output quality of directly prompting the full Qwen3-32B model. Memory footprint drops to roughly one-fiftieth, and on a MacBook M3 the system runs at 30 tokens per second without a network call. The team also released FuzzyBench, a 10-million-example dataset used to train the compiler.
The practical upshot is that tasks which resist tidy rule-writing — alerting on meaningful log lines, repairing malformed data, intent-based ranking — no longer have to be shipped to a remote API on every call. The compiler runs once per function definition and emits a small, reusable artifact; subsequent calls are cheap and fully offline. That flips the economic and privacy calculus for any team paying per-token for work that is repetitive and well-defined.
The approach is a direct challenge to the "just call GPT" default that has become standard practice — and a reminder that the interesting AI research right now is less about making models bigger and more about making the same capability cheaper to run.