A paper posted to arXiv proposes treating LLM agent behavior like compilable source code, so the predictable parts never have to hit a frontier model twice.
Researchers at RightNow AI built a system called Auto that watches a frontier model agent work, identifies which steps always produce the same output, and packages those as cognition binaries: WebAssembly artifacts whose manifests carry measured guarantees and whose declared capabilities are enforced by a sandbox. When a compiled binary encounters an unfamiliar input, it falls back to the full model, captures the new trace, and recompiles. On AUTO-BENCH, a benchmark the team introduces and pre-registers in the arXiv paper, 87.1% of 560 recorded frontier-agent spans are witnessed-deterministic (three of the four task families hit 100.0%); on a 300-item stream with three scheduled distribution shifts, Auto cut marginal cost from 59 to 2 micro-dollars per item, a 6.4x reduction end-to-end, at 96.9% parity on witnessed inputs. Code is on GitHub.
The paper's most useful contribution may be the failure taxonomy. A loose guard silently mislabeled 48.9% of compiled answers: costs drop, wrong answers go out, and nothing alerts you. The authors conclude that calibration and reference fidelity, not model capability, decide whether cheaper also means correct.
For repetitive, high-volume workloads the cost reduction is real on paper; the 48.9% silent failure rate is the figure that will determine whether it stays real in production.