A small AI model can close most of the gap between itself and a much larger one by asking yes-or-no questions — just 10 of them.
A new research paper examines how efficiently knowledge from a large language model can be transferred to a smaller one, testing both lossless and lossy compression techniques. The most striking result comes from a protocol called Question-Asking compression (QA), modeled on the game Twenty Questions. A small model iteratively asks binary questions to a stronger model; each answer transmits exactly one bit of information. Across eight benchmarks covering math, science, and code, 10 such exchanges recovered between 23% and 72% of the capability gap on standard benchmarks — and 7% to 38% on harder ones.
The compression ratios achieved — 0.0006 to 0.004 — are more than 100 times smaller than the best prior LLM-based compression work. That gap has real deployment implications: an on-device model could query a cloud-hosted system with minimal bandwidth, or developers could distill expensive reasoning into cheaper inference without shipping full responses. More fundamentally, the paper reframes knowledge transfer as an information-theoretic problem with measurable efficiency limits, which is a sharper framing than the prevailing "train a smaller model to mimic a bigger one" approach.
The harder benchmarks tell the more honest story: a 38% gap recovery at maximum difficulty still means most of the performance difference survives intact, and it suggests that binary exchanges have a ceiling — one that drops as tasks get harder.