A new method compresses the step-by-step reasoning that makes modern AI expensive to run.
The technique, called SuperThoughts, folds pairs of consecutive chain-of-thought tokens into a single latent representation and uses a lightweight module to decode two tokens per step, doubling throughput while cutting chain-of-thought length by 20-30%. A confidence-based fallback reverts to standard decoding when the model is uncertain, keeping accuracy degradation to 1-2 points on most tasks. Researchers tested it on Qwen2.5-Math models from 1.5B to 14B parameters against MATH500, AMC, OlympiadBench, and GPQA-Diamond.
Chain-of-thought reasoning has become the default approach for high-capability AI systems, but longer chains mean more tokens and higher inference costs. This approach also sidesteps the training instability that has plagued attempts to move reasoning into continuous latent spaces, where models think without generating discrete tokens at all. A 20-30% reduction in chain length, compounded by doubled per-step throughput, is real money at scale.
A 1-2 point accuracy drop for a 30% shorter reasoning chain looks like a reasonable trade in a controlled math benchmark. Whether it holds for open-ended, real-world prompts is the question the paper does not answer.