A new paper proposes a cheaper way to estimate expectations under language models, using information the model already produces for free.
Researchers describe a method for estimating the expectation of a test functional under an autoregressive language model, essentially averaging some property across the many possible outputs a model could generate rather than relying on one sample. They note that computing these expectations reliably usually requires expensive sampling. Their fix is to exploit the next-token conditional probabilities a model already calculates during normal sampling, using what they call potentials: functions that break the test functional into additive pieces tied to prefixes of the output. They derive the conditions under which a given potential actually reduces estimator variance, then build practical potentials for several use cases and report substantial variance reductions at the same computational cost.
Variance reduction sounds like a dry statistics problem, but it decides how many expensive samples you need before a model's output statistics can be trusted, directly affecting the cost of evaluating model behavior, estimating risk, or running Monte Carlo-style analyses on LLM outputs. Methods like this matter more as expectation-based techniques, such as reward estimation or uncertainty quantification, become routine parts of deploying large models rather than one-off research exercises.
It won't make models say anything new, but it's a reminder that squeezing a free byproduct of sampling, the next-token probabilities already sitting there, can be a bigger win than reaching for a flashier method.