A new inference-time technique makes AI alignment cheaper by improving the quality of responses before selection even begins.
The standard Best-of-N approach generates multiple responses from a language model and picks the best one using a reward model. The catch: if the model almost never produces high-quality answers on its own, picking the best of a bad lot still leaves you with a bad answer. Researchers behind Best-of-Better-N (BoBN) tackle that upstream problem. Their method retrieves high-reward examples relevant to the query, rewrites them in the reference model's own style, and feeds those rewritten examples back in as context — nudging the model's sampling distribution toward the good region before any selection happens.
This matters because training-based alignment is expensive, and inference-time alternatives have real limits. BoBN's restyling step is the novel piece: raw retrieved examples often come in the wrong format, so rewriting them makes the in-context signal actually land. The researchers provide analytical grounding for why this shift works and show gains on both safety alignment and mathematical reasoning benchmarks.
Inference-time alignment has attracted serious attention as a way to patch model behavior without retraining — but most prior work still assumes the base model can occasionally produce the right answer. BoBN is more honest about that assumption, and its fix is notably low-overhead. Whether it holds up outside controlled benchmarks, of course, remains the usual open question.