A new paper claims to fix one of reinforcement learning's most persistent headaches: the training instability that importance sampling introduces into LLM post-training.
The method, called Group Variance Policy Optimization (GVPO), builds on Group Relative Policy Optimization (GRPO), a post-training technique that compares a model's outputs against a group average. GRPO's reliance on importance sampling, a statistical reweighting trick, can make training runs unstable or hard to reproduce. GVPO replaces that trick with a formula derived directly from the math of reward maximization, which the authors say guarantees a single, well-defined optimal solution instead of a noisy approximation. They also show the same formula extends to on-policy distillation, where a smaller model learns by mimicking a larger one's live outputs.
That combination matters because post-training is where labs now spend significant effort refining reasoning ability, and unstable training runs waste compute and researcher time. A method that pairs a stability guarantee with a unified approach to distillation could simplify pipelines that currently rely on separate techniques for each job.
It's one arXiv preprint with no independent benchmarks yet, so treat the stability guarantee as promising math, not proven practice, until someone outside the author list reproduces it.