FP8 training promised cheaper reinforcement learning for large language models. It also kept breaking mid-run.
A new paper traces the failure to a specific mechanism. FP8 is a low-precision number format that speeds up training by using fewer bits per calculation, but that precision loss compounds. Researchers found the accumulated noise distorts a core RL signal called the importance ratio, which disproportionately pushes negative-advantage tokens (the model's bad moves) outside the algorithm's trust region. Those tokens then get their gradients zeroed out instead of penalized, so bad outputs go uncorrected and pile up, eventually showing up as entropy surges and garbled text. The fix, called Calibrated Clipping, dynamically realigns FP8's clipping bounds with high-precision BF16 distributions, matching the lower-bound quantile and adjusting the upper bound to match. Tested across two RL algorithms (GRPO and DAPO), model sizes from 8B to 32B parameters, and several FP8 scaling setups, it eliminated the entropy surges and matched BF16 baseline performance.
Prior fixes, like a correction method called TIS, targeted mismatches between training and inference precision but missed this root cause. That matters because RL is now the main lever for improving reasoning and agentic behavior in LLMs, and FP8 is the obvious way to make that training cheaper - if it can be trusted not to quietly degrade.
It's a narrow fix, but a telling one: a widely used efficiency shortcut had a stability bug hiding in plain sight for however long teams have been running FP8 RL. Worth remembering the next time a speedup ships without a matching stability analysis.