Reinforcement learning post-training doesn't just unlock latent abilities in AI models — it actively builds new ones by combining simpler skills.
Researchers tested a Transformer pretrained on basic symbol-rewrite operations, then post-trained it using reinforcement learning on a more complex reasoning task with only a binary right-or-wrong reward signal. RL solved problems the base model rarely cracked even with far more attempts. Trace analysis revealed a two-phase mechanism: the model first sharpens its primitive operations, then assembles them into new composite procedures — collapsing ordered chains into single steps or combining independent operations in parallel. Crucially, these aren't one-off tricks; the model reuses and consolidates them into a stable repertoire.
The finding cuts against the common framing that RL post-training is just "eliciting" capabilities already baked into pretraining. The study also pinpoints what makes RL outperform rejection fine-tuning: not more exploration, but more selective exploration. Rejection fine-tuning produces a flood of shortcut-like rewrites, many of them invalid; RL concentrates its search on valid, reusable structure.
For labs betting on post-training to squeeze more reasoning out of existing models, the implication is double-edged — RL can genuinely compose new strategies, but only if pretraining already organizes primitive skills into reduction procedures that RL has something to compress. Weak foundations don't get papered over; they just produce a weaker ceiling.