AI/ reinforcement-learning · compositional-reasoning · language-models · ai-research

Training AI on Hard Tasks Teaches Easy Ones Better

A new study finds language models trained on complex composed tasks pick up simpler skills for free, but the reverse does not hold.

Turns out the order you teach an AI model matters more than we thought.

A new paper builds a framework for measuring compositional reasoning in language models, the ability to chain simple skills into harder multi-step tasks. Using data-structure problems, which have clean right-or-wrong answers, the researchers ran reinforcement learning post-training and tested transfer in both directions. They found a consistent asymmetry: training a model on the full composed task also makes it better at the simpler sub-skills, but training it on the sub-skills first does not reliably make it better at the composed task. The pattern held up under length extrapolation, structural changes, and transfer to unseen skills, and a small pilot on tool-calling benchmarks suggested it shows up outside the lab too.

This cuts against a common assumption in how AI labs build training curricula, that breaking a hard task into easier pieces and mastering each one is the safe path to competence. If the composed-to-decomposed direction is the one that actually transfers, teams optimizing reasoning models might get more out of throwing harder, end-to-end tasks at RL post-training rather than scaffolding up from basics.

The caveat is that data-structure puzzles are a tidy testbed, not a messy production agent, and the tool-calling result is explicitly a pilot. Worth watching for a larger-scale replication before anyone rewrites their training pipeline.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →