Researchers have a new way to teach language models to reason without writing out every step in plain English first.
The method, called Abstract Token Curriculum (ATC), trains models to develop their own internal, continuous representations of intermediate reasoning steps instead of relying on explicit chain-of-thought text. Standard chain-of-thought prompting works, but it needs supervision on the reasoning tokens themselves, which means someone has to build task-specific scratchpad data. ATC skips that by gradually ramping up problem difficulty across a sequence of training distributions, pushing the model to form abstract internal "thoughts" on its own. The researchers back this with both theory and experiments: for parity functions learned with single-layer softmax attention, they show the model's attention naturally gravitates to whichever context tokens offer the easiest path to the right answer, and they test the approach on graph reachability and arithmetic tasks.
This matters because chain-of-thought's reliance on labeled reasoning data is a real bottleneck. Every new domain needs its own curated scratchpad examples, which is slow and expensive to produce at scale. If curriculum-based training can get models to develop useful internal reasoning representations without that hand-holding, it points toward cheaper training pipelines for reasoning-heavy tasks.
It is worth noting the paper's own scope: this is early-stage work tested on narrow, well-defined problems like graph reachability and arithmetic, not open-ended reasoning of the kind chain-of-thought is usually deployed for. Whether internal continuous thoughts hold up on messier, real-world tasks is still an open question.