Teaching an AI model to learn tasks in the wrong order might actually be the right call for safety.
Researchers published a study showing that "imbalanced" pretraining (where one task dominates early training before another is introduced) leads to cleaner internal representations inside transformer models. When tasks are balanced and learned simultaneously, both get routed through the same neural circuits, tangling them together. When one task comes first, the model develops separate circuits for each, making it easier to later target and suppress a specific behavior through fine-tuning. The effect held across synthetic copy tasks and a language task involving rule-following and rule-breaking data.
The safety implications are direct. Current approaches to refusal fine-tuning have a well-known side effect: they often suppress unintended behaviors too, or fail to suppress the targeted ones reliably. If pretraining order can be tuned to keep circuits cleanly separated, safety teams could adjust specific behaviors without collateral damage to other capabilities. That is a meaningfully different lever than anything available at the fine-tuning stage alone.
The research uses model scales far smaller than production systems, so the jump from synthetic copy tasks to 100-billion-parameter deployments is not trivial. Still, as labs race to show that safety measures can be applied with surgical precision rather than blunt force, curriculum design just got a seat at the table.