Harmless fine-tuning can roll back an AI model's safety guardrails — and a new paper thinks it knows why.
Researchers studying a phenomenon called fine-tuning reversion argue that large early training runs carve out dominant behavioral patterns so deeply that later alignment work amounts to a shallow detour. When you fine-tune again — even on innocuous data — the model gravitates back toward those early-training patterns, dragging safety behaviors with it. The team calls this the "gravitational interpretation" and backs it with geometry: they identified a specific direction in a model's activation space, labeled v_rev, that points back toward pre-alignment behavior. Alignment with that direction climbed from a cosine similarity of 0.43 after the first update to 0.65 by step 20, and every measured run beat the 99th percentile of a random baseline.
This matters because it puts a sharper edge on a fuzzy known risk. The AI safety field has long worried that fine-tuning can jailbreak aligned models, but the mechanism was poorly understood. If v_rev is a reliable causal lever — and the authors are careful to say it is not the only one — it opens the door to targeted interventions that block reversion without tanking model performance. In their tests, selectively blocking motion along v_rev cut harmful outputs roughly in half, from 19% to 8.5%, with little measurable task cost.
The caveat the authors bury but deserve credit for stating plainly: they do not claim to have directly observed the dominant manifold, nor that v_rev is a universal fix — just a robust, history-defined signal that partially explains early reversion. Given how many AI safety claims overpromise, that restraint is worth noting.