AI models that get fine-tuned on new tasks tend to forget what they already knew, and a new paper proposes a fix that works after the fact rather than during training.
The paper introduces JANUS (JAcobian NUll Space), a post-hoc, tuning-agnostic weight-rectification framework. Existing parameter-efficient fine-tuning methods try to prevent this forgetting with a condition the authors call Subspace Orthogonality, which they argue is overly restrictive. JANUS instead projects parameter updates into a Jacobian null space to hit a different target, Parameter Space Orthogonality, which the authors say is the necessary and sufficient condition for preserving old performance to a first-order approximation. To keep that math valid beyond small updates, the method adds a Multi-step Adaptive Rectification step that checks a trust region and adjusts step size as it goes, plus efficiency tricks like ghost projection and sequence-level singular-value-decomposition compression.
That's a technical way of saying JANUS aims to solve the stability-plasticity dilemma properly: keep old-task accuracy intact without dragging down accuracy on the new task, rather than trading one for the other. Because it's post-hoc and tuning-agnostic, it can in principle sit on top of whatever fine-tuning method a team is already using, instead of requiring a new training recipe. The authors report it recovers historical knowledge across multiple fine-tuning methods while preserving downstream task performance.
Still, these are the authors' own benchmarks in a preprint that hasn't been peer reviewed, so treat "significantly mitigates" as a claim to watch, not a verdict.