A new post-hoc editing method can strip a fine-tuned language model down to its reasoning essentials using under 1% of its parameters.
Researchers introduced Subspace-Aligned Rewiring (SAR), a technique that identifies where useful reasoning updates actually live inside a model's weight space after reinforcement learning fine-tuning. The core finding: those updates cluster in the model's spectral subspace — a compact geometric slice of parameter space — while the surrounding directions largely contribute noise or interfere with other capabilities. SAR removes those outer components after training, without touching the training process itself. Tested across multiple model families and scales, it retained over 99% of post-training performance while compressing the meaningful update to roughly 0.58% of total parameters.
The practical upshot matters for anyone trying to ship capable models cheaply or combine specialized models into one. SAR let researchers merge math and coding experts into a single model that outperformed both the prior merging baselines and the best individual specialists — a result that has eluded simpler averaging approaches. It also recovered suppressed coding ability in mixed-domain training runs without sacrificing math or instruction-following scores.
Model merging is a crowded research lane right now, with approaches like DARE and TIES already in wide use, so SAR will need head-to-head comparisons on standardized benchmarks to prove it has staying power — the paper's reliance on an in-house coding model for one key result makes that harder to verify independently.