Different transformer models, trained from scratch, keep arriving at the same internal algorithm — and now researchers have a tool to extract it.
A paper from arXiv introduces Algorithmic Core Extraction (ACE), a method for isolating the minimal computational subspace responsible for a given task inside a trained transformer. The researchers tested ACE across synthetic tasks and six pretrained language models — GPT-2 Small, Medium, and Large, LLaMA-3.1, Gemma-2, and Qwen2.5 — spanning more than two orders of magnitude in parameter count. The finding: for subject-verb agreement, all six models converge on a single steerable axis that aligns across architectures. Flip that axis, and the model inverts grammatical number throughout its output.
The implications cut against how most interpretability work is done today. Mechanistic interpretability typically dissects one model's weights, but ACE's results suggest those weights are incidental — one of many equivalent configurations that implement the same underlying computation. If the real object of study is the algorithm, not the parameters, then comparing circuits across runs is not just useful but necessary. The paper also finds an inverse scaling law for grokking: functional redundancy, where the same computation is distributed across many equivalent modes, actually speeds the transition from memorization to generalization.
None of this makes transformers suddenly legible — finding a shared axis is not the same as explaining why that axis works. But it does suggest interpretability researchers have been spending a lot of energy describing the wallpaper when they should be mapping the load-bearing walls.