A new arXiv preprint says grokking, the strange moment when a neural network flips from memorizing to actually generalizing, is not a switch flipping in some new brain module. It is a recode of circuitry that was there the whole time.
The paper, posted September 17, 2026 as arXiv:2609.17571 (cs.AI, full text) and not yet peer-reviewed, introduces what it calls Transition Games: controlled tests that isolate exactly which parts of a transformer are doing the generalizing work. The authors trace most of the gain to a block-0 attention bias, with a handful of degree-two frequency modes accounting for 67-92% of that effect across their test games. A separate path analysis found block-1's MLP layer mattered more than every other downstream component checked, in all 12 of 12 comparisons tested. Notably, the popular hunch that MLP layers memorize while attention layers generalize did not hold up: the data ran the opposite direction in all 12 tests.
That matters because grokking has become a stand-in explanation any time a model's behavior improves in a way researchers cannot fully account for, and this paper is a reminder that the mechanism behind it is still not settled. If generalization is really existing circuits getting spectrally recoded rather than new pathways switching on, that complicates simple stories about models having a sudden insight, and it means interpretability tools built around finding new modules may be looking in the wrong place.
Read literally, this is one preprint testing one architecture on synthetic tasks, so treat any grokking-solved claims with the same skepticism the paper applies to its own predictions, most of which it disproves.