[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-researchers-pinpoint-where-ai-grokking-actually-happens":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},6558,"researchers-pinpoint-where-ai-grokking-actually-happens","Researchers Pinpoint Where AI Grokking Actually Happens","An arXiv preprint argues the sudden shift from memorizing to generalizing in transformer models is a spectral recoding of existing circuits, not a new one.","A new arXiv preprint says grokking, the strange moment when a neural network flips from memorizing to actually generalizing, is not a switch flipping in some new brain module. It is a recode of circuitry that was there the whole time.\n\nThe paper, posted September 17, 2026 as arXiv:2609.17571 (cs.AI, [full text](https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.17571)) and not yet peer-reviewed, introduces what it calls Transition Games: controlled tests that isolate exactly which parts of a transformer are doing the generalizing work. The authors trace most of the gain to a block-0 attention bias, with a handful of degree-two frequency modes accounting for 67-92% of that effect across their test games. A separate path analysis found block-1's MLP layer mattered more than every other downstream component checked, in all 12 of 12 comparisons tested. Notably, the popular hunch that MLP layers memorize while attention layers generalize did not hold up: the data ran the opposite direction in all 12 tests.\n\nThat matters because grokking has become a stand-in explanation any time a model's behavior improves in a way researchers cannot fully account for, and this paper is a reminder that the mechanism behind it is still not settled. If generalization is really existing circuits getting spectrally recoded rather than new pathways switching on, that complicates simple stories about models having a sudden insight, and it means interpretability tools built around finding new modules may be looking in the wrong place.\n\nRead literally, this is one preprint testing one architecture on synthetic tasks, so treat any grokking-solved claims with the same skepticism the paper applies to its own predictions, most of which it disproves.","[\"ai\",\"research\",\"transformers\",\"arxiv\"]","2026-09-17T04:00:00.000Z","2026-09-18T00:03:02.178Z","2026-09-18T00:03:14.087Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Add explicit sourcing — name this as an arXiv preprint (arXiv:2609.17571, cs.AI) with a link and note it hasn't been peer-reviewed, since the draft currently attributes the findings only to unnamed 'researchers' with no traceable citation.","resolved","ai",[30,32,33,34],"research","transformers","arxiv",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.17571",0,{"sections":41},[42,46,50,55,60,64,68,73,78,82,87,92,97,102],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",3853,"2026-09-17T08:27:09.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":18},"Security","security",648,{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",338,"2026-09-11T04:00:00.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":18},"Hardware","hardware",154,{"name":65,"slug":66,"count":67,"latest_published_at":18},"Science","science",114,{"name":69,"slug":70,"count":71,"latest_published_at":72},"Consumer Tech","consumer-tech",99,"2026-09-09T17:27:33.000Z",{"name":74,"slug":75,"count":76,"latest_published_at":77},"Software","software",75,"2026-09-10T20:41:21.000Z",{"name":79,"slug":80,"count":81,"latest_published_at":18},"Dev Tools","dev-tools",73,{"name":83,"slug":84,"count":85,"latest_published_at":86},"Startups","startups",55,"2026-09-09T23:14:29.000Z",{"name":88,"slug":89,"count":90,"latest_published_at":91},"Gaming","gaming",43,"2026-09-10T12:18:06.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"General","general",41,"2026-09-08T01:57:23.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":103,"slug":104,"count":105,"latest_published_at":106},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]