[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-researchers-crack-the-math-behind-why-neural-nets-suddenly-get-smart":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},2785,"researchers-crack-the-math-behind-why-neural-nets-suddenly-get-smart","Researchers Crack the Math Behind Why Neural Nets Suddenly Get Smart","A new theoretical framework explains grokking - the delayed generalization effect in neural networks - using geometry and stochastic stopping-time analysis.","Neural networks sometimes memorize their training data for thousands of steps, then snap into genuine understanding almost overnight — and now researchers think they know why.\n\nA paper posted to arXiv lays out a stochastic-geometric theory for the phenomenon known as grokking. The researchers argue that the Adam optimizer, combined with weight-shrinkage regularization, structures the solution space into nested spherical shells: random starting weights land on the outermost shell, memorization solutions occupy an inner shell, and true generalization solutions cluster at the core. Training is essentially a waiting game — the optimization trajectory has to escape the memorization manifold before it can reach generalization, and the paper uses stopping-time theory to calculate how long that takes. From that framework, the team derives concrete scaling laws relating grokking delay to learning rate, batch size, and the strength of the L2 regularization coefficient, then validates those predictions against experiments.\n\nGrokking has been one of the more unsettling unexplained facts in deep learning since it was first documented — it complicates assumptions about what \"training complete\" actually means and raises questions about whether early-stopped models have truly learned anything. A theoretical handle on it could inform training recipes, regularization choices, and even interpretability work by making the memorization-to-generalization transition predictable rather than surprising.\n\nThe field has accumulated a lot of empirical observations about grokking but relatively few principled explanations, so a geometric account with testable scaling laws is a step forward — assuming the results hold outside the controlled settings the paper examines.","[\"machine learning\",\"deep learning\",\"ai research\",\"neural networks\"]","2026-06-30T04:00:00.000Z","2026-06-30T12:40:54.451Z","2026-06-30T12:40:57.371Z","published",null,[],"ai",[26,27,28,29],"machine learning","deep learning","ai research","neural networks",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.30388",0,{"sections":36},[37,41,46,51,56,61,66,71,76,81,86,90,95,100],{"name":38,"slug":24,"count":39,"latest_published_at":40},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":42,"slug":43,"count":44,"latest_published_at":45},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":87,"slug":88,"count":84,"latest_published_at":89},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]