[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-row-normalization-beats-adam-and-muon-in-new-optimizer-study":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},10994,"row-normalization-beats-adam-and-muon-in-new-optimizer-study","Row Normalization Beats Adam and Muon in New Optimizer Study","A new theoretical study shows a simple row normalization trick can out-generalize Adam and Muon optimizers under specific, well-defined data conditions.","A new paper argues that a simple trick, normalizing each row of a weight matrix, can beat two of the field's favorite optimizers at the thing that actually matters: how well a trained model handles data it has never seen.\n\nThe researchers tested the idea on multiclass classification under an idealized model where data forms Gaussian clusters around class centers. They compared row-wise normalization against full-batch Adam, a stand-in for the widely used random-reshuffling version, and an exact-SVD variant of Muon, the optimizer that has gained traction in some recent large-language-model training runs. Row normalization won because its class-wise geometry preserves the true decision-boundary directions as data scales up, while Adam's per-coordinate scaling and Muon's spectral scaling both introduce distortions that do not disappear. The advantage held under several more realistic variations, including power-law-distributed data and misaligned noise, though the authors also identified narrow conditions where Adam catches up.\n\nThis matters because optimizer choice usually gets judged on training speed, not on what it does to generalization. The paper offers a mechanistic reason, geometry distortion, for why one optimizer can outperform another even when both fit the training data equally well. That is a sharper explanation than the usual intuitions about why a given optimizer feels better.\n\nThe catch is that the strongest results come from a stylized Gaussian-cluster model plus small synthetic and last-layer experiments, not a frontier training run, so this is a compelling theory with real-world validation still pending.","[\"optimizers\",\"machine learning\",\"neural networks\",\"ai research\"]","2026-10-09T04:00:00.000Z","2026-10-10T01:05:36.194Z","2026-10-10T01:05:42.161Z","published",null,[],"ai",[26,27,28,29],"optimizers","machine learning","neural networks","ai research",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.11309",0,{"sections":36},[37,40,44,49,54,58,62,67,72,77,81,86,91,96],{"name":38,"slug":24,"count":39,"latest_published_at":18},"AI",6733,{"name":41,"slug":42,"count":43,"latest_published_at":18},"Security","security",931,{"name":45,"slug":46,"count":47,"latest_published_at":48},"Policy","policy",486,"2026-10-08T22:40:11.000Z",{"name":50,"slug":51,"count":52,"latest_published_at":53},"Deals","deals",474,"2026-10-08T22:00:00.000Z",{"name":55,"slug":56,"count":57,"latest_published_at":18},"Hardware","hardware",232,{"name":59,"slug":60,"count":61,"latest_published_at":18},"Science","science",193,{"name":63,"slug":64,"count":65,"latest_published_at":66},"Consumer Tech","consumer-tech",181,"2026-10-08T23:26:35.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":71},"Startups","startups",117,"2026-10-08T16:45:00.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Software","software",114,"2026-10-08T17:57:01.000Z",{"name":78,"slug":79,"count":80,"latest_published_at":18},"Dev Tools","dev-tools",106,{"name":82,"slug":83,"count":84,"latest_published_at":85},"General","general",66,"2026-10-09T04:46:11.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",58,"2026-10-08T20:08:45.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",34,"2026-10-08T14:00:22.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",8,"2026-10-05T09:00:00.000Z"]