[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-researchers-patch-a-flaw-in-multi-stream-residual-networks":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},8131,"researchers-patch-a-flaw-in-multi-stream-residual-networks","Researchers Patch a Flaw in Multi-Stream Residual Networks","A new spectral-norm constraint fixes training instability and expressivity limits in multi-stream neural network connections, replacing a slower prior fix.","Researchers have identified new flaws in a widely used technique for stabilizing multi-stream neural networks - and proposed a fix for the fix.\n\nHyper-Connections split a neural network's residual connections into several parallel streams, mixing information between them to boost model expressiveness. That mixing can destabilize training, so a follow-up method called Manifold-Constrained Hyper-Connections forced the mixing matrices to be \"doubly stochastic,\" using either an iterative algorithm called Sinkhorn-Knopp or permutation-based math. According to the paper, that constraint backfires in three ways: the matrices tend to collapse toward doing nothing (what the authors call identity degeneration), they cap how much the model can selectively adjust cross-stream signals, and the underlying math is either numerically unstable or scales explosively with model size. The proposed alternative, called Spectral-Sphere-Constrained Hyper-Connections, instead confines the matrices to a spectral norm sphere, which the authors say restores free control over which cross-stream signals to keep or dampen, without the instability or overhead of the earlier approach.\n\nFor anyone training large models, connection stability isn't cosmetic - it decides whether a bigger, more expressive architecture trains reliably or falls apart. If this constraint holds up, it's a cheaper and more flexible route to multi-stream mixing without babysitting a finicky training run.\n\nStill, this is one arXiv preprint proposing the third iteration of a fix for a fix. Sinkhorn-Knopp and permutation-based constraints looked solid on paper too, until their cracks showed - worth watching whether spectral-sphere constraints hold up outside the paper's own benchmarks.","[\"ai\",\"neural-networks\",\"research\",\"machine-learning\"]","2026-09-28T04:00:00.000Z","2026-09-28T10:41:20.893Z","2026-09-28T10:41:27.881Z","published",null,[],"ai",[24,26,27,28],"neural-networks","research","machine-learning",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2603.20896",0,{"sections":35},[36,39,43,48,53,58,62,67,72,77,82,87,91,96],{"name":37,"slug":24,"count":38,"latest_published_at":18},"AI",4798,{"name":40,"slug":41,"count":42,"latest_published_at":18},"Security","security",762,{"name":44,"slug":45,"count":46,"latest_published_at":47},"Policy","policy",399,"2026-09-27T18:39:02.000Z",{"name":49,"slug":50,"count":51,"latest_published_at":52},"Deals","deals",261,"2026-09-27T15:30:35.000Z",{"name":54,"slug":55,"count":56,"latest_published_at":57},"Hardware","hardware",188,"2026-09-27T20:46:36.000Z",{"name":59,"slug":60,"count":61,"latest_published_at":18},"Science","science",151,{"name":63,"slug":64,"count":65,"latest_published_at":66},"Consumer Tech","consumer-tech",135,"2026-09-26T14:30:00.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":71},"Software","software",91,"2026-09-25T20:55:00.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Dev Tools","dev-tools",84,"2026-09-26T04:20:58.000Z",{"name":78,"slug":79,"count":80,"latest_published_at":81},"Startups","startups",76,"2026-09-25T18:33:59.000Z",{"name":83,"slug":84,"count":85,"latest_published_at":86},"Gaming","gaming",48,"2026-09-25T18:35:21.000Z",{"name":88,"slug":89,"count":85,"latest_published_at":90},"General","general","2026-09-26T17:02:42.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Reviews","reviews",30,"2026-09-24T20:07:31.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]