[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-moe-models-may-not-need-a-learned-router-after-all":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},4412,"moe-models-may-not-need-a-learned-router-after-all","MoE Models May Not Need a Learned Router After All","A new parameter-free routing method called Self-Routing lets mixture-of-experts models assign tokens to experts without a dedicated routing layer.","A new paper argues that the learned router inside mixture-of-experts AI models is an unnecessary component — and shows competitive results without it.\n\nMixture-of-experts (MoE) architecture saves compute by routing each input token to only a small subset of \"expert\" sub-networks rather than the full model. Normally, a separate learned router module handles that assignment. The new technique, called Self-Routing, skips that module entirely, instead reading a designated slice of the token's existing hidden state and using it directly as routing scores. The researchers tested it across language modeling tasks at multiple scales and on image classification using DeiT-S\u002F16 on ImageNet-1K, benchmarking against standard learned routers, random routing, and dense non-MoE baselines.\n\nThe results are notable less for raw performance — Self-Routing roughly matches learned-router baselines — than for what they suggest about model architecture. Expert utilization was meaningfully more balanced, with about 17% higher average normalized routing entropy, and no explicit load-balancing loss was needed to achieve it. That matters because load-balancing is a persistent headache in MoE training; dedicated losses for it add complexity and can conflict with other training objectives.\n\nMoE has become a go-to architecture for scaling large language models without proportionally scaling inference cost — it underpins designs used across the industry. If routing quality emerges naturally from hidden representations, that opens a path to simpler, cheaper MoE implementations. The catch: \"competitive\" is not \"better\" on language tasks, and the gap between a paper result and production-scale deployment is where most architectural ideas quietly disappear.","[\"ai\",\"machine-learning\",\"model-architecture\",\"mixture-of-experts\"]","2026-07-08T04:00:00.000Z","2026-07-08T08:43:02.059Z","2026-07-08T08:43:05.501Z","published",null,[],"ai",[24,26,27,28],"machine-learning","model-architecture","mixture-of-experts",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2604.00421",0,{"sections":35},[36,40,45,50,55,60,65,70,75,80,85,89,94,99],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Dev Tools","dev-tools",59,"2026-07-07T04:00:00.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":86,"slug":87,"count":83,"latest_published_at":88},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]