[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-new-scaling-law-lets-looped-moe-models-match-bigger-rivals":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},9213,"new-scaling-law-lets-looped-moe-models-match-bigger-rivals","New Scaling Law Lets Looped MoE Models Match Bigger Rivals","A new scaling law shows a looped mixture-of-experts model matching the reasoning performance of a non-looped MoE twice its size, at equal training compute.","A new paper gives looped mixture-of-experts models a formula for punching above their weight.\n\nResearchers built what they call Loop Scaling Laws, the first framework to jointly model two efficiency tricks usually studied separately: looping (reusing the same transformer layers multiple times to add computational depth without adding parameters) and mixture-of-experts sparsity (activating only a fraction of a model's parameters per token). The laws predict held-out loss for looped models more accurately than prior scaling laws, and they collapse back into the standard dense and MoE scaling laws as special cases when either axis is turned off. Tested at trillion-token scale, a looped MoE model sized using the new law matched the reasoning-benchmark performance of a non-looped MoE model roughly twice its size, at the same training compute. The looped version also supports test-time scaling - letting it loop more at inference to trade latency for accuracy.\n\nThat's a real efficiency win, not a marginal one: the paper reports sparsity alone buys about 3x active-parameter efficiency, and recurrence alone buys about 2x total-parameter efficiency on reasoning tasks, with the two compounding when combined. For anyone training large models under a fixed compute or memory budget, that's a concrete recipe for getting more reasoning capability out of the same hardware, rather than just a reason to buy more of it.\n\nIt's a formula, not a shipped model - whether any lab actually bakes a law-derived recurrence depth into its next release is the part worth watching.","[\"ai\",\"mixture-of-experts\",\"scaling-laws\",\"model-efficiency\"]","2026-10-01T04:00:00.000Z","2026-10-02T02:07:42.500Z","2026-10-02T02:07:45.939Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Fix the dek: the source compares the looped MoE model to a non-looped MoE model roughly twice its size, not to a 'dense' model — correct the dek so it doesn't misstate the comparison baseline given in the body.","resolved","ai",[30,32,33,34],"mixture-of-experts","scaling-laws","model-efficiency",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.40316",0,{"sections":41},[42,45,49,53,58,63,67,72,77,81,86,91,96,101],{"name":43,"slug":30,"count":44,"latest_published_at":18},"AI",5614,{"name":46,"slug":47,"count":48,"latest_published_at":18},"Security","security",815,{"name":50,"slug":51,"count":52,"latest_published_at":18},"Policy","policy",430,{"name":54,"slug":55,"count":56,"latest_published_at":57},"Deals","deals",298,"2026-09-30T21:00:26.000Z",{"name":59,"slug":60,"count":61,"latest_published_at":62},"Hardware","hardware",196,"2026-09-30T13:00:00.000Z",{"name":64,"slug":65,"count":66,"latest_published_at":18},"Science","science",163,{"name":68,"slug":69,"count":70,"latest_published_at":71},"Consumer Tech","consumer-tech",149,"2026-09-30T22:57:11.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Dev Tools","dev-tools",93,"2026-10-01T02:30:48.000Z",{"name":78,"slug":79,"count":75,"latest_published_at":80},"Software","software","2026-09-30T21:41:11.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Startups","startups",84,"2026-09-30T20:39:09.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",51,"2026-09-30T16:24:30.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":102,"slug":103,"count":104,"latest_published_at":105},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]