[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-elastic-gang-scheduler-beats-static-core-splits-by-up-to-175x":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},3977,"elastic-gang-scheduler-beats-static-core-splits-by-up-to-175x","Elastic Gang Scheduler Beats Static Core Splits by Up to 1.75x","Anima OS uses an ACK-latched epoch protocol to let LLM inference cores flex between tokens, beating static splits by up to 1.75x general throughput.","A new kernel called Anima OS solves the oldest problem in on-device LLM inference: sharing CPU cores between a hard-barriered gang and a general-purpose OS without deadlocking or corrupting output.\n\nResearchers built Anima OS, a bare-metal x86-64 kernel written in Rust, around an \"elastic gang\" scheduler that lets the set of CPU cores assigned to an LLM inference job change between each generated token. The mechanism is an ACK-latched epoch protocol: instead of waiting on a specific core to acknowledge, the gang takes the intersection of requested cores and those that have actually acked the current epoch. A core that misses the ack window sits out one token and rejoins on the next; general OS processes displaced from borrowed cores keep running on whatever is available and get their cores back the moment an inference epoch ends. On an AMD Zen 5 eight-core machine, the system was tested against 135M and 7B parameter models, with bit-exact output verified under per-token core membership changes.\n\nOn static core partitions — the standard approach — cores committed to inference sit idle when there is no inference work, and cores committed to the OS stay off-limits when the model needs them. Elastic gang scheduling breaks that tradeoff: at intermediate inference duty cycles, the paper reports 1.75x (25% duty), 1.52x (50% duty), and 1.28x (75% duty) general-process throughput compared to a fair static eight-core split, at equal or better inference throughput. The cost of returning a core is 0.22 microseconds at the median; the worst case for acquiring an occupied core is one scheduling quantum, roughly 16 milliseconds, because a running process is never preempted mid-slice.\n\nDecode throughput maxes out at eight cores on this machine anyway, so ceding cores past the saturation point costs almost nothing — a convenient result, but one that only holds until someone runs this on hardware with more cores than the model can actually use.","[\"on-device-ai\",\"inference\",\"kernel\",\"scheduling\"]","2026-07-07T04:00:00.000Z","2026-07-07T13:59:06.679Z","2026-07-07T13:59:09.510Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The title and dek are vague and placeholder-quality — 'On-Device AI Gets Smarter Core Sharing' does not state the actual news; rewrite the headline and dek to name the elastic gang scheduler, the specific throughput gains, and the kernel-level mechanism, matching the specificity and dry tone required by the brand.","resolved","ai",[32,33,34,35],"on-device-ai","inference","kernel","scheduling",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.04668",0,{"sections":42},[43,47,52,57,62,67,72,77,82,86,91,95,100,105],{"name":44,"slug":30,"count":45,"latest_published_at":46},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":48,"slug":49,"count":50,"latest_published_at":51},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":56},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":58,"slug":59,"count":60,"latest_published_at":61},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":63,"slug":64,"count":65,"latest_published_at":66},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":68,"slug":69,"count":70,"latest_published_at":71},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":73,"slug":74,"count":75,"latest_published_at":76},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":78,"slug":79,"count":80,"latest_published_at":81},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":83,"slug":84,"count":85,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":87,"slug":88,"count":89,"latest_published_at":90},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":92,"slug":93,"count":89,"latest_published_at":94},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":101,"slug":102,"count":103,"latest_published_at":104},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":106,"slug":107,"count":108,"latest_published_at":109},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]