[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-selective-importance-sampling-tames-rl-variance-in-llm-training":10,"sections":44},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":34,"tags":35,"sources":39,"feedback":43,"feedback_at":22,"cost_usd":43,"total_tokens":43},3985,"selective-importance-sampling-tames-rl-variance-in-llm-training","Selective Importance Sampling Tames RL Variance in LLM Training","A plug-in technique called SIS cuts the variance problem that plagues importance sampling during reinforcement learning post-training for large language models.","A new method called Selective Importance Sampling wants to fix one of the messier plumbing problems in modern AI training.\n\nWhen labs fine-tune large language models with reinforcement learning, they use a \"rollout then update\" loop: the model generates responses, then learns from them. The catch is that by the time the update runs, the data is already stale — generated by a slightly older version of the model. Importance sampling is the standard statistical fix, reweighting old data to approximate what a current model would have produced. But token-level importance ratios multiply across long sequences, and the variance compounds fast enough to destabilize training. Selective Importance Sampling, or SIS, sidesteps this by borrowing from rejection sampling: it runs a token-by-token test and treats accepted tokens as effectively on-policy, assigning them a unit importance score. Only rejected tokens get the full IS correction, keeping the math honest without letting variance explode.\n\nThe practical upside is narrow but real. SIS is a drop-in modification to the policy loss function — it adds no meaningful compute overhead and slots into existing RL algorithms without redesign. The authors tested it on both dense and mixture-of-experts LLMs across math and agent benchmarks, reporting consistent gains and better stability when training data drifts off-policy.\n\nImportance sampling variance is a known headache in RL research, not a novel discovery, but workable plug-in fixes for the LLM-scale version are scarcer than the problem's prominence would suggest — which is either an opportunity or a sign that the real solutions are still messier than a single loss tweak.","[\"ai\",\"reinforcement-learning\",\"llm\",\"research\"]","2026-07-07T04:00:00.000Z","2026-07-07T14:09:44.416Z","2026-07-07T14:09:47.221Z","published",null,[24,30],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The article attributes the research to 'Researchers at arXiv' — arXiv is a preprint server, not an institution; the actual authors must be identified or the attribution corrected to something like 'researchers in a preprint posted to arXiv.'","resolved",{"id":31,"reviewer":26,"round":32,"reason":33,"status":29},"editor-r2",2,"The headline and dek are too vague and read as working placeholders — neither names the technique (Selective Importance Sampling) nor the specific problem solved (importance sampling variance explosion in RL post-training), failing the requirement that headline and dek clearly state the actual news.","ai",[34,36,37,38],"reinforcement-learning","llm","research",[40],{"name":41,"url":42},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.04728",0,{"sections":45},[46,50,55,60,65,70,75,80,85,89,94,98,103,108],{"name":47,"slug":34,"count":48,"latest_published_at":49},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":95,"slug":96,"count":92,"latest_published_at":97},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":104,"slug":105,"count":106,"latest_published_at":107},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":109,"slug":110,"count":111,"latest_published_at":112},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]