[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-a-smarter-data-diet-makes-basic-ai-finetuning-rival-rl":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},9787,"a-smarter-data-diet-makes-basic-ai-finetuning-rival-rl","A Smarter Data Diet Makes Basic AI Finetuning Rival RL","A new MCMC sampling method reshapes training data so supervised finetuning can match reinforcement learning's generalization without its sampling overhead.","A new paper argues that supervised finetuning, the plainer and cheaper way to retrain AI models, was never as weak as assumed; the data feeding it was the problem.\n\nThe researchers built a sampling algorithm based on Markov chain Monte Carlo methods that reshapes off-policy training examples, meaning data not generated by the model itself, so they resemble what the model would produce on its own. That reshaped data is then used for standard supervised finetuning (SFT), not reinforcement learning (RL). Across tests in scientific skill acquisition, mathematical reasoning, and open-ended expertise tasks, models finetuned this way matched or beat RL baselines on generalizing to new tasks. They also forgot less of what the base model already knew, and in some cases learned skills beyond what the base model's own outputs could have taught them.\n\nThe standard pitch for RL in posttraining has been that it generalizes better because models learn from their own trial and error, while SFT just copies expert examples and overfits. This result suggests that gap may be a data preparation problem, not a fundamental limit of SFT. If a cheaper preprocessing step can close most of the gap, it undercuts a key justification labs use for RL's extra infrastructure and compute cost.\n\nThat is a big claim from one paper testing specific tasks, not a frontier model shipped to millions of users, so the usual rule applies: wait for someone else to reproduce it before retiring your RL pipeline.","[\"ai\",\"llm-training\",\"research\",\"reinforcement-learning\"]","2026-10-02T04:00:00.000Z","2026-10-03T10:28:57.555Z","2026-10-03T10:29:03.284Z","published",null,[],"ai",[24,26,27,28],"llm-training","research","reinforcement-learning",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2610.02140",0,{"sections":35},[36,39,43,47,52,56,60,65,70,75,80,85,90,95],{"name":37,"slug":24,"count":38,"latest_published_at":18},"AI",6058,{"name":40,"slug":41,"count":42,"latest_published_at":18},"Security","security",848,{"name":44,"slug":45,"count":46,"latest_published_at":18},"Policy","policy",439,{"name":48,"slug":49,"count":50,"latest_published_at":51},"Deals","deals",317,"2026-10-01T22:00:00.000Z",{"name":53,"slug":54,"count":55,"latest_published_at":18},"Hardware","hardware",199,{"name":57,"slug":58,"count":59,"latest_published_at":18},"Science","science",176,{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",155,"2026-10-01T19:54:10.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Dev Tools","dev-tools",96,"2026-10-01T16:57:03.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Software","software",93,"2026-09-30T21:41:11.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Startups","startups",90,"2026-10-01T21:55:22.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":84},"Gaming","gaming",53,"2026-10-02T02:50:39.000Z",{"name":86,"slug":87,"count":88,"latest_published_at":89},"General","general",50,"2026-09-30T21:37:54.000Z",{"name":91,"slug":92,"count":93,"latest_published_at":94},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":96,"slug":97,"count":98,"latest_published_at":99},"How-To","how-to",7,"2026-10-01T09:00:00.000Z"]