[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-a-smarter-way-to-train-diffusion-language-models":10,"sections":34},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":29,"feedback":33,"feedback_at":22,"cost_usd":33,"total_tokens":33},3941,"a-smarter-way-to-train-diffusion-language-models","A Smarter Way to Train Diffusion Language Models","A new technique lets diffusion language models teach themselves using their own decoding process, improving math and code reasoning without external labels.","Diffusion language models just got a better training method — one that doesn't need outside help to work.\n\nResearchers introduced dOPSD, a training approach for diffusion large language models (dLLMs) — a class of models that generate text by repeatedly cleaning up a masked, noisy sequence rather than producing tokens one at a time. The core problem they tackled: existing post-training methods are either off-policy (trained on data the model itself wouldn't produce) or rely on sparse feedback that's hard to apply to these non-autoregressive models. On-policy self-distillation looked promising, but required feeding the teacher model ground-truth answers that wouldn't be available at actual inference time — limiting real gains. dOPSD fixes this by deriving the teacher's advantage from the student model's own denoising steps, using later, more-decoded positions in the same sequence to evaluate earlier masked ones.\n\nThis matters because diffusion language models have stalled as a serious alternative to standard autoregressive models partly due to weak post-training. If dOPSD holds up beyond the lab, it closes one of the practical gaps keeping dLLMs on the research shelf. Tested on Dream and LLaDA — two established dLLM benchmarks — the method improved both in-domain math reasoning and out-of-domain code generation, outperforming supervised and on-policy baselines.\n\nThe approach is clever precisely because it avoids the usual cheat: instead of leaning on labels the model won't have in the wild, it lets the model's own decoding process do the teaching. Whether that scales to harder tasks is the next question nobody's answered yet.","[\"ai\",\"machine-learning\",\"language-models\",\"research\"]","2026-07-07T04:00:00.000Z","2026-07-07T12:44:45.939Z","2026-07-07T12:44:48.921Z","published",null,[],"ai",[24,26,27,28],"machine-learning","language-models","research",[30],{"name":31,"url":32},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.04428",0,{"sections":35},[36,40,45,50,55,60,65,70,75,79,84,88,93,98],{"name":37,"slug":24,"count":38,"latest_published_at":39},"AI",2590,"2026-07-16T04:00:00.000Z",{"name":41,"slug":42,"count":43,"latest_published_at":44},"Security","security",294,"2026-07-15T19:59:48.000Z",{"name":46,"slug":47,"count":48,"latest_published_at":49},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",158,"2026-07-16T00:02:48.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Hardware","hardware",122,"2026-07-14T19:46:26.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Consumer Tech","consumer-tech",93,"2026-07-13T13:20:48.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Software","software",70,"2026-07-13T19:52:25.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",66,"2026-07-10T10:29:37.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":18},"Dev Tools","dev-tools",59,{"name":80,"slug":81,"count":82,"latest_published_at":83},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":85,"slug":86,"count":82,"latest_published_at":87},"Startups","startups","2026-06-29T20:55:50.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",29,"2026-07-10T22:28:58.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]