[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-gemini-25-flash-best-at-turning-game-bugs-into-pddl":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},6540,"gemini-25-flash-best-at-turning-game-bugs-into-pddl","Gemini 2.5 Flash Best at Turning Game Bugs Into PDDL","A new arXiv preprint pits six LLMs on translating game testers' plain-English goals into PDDL, and Gemini 2.5 Flash comes out most accurate.","A new arXiv preprint puts six large language models to the test on a narrow but useful job: turning a game tester's plain-English bug report into a formal PDDL planning goal.\n\nThe paper (arXiv:2609.18731) built a prompt template through iterative testing, then measured how well GPT-4.1, Gemini 2.5 Flash, and four other contemporary models could convert real tester language into valid PDDL, the formal syntax classical planners use to define goals and states. All six models got it right more than 92% of the time. Gemini 2.5 Flash led on accuracy at 96% and had the fewest false positives, while GPT-4.1 answered fastest. Most of the remaining errors traced back to ambiguous phrasing from testers or gaps in how a given game's domain was represented in PDDL.\n\nThat distinction matters because PDDL has stayed a specialist's tool since the 1990s, useful mainly to people who already know its syntax. If a prompted LLM can reliably do that translation, QA teams could hook automated planners into bug triage without training testers to write formal goals themselves.\n\nNone of the six models cleared the ambiguity problem entirely, which is the part natural-language translation keeps rediscovering it can't just prompt its way past.","[\"ai\",\"llm-benchmarks\",\"automated-planning\",\"video-game-testing\"]","2026-09-17T04:00:00.000Z","2026-09-17T23:13:03.517Z","2026-09-17T23:13:15.428Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Attribute the study explicitly as an arXiv preprint (cite the paper, e.g. arXiv:2609.18731) instead of vague phrases like 'a new study says' and 'researchers evaluated,' since no institution or publication is otherwise named.","resolved","ai",[30,32,33,34],"llm-benchmarks","automated-planning","video-game-testing",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.18731",0,{"sections":41},[42,46,50,55,60,64,68,73,78,82,87,92,97,102],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",3853,"2026-09-17T08:27:09.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":18},"Security","security",648,{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",338,"2026-09-11T04:00:00.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":18},"Hardware","hardware",154,{"name":65,"slug":66,"count":67,"latest_published_at":18},"Science","science",114,{"name":69,"slug":70,"count":71,"latest_published_at":72},"Consumer Tech","consumer-tech",99,"2026-09-09T17:27:33.000Z",{"name":74,"slug":75,"count":76,"latest_published_at":77},"Software","software",75,"2026-09-10T20:41:21.000Z",{"name":79,"slug":80,"count":81,"latest_published_at":18},"Dev Tools","dev-tools",73,{"name":83,"slug":84,"count":85,"latest_published_at":86},"Startups","startups",55,"2026-09-09T23:14:29.000Z",{"name":88,"slug":89,"count":90,"latest_published_at":91},"Gaming","gaming",43,"2026-09-10T12:18:06.000Z",{"name":93,"slug":94,"count":95,"latest_published_at":96},"General","general",41,"2026-09-08T01:57:23.000Z",{"name":98,"slug":99,"count":100,"latest_published_at":101},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":103,"slug":104,"count":105,"latest_published_at":106},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]