[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-benchmark-shows-llms-struggle-to-tune-physics-simulations-fast":10,"sections":40},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":35,"feedback":39,"feedback_at":22,"cost_usd":39,"total_tokens":39},5593,"benchmark-shows-llms-struggle-to-tune-physics-simulations-fast","Benchmark Shows LLMs Struggle to Tune Physics Simulations Fast","LLMs get first-guess physics simulation parameters right only 45-62% of the time, and refined answers are still 1.5-2.7x slower than traditional scanning.","A new benchmark says frontier language models are shaky at guessing the right settings for physics simulations, and slow to fix their guesses once they're wrong.\n\nThe benchmark, called SimulCost, tests LLMs on 11 physics simulators covering fluid dynamics, solid mechanics, and plasma physics, plus a twelfth production plasma code scored separately. Across 2,643 single-round tasks, where a model gets one shot at picking simulation parameters, frontier models succeeded 45-62% of the time, falling to 34-50% when the task demanded high accuracy. Letting models revise their guesses through trial and error, across 2,304 multi-round tasks, pushed success up to 66-81%. That improvement came at a cost: even in multi-round mode, the models ran 1.5-2.7x slower than a traditional scanning approach that simply sweeps the parameter space.\n\nThat tradeoff undercuts a common pitch for AI research agents: that they can replace tedious manual parameter tuning outright. SimulCost's numbers suggest LLMs are only worth deploying here once they are both more accurate on the first guess and cheaper than brute-force search, not just accurate eventually.\n\nA model that gets the physics right on the third try, slower than a for-loop, is not much of a shortcut.","[\"ai\",\"benchmarks\",\"physics-simulation\",\"llm-agents\"]","2026-08-18T04:00:00.000Z","2026-08-19T02:51:06.801Z","2026-08-19T02:51:18.656Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"The dek says LLMs 'succeed on physics simulation tuning barely half the time,' but the body shows multi-round success actually reaches 66-81% — the dek conflates single-round accuracy with the overall finding and needs to specify it's the single-round\u002Ffirst-guess rate, with the slower-than-scanning claim clearly tied to multi-round mode.","resolved","ai",[30,32,33,34],"benchmarks","physics-simulation","llm-agents",[36],{"name":37,"url":38},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2603.20253",0,{"sections":41},[42,46,50,55,60,65,70,75,80,84,89,94,99,104],{"name":43,"slug":30,"count":44,"latest_published_at":45},"AI",3293,"2026-08-20T04:00:00.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":45},"Security","security",435,{"name":51,"slug":52,"count":53,"latest_published_at":54},"Policy","policy",210,"2026-08-19T09:32:27.000Z",{"name":56,"slug":57,"count":58,"latest_published_at":59},"Deals","deals",179,"2026-06-29T20:02:07.000Z",{"name":61,"slug":62,"count":63,"latest_published_at":64},"Hardware","hardware",140,"2026-08-19T18:25:42.000Z",{"name":66,"slug":67,"count":68,"latest_published_at":69},"Consumer Tech","consumer-tech",95,"2026-08-18T16:05:00.000Z",{"name":71,"slug":72,"count":73,"latest_published_at":74},"Science","science",90,"2026-08-19T18:41:02.000Z",{"name":76,"slug":77,"count":78,"latest_published_at":79},"Software","software",73,"2026-08-18T07:51:50.000Z",{"name":81,"slug":82,"count":83,"latest_published_at":18},"Dev Tools","dev-tools",69,{"name":85,"slug":86,"count":87,"latest_published_at":88},"Startups","startups",47,"2026-08-19T19:13:46.000Z",{"name":90,"slug":91,"count":92,"latest_published_at":93},"Gaming","gaming",41,"2026-07-09T04:00:00.000Z",{"name":95,"slug":96,"count":97,"latest_published_at":98},"General","general",33,"2026-08-18T22:18:13.000Z",{"name":100,"slug":101,"count":102,"latest_published_at":103},"Reviews","reviews",20,"2026-06-24T12:00:01.000Z",{"name":105,"slug":106,"count":107,"latest_published_at":108},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]