[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-robot-ai-models-look-alike-until-instructions-get-tricky":10,"sections":35},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":24,"tags":25,"sources":30,"feedback":34,"feedback_at":22,"cost_usd":34,"total_tokens":34},11147,"robot-ai-models-look-alike-until-instructions-get-tricky","Robot AI Models Look Alike Until Instructions Get Tricky","A new benchmark finds six open robot policy models score nearly identically on clean tasks but diverge sharply once instructions change.","Robot AI models that look nearly identical on paper fall apart differently once you change how you phrase a command.\n\nA new benchmark called IndustrialVLA-Bench tested six released robot policy systems, a mix of vision-language-action models and world-action models, under one unified reporting schema instead of each vendor's own protocol. On clean, standard tasks from the LIBERO benchmark, the six systems scored within 1.58 points of each other. But on LIBERO-Plus, which tests robustness to non-language changes like lighting or object position, scores spread across 14.62 points. On LIBERO-Para, which paraphrases the same instructions, the spread widened to 31.08 points. The researchers also logged inference latency, memory use, and flagged which results were independently reproduced versus self-reported.\n\nThe real finding here is not which model wins. It is that raw task-completion scores, the number every lab leads with, barely separate these systems at all. The actual differentiator is how a model handles a worker saying \"grab the part on the left\" instead of \"pick up the item to the left,\" and that is exactly the scenario a factory floor guarantees.\n\nIt is the same lesson most benchmarks eventually teach: the leaderboard number tells you less than how the system behaves the first time someone phrases a request in a way the demo did not rehearse.","[\"robotics\",\"ai-benchmarks\",\"vision-language-action\",\"open-source-ai\"]","2026-10-09T04:00:00.000Z","2026-10-10T08:24:09.647Z","2026-10-10T08:24:13.005Z","published",null,[],"ai",[26,27,28,29],"robotics","ai-benchmarks","vision-language-action","open-source-ai",[31],{"name":32,"url":33},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.25562",0,{"sections":36},[37,41,46,51,56,61,66,71,76,81,86,91,96,101],{"name":38,"slug":24,"count":39,"latest_published_at":40},"AI",6841,"2026-10-09T19:36:56.000Z",{"name":42,"slug":43,"count":44,"latest_published_at":45},"Security","security",941,"2026-10-09T19:25:43.000Z",{"name":47,"slug":48,"count":49,"latest_published_at":50},"Policy","policy",490,"2026-10-09T14:25:43.000Z",{"name":52,"slug":53,"count":54,"latest_published_at":55},"Deals","deals",483,"2026-10-09T11:20:39.000Z",{"name":57,"slug":58,"count":59,"latest_published_at":60},"Hardware","hardware",233,"2026-10-09T16:07:54.000Z",{"name":62,"slug":63,"count":64,"latest_published_at":65},"Science","science",195,"2026-10-09T11:00:57.000Z",{"name":67,"slug":68,"count":69,"latest_published_at":70},"Consumer Tech","consumer-tech",183,"2026-10-09T14:50:57.000Z",{"name":72,"slug":73,"count":74,"latest_published_at":75},"Startups","startups",120,"2026-10-09T17:02:31.000Z",{"name":77,"slug":78,"count":79,"latest_published_at":80},"Software","software",114,"2026-10-08T17:57:01.000Z",{"name":82,"slug":83,"count":84,"latest_published_at":85},"Dev Tools","dev-tools",109,"2026-10-09T13:03:48.000Z",{"name":87,"slug":88,"count":89,"latest_published_at":90},"General","general",69,"2026-10-09T19:10:07.000Z",{"name":92,"slug":93,"count":94,"latest_published_at":95},"Gaming","gaming",59,"2026-10-09T11:43:43.000Z",{"name":97,"slug":98,"count":99,"latest_published_at":100},"Reviews","reviews",34,"2026-10-08T14:00:22.000Z",{"name":102,"slug":103,"count":104,"latest_published_at":105},"How-To","how-to",8,"2026-10-05T09:00:00.000Z"]