[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"branding":3,"analytics":7,"article-new-benchmark-shows-voice-ai-stumbles-in-korean-and-mandarin":10,"sections":41},{"siteName":4,"siteTagline":5,"publisherName":4,"contactEmail":6},"The Revision","Tech news, decoded.","editor@therevision.news",{"gaMeasurementId":8,"adsenseClientId":9},"G-ZW2MV82GYR","ca-pub-8533917693782264",{"article":11},{"id":12,"slug":13,"title":14,"dek":15,"body_md":16,"tags_json":17,"published_at":18,"created_at":19,"updated_at":20,"status":21,"review_note":22,"review_notes":23,"image_url":22,"persona_id":22,"persona_name":22,"section":30,"tags":31,"sources":36,"feedback":40,"feedback_at":22,"cost_usd":40,"total_tokens":40},8558,"new-benchmark-shows-voice-ai-stumbles-in-korean-and-mandarin","New Benchmark Shows Voice AI Stumbles in Korean and Mandarin","A benchmark posted to arXiv on September 30 finds voice agents falter in Korean and Mandarin, and Grok's top task scores come with weaker speech quality.","A new benchmark finds today's voice AI assistants still stumble badly once you leave English, especially in Korean and Mandarin.\n\nThe arXiv preprint arXiv:2609.35820, posted September 30, 2026, introduces tau-Multilingual, which extends the earlier tau-Voice benchmark to Spanish, Brazilian Portuguese, Hindi, Korean, and Mandarin. Native speakers reviewed both the generated language and the spoken output across 4,500 full-duplex calls and five voice configurations. Spanish, Portuguese, and Hindi stayed within 3.2 points of English task-completion scores, but Korean dropped 14.7 points and Mandarin fell 8.4 points. The paper also breaks down failure modes: Korean systems miss more responses outright, Mandarin systems interrupt callers more often, and both struggle with tools and named entities. Grok topped the task-completion leaderboard but scored lowest on generation quality, which is why the authors report task, interaction, and generation performance as separate scores instead of one blended number.\n\nThat separation matters more than it sounds. A single leaderboard number would have let Grok's task-completion lead paper over its weaker speech quality, and it would have hidden that Korean and Mandarin support is not a rounding-error gap but a double-digit one. For any company selling voice agents into East Asian markets, this is the first independently reviewed evidence of how much ground those systems still have to make up.\n\nMost voice-agent benchmarks to date have been English-only, so vendors touting broad language support have had little external checking. This one gives buyers a reason to ask which languages, specifically, before they believe the pitch.","[\"voice-agents\",\"benchmarks\",\"multilingual-ai\",\"grok\"]","2026-09-30T04:00:00.000Z","2026-09-30T11:08:46.551Z","2026-09-30T11:08:50.744Z","published",null,[24],{"id":25,"reviewer":26,"round":27,"reason":28,"status":29},"editor-r1","editor",1,"Attribute the benchmark to its source explicitly — name the arXiv preprint (arXiv:2609.35820) and when it was posted, since currently every figure is sourced only to vague 'researchers'\u002F'the team' with no named institution, paper, or date a reader could verify.","resolved","ai",[32,33,34,35],"voice-agents","benchmarks","multilingual-ai","grok",[37],{"name":38,"url":39},"arXiv cs.AI","https:\u002F\u002Farxiv.org\u002Fabs\u002F2609.35820",0,{"sections":42},[43,46,50,54,59,64,69,74,79,83,88,93,98,103],{"name":44,"slug":30,"count":45,"latest_published_at":18},"AI",5105,{"name":47,"slug":48,"count":49,"latest_published_at":18},"Security","security",785,{"name":51,"slug":52,"count":53,"latest_published_at":18},"Policy","policy",417,{"name":55,"slug":56,"count":57,"latest_published_at":58},"Deals","deals",284,"2026-09-29T21:00:00.000Z",{"name":60,"slug":61,"count":62,"latest_published_at":63},"Hardware","hardware",194,"2026-09-29T13:16:04.000Z",{"name":65,"slug":66,"count":67,"latest_published_at":68},"Science","science",154,"2026-09-28T13:19:18.000Z",{"name":70,"slug":71,"count":72,"latest_published_at":73},"Consumer Tech","consumer-tech",142,"2026-09-29T18:38:03.000Z",{"name":75,"slug":76,"count":77,"latest_published_at":78},"Software","software",91,"2026-09-25T20:55:00.000Z",{"name":80,"slug":81,"count":82,"latest_published_at":18},"Dev Tools","dev-tools",90,{"name":84,"slug":85,"count":86,"latest_published_at":87},"Startups","startups",83,"2026-09-29T21:51:36.000Z",{"name":89,"slug":90,"count":91,"latest_published_at":92},"General","general",49,"2026-09-28T16:44:57.000Z",{"name":94,"slug":95,"count":96,"latest_published_at":97},"Gaming","gaming",48,"2026-09-25T18:35:21.000Z",{"name":99,"slug":100,"count":101,"latest_published_at":102},"Reviews","reviews",31,"2026-09-28T14:31:34.000Z",{"name":104,"slug":105,"count":106,"latest_published_at":107},"How-To","how-to",6,"2026-06-16T09:00:00.000Z"]