Automated robot video graders keep failing — RoboGaze is a research attempt to fix that.
Researchers have released RoboGaze, a training-free framework that uses multiple vision-language models in a three-stage pipeline to evaluate videos generated by robot world models. The system grounds a task instruction in the scene, routes clips to dimension-specific specialist models, then runs a critic pass to catch false alarms. It outputs timestamped failure reports organized across a 6-dimension, 30-type taxonomy covering physics violations, temporal inconsistencies, and task logic errors. Testing across 382 human-validated clips and eight VLM backbones, the framework improved description accuracy by up to 43 F1 points and temporal alignment by up to 37 points over zero-shot baselines.
The false-positive problem it targets is real and underappreciated: standard vision-language model judges tend to flag clean clips as broken, hitting accuracy below 25% on unflawed video. RoboGaze's critic stage pushes that to above 80%, which matters because a grader that cries wolf on good footage is useless for filtering training data at scale. Better evaluation tooling is the unglamorous prerequisite for better robots.
Robot world models — systems that synthesize plausible video of future states to help robots plan — have advanced quickly, but the evaluation gap has been a quiet bottleneck. RoboGaze does not make robot videos better; it makes it harder for bad ones to pass unnoticed, which is a different and arguably more durable contribution.