Robots that rely on today's vision-language-action models can flip their next move if a camera rotates by as little as one degree - and the usual way of testing these systems would likely miss it.
Researchers built a validation tool called H2V-M, an upgrade to an earlier framework named H2V, that checks entire continuous ranges of camera perturbations instead of spot-checking a handful of examples. They ran it against six vision-language models from the Gemma, InternVL, LLaVA, and Qwen families, plus five vision-language-action models from the GR00T, OpenVLA, and pi families. The perturbations covered brightness shifts, camera rotations, and combinations of both. H2V-M needed roughly a tenth as many model queries as the original H2V method and found failure cases faster than random sampling, all while still guaranteeing it hadn't missed a failure anywhere in the tested range.
The interesting result isn't the tool - it's what it found. Robustness tracked the type of perturbation far more than which model was tested, and robot-control models buckled under camera rotations that vision-only chatbots shrugged off. That matters for anyone bolting one of these models onto a robot arm or a warehouse picker, where a slightly crooked camera mount is a Tuesday, not an edge case.
Bigger models weren't safer models, either: robustness tracked with which model family a system came from, not how many parameters it had, undercutting the assumption that scale buys reliability for free.
Sampling-based robustness claims should be read with that one-degree number in mind - a benchmark that never stumbled onto the wrong rotation angle isn't evidence of safety, just evidence of bad luck avoided.