A research benchmark called AnyGroundBench exposes a wide gap between what video AI can do in a lab and what it can do when the footage gets specialized.
Researchers built AnyGroundBench to test 15 leading vision-language models on spatio-temporal video grounding — the task of locating a specific object or subject in both space and time within a video clip. The benchmark covers five domains: animal behavior, industrial settings, sports, surgery, and public security. It includes newly captured footage, such as expert-annotated mouse behavior videos, alongside established datasets, all unified under dense, high-quality annotations. Crucially, it also provides training subsets so researchers can measure not just out-of-the-box performance but how well a model adapts when given new domain examples.
The results are blunt: every model tested failed, and failed consistently, whether given zero prior exposure to the domain or a handful of in-context examples to learn from. That second failure mode is the more telling one. In-context learning is supposed to be a shortcut — show the model a few examples and it adjusts. The fact that it did not work here suggests the models lack the underlying spatio-temporal reasoning required, not just the domain-specific vocabulary.
Most public benchmarks for video AI use everyday footage — people walking, cars driving, generic household scenes. Models trained and evaluated on that distribution look capable until they meet a surgeon's hands or a mouse in a research enclosure. AnyGroundBench does not fix that problem, but it makes the gap measurable, which is the first step toward fixing it.