A research team has proposed a way to make AI systems smarter about when to respond to questions during live video — without forcing the model to make a judgment call on every single frame.
Current streaming video AI models work by checking each incoming frame against an open query, deciding on the spot whether it's time to respond. Em-Garde breaks that into two stages. At query time, a component called the Instruction-Guided Proposal Parser converts the user's question into a set of structured visual checkpoints — specific things to look for. Then, as video streams in, a lightweight matching module compares incoming frames against those checkpoints using fast embedding comparisons rather than full model inference. The result is a system that does the heavy thinking up front rather than on every frame.
This matters because per-frame inference is expensive, and the efficiency-accuracy tradeoff has been a genuine obstacle to deploying proactive video AI at scale. Em-Garde's approach sidesteps that tradeoff structurally rather than papering over it with faster hardware. Benchmarks on StreamingBench and OVO-Bench show improvements on both accuracy and efficiency metrics over prior models.
The approach echoes a broader pattern in AI systems design: separate the slow, smart reasoning pass from the fast, cheap recognition pass. It is the same intuition behind retrieval-augmented generation in text models — do the semantic work once, then match cheaply at runtime.