A new technique lets robot AI systems process video nearly eight times faster by quietly discarding redundant visual data before it clogs the pipeline.
Researchers published ST-Merge, a plug-and-play framework that works during the visual encoding phase — no retraining required. The method builds three-dimensional spatiotemporal coordinates across video frames, then merges duplicate or near-duplicate visual tokens using parallel matching and weighted averaging. A correction step recalculates positional data after the merge to prevent the model from losing track of where things are in space. On the vision-language model Qwen2.5-VL, it delivers a 2x inference speedup with roughly 1% precision loss. On the pi0.5 vision-language action policy at 1024x1024 resolution, speedup reaches 8.3x while matching baseline success rates — though lower resolutions see a small accuracy dip.
Real-time robot control has been a quiet ceiling for vision-language models: the same high-resolution video that makes a robot smarter also buries it in compute. ST-Merge attacks that bottleneck without touching the underlying model weights, which means it could drop into existing deployments rather than requiring labs to retrain expensive models from scratch.
The 8.3x figure is striking, but it applies specifically to the high-resolution case where token bloat is worst — the more modest gains at lower resolutions suggest the method earns its keep mainly when image quality is cranked up, which is exactly where dexterous manipulation tends to need it most.