Researchers have built a far larger, more rigorous test set for tracking targets as they cross between air and water, and paired it with a fix that helps an existing tracker recover faster when splashes and reflections scramble its view.
The team tackled two problems that have long dogged air-water robotic vision. First, judging whether a tracker is right is tough when splashes, bubbles, and light refraction make it impossible to pin down a target's true position from video alone. They solved that by syncing camera footage with motion-capture data, projecting the target's known shape onto each frame, and correcting for underwater distortion, then manually checking every result to build a 22,346-frame evaluation set. Second, they built what they call a Cross-Medium Recovery Policy, which feeds an existing tracking algorithm called MixFormerV2 three reference images (its original template, the best recent snapshot before things went wrong, and a motion-predicted crop of the frame), so it can relocate the target without retraining.
This matters because most tracking benchmarks assume the camera's view is a straight line to the truth, which breaks down the moment water gets involved. By refusing to trust image-only labels during underwater blind spots, this test set exposes a failure mode that polished demo reels usually hide. The fix itself is incremental: accuracy rose by just 2.95 points to a 49.90 score, and recovery time dropped from 55.3 to 49.3 frames, which says less about the fix failing and more about how hard splashes and bubbles really are to see through.
For now, this is a narrow, lab-grade patch bolted onto an existing tracker, not a general answer for the drones and amphibious robots that actually need to operate in both air and water.