A robot navigation system trained on 8.7 hours of data outperforms approaches that needed thousands of hours to train.
Researchers introduced GemNav, a visual navigation policy that skips the standard playbook for robot navigation AI. Instead of pairing a dedicated visual encoder with a custom action head and feeding it massive cross-embodiment datasets, GemNav takes a frozen multimodal large language model and fine-tunes only its language tower using Low-Rank Adaptation, a technique that modifies a small fraction of model weights. Waypoints and navigation signals are expressed as discrete tokens — the same kind the language model already produces — and an auxiliary loss keeps the metric geometry intact so the system still understands physical distances. Trained on a single open corpus, it stopped within 0.25 to 0.42 meters of target goals across 20 real-world trials in four distinct environments: an open carpark, an obstacle-filled carpark, an outdoor chemical yard, and an indoor warehouse.
The data efficiency claim is the headline here. The standard recipe for robot foundation models has required enormous training sets, which concentrates the field around a handful of labs with the compute and data pipelines to match. GemNav's results suggest the pretrained vision features inside a frozen multimodal model already carry enough spatial understanding that navigation can be layered on top cheaply — zero-shot transfer across four unseen environments on less than nine hours of training data is a meaningful data point against that assumption.
One result worth flagging: adding longer image histories improved offline benchmark scores but made no difference on actual robots, a reminder that leaderboard gains and real-world gains are still different currencies.