A new robot-learning system called JEPA-WAM teaches robots to follow instructions by first imagining what success should look like.
World Action Models pair pretrained video generators with action experts to control robots, but they tend to ignore the actual words in an instruction. Researchers argue this happens because training data pairs rich visual-action trajectories with thin, repetitive language labels, so the model learns to read the scene instead of the sentence. JEPA-WAM's fix uses an off-the-shelf, untrained text-to-image generator to produce several sample images of what the finished task should look like, then encodes those images with a frozen V-JEPA 2.1 model into compact goal tokens that steer both the video and action components through cross-attention. On a new real-robot benchmark, JEPA-WAM hit 87.3% success on familiar setups, 74.5% on unfamiliar scenes, and 80.9% on unfamiliar instructions, beating rivals pi0 and Fast-WAM by 10 to 27 percentage points.
This is a workaround for a data problem, not a new capability built from scratch. It repurposes an image generator nobody had to train for the job. The out-of-distribution numbers matter most, since real warehouses and homes rarely look like a lab bench, and this is one of the more concrete demonstrations that grounding language in imagined outcomes, rather than scene cues alone, helps a policy generalize.
Whether this scales past curated lab benchmarks to messier real deployments, where a text-to-image model's guess could be flatly wrong, is the harder question the paper does not yet answer.