A new robotics framework gives vision-language models a rehearsal space to preview and fix their moves before a robot arm actually commits to them.
The system, called World Action Agent (WAA), is a multi-agent harness that lets vision-language models pilot robots directly, rather than just predicting constraints or writing control code. It builds a visual workspace around the robot's current interaction, automatically selecting contact-point camera views, then turns every planned action into an editable proposal the agent can preview and revise, with an in-view correction step that catches positioning errors before they cause a failure. WAA also builds up a library of skills from expert videos and human demonstrations, which a dedicated Skill Agent can call on, and uses its own interaction logs to train smaller models to run the same harness. On the LIBERO-Pro benchmark, WAA hit a state-of-the-art 75.6% average success rate, beating end-to-end vision-language-action models and code-generation approaches built on the same underlying model.
Most VLM-for-robotics work treats the model as an advisor that writes a plan or script and then hands off to a separate controller with no way to check its work before things go wrong. WAA instead keeps the model in the loop through execution, rehearsing and correcting in the same visual space it observes from. The clearest evidence this matters: fine-tuning a smaller model, Qwen3.5-9B, on WAA's own interaction traces pushed its out-of-domain success rate from 1.7% to 43.3%, a sign the harness captures reusable procedural knowledge rather than memorized benchmark tricks.
Those numbers come from simulated benchmarks like LIBERO-Pro and robosuite, not a warehouse floor or someone's kitchen - simulated pick-and-place tasks have a long history of not surviving contact with real clutter, lighting, and hardware noise, so the real test for WAA is still ahead.