AI/ robotics · ai research · latent action models · cross-embodiment transfer

Better Training Helps Robot AI Transfer Skills Across Bodies

Training robot AI to compare action similarity, rather than predict exact motions, transfers skills better between different robot bodies, researchers found.

A new training trick makes it easier for one robot's AI to learn from motions it only ever saw another robot perform.

Researchers tested latent action models, systems that learn a robot's motions as compressed "latent" representations from ordinary video, without needing action data tied to a specific machine. The catch is that identical motions from two different robots often end up encoded as different latents, which breaks skill transfer. The team compared two ways of using real robot action labels during training: one predicts the exact robot action from the latent, the other, action-similarity supervision, only trains latent actions to be as similar or different as the ground-truth action sequences they came from, without ever predicting an actual action. They tested this on RoboTwin 2.0, using two bimanual robots that each demonstrated a different set of tasks, then evaluated closed-loop on the tasks only the other robot had shown.

The big number here, more than double the cross-embodiment success rate, comes from using latent actions at all instead of training directly on raw robot actions, not from the new supervision trick specifically. Within that setup, though, action-similarity supervision still beat the older, prediction-based approach at transferring skills between robots, especially when similarity was measured on end-effector motion and compared directly across the two robots.

It is one benchmark with two robot types and a tidy academic setup, not the messy reality of a warehouse full of mismatched arms, so call this a solid data point for cross-robot training, not a finished solution.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →