Robots are notoriously bad students when the classroom is the real world, but a new training method suggests you can get away with teaching them less.
A team of researchers built a co-training recipe called SimHum that mixes two cheap-but-flawed data sources: simulated demonstrations, which show robot-valid actions but look nothing like the real world, and videos of humans performing tasks, which look real but come from the wrong body. SimHum extracts action patterns from simulation and visual patterns from human footage, then fine-tunes the combination on a small set of real-robot recordings, 80 episodes per task. Tested across four bimanual tabletop tasks, robots trained this way hit 62.5% success on scenes they had never seen before, a 53.7 percentage point jump over training on real data alone. In a separate test that controlled for how much time was spent collecting data, SimHum beat the best single-source pretraining approach by 35.0 percentage points.
The real bottleneck in robot learning has always been data: real-robot demonstrations require a physical robot, a physical setup, and a human operator, which makes them slow and expensive to scale. SimHum's numbers suggest that pairing cheap simulation with cheap human video can substitute for a lot of that expensive real-robot collection, without needing more robot time to close the gap.
Four tabletop tasks in a research paper is a long way from a robot doing your dishes, so treat the percentages as a promising lab result, not a shipped capability.