A new robotics AI model predicts not just what happens next in a scene, but exactly where an object can be grabbed, poked, or moved.
AffordanceWAM, described in a paper posted to arXiv on September 23, 2026, forecasts future video frames alongside two new signals: a "Scalar Affordance" score and an "Affordance Heatmap" marking where interaction is possible in the scene. It runs on a pretrained video diffusion transformer split into a "World" expert and an "Action" expert, kept in sync through shared attention layers. Because human videos carry no robot action labels, they only train the World side, while a smaller set of real robot trajectories teaches the Action side what to actually do. On the RoboCasa and CALVIN ABC-to-D benchmarks, plus real-world tests, the combined system beat baselines trained on video alone or robot data alone, and results kept improving as more affordance-labeled human video was added.
Robot learning has long been bottlenecked by the cost of action-labeled demonstrations, since every clip requires a human physically operating an arm. AffordanceWAM's bet is that the far cheaper supply of ordinary human video can substitute for some of that data, provided the model has a way to translate "a hand did this" into "a gripper could do this here" - which is what the heatmap and scalar fields are for.
It's one paper's benchmark numbers, not proof the approach holds up outside the test suites the authors chose. Still, in my read, a method that gets measurably better every time you feed it more cheap video - rather than more expensive robot data - is the more interesting result here, even if it's not yet shown to be the industry's answer.