An AI training method that ditches explicit reward functions just showed it can match or beat existing approaches across four benchmark tasks.
Researchers introduced Rank-Then-Act (RTA), a framework that trains a Vision-Language Model offline to judge how far along a task is — not by watching a clock, but by sorting shuffled video frames into the right order. The model learns what "progress" looks like purely from visual context. At training time, RTA measures how well the model's predicted rankings match the true frame order using Spearman rank correlation, a bounded, scale-invariant signal that sidesteps the usual headaches of reward calibration. The team tested it on discrete tasks (two Game Boy games) and continuous control benchmarks (PointMaze and MetaWorld), finding that a single pretrained scorer transferred across tasks without retraining.
The result matters because reward engineering is one of the quieter tax lines in reinforcement learning: humans spend significant time hand-crafting signals that tell an agent whether it's doing well, and those signals often break when the environment changes. RTA suggests you can instead extract a usable learning signal from raw demonstration video — the kind that already exists in abundance — and reuse it broadly. That's a meaningful reduction in the labor cost of deploying RL agents in new settings.
The approach still lives in benchmark territory, and benchmark performance rarely survives contact with messier real-world video; the gap between a MetaWorld simulation and a factory floor is not small.