Robotics research finally has a benchmark that refuses to pick sides between simulation and the real world.
RoboDojo is a new evaluation framework that runs manipulation policies through 42 simulated tasks and 18 physical-world tasks in a single unified system. The simulation side scores policies across five dimensions — generalization, memory, precision, long-horizon execution, and open-vocabulary instruction following — while the real-world component, called RoboDojo-RealEval, uses standardized hardware and remote cloud access to make physical testing reproducible. The team ran 30 existing policies through the full suite and published the results as a public leaderboard.
The sim-to-real gap is one of robotics' oldest headaches: a policy that aces a simulated kitchen can fail the moment it meets an actual countertop. Most benchmarks pick one environment or the other, which means either cheap-but-misleading simulation scores or expensive, hard-to-replicate lab runs that other teams cannot easily verify. RoboDojo's paired design forces both numbers into the open at once, making the gap itself measurable rather than something researchers quietly acknowledge and move on from.
Thirty policies is a reasonable starting set, but the leaderboard's value depends entirely on whether the robotics community adopts the standardized hardware stack — a coordination problem that has sunk more than a few would-be benchmark standards before it.