A new report ties simulated robot testing directly to matching trials on real hardware, using one shared record for each run instead of separate benchmarks for each stage.
DeepInsight II builds on the original DeepInsight framework, which standardized evaluation around three ideas: task, resource, and result. That first report mostly covered foundation models; navigation, manipulation, and whole-body control were tested only in simulation. This second report fills in that gap. It reproduces results from released model checkpoints across two navigation and four manipulation benchmarks, and introduces MotionBench, which runs four existing whole-body controllers through the same workload and metrics. A subset of those controllers then moves from simulated runs to matched real-robot trials that share a parent trace, so simulated and physical results can be compared directly instead of reconciled across separate toolchains. The report also defines five labeled failure points across the perception-to-control pipeline, each tied to a specific fix, and tests them on physical robots.
This matters because embodied AI evaluation has lagged far behind the standardized, leaderboard-driven benchmarks that language models get. That gap is part of why robots that ace simulation still stumble in the real world, and why comparing 'sim-to-real gap' claims across papers has mostly been guesswork. Tying simulated and physical results to the same trace turns that gap into a specific, measurable number rather than an anecdote in a discussion section.
Whether other labs adopt this shared-trace approach, or keep publishing benchmark scores next to unrelated real-robot demo videos, will say more about the state of robotics accountability than any single paper can.