AI/ robotics · ai · simulation · benchmarks

Researchers Build a Fidelity Check for Robot Simulation Tests

ReVeal measures how closely a reconstructed 3D environment matches reality, and the closer the match, the more trustworthy simulated robot testing becomes.

A new benchmark grades a simulated copy of a room before anyone trusts it to test a robot.

Researchers built ReVeal, a framework that combines workspace reconstruction, reconstruction-level assessment, and matched closed-loop policy evaluation. It introduces two fidelity metrics: Novel-View Mesh Fidelity, which checks how the reconstruction holds up from new camera angles, and Annotated Planar Geometry Fidelity, which checks how well flat surfaces like tables and floors are captured, since manipulation robots depend on accurate geometry to grasp objects. The team also built PGSR-D, a reconstruction pipeline that adds monocular depth supervision to fill in geometry gaps when multiple camera views aren't enough. Across 8 test scenes, the metrics consistently ranked three reconstruction pipelines, 2DGS, PGSR, and PGSR-D, by fidelity, and across 8 humanoid manipulation tasks, three vision-language-action policies, GR00T, SmolVLA, and pi0.5, showed a consistent pattern: higher reconstruction fidelity tracked with closer agreement between simulated and real-world results.

Real-world robot testing is slow and expensive, so labs are leaning harder on simulated recreations of real workspaces to evaluate policies. That shortcut only works if the simulation is honest about what it approximates. A reconstruction with subtle geometry errors can make a mediocre policy look great in sim and then fail on actual hardware.

Call it the real-to-sim gap, the sim-to-real gap's quieter cousin, and one worth measuring before trusting any simulated leaderboard.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →