Robots that finish the job aren't necessarily doing it safely — and most benchmarks have been too polite to say so.
Researchers have released SoftVTBench, a benchmark built in Isaac Sim that tests robotic manipulation of deformable objects under physical safety constraints, not just task completion. The benchmark tracks two separate scores: Goal Success (did the robot finish the task?) and Safety Success (did it do so without dropping the object or deforming it past a calibrated threshold?). It uses multi-view RGB cameras, tactile sensors with marker motion, and finite-element simulation to measure deformation from states the policy itself cannot see. Four task suites cover deformable and rigid objects across object-centric and spatial variation.
The gap between those two scores is the point. Experiments with pi0.5-based baselines show that a large share of successful task completions still fail on safety grounds — meaning the field has been grading robots on a curve. Tactile sensing narrows that gap meaningfully: Safety Success on object-centric deformable tasks climbed from 21.4% to 35.6% when tactile input was added, with no meaningful drop in Goal Success.
Most manipulation benchmarks reward getting the cup from A to B; SoftVTBench also asks whether the cup survived the trip — a distinction that matters considerably more outside the lab than inside it.