Researchers have built a compact robot monitoring model that judges task progress rather than just narrating what it sees.
The team behind PRIMO R1 (Process Reasoning Induced Monitoring) argues that most video models trained to watch robots are passive — they describe what's happening but can't assess whether the robot is actually on track. PRIMO R1 is a 7B model that uses outcome-based reinforcement learning to generate chain-of-thought reasoning about task progress, comparing the robot's current state against its starting point and goal. The researchers trained and evaluated the model on their own dataset and benchmark, called RoboFail, and ran zero-shot tests on real-world humanoid robot scenarios. On RoboFail, PRIMO R1 hit 67.0% accuracy — 6 percentage points above OpenAI o1 — and cut mean absolute error by 50% compared to specialized reasoning baselines.
The headline o1 comparison needs context: o1 and the 72B-scale general multimodal models PRIMO R1 also outperformed aren't purpose-built for robotic process supervision. A 7B task-specific model beating larger generalists at a task they weren't designed for isn't shocking — the more meaningful result is the 50% error reduction against specialized baselines. The architectural choice is worth noting: anchoring the video between initial-state and current-state images gives the model an explicit reference frame for measuring progress.
RoboFail is the team's own benchmark, not an independent industry standard, so how much to read into the leaderboard position depends on how much you trust the authors to design a fair test.