A research team has released CaP-X, an open framework for measuring how well AI coding agents can direct physical robots without relying on mountains of training data.
The project bundles three components: CaP-Gym, a simulation environment where agents control robots by generating executable programs; CaP-Bench, a benchmark that tested 12 frontier language and vision-language models across tasks of varying complexity; and CaP-Agent0, a training-free system derived from those findings. A fourth piece, CaP-RL, applies reinforcement learning with verifiable rewards and shows the resulting skills transfer from simulation to real hardware with minimal degradation. The core finding: models perform well when researchers hand-craft helpful abstractions for them, but performance drops sharply when that scaffolding is stripped away.
That scaffolding dependency matters because it is the central unsolved problem in deploying AI-driven robots outside controlled lab settings. The paper shows the gap can be narrowed - not by more training data, but by scaling what the researchers call agentic test-time computation: multi-turn interaction, structured error feedback, visual comparison, and ensembled reasoning working together at inference time.
The "Code-as-Policy" approach the paper builds on is a direct alternative to Vision-Language-Action models, which require vast labeled datasets to learn manipulation skills end-to-end. CaP-X is essentially an argument that writing code might be a cheaper path to reliable robot control - though the scaffolding findings suggest the shortcut still has a toll booth in the middle.