A robotics research team has published Cortex, a framework designed to stop AI-driven robot arms from losing the plot halfway through a complex task.
Most current Vision-Language-Action models treat each moment in isolation — they look at what's in front of them now and act, with no memory of the broader plan. Cortex attacks that problem by inserting a structured interface between a high-level Vision-Language Model that plans and a low-level VLA that moves. The team codified manipulation into 32 canonical skill primitives and used that vocabulary to annotate more than 4,000 hours of open-source video and generate 30 hours of simulation data. An event-balanced sampling strategy helps the system handle the messy transitions between subtasks, where ambiguity tends to pile up.
The numbers are modest but meaningful: Cortex beats monolithic baselines by 3.1% on the Libero-long benchmark and 4.1% on RoboTwin. More interesting is the zero-shot result — the generalist planner completed multi-stage chemistry experiments it had never seen before, simply by pairing with a fine-tuned VLA. That's a capability the researchers say is out of reach for VLA fine-tuning alone.
The dual-system approach is not new — hierarchical planners have been a robotics staple for years — but the coordination gap between planning and execution has always been the hard part. Cortex's contribution is essentially a cleaner translation layer, and whether that holds up outside tidy lab benchmarks is the question every robotics paper has to answer eventually.