A new agent for robots splits "understand the room" from "figure out what to do," and household task success jumps past 90 percent.
Researchers built a neurosymbolic agent that pairs a vision-language model with formal step-by-step planning. First, the model explores a simulated home and turns what it sees into symbolic facts, like what's in a drawer or whether a stove is on. Then a planner built on PDDL, a decades-old formal planning language, only considers moves the rules say are actually possible, and Monte Carlo tree search picks the best sequence among them. Tested on the VirtualHome and ALFWorld simulators, versions running open models between 4 billion and 27 billion parameters cleared 90 percent success in both, and even the smallest version beat a 27B model that tried to plan directly from images.
Plain language-model planners are prone to inventing actions that do not work in the real environment, like opening a fridge that is already open or grabbing an object that is not where the model thinks it is. This paper's fix is not a bigger model. It is separating perception from planning so a formal checker can throw out anything the environment rules forbid before it is ever executed. In ALFWorld, symbolic constraints alone or search alone solved under a third of tasks; combined, they solved more than 95 percent, so neither piece is doing the work alone.
When the system does fail, the study traces the errors to state acquisition, meaning what the agent sees and grounds, not to the planner itself. That just moves the hard problem from thinking back to looking, and looking is still the part robotics has not solved.