Researchers have built a robot planning system that tries to anticipate what nearby humans are about to do, not just avoid bumping into them.
The system, called HINT-Plan, uses vision language models to watch a scene from a third-person camera and predict a person's high-level intention, like what task they're likely trying to complete. It converts that guess into a goal state, then feeds it into a planner alongside a hierarchical scene graph that maps out the environment's objects and layout. The planner turns all of that into a formal planning language so the resulting robot actions are actually executable. In photorealistic simulation testing, HINT-Plan hit a 69.71% success rate on joint human-robot task planning, beating baseline approaches by as much as 35.29 percentage points.
Most human-aware robotics work stops at collision avoidance: don't run into the person, steer around them, stay a safe distance away. HINT-Plan is aimed at a different problem: figuring out what a person is trying to accomplish and planning the robot's work so it doesn't get in the way or duplicate effort. That's the harder, more useful problem if robots are ever going to share kitchens, warehouses, or offices with people instead of just avoiding them.
A 70% success rate is progress, not a solved problem, and it's a simulation result, not a robot working in a real cluttered room with real lighting and real unpredictable people. The gap between photorealistic sim and messy reality has sunk plenty of promising planning papers before. Still, treating human intent as an input to task planning, not just an obstacle to dodge, is a reasonable next step for robots meant to work alongside us rather than around us.