Researchers built a two-agent AI system that lets small social robots follow plain-English commands without custom code for every task.
MistyPilot is a new framework that lets researchers control the Misty social robot using natural-language instructions instead of hand-coded API calls. A Task Router agent decides whether an instruction should go to a Physically Interactive Agent, which handles sensor-triggered actions and direct skill invocation, or a Social Interaction Agent, which manages dialogue state and generates context-aware responses. The Social Interaction Agent also reuses prior results when it can, rather than regenerating output for every similar request. The team tested the system on five component-level suites tied to a physical Misty robot, plus a user study with 12 participants.
The results matter because programming interactive robots has historically meant manually wiring sensor events to skills and tracking task state by hand, tedious work that keeps social robots out of the hands of non-specialists. MistyPilot reportedly held up to 100 skills with high routing and binding accuracy, and showed lower variance than a single-agent version of the same system, suggesting the two-agent split does real work rather than just adding complexity.
Still, a 12-person user study and a single robot platform is a narrow test bed. The code isn't public yet, so the 100-skill claim and the usability gains will need outside verification before this looks like more than a promising lab result.