AI/ ai planning · llm agents · benchmarks · arxiv research

New Framework Gets AI to Write Verifiable Planning Code

A new agentic framework drafts and self-corrects planning code, lifting valid-plan generation to 89.6 percent, beating direct LLM planning's 74.5 percent.

A new AI system writes formal planning code instead of just guessing at multi-step plans.

Researchers built PDDLCoder, an agentic framework that turns plain-language planning problems into the Planning Domain Definition Language, or PDDL, the format symbolic planners use to check whether a plan will actually work. Instead of generating a PDDL specification in one pass, the system iterates: it drafts a specification, analyzes it, and refines it until a symbolic planner can execute it. The team also built NL-pddlgym, a benchmark of 711 planning problems across 23 domains with executable environments that automatically verify whether a generated plan is applicable. On a held-out test set of 106 problems spanning four domains the system had never seen, PDDLCoder produced applicable plans 89.6 percent of the time.

That beats prior PDDL-generation approaches, which the researchers say topped out at 45.3 percent in their tests, and it beats direct LLM planning, where a model just outputs a plan in natural language, which reached 74.5 percent. The gap matters because long-horizon planning is one of the places LLMs reliably fall apart, producing plans that sound plausible but skip steps or violate constraints. Routing the problem through a symbolic planner means a plan gets checked, not just phrased convincingly.

Still, four held-out domains and 106 test problems is a narrow proving ground, not a verdict on planning writ large. The real test is whether this holds up on messier domains that do not already come with a ready-made gym environment.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →