An LLM-based policy editor nearly matched a benchmark hotel-pricing strategy — but the finding that matters is what the experiment broke.
Researchers built a hotel-pricing simulator where an agentic editor receives only summary feedback: how its price distribution differs from a benchmark across time, inventory, and market regions. The editor never sees benchmark actions, source code, or reward numbers. Running across 5,000 held-out episodes, a multi-restart LLM editor achieved a RevPAR of 108.47 against the benchmark's 108.75 — a gap of -0.276, within the margin of uncertainty. It also cut episode composition distance nearly in half, from 1.153 to 0.609. Non-semantic proposers given up to 2,500 evaluations fell 8.77 to 14.57 RevPAR points short, suggesting the LLM's grasp of diagnostic structure is doing real work.
The catch comes from a tree-based editor that outperforms the LLM on behavioral alignment metrics yet earns revenue of only 98.91 — nearly 10 points below. That gap illustrates the paper's core argument: optimizing for aggregate behavioral distance can mislead. A policy that looks more similar to the benchmark on paper can still lose money. The authors argue evaluation should track whether diagnostic feedback produces reliable closed-loop outcomes, not just how closely actions mirror a reference.
The result lands at a moment when agentic AI systems are being quietly embedded in operational decisions — pricing, logistics, resource allocation — where the cost of a misaligned metric is real revenue, not a benchmark score. Goodhart's Law has entered the policy editor.