A research paper proposes replacing numeric feedback scores with written critiques to train AI agents on imperfect data.
Most imitation learning systems distill human feedback into a single number — a confidence score, a discriminator output, an importance weight. Researchers behind this paper argue that flattening feedback into a scalar throws away too much information. Their alternative, called language-critique imitation learning, builds text labels from demonstrations that describe what went wrong, why it was suboptimal, and what the agent should do differently. Those labels feed directly into training without being reduced back to a number. The team tested two variants — LC-BC for behavior cloning and LC-DP for diffusion policies — across navigation, manipulation, and gameplay tasks.
The practical stakes are real: most real-world training data is messy, and scalar signals routinely fail to explain why a behavior was poor. If language critiques can close that gap, it could make it cheaper to train capable agents without needing pristine expert demonstrations. The authors also provide a theoretical bound showing the method limits how far learned performance can stray from expert-level behavior.
Imitation learning from imperfect data has been a stubborn problem for years; prior approaches leaned on reinforcement learning hybrids or data filtering rather than richer supervision. Using language as structured feedback is a bet that the same properties making LLMs useful for reasoning can be recycled inside the training loop itself — a plausible idea that still needs to survive contact with messier real-world conditions than a benchmark provides.