AI agents are getting better benchmark scores while repeating the same mistakes.
Researchers reviewed 27 papers published between 2023 and 2026, covering 19 benchmarks, and distilled the findings into a unified taxonomy of how large language model agents break down. They found six recurring failure clusters: errors in tool invocation and parameter handling, failures in planning and constraint satisfaction, degraded reasoning over long contexts, coordination problems in multi-agent setups, safety and security failures under adversarial conditions, and flawed measurement practices that make agents look better than they are. The paper claims to be the first synthesis to span all these dimensions at once.
The most pointed finding is that failures compound nonlinearly as tasks get longer — meaning a small weakness early in a task can cascade into full breakdown, not just a proportional dip in quality. Strong performance on individual sub-tasks also does not reliably translate into end-to-end success, which means the fragmented benchmarks labs use to tout progress may be measuring the wrong things entirely. Bolting on more scaffolding, the authors note, does not consistently help.
The one honest bright spot: agents have made real gains on single-turn tool use, short-horizon web navigation, and tightly scoped coding tasks — the exact conditions most leaderboards are designed to test.