A research framework called skill coverage gives testers a way to measure whether AI agents actually follow the procedural instructions they are given.
LLM agents increasingly rely on reusable skills — stored instructions for how to handle recurring tasks. Current benchmarks score agents on whether they complete a task, not on whether they followed the skill instructions that were supposed to guide them. The new framework changes that by extracting behavior constraints from skill instructions, checking whether an agent trajectory triggers each constraint, and returning a Pass or Fail verdict for each one. Applied to SkillsBench, a public agent benchmark, the framework found that agents on the leaderboard cover only 38.66 to 45.51% of extracted skill behavior constraints on average.
That gap matters because a passing task score can hide systematic non-compliance with skill instructions — meaning developers have no reliable signal for why an agent failed or which part of a skill to fix. The researchers tested a simple repair: re-emphasizing only the original instructions that the agent had failed to follow, then rerunning the same tasks. That alone recovered 16% of previously failed tasks.
The result is a quiet indictment of how AI agents are benchmarked today: task-level pass rates look clean while the underlying skill adherence is largely untested. As agent skill libraries grow, the gap between "completed the task" and "followed the instructions" is likely to widen.