AGENTS.md files, the repo-level briefing documents meant to make AI coding agents smarter, mostly don't.
A study titled "Evaluating AGENTS.md: Are Repository-Level Context Files Helpful for Coding Agents?" (arXiv:2602.11988) tested the practice in two settings: SWE-bench tasks paired with LLM-generated context files, and a separate set of repository issues that already had developer-written context files attached. Across both, adding a context file did not generally raise task success rates. It did raise inference costs by more than 20 percent on average. The pattern held across multiple LLMs and coding agents, and for both AI-generated and developer-written files.
There was one exception: agents reliably followed explicit instructions in the files, like which test command to run or which style rules to respect. What didn't help was the part everyone recommends: a plain-English repository overview. That's the opposite of how most teams write AGENTS.md today, which lean heavily on describing architecture and purpose rather than giving direct instructions.
Every major agent vendor recommends shipping one of these files as a performance boost. This paper is among the first rigorous tests of that advice, and for now it looks more like a cost increase dressed up as an optimization.