AI/ ai · llm-agents · context-windows · prompt-engineering

Research Shows Bloated AI Instruction Files Backfire

A new study models AI agents' always-loaded instruction files as a shelf-space problem, showing that more rules can make agents perform worse, not better.

Stuffing your AGENTS.md file with every rule you've ever thought of is quietly making your AI agent worse at its job.

A new paper formalizes the problem of curating always-loaded context files, like AGENTS.md, as a "capacitated assortment problem" - the same kind of math retailers use to decide what fits on a limited shelf. Each instruction eats into a fixed token budget and adds a small setup cost every session, while adding one instruction never improves compliance with the others already loaded. The authors prove there's a hard ceiling on how large a well-curated instruction file should ever get, regardless of how many candidate rules exist, and show that appending every individually useful instruction can leave you with a file that performs arbitrarily worse than a smaller, deliberately chosen subset. They also flag a trickier problem: feedback on removed instructions is censored, meaning you only learn a rule mattered once its absence causes a visible failure, so deleting rules that look unused can quietly strip out ones that were actually helping.

This matters because the default behavior for most teams maintaining these files is to append whenever something goes wrong, on the assumption that more guidance can't hurt. The paper's experiments, run on real-world context files, show that assumption is backwards: irrelevant instructions measurably reduce how well language models follow the rules that remain.

It's shelf-space economics applied to prompt engineering, and the conclusion that less can beat more is one anyone tending a sprawling instructions file should take personally.

TR

The Revision

Written by an AI system from the public sources credited above. How we write →