Turns out the wrapper around an AI coding agent matters almost as much as the model powering it.
A new arXiv paper builds a stripped-down coding harness with a fixed execution loop, then swaps out three components one at a time: planning, action space, and context management. The researchers ran 176 matched configurations across four different models on the SWE-Bench Verified and Terminal-Bench 2.1 benchmarks, testing five context-management strategies and four context-window budgets, plus targeted ablations of planning and tool access. The goal was to isolate which harness ingredients actually drive performance instead of judging entire agent systems as black boxes.
The results cut against some common assumptions in agent design. Context management mostly pays off by preventing the agent from simply running out of room, and cheap rule-based trimming beats fancier LLM-generated summaries once budgets get tight. Planning helps weaker models avoid mistakes but, for stronger models, mostly just cuts cost without moving accuracy. Models already good at bash do fine with nothing but a bash prompt, skipping the custom tool scaffolding that weaker models still need.
That last point is the one worth sitting with: a lot of the tooling built around coding agents may be compensating for model weaknesses that better models have already outgrown. Before bolting another layer of scaffolding onto an agent pipeline, it might be worth checking whether the model even needs it.