Teach an AI agent to search in parallel, then rewrite its system prompt, and it may quietly forget how.
Researchers studying reinforcement-learning-trained search agents found that models taught to fire off several search queries simultaneously, a technique known as parallel search that boosts both speed and accuracy, tend to learn that skill in a way that is tightly bound to the exact wrapper, or harness, they trained under. The harness is the surrounding scaffolding, things like the system prompt and tool-calling format, that a production app wraps around a model. Change the wording of that prompt and the agent often reverts to slower, one-query-at-a-time serial search, even though the underlying task hasn't changed. Simply training on a bigger variety of harnesses doesn't fix it either: a small set of harnesses gets mastered too quickly to teach the general skill, while a large set spreads the training signal so thin that no single harness fully sticks.
The fix, called CHART, rotates the training harnesses on a schedule, retiring ones the model has mastered and swapping in fresh, still-difficult ones to keep the learning signal alive. The result: the agent generalizes parallel search across 89% of harnesses it never saw during training, versus at most 5% for the best fixed-pool approach, and it even carries the skill to a new question-answering task and search environment, gaining 5.6 percentage points in accuracy. That matters because production teams rewrite system prompts constantly, and this work suggests a lot of the behavior we assume is 'learned' by an agent is actually just memorized surface pattern-matching to one particular prompt.
It's a useful reminder that a benchmark score earned under one prompt says less about an agent's actual competence than about how well it memorized that prompt's shape.