A new training method squeezes more success out of AI agents tackling multi-step tasks, without collecting a single extra piece of data.
Researchers describe Matched-Context Group Policy Optimization (MCGPO), a technique for training language model agents that must string together many decisions before getting any feedback, like navigating a simulated house in ALFWorld or completing a purchase in the WebShop benchmark. Instead of adding a separate critic model, MCGPO reuses rollouts the agent already generated and compares sibling attempts that faced matching situations, using state summaries and task-rubric progress to judge which actions deserved more credit. Those matched comparisons feed into an existing critic-free method called GraphGPO, weighted by how strong the evidence for one action over another actually was. The team tested the approach across six combinations of environment and model backbone, each run three times with different random seeds.
Credit assignment, figuring out which of dozens of steps in a long task actually mattered, is one of the stubborn problems in training agents from sparse, end-of-task rewards. MCGPO's gains were uneven: tiny on ALFWorld, where baseline success was already above 92%, but far larger on WebShop running the newer Qwen3 models (3.96 and 5.16 percentage points) than on the older Qwen2.5 models (0.47 to 1.46 points). That gap hints that better credit assignment pays off more as the underlying model gets more capable, not less.
It's a single preprint tested on two lab benchmarks, not production agents, so treat the percentage-point gains as a result worth tracking rather than a reason to retrain anything today.