A new reward-shaping method is trying to fix a basic flaw in how AI agents learn from trial and error.
Researchers built Dependency-Aware Reward Shaping, or DARS, which breaks a task into a graph of prerequisite steps instead of judging only the final outcome. An annotator marks whether each step verifies, breaks, or repairs part of that graph, and credit is discounted based on distance from the nearest broken prerequisite rather than zeroed out entirely. Tested on models from 1.5B to 8B parameters across five task families, DARS improved ALFWorld success by up to 10 points over the GiGPO method under the same training budget, raised scores on WebShop and Search-R1 question answering, and beat the OmniOPD baseline on tool-free reasoning tasks at 1.7B and 4B scale. The team also showed a distilled 8B model can do the step-annotation job as well as a larger API-based judge, and published the code on GitHub.
The point is that standard reinforcement learning treats a failed task as a total loss, even when an agent nailed three out of four steps before one mistake sank the whole episode. That is a particularly bad fit for the multi-step agentic tasks AI labs are racing to automate, like web shopping, tool use, and search-driven question answering, where progress is rarely all-or-nothing. By scoring steps against a dependency graph instead of a single pass or fail signal, DARS squeezes more useful training signal out of the same rollouts.
The gains are real but narrow: they are measured against specific baselines like GiGPO and OmniOPD on benchmark tasks such as ALFWorld and WebShop, not against the production training stacks that major AI labs actually run. Whether dependency graphs scale cleanly to messier, open-ended real-world tasks is the question this paper doesn't answer yet.