AI web agents can finish a job, file a clean answer, and still be wrong — and most benchmarks won't catch it.
Researchers introduced Parallel WebBench, a benchmark of 1,679 verified records designed to test AI agents navigating the web across multiple parallel paths. They trained agents using a reinforcement learning method called GRPO on mixes of human and synthetic data. The best-performing model jumped completion rates from 50.7% to 96.0% compared to the baseline WebExplorer-8B — but binary accuracy, meaning whether the final answer was actually correct, stayed well behind. The gap between "finished" and "right" is the whole story.
The finding matters because most agent evaluations stop at whether the agent produced a well-formed answer, not whether that answer is grounded in what it actually found. This research names three specific failure modes — agents looping on searches they already ran, quitting too early on partial answers, and losing coherence when combining evidence — which gives builders something concrete to fix rather than a vague accuracy score to chase.
Synthetic training data helped agents stop abstaining and improved partial correctness, but it didn't close the gap between completing a task and getting it right. That's a useful caution for anyone betting that more synthetic data is the path to reliable agents.