Researchers have a new way to evaluate how well AI agent training signals work — without actually training anything.
When an AI agent operates over hundreds or thousands of steps, a single reward at the end tells the model almost nothing about which intermediate actions were good or bad. Dense supervision methods try to fix this by scoring each step along the way, but comparing them has been a mess: the standard approach bakes them into a full training run, which is expensive, slow, and tangled up with unrelated engineering choices. QVal sidesteps all of that. Given a state-action pair, it checks whether a method's scores align with the Q-values from a strong reference policy — essentially asking whether the signal correctly ranks actions by their actual worth. No training run required.
The practical upside is that researchers can now stress-test a dense supervision method in hours rather than weeks, and compare approaches from completely different methodological families on equal footing. That kind of common benchmark has been notably absent from this corner of AI research, which helps explain why the field has struggled to build on itself.
The first release, QVal-v1.0, covers 21 methods across seven methodological families, four environments, and six open-weight model backbones — over 1,200 evaluation experiments in total. The headline finding is quietly damning for the field: simple prompting baselines beat the more elaborate dense supervision methods consistently across model sizes, environments, and input types. It is a familiar pattern in AI research — the clever approach loses to the boring one — and it suggests that a lot of recent work on step-level agent feedback may be solving a problem that a well-worded prompt already handles.