Researchers have a new way to tell when an AI agent is confidently making things up.
The method, called GRUET, targets so-called ReAct agents: systems that alternate between reasoning in text and taking actions, like searching the web or calling a tool, across multiple turns. Give the same task to one of these agents twice and you can get wildly different paths to an answer. The researchers argue that inconsistency traces back to uncertainty piling up turn by turn, so they built a graph of the possible reasoning branches at each step and used the graph's complexity as a proxy for how shaky that step really is. Add up those per-turn scores and you get a confidence rating for the whole trajectory. The team tested it across nine different LLMs and five benchmarks, checking whether it could reliably flag which answers to trust.
This matters because "the agent seemed sure of itself" is currently the closest thing most people have to a trust signal, and it's a bad one. Confident-sounding chains of thought and correct answers are not the same thing, and as agents get handed real tasks like booking travel or managing infrastructure, an unreliable trajectory that looks fine on the surface is the failure mode that actually costs money. A working uncertainty score means a system could flag or halt a risky run before it acts, rather than after.
It's not a fix for hallucination, just a smoke detector for it. Whether it holds up outside benchmark conditions, where agents chase messier goals with real consequences, is the open question.