Ask an AI model to explain a failure that almost never happens, and it gets more talkative, not less, at least for a while.
Researchers built a free, local test harness running three open-weight models, qwen3:8b, llama3.1:8b, and mistral:7b, through a repeated tool-call task where one call failed at a controlled rate. They swept that failure probability from 0.2 down to 0.0001 across five prompting setups, ranging from demanding an explanation the instant a failure occurred to asking for nothing at all. Under the most demanding setup, explanation length rose as failures grew rarer, peaking at 28.4 words when failures hit about 1 in 20 calls, then leveled off at 17 to 19 words at the rarest rates rather than collapsing. Self-reported confidence climbed too, unevenly, from around 53 percent into the 70s and 90s.
That plateau instead of a collapse matters for anyone building AI agents meant to flag their own rare failures, because it suggests the prompting structure, not just the model, decides whether that self-monitoring signal survives as failures become vanishingly rare. The researchers also found llama3.1:8b would volunteer structured confidence reports without being asked, sometimes growing less confident as trials piled up, while the other two models did that only once, as boilerplate.
That is a useful reminder that "does the model notice something is wrong" and "does it explain that clearly" are two different questions, and a system prompt that never asks for an explanation may be quietly discarding a signal the model was willing to give for free.