Researchers have built a statistical circuit breaker for AI systems that diagnose patients, designed to know when to stop testing and commit to an answer.
The tool, called Cros, sits on top of clinical diagnosis agents that decide what test to run next and when to stop. On a 1,834-episode benchmark built from MIMIC hospital records covering abdominal pain, it ranked which patient states were risky enough to keep investigating far better than existing methods: 0.853 AUROC versus 0.715 for a simple confidence score and 0.552 for the underlying agent's own stopping judgment. On a held-out 367-episode split, it cut selective diagnostic error to 16.9% at 78.8% coverage, versus 30.8% error when the agent was forced to run until it hit 100% coverage.
This matters because "let the model keep gathering data" is the default intuition for making AI diagnosis safer, and the paper shows that intuition is wrong. Forcing continuation was non-monotone: error was 28.3% using history alone but rose to 34.3% after a full workup, meaning more tests sometimes made the agent's judgment worse, not better. A calibrated stopping rule, not just a smarter agent, is what actually reduced error here.
The authors are unusually candid about the catch: a cheaper, unweighted version of their method beat the full system on this test split, and Cros itself only satisfied its own safety criterion in 6 of 20 development resplits. Because evaluation labels were seen during earlier development, they call this exploratory audit evidence, not a certified safety result -- a caveat worth remembering the next time a vendor claims an AI diagnostic tool is production-ready.