Estimating how often an AI system fails in rare but dangerous ways just got meaningfully cheaper.
Researchers have published SCARCE, a method for quantifying the probability of rare AI failures without the massive computational budgets that brute-force testing requires. Classical approaches — called Subset Simulation — need a hand-crafted scoring function that essentially describes what failure looks like in advance, a requirement that limits how well the method transfers across domains. SCARCE sidesteps that by learning latent representations of failure regions directly from data, then using geometric distance to score how close any given input is to going wrong. The system also provides a mathematically grounded upper bound on failure probability, valid even when you stop testing early.
The practical payoff is significant for anyone trying to measure safety margins at scale. On a standard image misclassification benchmark, SCARCE achieved 400-500 times lower mean absolute error than grid-searched traditional Subset Simulation. On LLM jailbreak detection using Llama-Guard-3-8B's internal states, the method hit 2.6% mean relative error for adversarial fractions as low as 1-in-1,000 — a regime where brute-force sampling would be prohibitively expensive.
AI safety testing has long been hobbled by a fundamental mismatch: the failures that matter most are the ones that happen least, making them hardest to measure. SCARCE doesn't eliminate that tension, but it dramatically reduces the sample budget needed to get a credible estimate — which matters most for developers who need to certify system behavior before deployment, not just after an incident.