A new benchmark grades AI not just on catching anomalies in data, but on explaining them in plain language.
Researchers introduced SHAD, a benchmark built from 215 real-world, multivariate time series collected from distributed cloud storage systems operated by Scality. The data covers three families of anomalies at varying severity levels, with rich contextual annotations attached to each one. The team first benchmarked a range of existing anomaly detectors on the dataset, then tested whether measuring each dimension's contribution to an anomaly score produces an accurate attribution of what went wrong. Finally, they checked whether frozen large language models, used as-is with no extra training, can localize and interpret the anomalies in human-readable terms.
Most anomaly detection research stops at "did it find the glitch." For an engineer debugging a storage cluster, a detector that flags a spike without saying why is only half a tool. By building explainability and interpretability directly into the evaluation, SHAD pushes the field toward systems that hand over an answer, not just an alarm.
Whether off-the-shelf LLMs are actually any good at that interpretive leap is exactly the question the benchmark is built to test, not assume.