A training-free calibration layer consistently outpicked standard model selection on a major anomaly detection benchmark — no labeled test data, no retraining.
Researchers built a post-hoc layer that sits on top of frozen audio embeddings for DCASE Challenge Task 2, a yearly benchmark where systems must flag anomalies in audio from machine types they have never encountered. The core challenge: source domains have 990 normal training clips; target domains have 10. Across competing systems, source-domain accuracy and target-domain accuracy tend to move in opposite directions, so picking a good configuration is hard. The new layer applies per-domain quantile calibration balanced against a pooled estimate, then ranks candidate configurations using a label-free, cross-validated criterion derived entirely from normal training data.
On DCASE 2025, that criterion predicted official evaluation scores across a 45-configuration grid at Spearman rho = +0.91 (bootstrapped 95% CI: +0.83 to +0.95), while the conventional development-set score was effectively noise at rho = +0.06. Criterion-based selection raised the evaluation score from 55.83 to 59.34 on the standard grid — a jackknife confidence interval of 2.2 to 4.8 points. On an extended configuration grid the score reached 61.05, retrospectively ranking fourth among 35 teams.
In 2023 and 2024, a fixed full-equalization default matched or beat criterion-based selection — so the 2025 result depends on a frozen DCASE 2026 forward test to prove it was not an outlier.