Scientists routinely pick a surrogate model once, then stop checking it. A new study says that's the actual bug, not the fix.
Researchers built a benchmark called RegimeShift-Surrogates: eight analytic and dynamical tasks, four stationary or shifting regimes, ten held-out seeds, and eight model types spanning classical methods, multilayer perceptrons, and Kolmogorov-Arnold networks. The confirmatory run required 30,720 model fits scored across 3,200 deployment windows. Simply re-validating candidate models on each new batch and picking the current best produced a mean log regret of 0.091, versus 0.192 for the best fixed model chosen with hindsight - revalidation won in 26 of 32 task-scenario combinations (Holm-adjusted p = 0.0469). None of the fancier alternatives tested, including exponential smoothing, dual-timescale adaptation, Page-Hinkley change detection, or margin gating, beat plain revalidation, and delayed bias correction actively hurt results. The paper (arXiv:2609.29715, https://arxiv.org/abs/2609.29715, posted 2026-09-25) does not list individual author names or an institution in its abstract.
That's a quiet rebuke to a lot of MLOps orthodoxy. The industry default for handling model drift is to bolt on stateful monitoring - drift detectors, adaptive weighting, rolling retraining schedules - treating a deployed model like a patient on an IV drip. This result suggests that for scientific surrogate models, at least, that machinery is mostly overhead: just re-run validation on fresh data and swap in whichever candidate wins.
It also echoes a pattern seen elsewhere in machine learning deployment, where a simple baseline re-evaluated often beats a complex adaptive system tuned once and trusted forever. If your surrogate pipeline hasn't been re-validated since it shipped, consider this paper a nudge to check it before something quietly drifts.