Streaming anomaly detection just got a reality check.
Researchers ran what they call the first large-scale, apples-to-apples comparison of streaming and static time series anomaly detection (TSAD) methods. The setup mimics real-world deployment: a model trains on an initial batch of data, then gets evaluated online on both accuracy and computational cost as new data streams in. The team also built a new dataset, TSB-drift, made of real time series chosen specifically to include the kind of distribution drift that should, in theory, favor streaming methods. Across the board, static methods that don't update on the fly beat the streaming alternatives in most scenarios.
That's a problem for an entire subfield. Many streaming TSAD methods were adapted from generic streaming outlier detection research, which doesn't account for what actually makes a time series anomaly look like an anomaly. The paper argues that bolting incremental updates onto detection logic built for a different problem doesn't automatically produce something better suited to drifting data.
Continuous adaptation sounds good in a product pitch. This benchmark is a reminder that it still has to earn its keep against the boring static baseline.