A new pretraining method teaches AI models to recognize patterns in time series data instead of chasing noise.
Researchers introduced WinoTS, a self-distillation pretraining approach built specifically for time series data such as sensor readings, financial series, or energy usage. Rather than predicting the next data point or reconstructing missing values - the standard approach for time series models - WinoTS trains models to recognize the same underlying structure across different transformed views of a signal. It builds those views using wavelet-based time-frequency transformations instead of the cropping and jittering tricks borrowed from image-based self-supervised learning, which can distort a signal's timing or offer too little variation to be useful. In testing, WinoTS beat existing state-of-the-art baselines on long-term forecasting, zero-shot transfer across different domains, and unsupervised anomaly detection.
Most self-supervised time series models still waste capacity memorizing high-frequency noise rather than learning the repeating cycles that actually generalize. WinoTS borrows the self-distillation approach that has worked well for vision models, but swaps out spatial crops and jitter for time-frequency transforms better suited to signals, where naive cropping can scramble the timing of a repeating cycle. The researchers report that simple linear probes on WinoTS's frozen representations often beat fully supervised models trained from scratch, a sign the self-supervised features are doing real work.
If it holds up outside the paper's own benchmarks, it's a template other labs will likely borrow rather than a one-off trick.