The benchmarks used to test anomaly detection in time series data have a labeling problem - and a new open research tool is trying to fix it.
Researchers introduced Fun-TSG, a configurable time series generator built specifically to benchmark anomaly detection models on multivariate data. Existing datasets, the paper argues, lack fine-grained anomaly annotations and hide their internal dependency structures, making it hard to know whether a model is genuinely learning patterns or just getting lucky. Fun-TSG lets researchers either auto-generate data from randomized dependency graphs and anomaly types, or hand-craft scenarios using explicit equations - both modes expose ground-truth labels at the individual variable and timestamp level.
That granularity matters because most real-world anomaly detection fails quietly: a model flags something without explaining which variable triggered it or when. Without benchmarks that track those specifics, it is nearly impossible to compare classical statistical methods against newer neural approaches on equal footing. Fun-TSG is designed to make those comparisons honest.
Anomaly detection benchmarking has been a known weak spot for years - the same handful of aging datasets circulate through papers, and their fuzzy labels let mediocre models look competitive. A synthetic generator with transparent internals does not solve every problem, but it removes one convenient excuse for inflated benchmark numbers.