Synthetic tabular data looks convincing on paper — until you ask whether it actually respects cause and effect.
A team of researchers has released a benchmark framework designed to expose a blind spot in how synthetic tabular data is evaluated. Current tests lean on low-order statistics — things like column means and pairwise correlations — and downstream machine learning performance. The new framework instead probes high-order structural causal information: the multivariate chains of cause and effect that simple correlation checks routinely miss. The researchers open-sourced the code, data, and documentation to invite broader use.
This matters because synthetic data is increasingly used as a stand-in for sensitive real-world records in healthcare, finance, and other regulated industries. If the synthesis models quietly scramble causal relationships while preserving surface statistics, any model trained on that data inherits a distorted picture of reality. The benchmark gives practitioners a controlled way to measure that distortion before it reaches production.
The results from testing current state-of-the-art tabular synthesis models are not flattering: the paper reports significant gaps between ideal and actual causal fidelity. That finding puts the field's standard evaluation metrics in an awkward position — models can score well on existing benchmarks while quietly failing at the thing the synthetic data is supposed to replicate.