Differential privacy is supposed to protect everyone in a dataset equally. A new audit says it doesn't.
Researchers built a subgroup-targeted membership inference test and ran it against 32 proxy groups under three levels of attacker knowledge, across four datasets and three ways of generating synthetic text: DP-SGD fine-tuning, API-based prompting, and activation steering. They tested five different privacy budgets. The synthetic releases leaked which subgroup members were in the original training data, and standard membership inference attacks systematically underestimated how much. Differential privacy did cut average leakage at every budget tested, but the leakage that remained was lopsided: about a tenth of records accounted for roughly 40% of it.
That concentration is the real finding. DP's noise scrubbed more signal from random, low-risk records than from the high-risk ones attackers actually care about, meaning the aggregate privacy number can look reassuring while the riskiest people stay exposed. The researchers also found that whether a given record leaks depends on how the synthetic data was generated, not on some fixed property of the record itself - so you can't judge a single record's risk without knowing the release mechanism behind it.
Worth remembering next time a vendor cites a differential privacy budget as proof a synthetic dataset is safe: that number describes a worst-case ceiling averaged across everyone, not a guarantee for the specific people most likely to be re-identified.