A new research paper argues that clinical AI models can leak patient information even when hospitals follow every data-privacy rule on the books.
Researchers behind a paper posted to arXiv propose a framework for assessing privacy risk in clinical foundation models - the large AI systems increasingly used for decision support, screening, and public health planning. The paper argues that these models can leak sensitive details from their training data, creating a path to patient re-identification that has nothing to do with how the underlying data was stored or shared. The authors map out realistic leakage scenarios across different deployment settings, then connect each one to the legal regimes - including HIPAA and GDPR - that would normally apply. They close with a mix of technical and legal mitigations meant to catch what those laws currently miss.
That's the real finding here: HIPAA and GDPR were built to govern how patient data is collected, stored, and shared, not how a trained model might reproduce it. A hospital can pass every audit on data handling and still deploy a model that memorized and can regurgitate identifiable patient details. As more health systems lean on foundation models for triage and screening, that blind spot gets more consequential, not less.
It's the same memorization problem researchers have flagged in general-purpose language models for years, just transplanted into a setting where a leak isn't an embarrassing tweet - it's a HIPAA violation waiting to happen.