Researchers have proposed a federated learning framework called Mosaic that trains a global AI model using only synthetic data, closing a privacy gap that conventional distributed training leaves open.
Federated learning lets organizations or devices train AI models together without pooling raw data — useful when regulations or competitive concerns make sharing impossible. The long-standing problem: participants run different hardware and hold different local datasets, which pushes each client's model in a different direction and produces a global model that fits no one's data well. Mosaic's solution is two-stage. Each client first trains a small local generative model to approximate its own data distribution, then samples synthetic examples from it — real data never leaves the device. Those synthetic samples feed a mixture-of-experts ensemble assembled from all the client models; a lightweight meta model learns to weight each expert's contribution before the whole ensemble is distilled into a single deployable global model. The code has been released publicly on GitHub.
The privacy angle is more meaningful than it sounds. Standard federated learning still leaks information through shared gradient updates — a class of attacks called gradient inversion can partially reconstruct training samples from those updates alone. A synthetic-data-only pipeline adds a structural barrier that gradient-level defenses cannot. The paper reports consistent outperformance against current state-of-the-art approaches on image and multimodal benchmarks.
The catch: a generative model trained on a small, skewed local dataset may produce synthetic noise rather than useful signal, and benchmark datasets tend to be tidier than real federated deployments. Whether Mosaic's gains hold when client data is genuinely messy is a question the paper doesn't answer.