Mosaic lets federated learners improve a shared model without exchanging any real data.
The paper introduces a two‑step process. First, each client trains a small generative model to mimic its own data distribution, then uses that model to create synthetic samples that never leave the device. Second, the system stitches together a mixture‑of‑experts from the client models and distills their combined knowledge into a single global model, using the synthetic data as a teaching set. A lightweight meta model aggregates expert predictions based on a handful of representative prototypes.
Why it matters: federated setups often suffer from both model and data heterogeneity, which makes a single global model hard to train. By generating privacy‑preserving data locally and pooling specialist models, Mosaic sidesteps the need for costly data alignment or heavy communication. The authors report consistent gains on image and multimodal benchmarks compared with prior state‑of‑the‑art methods.
The approach is a step toward practical federated AI, but it still relies on extra compute for generator training and a meta model, which could limit adoption on very low‑power devices.