A new academic method wrings extra accuracy out of merged AI models without needing a single real training example.
Researchers describe Merge++, a post-hoc refinement step for model merging, the practice of combining several fine-tuned expert models into one multi-task model without retraining. Existing data-free merging methods only average or combine model weights directly, which means they never actually observe how each expert behaves on real inputs. Merge++ gets around that by inverting each expert checkpoint to synthesize representative images, then using those synthetic images to distill the experts' behavior into the merged model. It works as an add-on layer on top of any existing merging algorithm, from basic task arithmetic to newer spectral methods.
That matters because data-free merging has always been blind to actual model behavior, and gathering real task data to fix that is often impractical or restricted by data licensing. According to the paper, Merge++ improved results across every merging method tested, with average gains of 2 to 8 points and spikes as high as 25.9 points on individual configurations.
Whether synthetic, model-generated images hold up as a substitute for real-world data at larger scale is the open question this line of research will have to answer next.