Researchers say the standard way of combining AI models is leaving performance on the table — and they have a framework to fix it.
Most model merging techniques blend the weights of several specialist models into one generalist using what mathematicians call convex combinations — basically weighted averages that stay inside the boundaries defined by the originals. A new paper introduces MERGEvolve, which treats that merged model not as the finished product but as a starting point. From there, an evolutionary algorithm perturbs the parameters with random noise, hunting for high-performing configurations that no weighted average of the originals could reach. The authors report that MERGEvolve matches or beats existing merging baselines on both single-task and multi-task benchmarks, and they show theoretically that it genuinely escapes the convex hull — not just as a claim, but as a provable property of the method.
Model merging has gained traction as a way to get multi-skill models without the compute cost of full retraining or fine-tuning. If you can stitch together a coding specialist and a reasoning specialist for free, why spend GPU hours training a combined model? MERGEvolve extends that logic one step further: you still skip retraining, but you add a cheap evolutionary search pass that can recover performance that naive blending loses.
The caveat the paper buries in its ablation studies is worth surfacing: the quality of that initial merged model matters a lot. Evolutionary search from a weak starting point is just expensive wandering — which means MERGEvolve is an amplifier, not a rescue operation for bad merges.