A new diagnostic tool catches something ML practitioners have quietly sidestepped: two models can post identical accuracy and still explain their predictions through entirely different internal logic.
Researchers introduced EvoXplain, a framework that treats model explanations not as properties of a single trained model but as outputs drawn from an entire training pipeline. Testing on cancer genomics data — a TCGA pan-cancer cohort and a breast-cancer subtype task — they found that logistic regression models hitting 98% accuracy split into several distinct "explanatory basins," each pointing to different gene sets. Gradient-boosted trees, given the same data, converged on a single basin. The split emerged from varying regularization strength alone, with no change to the training data.
This matters most in genomics, where model explanations get published as biological findings — specific genes flagged as relevant to a cancer subtype. If different training runs point to different genes with equal confidence, the standard practice of averaging explanations across runs is masking a disagreement, not resolving one. EvoXplain also shows that a consensus explanation can correspond to none of the actual trained models.
Most interpretability research focuses on making single models more legible. This work asks a prior question: are the explanations stable across runs at all? For anyone who has published findings derived from ML pipelines, that is a more uncomfortable question than it might sound.