AI tools meant to show why a vision model made a decision have been producing noisy, unreliable maps, and a new arXiv paper says it has found the specific flaw causing it.
In a new arXiv paper, researchers introduce ResLRP, short for Residual-aware Layer-wise Relevance Propagation, an extension of the existing Layer-wise Relevance Propagation technique used to highlight which parts of an image drove a vision transformer's output. The paper traces the noisy, unfaithful maps that standard LRP produces on vision transformers to cancellation effects inside residual connections, the pathways that carry information around each transformer layer, which cause attribution scores to blow up. The authors found these cancellations are substantially stronger in vision transformers than in language transformers. ResLRP's propagation rules explicitly correct for that cancellation, stay mathematically conservative, and provably limit how far the explosion can go, and causal channel-wise interventions confirmed the residual pathway, not a generic regularization effect, is the actual driver.
The numbers in the new arXiv paper are the part worth noting: 27-29% better localization and faithfulness scores up to 3.4x higher on modern vision-language models, tested across supervised, self-supervised, contrastive, hierarchical, and multimodal architectures. For anyone using attribution maps to audit or debug a vision model, that is the difference between a tool that roughly points at the right pixels and one that reliably does.
It is a useful reminder that a good chunk of what gets called AI interpretability is really just interpreting the quirks of the interpretability method itself.