A new arXiv paper shows that a single unreported convention in how researchers measure transformer attention can flip their conclusions.
The researchers tested ten pretrained models across five model families and found that whether you keep or drop the "sink" token, the token, usually the first, that soaks up most of a row's attention probability, changes verdicts about which attention heads are most similar in 17 to 47 percent of cases. They also found that a widely cited clustering pattern in BERT's attention heads is not a real structural signal but an artifact of that convention. By treating each attention row as compositional data and splitting it into a sink term and a content term, they show most of what looks like entropy collapse during training is actually the sink token growing, not attention getting sharper: 30 percent of the drop at 70M parameters, 95 percent at 1B, and 79 percent at 1.4B. Pruning heads using the wrong convention, they add, can inflate a model's perplexity more than a hundredfold.
Interpretability work built on attention often assumes its metrics measure the model, not an unreported preprocessing choice. If normalizing away one token changes nearly half of your similarity verdicts, a fair amount of published head-clustering and head-pruning work may deserve a second look.
It is a reminder that in interpretability research, the metric is not the message. A paper that does not say whether it kept the sink token may not fully know what it measured.