A new metric suggests the usual way of measuring redundancy in AI models has been missing the point.
Researchers studied Transformers, the architecture behind most of today's large AI models, to figure out which internal parts are truly redundant. The usual approach grades a component as redundant if it looks unimportant or looks similar to another component, but the authors say that's an indirect and unreliable proxy, since two components can look alike yet behave completely differently depending on what input they get. Their alternative, called Conditional Functional Substitutability (CFS), instead checks whether swapping one part for another produces the same output for the same input. Tested across different model families and data types, CFS showed that how a model's internal computation is organized shifts in predictable patterns as it scales up.
The more interesting finding is that substitutability and performance don't move together: models whose parts stay more independent from each other perform better at a fixed size, even though they look more redundant by older metrics. That's a concrete, testable explanation for why making a model bigger doesn't always deliver proportional gains. The team also used CFS predictions to decide, in real time, which parts of a network to skip, and that beat older importance-based shortcuts on the tradeoff between speed and accuracy.
It's a research result, not a shipping feature, but it points at something practical: the industry's add-more-parameters instinct may be masking inefficiency that a sharper redundancy test can actually find and cut.