Researchers have derived a formula that predicts exactly when AI interpretability tools will blur a concept and its subtypes into one confused feature.
Sparse autoencoders (SAEs) are a popular tool for peering inside large language models, breaking their internal activations down into sparse "features" that are supposed to map to single, human-understandable concepts. A known failure mode called feature absorption sometimes merges a parent concept - fruit, say - with its children - apple, banana, pear - into one shared direction instead of keeping them separate. A new paper derives a closed-form "phase boundary" describing exactly when that merger becomes the mathematically cheapest way to encode a set of active child concepts, based on how many children are active and how much residual signal remains. The researchers then tested the prediction, an evaluation protocol they call HiPACE, against real SAE dictionaries trained on Pythia-160m, using WordNet word families and a locked holdout set to avoid cherry-picking results.
Feature absorption has been a nagging, poorly understood glitch for anyone trying to trust SAE-based interpretability claims - it is part of why "interpretable" AI features do not always mean what they say. This paper does not eliminate the problem, but it gives researchers a way to predict, before training, when a given setup will merge concepts rather than split them, turning a mysterious artifact into a calculable tradeoff.
The equation held up within 15 percent accuracy in synthetic tests and matched real-model behavior with correlations as strong as -0.93, tighter agreement between theory and messy neural-network reality than interpretability research usually manages.