Most popular tools for interpreting AI models may be giving researchers a distorted picture of what's actually happening inside them.
Sparse autoencoders, or SAEs, are widely used to extract interpretable concepts from the internal activations of large language models. They work by forcing a model's messy internal representations into a sparse set of discrete features - ideally ones that correspond to human-understandable ideas. A new paper argues that a core training hyperparameter called L0 - which controls how many features fire per token on average - is not the neutral dial researchers have assumed it to be. Set it too low, and the SAE blends correlated features together to compensate, producing outputs that look clean but are actually conflated. Set it too high, and the model finds degenerate shortcuts that also mix features. The authors introduce a proxy metric to help practitioners identify the correct L0 for a given training setup, validating it on toy models and confirming it aligns with peak performance on real LLM SAEs.
The implication is uncomfortable: most widely used SAEs, the paper finds, are running with L0 set too low. That means the interpretable features researchers have been pointing to as evidence of what a model "knows" or "represents" may be artifacts of miscalibration, not genuine reflections of the model's internal structure. For the AI interpretability field - which is already struggling to prove its findings translate to real safety insights - this is a methodological problem, not a minor tuning note.
Interpretability research has been pitched as a path toward understanding and auditing AI systems before they cause harm. If the primary measurement instrument is systematically biased, that pitch gets harder to make.