OpenAI has identified 16 million distinct patterns inside GPT-4 using a scaled version of sparse autoencoders.
The technique involves training a secondary network to decompose a model's internal activations into as few active components as possible. Done at scale, the method tends to surface features that correspond to interpretable concepts rather than the tangled, overlapping representations embedded in raw weights. OpenAI says the 16 million patterns were extracted automatically, without hand-labeling by researchers. The effort is one of the larger mechanistic interpretability sweeps applied to a production language model.
Interpretability research sits at the intersection of AI safety and model auditing. Knowing what concepts a model internally represents gives researchers a way to look for dangerous or deceptive behavior before it surfaces in outputs. Anthropic has been the most visible actor in this space, publishing a series of papers that cataloged millions of features in its own models using similar methods. OpenAI's work at this scale suggests sparse autoencoders are settling in as the field's standard tool.
Sixteen million patterns sounds like a complete picture. Whether any of them explain the outputs researchers actually want to understand is a different question.