Researchers have found a way to stack interpretability tools for AI vision systems instead of settling for one flat list of features.
A team built a cascaded sparse autoencoder system, called SAE++, that trains a second interpretability model on the internal weights of a first one, rather than on raw model output. The first-level sparse autoencoder learns simple visual features from a multimodal large language model's activations. The second level takes those learned feature directions as its own input, letting it find "concepts of concepts" without the overlap problems that come from nesting or Matryoshka-style tricks. The team tested the approach on three multimodal models - Qwen3-VL, Gemma-3, and LLaVA - across multiple visual datasets, and released the code on GitHub.
Sparse autoencoders have become a standard tool for peering inside large language models, but most versions produce one undifferentiated pile of features with no sense of hierarchy. SAE++ reports better coherence across levels of abstraction, and shows those concept groups can be used to steer a model's output as a group rather than feature by feature - useful for anyone trying to adjust what a vision-language model "sees" without retraining it.
Interpretability research keeps promising to make black-box models legible; a second autoencoder stacked on the first is a more organized map of the fog, not proof the fog has cleared.