A new study suggests language models don't just agree on where concepts live — they agree on how context moves them.
Researchers analyzed six families of large language models at varying scales and found that when context changes the meaning of a concept, the direction and magnitude of that shift follows a consistent pattern across models. The team formalized this using tools borrowed from neural population geometry: concepts become point-cloud manifolds, and contextual transformations become vector fields. A displacement structure learned from one model could predict held-out displacements in a separate model significantly above chance. The variance in these shifts correlates with lexical concreteness — abstract words move more unpredictably than concrete ones.
The finding matters because the dominant assumption in interpretability research has been that concept representations are static geometric objects. If context also transforms concepts in a shared, transferable way, that opens a path toward model-agnostic interpretability tools — probes or steering methods trained on one model that generalize to others without retraining. It also raises harder questions about whether this shared geometry is a property of language itself or an artifact of training on overlapping data.
Either way, the result is a quiet challenge to the idea that each model is its own black box.