A research team has found a way to trace exactly what concept a language model is holding at any point in its reasoning chain — no training required.
The paper, posted to arXiv, argues that existing ways of representing concepts inside large language models — as single points, directions, or clusters — are too coarse and lack a principled explanation for why they form. The authors propose using Laguerre Geometry instead, defining concepts as regions called Laguerre-Voronoi cells. They then decompose each transformer layer into piecewise-linear operators, revealing that a token's path through the model is shaped by two things: a static tree of local flows, and a dynamic jump triggered by cross-token attention. From that decomposition they build Geometric Lens, a method that can read out the exact concept a hidden vector encodes at any layer without any hyperparameter tuning. A companion tool, Laguerre Autoencoder, visualizes both the decision geometry and the full reasoning path in a single 2D view.
Most interpretability research either explains behavior after the fact or requires retraining the model to add probes. A tool that works on an already-deployed model and makes no changes to it is more useful in practice — and the authors back the claim with a concrete test, showing Geometric Lens can recover the correct factual token when the model is fed misleading context. That last result is the one practitioners will want to stress-test.
Mechanistic interpretability is a crowded field right now, with competing geometric and circuit-based frameworks all claiming to illuminate what happens inside transformers — this one will need independent replication before it earns a place in the standard toolkit.