A new research paper treats a neural network's training run as a shape that changes over time, rather than a single frozen snapshot.
The researchers built temporal parameter graphs: snapshots of a multilayer perceptron's weights taken at many points during training. They then embedded those graphs in hyperbolic space, specifically the Poincare ball model, a geometry well-suited to tree-like, hierarchical structures. A metalearning system learns from this unfolding geometry while staying indifferent to the arbitrary ordering of neurons within each snapshot and the order of past snapshots themselves. In tests on regression and classification tasks, the resulting models tracked how network structure self-organizes as training proceeds.
Most explainability work examines a trained network as a finished object, measuring weights or activations at one point in time. This method argues that the path a network takes to get there carries information too, which could matter for debugging training instability or understanding why two networks with similar final accuracy behave differently. It is an argument for treating training as a process worth modeling, not just an input to a result.
The catch: these experiments ran on modest MLPs, not the billion-parameter models reshaping the industry. Whether hyperbolic trajectory-tracking holds up, or holds any practical value, at transformer scale is still an open question.