A research paper reframes gradient descent as a discrete dynamical system — and says that reframing explains training behaviors that standard theory largely hand-waves.
The paper studies fixed-step gradient descent through a hierarchy of exactly solvable models designed to preserve core features of deep learning: depth, width, data coupling, nonlinear activations, and stochasticity. The authors start with a simplified deep linear chain that yields a quartic loss and a cubic gradient map. Under large-depth scaling, the dynamics converge to a universal Ricker-type map — a well-studied object in nonlinear dynamics. That connection lets them treat the "edge of stability" (the point where loss starts oscillating rather than falling cleanly) not as a numerical glitch but as the first bifurcation of the training map itself.
The practical implication cuts against how most practitioners think about learning rates. If the learning rate is a structural parameter that determines which attractors the training dynamics settle into, then tuning it is not just a stability exercise — it is a representation-selection decision. The paper shows that finite-step oscillations actively contract factorization imbalance and push parameters toward flatter, more balanced representations, effects that the continuous gradient-flow approximation misses entirely.
Most ML theory still leans on gradient flow as its default model of training, largely because continuous-time analysis is cleaner. This paper is part of a smaller tradition trying to take the discrete, finite-step nature of real optimizers seriously — a tradition that has been gaining ground as phenomena like sharpness oscillations and catapult phases piled up without clean explanations under the old framework.