A paper out of arXiv argues that most neural network compression is attacking the wrong thing.
The standard playbook for shrinking a model — pruning weights, quantizing to INT8, factoring matrices with SVD — treats compression as a weight problem. The researchers behind this work treat it as a control theory problem instead. They reframe a trained network as a nonlinear dynamical system, then borrow two classical tests — controllability and observability — to measure which internal states actually influence the network's outputs and which are just along for the ride. States that are neither reachable nor observable get cut. What remains is what the authors call a "minimal realization" of the original network. On MNIST, a four-layer network shrank from state order 1024 to 277 — a 72.95% reduction in states and 73.48% reduction in parameters — while dropping accuracy only from 96.60% to 95.45%. On the harder CIFAR-10 benchmark, they cut parameters by 83.09% and shaved CUDA inference latency by roughly 3x, with accuracy holding essentially flat at 54.44% versus 54.45%.
The framing matters because it reframes what compression actually is: not a lossy approximation, but a principled elimination of states the network was never really using. That is a different claim than "our pruning hurts less," and if it holds up on larger, modern architectures, it could change how engineers think about model design from the start. The technique also produces a diagnostic — layer-wise redundancy scores — that could be useful even without compression.
The experiments run on MNIST and CIFAR-10, benchmarks that have not stressed the field in years. Whether controllability-observability analysis scales to transformer-sized models with billions of parameters, or survives contact with real-world tasks, remains entirely open.