A new neural network skips gradient descent entirely, solving for its output layer with a single matrix calculation instead of training over many epochs.
The method, called Green-ELM, projects input data into a very high-dimensional random feature space, then solves for the output layer weights in one step using the Moore-Penrose pseudoinverse plus LU and Cholesky decomposition, skipping backpropagation entirely. On MNIST digit recognition, a 4,000-dimensional version hits 98.10% accuracy, and a Fashion-MNIST test reaches 86.63%. A leaner CPU setup at 2,000 dimensions trains in 1.5 seconds and still scores 97.15% accuracy, which the researchers say is 11.6 times faster than a comparable SGD-trained baseline. They also paired the method with a pretrained ResNet-18 to extract features first, showing the one-shot solver works beyond simple digit datasets.
The pitch is real-time edge AI: devices that need a working model immediately, not after a lengthy training run. If a closed-form solve can really substitute for backprop on constrained hardware, that changes the math for on-device personalization and fast retraining. The researchers also propose an "Empirical Scaling Hypothesis" linking accuracy to feature-space dimensionality and dataset complexity, suggesting they see this as more than a one-off trick.
Fast, one-shot output layers are not new. Extreme learning machines have solved problems this way since the mid-2000s. Green-ELM's contribution is scaling that idea to far higher dimensions and pairing it with modern pretrained backbones. But MNIST and Fashion-MNIST are the easiest benchmarks in machine learning, and the edge-AI pitch still needs proving on harder, real-world data.