A new proof says fattening the very first layer of a neural network eventually makes its output boringly predictable, as long as the rest of the network stays the same size.
The paper, posted to arXiv, fixes the number of layers in a feedforward network and grows only the input layer, from n neurons toward infinity. It assumes two things: the weights connecting each layer to the next follow a distribution that settles onto a fixed curve as the network widens, and the n input values are drawn independently from the same continuous probability distribution. Under those conditions, the author shows there is a single number, call it psi, that the network's one output neuron converges to. As the input layer grows, the probability that the output lands within any margin of error around psi approaches 100 percent.
That is a concentration result, the same flavor of math that explains why flipping a million coins gives a predictable ratio of heads to tails even though any one flip is random. It matters because it gives theorists another rigorous foothold on why very wide networks behave more predictably than intuition suggests, this time without the usual trick of scaling every layer at once, as neural tangent kernel and mean field analyses typically do.
Worth noting: this is pure math, not an experiment on a trained model. Real networks are not initialized with input layers that stretch to infinity while everything else holds still, so how much this shapes actual training practice remains an open question.