Evolutionary algorithms designing neural networks hit a wall on a simple logic puzzle, unless one neuron breaks the mold and oscillates instead of behaving like the rest.
Researchers ran more than 4,500 experiments evolving neural networks in which each node gets its own activation function (the math rule that decides when a neuron fires), picked from a palette of 18 options, instead of the usual practice of applying one function to every neuron in the network. The setups were tested on Boolean logic, regression, and spatial classification tasks. On a four-input parity test, which asks a network to correctly flag whether four inputs sum to an odd or even number, networks built entirely from the nine monotonic functions (the smooth, always-increasing curves standard in deep learning) solved it 0% of the time. Give the evolutionary search access to even one oscillating neuron, and the success rate jumped to 100%.
The failure is not about what these networks can represent. Gradient descent, the training method behind ordinary deep learning, finds monotonic solutions to parity without trouble, and adding recurrence closes the gap too. The problem is specific to evolutionary search through sparse, indirectly encoded networks, which needs odd neuron types to find solutions that gradient-based training reaches easily.
It is a reminder that most artificial networks still run every neuron on the same rule, unlike real brains, which mix tonic, bursting, and fast-spiking cells freely, and that this uniformity, not just network shape, can decide what evolutionary search is able to find.