Deep learning's oldest workhorses just got a biology-inspired tune-up.
A new paper describes Purin, a mechanism that adds synaptic efficacy modulation to conventional convolutional neural networks. Instead of the fixed weights ANNs typically use during a training batch, Purin models both short-term and long-term changes in connection strength between neurons, using a time-interval-based abstraction rather than the discrete time-steps most architectures rely on. The method adds a bounded factor for temporary efficacy shifts, plus two additional weight matrices trained via standard backpropagation that capture the longer-term changes. The researchers tested Purin by adding it to three established CNN architectures: AlexNet, VGG11, and GoogLeNet.
Accuracy improved across every architecture and dataset tested, once the researchers controlled for confounding factors in the baselines. That is a meaningful proof of concept. AlexNet, VGG11, and GoogLeNet are decade-old designs that thousands of researchers have already optimized to the hilt, so squeezing out further accuracy gains without changing the underlying architecture is not trivial.
Purin belongs to the same family as spiking neural networks, which also try to import biological timing into ANNs. Unlike those models, it skips the computational overhead of simulating discrete time-steps, making synaptic-style modeling more plausible for mainstream architectures rather than a research curiosity.