A new fine-tuning method lets CNNs retrain themselves on-device without hoarding memory.
Researchers introduced MemFLoRA (Memory-Floor LoRA), a low-rank adapter built specifically for convolutional neural networks rather than transformers. Instead of just shrinking the number of trainable parameters the way standard LoRA does, it targets the activation memory a model has to hold onto until the backward pass runs. The adapter freezes the down-projection, trains a scale-matched up-projection, and uses eval-mode backbone normalization so the only state it needs to save is the low-rank branch itself. Tested on three human-activity-recognition datasets and two CNN backbones across subject, body-location, and sensor-placement shifts, it cut saved-activation memory by 98.5-98.7% and peak training-state memory by 94.9-97.3% compared with full fine-tuning, while matching or beating existing CNN adaptation baselines.
Most on-device learning work borrows LoRA's math wholesale from language models, where parameter count is the bottleneck. CNNs have a different problem: activations pile up during the forward pass and have to stick around for backpropagation, and that is what actually blows the memory budget on constrained hardware. By redesigning the adapter around that constraint instead of retrofitting a transformer trick, this closes a gap that parameter-counting approaches kept missing.
The benchmarks here are all human-activity-recognition datasets, not image classification at scale, so the real test is whether this memory floor holds up outside that sensor-data setting.