A new controlled study pitted nine lightweight CNN architectures against each other under identical conditions — and the newest models didn't reliably win.
Researchers tested nine lightweight convolutional neural network packages across three standard image classification benchmarks: CIFAR-10, CIFAR-100, and Tiny ImageNet. They measured accuracy, parameter count, memory use, compute cost, and latency on both GPU and CPU hardware. EfficientNetV2-S posted the highest accuracy on two of three datasets, but EfficientNet-B0, an older design, came within 0.22 to 1.79 percentage points while using roughly 79% fewer parameters and 86% fewer floating-point operations. MobileNetV3-Small also outperformed the newer MobileNetV4-Conv-S on all three datasets, leading by between 0.99 and 2.55 accuracy points under random initialization.
The benchmark industry has a habit of declaring each new architecture generation a clear step forward, but a controlled comparison under fixed training budgets tells a more complicated story. For engineers shipping models on edge devices or cost-limited cloud infrastructure, architecture age alone is a poor guide to deployment suitability.
EfficientNet-B0 still trails its pretrained counterpart by up to 17.54 accuracy points after 100 epochs of scratch training — a reminder that the training recipe matters as much as the model design.