A new paper argues you can make Vision Transformers significantly cheaper to run without sacrificing accuracy — by baking geometry into the math.
Researchers behind "Quick ViTs" noticed that natural images repeat the same edges, corners, and textures across eight orientations — the rotations and reflections that form what mathematicians call the dihedral group D8. Standard ViTs ignore this structure entirely, recomputing equivalent information for every orientation. Quick ViTs implement their linear layers in the Fourier domain of this symmetry group, letting the network share computation across orientations rather than repeat it. The result: 5.33x fewer floating-point operations and up to 8x lower memory usage compared to ordinary linear layers.
Efficiency gains in deep learning often come with an asterisk — lower FLOPs don't always translate to faster wall-clock time on real hardware. The researchers are upfront about this: GPU arithmetic intensity limits how much the FLOP savings convert to actual throughput at smaller embedding sizes. The caveat fades as embedding dimensions grow, which matters because modern ViTs tend to run large. More importantly, Quick ViTs trained on ImageNet-1K under both supervised (DeiT-III) and self-supervised (DINOv2) regimes matched or beat baseline accuracy — meaning the efficiency isn't borrowed from performance.
The vision efficiency space is crowded, with approaches ranging from token pruning to linear attention approximations. What distinguishes this work is that the savings come from the architecture's mathematical structure rather than heuristics or approximations — a harder claim to make, and a more durable one if it holds at scale.