Researchers have cut the energy cost of running neural networks on FPGAs by up to 4.2 times compared to GPU-based CUDA implementations - by pruning the lookup tables that replace traditional multiply-accumulate operations.
The work builds on MADDNESS, an existing algorithm that substitutes multiply-accumulate operations inside neural networks with faster lookup-table queries. The catch with MADDNESS has always been scalability: resource use balloons as problem size and precision requirements grow. The new architecture adds a pruning step that trims the lookup tables themselves, capping that expansion. Tested on two Xilinx FPGA boards across MNIST, CIFAR-10, and ImageNet benchmarks, the proposed unit delivered up to 1.6 times better throughput and 4.2 times better energy efficiency than CUDA equivalents. It also beat leading quantized network implementations by 1.8 times on energy while consuming 1.3 to 2.6 times fewer FPGA resources than unoptimized MADDNESS.
For anyone paying GPU inference bills at scale, compute costs are a real and growing line item, and quantization - shrinking numerical precision - is the dominant efficiency play right now. Lookup-table approaches take a different bet: skip the multiply entirely rather than shrink the numbers. FPGAs are most relevant for edge deployments where routing everything through a GPU server isn't practical.
The paper is candid that accuracy takes a "moderate" hit - and that word is doing a lot of work. How moderate is moderate will decide whether this moves out of research papers and onto actual production boards.