Optimizing FPGA-based CNN accelerator for energy efficiency with an extended Roofline model

Abstract

In recent years, the convolutional neural network (CNN) has found wide acceptance in solving practicalcomputer vision and image recognition problems. Also recently, due to its flexibility, faster development time, andenergy efficiency, the field-programmable gate array (FPGA) has become an attractive solution to exploit the inherentparallelism in the feedforward process of the CNN. However, to meet the demands for high accuracy of today’s practicalrecognition applications that typically have massive datasets, the sizes of CNNs have to be larger and deeper. Enlargementof the CNN aggravates the problem of off-chip memory bottleneck in the FPGA platform since there is not enough spaceto save large datasets on-chip. In this work, we propose a memory system architecture that best matches the off-chipmemory traffic with the optimum throughput of the computation engine, while it operates at the maximum allowablefrequency. With the help of an extended version of the Roofline model proposed in this work, we can estimate memorybandwidth utilization of the system at different operating frequencies since the proposed model considers operatingfrequency in addition to bandwidth utilization and throughput. In order to find the optimal solution that has the bestenergy efficiency, we make a trade-off between energy efficiency and computational throughput. This solution saves18% of energy utilization with the trade-off having less than 2% reduction in throughput performance. We also proposeto use a race-to-halt strategy to further improve the energy efficiency of the designed CNN accelerator. Experimentalresults show that our CNN accelerator can achieve a peak performance of 52.11 GFLOPS and energy efficiency of 10.02GFLOPS/W on a ZYNQ ZC706 FPGA board running at 250 MHz, which outperforms most previous approaches.