Abstract:
With the rapid growth of neural network model sizes and computational demands, traditional computing architectures face the challenges of the "power wall" and "memory wall". Although FPGAs are an ideal choice for neural network acceleration due to their high energy efficiency and reconfigurability, the resource limitations of a single FPGA struggle to meet the computational requirements of large-scale models. FPGA clusters can overcome this limitation but still face challenges such as cluster architecture design, distributed acceleration methods, and the lack of automated toolchains. To address these issues, this paper proposes TH-FCAD, an end-to-end automated deployment framework for neural network on FPGA clusters, enabling seamless compilation and deployment from PyTorch/TensorFlow-trained models to FPGA clusters. First, a TH-Express network-based two-level interconnect is proposed, combining a high-speed global GTH network with a local LVDS full-mesh interconnect to achieve high-bandwidth, low-latency cross-node communication. Second, A latency-balancing optimization strategy and model partitioning method are introduced to evenly split large-scale neural networks into multiple acceleration kernels while maintaining balanced inter-kernel delays. Finally, A communication-aware mapping algorithm automatically generates optimal placement schemes by jointly considering FPGA resource constraints and cluster communication topology, minimizing communication overhead while maximizing performance. Additionally, a complete automated toolchain is developed to streamline the entire deployment process. A 16-node FPGA cluster was built for evaluation. When deploying ResNet-8, SqueezeNet and ResNet-50, TH-FCAD achieves 0.069ms, 1.24ms and 1.36ms inference latency, respectively, with a peak computational performance of 6.42 TOPS. Compared to other multi-FPGA methods, it delivers a 28.4% performance improvement while outperforming CPU and GPU implementations.