TH-FCAD: An Automated Deployment Framework for Large-Scale Neural Networks on FPGA Clusters

With the rapid growth of neural network model sizes, single FPGA resources have become insufficient for accelerating large-scale models, while FPGA clusters face critical challenges, including complex architecture design, inefficient task partitioning, and the lack of automated toolchains. To address these issues, an end-to-end automated deployment framework for neural networks on FPGA clusters, named TH-FCAD(automatic neural network acceleration deployment framework on th-based FPGA cluster), is proposed. First, a two-level interconnect FPGA cluster architecture based on TH-Express is constructed. By integrating a high-speed global GTH(gigabit transceiver high-speed) network with a local LVDS(low voltage differential signaling) full-mesh interconnect, it achieves high-bandwidth and low-latency cross-node communication. Second, a latency-balancing optimization and model partitioning method is introduced to evenly partition large-scale neural networks into multiple acceleration kernels while maintaining inter-kernel delay equilibrium, which is essential for pipeline efficiency. Finally, a cluster-aware mapping algorithm is designed to automatically generate optimal placement schemes by jointly considering FPGA resource constraints and the cluster communication topology, thereby minimizing communication overhead. Furthermore, a complete automated toolchain is developed to streamline the entire deployment process, enabling seamless compilation from PyTorch/TensorFlow trained models to FPGA cluster configurations. Evaluations on a built 16 node FPGA cluster demonstrate that when deploying ResNet-8, SqueezeNet, and ResNet-50, TH-FCAD achieves inference latencies of 0.051 ms, 1.24 ms, and 1.36 ms, respectively, alongside a peak computational performance of 5.95 TOPS. Compared to state-of-the-art multi-FPGA approaches, the proposed framework delivers a 28.4% performance improvement and outperforms corresponding CPU and GPU implementations.