高级检索

    TH-FCAD:一个面向FPGA集群的大规模神经网络自动化部署框架

    TH-FCAD: An Automated Deployment Framework for Large-Scale Neural Networks on FPGA Clusters

    • 摘要: 随着神经网络模型规模和计算需求的快速增长,传统计算架构面临严重的功耗和存储瓶颈。尽管FPGA因其高能效和可重构性成为神经网络加速的理想选择,但单FPGA的资源限制难以满足大规模模型的计算需求。FPGA集群可突破这一限制,但仍面临集群架构设计复杂度高、任务到集群的划分缺乏高效方法,以及全流程自动化工具链缺失等挑战。为此,本文提出面向FPGA集群的神经网络端到端自动化部署框架TH-FCAD,实现了PyTorch/TensorFlow训练模型到FPGA集群的自动化编译部署。首先,提出一种基于天河高速互连网络TH-Express的两级互连FPGA集群架构,通过设计全局GTH高速网络与局部LVDS全互连的分层通信机制,实现了高带宽、低延迟的跨节点通信;其次,提出延迟平衡优化策略与模型切分方法,将大规模神经网络均匀切分成多个加速核,并保持核间的延时平衡;最后,提出集群感知的加速核映射算法,综合考虑FPGA资源约束和集群通信拓扑,并自动生成最优映射方案,在满足资源约束的同时最小化通信开销,提升系统性能。此外,构建了完整的自动化工具链,实现了部署全流程的自动化。本文构建了一个16节点的FPGA集群,并在该集群上进行了实验。结果表明,在部署ResNet-8、SqueezeNet和ResNet-50模型时,TH-FCAD能够分别实现0.069ms、1.24ms和1.36ms的推理延迟以及6.42TOPS的峰值计算性能,相比现有的其它多FPGA实现性能提升了28.4%,且优于相应的CPU、GPU实现。

       

      Abstract: With the rapid growth of neural network model sizes and computational demands, traditional computing architectures face the challenges of the "power wall" and "memory wall". Although FPGAs are an ideal choice for neural network acceleration due to their high energy efficiency and reconfigurability, the resource limitations of a single FPGA struggle to meet the computational requirements of large-scale models. FPGA clusters can overcome this limitation but still face challenges such as cluster architecture design, distributed acceleration methods, and the lack of automated toolchains. To address these issues, this paper proposes TH-FCAD, an end-to-end automated deployment framework for neural network on FPGA clusters, enabling seamless compilation and deployment from PyTorch/TensorFlow-trained models to FPGA clusters. First, a TH-Express network-based two-level interconnect is proposed, combining a high-speed global GTH network with a local LVDS full-mesh interconnect to achieve high-bandwidth, low-latency cross-node communication. Second, A latency-balancing optimization strategy and model partitioning method are introduced to evenly split large-scale neural networks into multiple acceleration kernels while maintaining balanced inter-kernel delays. Finally, A communication-aware mapping algorithm automatically generates optimal placement schemes by jointly considering FPGA resource constraints and cluster communication topology, minimizing communication overhead while maximizing performance. Additionally, a complete automated toolchain is developed to streamline the entire deployment process. A 16-node FPGA cluster was built for evaluation. When deploying ResNet-8, SqueezeNet and ResNet-50, TH-FCAD achieves 0.069ms, 1.24ms and 1.36ms inference latency, respectively, with a peak computational performance of 6.42 TOPS. Compared to other multi-FPGA methods, it delivers a 28.4% performance improvement while outperforming CPU and GPU implementations.

       

    /

    返回文章
    返回