高级检索

    TH-FCAD:面向FPGA集群的大规模神经网络自动化部署框架

    TH-FCAD: An Automated Deployment Framework for Large-Scale Neural Networks on FPGA Clusters

    • 摘要: 针对单颗FPGA资源受限难以加速大规模神经网络,且FPGA集群面临架构设计复杂、任务划分低效及自动化工具链缺失等挑战,提出面向FPGA集群的端到端自动化部署框架TH-FCAD(automatic neural network acceleration deployment framework on th-based FPGA cluster)。首先,构建基于TH-Express的2级互连集群架构,融合全局互连与局部互连,实现高带宽、低延迟的跨节点通信;其次,提出延迟平衡优化与模型切分方法,将大规模模型均匀切分为多加速核并保持核间延时均衡;最后,设计集群感知映射算法,联合FPGA资源约束与通信拓扑自动生成最优映射,以最小化通信开销。构建了完整的工具链,实现了从PyTorch/TensorFlow模型到集群的端到端自动化编译部署。基于16节点FPGA集群的评估表明,部署ResNet-8,SqueezeNet,ResNet-50时,推理延迟分别为0.051 ms,1.24 ms,1.36 ms,峰值算力达5.95TOPS;较现有多FPGA方案性能提升28.4%,且优于CPU与GPU平台。

       

      Abstract: With the rapid growth of neural network model sizes, single FPGA resources have become insufficient for accelerating large-scale models, while FPGA clusters face critical challenges, including complex architecture design, inefficient task partitioning, and the lack of automated toolchains. To address these issues, an end-to-end automated deployment framework for neural networks on FPGA clusters, named TH-FCAD(automatic neural network acceleration deployment framework on th-based FPGA cluster), is proposed. First, a two-level interconnect FPGA cluster architecture based on TH-Express is constructed. By integrating a high-speed global GTH(gigabit transceiver high-speed) network with a local LVDS(low voltage differential signaling) full-mesh interconnect, it achieves high-bandwidth and low-latency cross-node communication. Second, a latency-balancing optimization and model partitioning method is introduced to evenly partition large-scale neural networks into multiple acceleration kernels while maintaining inter-kernel delay equilibrium, which is essential for pipeline efficiency. Finally, a cluster-aware mapping algorithm is designed to automatically generate optimal placement schemes by jointly considering FPGA resource constraints and the cluster communication topology, thereby minimizing communication overhead. Furthermore, a complete automated toolchain is developed to streamline the entire deployment process, enabling seamless compilation from PyTorch/TensorFlow trained models to FPGA cluster configurations. Evaluations on a built 16 node FPGA cluster demonstrate that when deploying ResNet-8, SqueezeNet, and ResNet-50, TH-FCAD achieves inference latencies of 0.051 ms, 1.24 ms, and 1.36 ms, respectively, alongside a peak computational performance of 5.95 TOPS. Compared to state-of-the-art multi-FPGA approaches, the proposed framework delivers a 28.4% performance improvement and outperforms corresponding CPU and GPU implementations.

       

    /

    返回文章
    返回