ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2017, Vol. 54 ›› Issue (11): 2534-2546.doi: 10.7544/issn1000-1239.2017.20151069

• 系统结构 • 上一篇    下一篇

面向大规模计算集群的多轨分割网络

邵恩1,2,元国军1,2,郇志轩1,2,曹政1,孙凝晖1   

  1. 1(计算机体系结构国家重点实验室(中国科学院计算技术研究所) 北京 100190); 2(中国科学院大学 北京 100049) (shaoen@ncic.ac.cn)
  • 出版日期: 2017-11-01
  • 基金资助: 
    国家重点研发计划项目(2016YFB0200300,2016YFGX030148,2016YFB0200205,2016GZKF0JT006);国家自然科学基金项目(61572464,61331008,61402444);国家“八六三”高技术研究发展计划基金项目(2015AA01A301);华为科研基金项目(YB2015070066);中国科学院战略性先导科技专项(XDB24060600)

A Sliced Multi-Rail Interconnection Network for Large-Scale Clusters

Shao En1,2, Yuan Guojun1,2, Huan Zhixuan1,2, Cao Zheng1, Sun Ninghui1   

  1. 1(State Key Laboratory of Computer Architecture (Institute of Computing Technology, Chinese Academy of Sciences), Beijing 100190); 2(University of Chinese Academy of Sciences, Beijing 100049)
  • Online: 2017-11-01

摘要: 在千万亿次规模的系统中,互连网络设计面临新的挑战.高性能节点和大规模是构建千万亿次系统的主要技术趋势,不断提高的节点计算能力要求互连网络提供更高的性能,而不断增大的规模又对互连网络扩展性提出了更高的要求.此外,随着系统规模的增大,集合通信的执行时间也在不断增长,制约了应用的扩展性,集合通信的性能需要得到进一步优化.除性能之外,可靠性问题也随着系统规模的扩大而日益严重.而随着计算节点性能的不断提高,互连网络逐渐成为限制大规模计算机系统性能的瓶颈.互连网络核心部件交换芯片可提供的聚合网络带宽受到工艺和封装技术的限制.从网络结构与交换机结构的协同设计思想出发,提出了一种在交换机聚合带宽限定的条件下多轨分割网络结构和设计方法.通过数学建模和网络模拟仿真,分析了该多轨分割网络的性能边界.评测结果表明:该网络可将短消息(长度小于128B)的平均延迟性能提高10倍以上,为以短消息占多数的数据中心网络的性能优化提供了新思路.

关键词: 大规模计算集群, 多轨网络, 带宽分割, 数据中心网络, 大规模网络模拟

Abstract: In large-scale clusters, the design of interconnection network is facing greater challenges. Firstly, the increasing computing capacity of a single node requires the network providing higher bandwidth and lower latency. Secondly, the increasing number of nodes requires the network to have extremely better scalability. Thirdly, the increasing scale of system leads to worse performance of collective communication, which is harmful to the performance and scalability of applications. Fourthly, the increasing number of devices requires the network to have better reliability. As the performance of computing nodes keeps increasing, interconnection network has gradually become the bottleneck of large-scale computing system. However, switch chip, the core component of interconnection network, can offer limited aggregate bandwidth because of the constraint of physical processes and packaging technologies. With the co-design of network architecture and switch micro-architecture, this paper proposes a sliced multi-rail network architecture regarding the given aggregate bandwidth. Through mathematical modeling and network simulation, we studies the performance boundaries of sliced multi-rail network. Evaluation results show that the average latency of the short message (less than 128B)can be increased by more than 10 times.

Key words: large-scale clusters, multi-rail network, bandwidth division, data center network, large-scale network simulation

中图分类号: