A Sliced Multi-Rail Interconnection Network for Large-Scale Clusters

In large-scale clusters, the design of interconnection network is facing greater challenges. Firstly, the increasing computing capacity of a single node requires the network providing higher bandwidth and lower latency. Secondly, the increasing number of nodes requires the network to have extremely better scalability. Thirdly, the increasing scale of system leads to worse performance of collective communication, which is harmful to the performance and scalability of applications. Fourthly, the increasing number of devices requires the network to have better reliability. As the performance of computing nodes keeps increasing, interconnection network has gradually become the bottleneck of large-scale computing system. However, switch chip, the core component of interconnection network, can offer limited aggregate bandwidth because of the constraint of physical processes and packaging technologies. With the co-design of network architecture and switch micro-architecture, this paper proposes a sliced multi-rail network architecture regarding the given aggregate bandwidth. Through mathematical modeling and network simulation, we studies the performance boundaries of sliced multi-rail network. Evaluation results show that the average latency of the short message (less than 128B)can be increased by more than 10 times.