ISSN 1000-1239 CN 11-1777/TP

• 网络技术 •

### 数据中心内Incast流量的网内聚合研究

1. (信息系统工程国防科技重点实验室(国防科学技术大学) 长沙 410073) (dekeguo@nudt.edu.cn)
• 出版日期: 2016-01-01
• 基金资助:
国家自然科学基金优秀青年科学基金项目(61422214)

### Aggregating Incast Transfers in Data Centers

Guo Deke, Luo Lailong, Li Yan, Hu Zhiyao, Ren Bangbang

1. (Key Laboratory on Information System and Engineering (National University of Defense Technology), Changsha 410073)
• Online: 2016-01-01

Abstract: Data transfers, such as the common shuffle and incast communication patterns, contribute most of the network traffic in MapReduce like working paradigms and thus have severe impacts on application performance in modern data centers. This motivates us to bring opportunities for performing the inter-flow data aggregation during the transmission phase as early as possible rather than just at the receiver side. In this paper, we first examine the gain and feasibility of the inter-flow data aggregation with novel data center network structures. To achieve such a gain, we model the incast minimal tree problem. We propose two approximate incast tree construction methods, RS-based and ARS-based incast trees. We are thus able to generate an efficient incast tree solely based on the labels of incast members and the data center topology. We further present incremental methods to tackle the dynamic and fault-tolerant issues of the incast tree. Based on a prototype implementation and large-scale simulations, we demonstrate that our approach can significantly decrease the amount of network traffic, save the data center resources, and reduce the delay of job processing. Our approach for BCube and FBFLY can be adapted to other data centers structures with minimal modifications.