• 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
Advanced Search
Zhan Hanglong, Liu Lantao, Kang Lianghuan, Cao Donggang, Xie Bing. An Elastic Scalable Stream Processing Framework Based on Actor Model[J]. Journal of Computer Research and Development, 2017, 54(5): 1086-1096. DOI: 10.7544/issn1000-1239.2017.20151044
Citation: Zhan Hanglong, Liu Lantao, Kang Lianghuan, Cao Donggang, Xie Bing. An Elastic Scalable Stream Processing Framework Based on Actor Model[J]. Journal of Computer Research and Development, 2017, 54(5): 1086-1096. DOI: 10.7544/issn1000-1239.2017.20151044

An Elastic Scalable Stream Processing Framework Based on Actor Model

More Information
  • Published Date: April 30, 2017
  • In the era of big data, stream processing has been widely applied in financial industry, advertising, Internet of things, social networks and many other fields. In streaming scenarios, the generation speed of stream data tends to be fluctuant and difficult to predict. If the streaming peak is larger than system capacity, the system may run slowly or even crash, which leads to job failure. If excessive resources are provided in case of streaming peak, there can be unnecessary waste under light load. In order to solve the matching problem between stream processing load and resources, stream processing system should be elastically scalable, which means that provided resources can be adjusted automatically according to the real-time change of stream flow. Although some researches have made great progress in stream processing, it is still an open problem that how to design an elastic scalable system. This paper introduces eSault, an elastically scalable stream processing framework based on Actor model. eSault firstly manages the processing units stratified hierarchically based on Actor model, and realizes scalability with two-layer routing mechanism. On this basis, eSault proposes an overload judgment algorithm based on data processing delay and light load judgment algorithm based on the data processing speed to efficiently allocate the resources, and achieve elastically scalable stream processing. Experiments show that eSault has good performance, and can achieve flexible scalability well.
  • Related Articles

    [1]Wang Yuwei, Liu Min, Ma Cheng, Li Pengfei. High Performance Load Balancing Mechanism for Network Function Virtualization[J]. Journal of Computer Research and Development, 2018, 55(4): 689-703. DOI: 10.7544/issn1000-1239.2018.20170923
    [2]Chen Qi, Chen Zuoning, Jiang Jinhu. MDDS: A Method to Improve the Metadata Performance of Parallel File System for HPC[J]. Journal of Computer Research and Development, 2014, 51(8): 1663-1670. DOI: 10.7544/issn1000-1239.2014.20121094
    [3]Wang Peng, Huang Yan, Li Kun, Guo Youming. Load Balancing Degree First Algorithm on Phase Space for Cloud Computing Cluster[J]. Journal of Computer Research and Development, 2014, 51(5): 1095-1107.
    [4]Shen Zhijun, Zeng Huashen. A Load Balanced Switch Architecture Based on Implicit Flow Splitter[J]. Journal of Computer Research and Development, 2012, 49(6): 1220-1227.
    [5]Liu Xinhua, Li Fangmin, Kuang Hailan, Fang Yilin. An Distributed and Directed Clustering Algorithm Based on Load Balance for Wireless Sensor Network[J]. Journal of Computer Research and Development, 2009, 46(12): 2044-2052.
    [6]Liu Ying, Wang Qirong, Sun Ninghui. Study of Loading Strategy in Shared-Nothing Event Stream Parallel Database Systems[J]. Journal of Computer Research and Development, 2009, 46(1): 159-166.
    [7]Wang Xianghui, Zhang Guoyin, and Xie Xiaoqin. A Load Balance Clustering Algorithm for Multilevel Energy Heterogeneous Wireless Sensor Networks[J]. Journal of Computer Research and Development, 2008, 45(3): 392-399.
    [8]Li Zhenyu, Xie Gaogang. A Load Balancing Algorithm for DHT-Based P2P Systems[J]. Journal of Computer Research and Development, 2006, 43(9): 1579-1585.
    [9]Tian Junfeng, Liu Yuling, and Du Ruizhong. Research of a Load Balancing Model Based on Mobile Agent[J]. Journal of Computer Research and Development, 2006, 43(9): 1571-1578.
    [10]Zhang Xiangquan, Guo Wei. A Bidirectional Path Re-Selection Based Load-Balanced Routing Protocol for Ad-Hoc Networks[J]. Journal of Computer Research and Development, 2006, 43(2): 218-223.

Catalog

    Article views (1419) PDF downloads (793) Cited by()

    /

    DownLoad:  Full-Size Img  PowerPoint
    Return
    Return