Continuous Learning-Based Task Demand Understanding and Scheduling Method for Video Internet of Things

Xu Dongzhu; Zhou Anfu; Ma Huadong; Zhang Yuan

doi:10.7544/issn1000-1239.202440403

Journal of Computer Research and Development > 2024 > 61(11): 2793-2805. > DOI: 10.7544/issn1000-1239.202440403 CSTR: 32373.14.issn1000-1239.202440403

Xu Dongzhu, Zhou Anfu, Ma Huadong, Zhang Yuan. Continuous Learning-Based Task Demand Understanding and Scheduling Method for Video Internet of Things[J]. Journal of Computer Research and Development, 2024, 61(11): 2793-2805. DOI: 10.7544/issn1000-1239.202440403

Citation:

PDF (2724 KB)

Continuous Learning-Based Task Demand Understanding and Scheduling Method for Video Internet of Things

Xu Dongzhu^{1, 2,},
Zhou Anfu^{1, 2},
Ma Huadong^{1, 2, ,},
Zhang Yuan³

1.
School of Computer Science (National Pilot Software Engineering School), Beijing University of Posts and Telecommunications, Beijing 100876
2.
State Key Laboratory of Networking and Switching Technology (Beijing University of Posts and Telecommunications), Beijing 100876
3.
China Telecommunications Corporation, Shanghai 200125

Funds: This work was supported by the Innovation Research Group Project of the National Natural Science Foundation of China (61921003) the National Natural Science Foundation of China for Young Scientists (6240072128), and the China National Postdoctoral Program for Innovative Talents (BX20240046).

More Information

Author Bio:
Xu Dongzhu: born in 1996. Postdoc. His main research interests include VIoT and time-sensitive networking

Zhou Anfu: born in 1981. PhD, professor. His main research interests include IoT and mobile computing

Ma Huadong: born in 1964. PhD, professor. His main research interests include IoT, sensor network, and multimedia computing

Zhang Yuan: born in 1986. Master. Her main research interests include AI and IoT, and image/video coding
Received Date: May 30, 2024
Revised Date: August 04, 2024
Available Online: August 13, 2024

Graphical Abstract

Abstract

Abstract

Efficient scheduling between cloud-network resources and video tasks is crucial for the performance of video Internet of things (VIoT) applications. However, the current scheduling algorithms used in operational VIoT systems are insufficiently adaptable to differentiated task demands and highly dynamic changes in cloud-network resources, resulting in poor performance of VIoT applications. To overcome the aforementioned problem, we propose a continuous learning-based task demand understanding and scheduling method for VIoT (CLTUS) Unlike traditional heuristic or machine learning-driven scheduling algorithms, CLTUS integrates the continuous learning into the matching between cloud-network resources and video task demands. Specifically, it first employs a continuous learning framework to accurately comprehend various video task demands. Subsequently, based on the dependency relationships among video tasks, it achieves an optimal match between tasks and servers, thereby refining the scheduling of cloud-network resources. Finally, the proposed method is deployed on a software-defined VIoT experimental platform. Compared with conventional methods, CLTUS not only improves the average processing efficiency of video tasks by 127.73% but also increases the balanced utilization rate of cloud-network resources to 67.2% on average, effectively improving the performance of VIoT applications.
- video Internet of things,
- continuous learning,
- task demand understanding,
- resource scheduling,
- operational network measurement

FullText(HTML)

References (25)

References

[1]	Zhang Huanhuan, Zhou Anfu, Hu Yuhan, et al. Loki: Improving long tail performance of learning-based real-time video adaptation by fusing rule-based models[C]//Proc of the 27th Annual Int Conf on Mobile Computing and Networking (MobiCom’21). New York: ACM, 2021: 775–788
[2]	Xu Dongzhu, Zhou Anfu, Wang Guixian, et al. Tutti: Coupling 5G RAN and mobile edge computing for latency-critical video analytics[C]//Proc of the 28th Annual Int Conf on Mobile Computing and Networking (MobiCom’22). New York: ACM, 2022: 729–742
[3]	Liu Luyang, Li Hongyu, Gruteser M. Edge assisted real-time object detection for mobile augmented reality[C]//Proc of the 25th Annual Int Conf on Mobile Computing and Networking (MobiCom’19). New York: ACM, 2019: 1–16
[4]	Machina Research. The number of IoT connections worldwide is expected to reach 27 billion by 2025[EB/OL]. [2016-08-09]. https://www.199it.com/archives/505556.html
[5]	Tang Xiongyan, Cao Chang, Wang Youxiang, et al. Computing power network: The architecture of convergence of computing and networking towards 6G requirement[J]. China Communications, 2021, 18(2): 175−185 doi: 10.23919/JCC.2021.02.011
[6]	Coronado E, Khan S N, Riggio R. 5G-EmPOWER: A software-defined networking platform for 5G radio access networks[J]. IEEE Transactions on Network and Service Management, 2019, 16(2): 715−728 doi: 10.1109/TNSM.2019.2908675
[7]	Coronado E, Yousaf Z, Riggio R. LightEdge: Mapping the evolution of multi-access edge computing in cellular networks[J]. IEEE Communications Magazine, 2020, 58(4): 24−30 doi: 10.1109/MCOM.001.1900690
[8]	Grandl R, Ananthanarayanan G, Kandula S, et al. Multi-resource packing for cluster schedulers[J]. ACM SIGCOMM Computer Communication Review, 2014, 44(4): 455−466
[9]	Liu Libin, Xu Hong. Elasecutor: Elastic executor scheduling in data analytics systems[C]//Proc of the ACM Symp on Cloud Computing (SoCC’18). New York: ACM, 2018: 107–120
[10]	Zhang Haitao, Geng Xin, Ma Huadong. Learning-driven interference-aware workload parallelization for streaming applications in heterogeneous cluster[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(1): 1−15
[11]	姜玉龙,东方,郭晓琳,等. 算力网络环境下基于势博弈的工作流任务卸载优化机制[J]. 计算机研究与发展,2023,60(4):797−809 doi: 10.7544/issn1000-1239.202330021 Jiang Yulong, Dong Fang, Guo Xiaolin, et al. Potential game based workflow task offloading optimization mechanism in computing power network[J]. Journal of Computer Research and Development, 2023, 60(4): 797−809 (in Chinese) doi: 10.7544/issn1000-1239.202330021
[12]	Rebuffi S, Kolesnikov A, Sperl G, et al. iCaRL: Incremental classifier and representation learning[C]//Proc of 2017 IEEE Conf on Computer Vision and Pattern Recognition (CVPR). Piscataway, NJ: IEEE, 2017: 5533−5542
[13]	Xu Dongzhu, Zhou Anfu, Zhang Xinyu, et al. Understanding operational 5G: A first measurement study on its coverage, performance and energy consumption[C]//Proc of the Annual Conf of the ACM Special Interest Group on Data Communication on the Applications, Technologies, Architectures, and Protocols for Computer Communication (SIGCOMM’20). New York: ACM, 2020: 479–494
[14]	Jocher G, Chaurasia A, Jing Qiu. Ultralytics YOLO [EB/OL]. [2023-07-02]. https://github.com/ultralytics/ultralytics
[15]	He Kaiming, Gkioxari G, Dollár P, et al. Mask R-CNN[C]//Proc of the IEEE Int Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2017: 2961–2969
[16]	Yu Changqian, Gao Changxin, Wang Jingbo, et al. 2021. BiSeNet V2: Bilateral network with guided aggregation for real-time semantic segmentation[J]. Computer Vision and Pattern Recognition, 2021, 129(11): 3051−3068
[17]	Liu Peiye, Wu Bo, Ma Huadong, et al. MemNAS: Memory-efficient neural architecture search with grow-trim learning[C]//Proc of 2020 IEEE/CVF Conf on Computer Vision and Pattern Recognition (CVPR). Los Alamitos, CA: IEEE Computer Society, 2020: 2105−2113
[18]	Mucha M, Sankowski P. Maximum matchings via Gaussian elimination[C]//Proc of 45th Annual IEEE Symp on Foundations of Computer Science. Los Alamitos, CA: IEEE Computer Society, 2004: 248−255
[19]	Nie Yuqi, Nguyen N H, Sinthong P, et al. A time series is worth 64 words: Long-term forecasting with transformers[C]//Proc of the 11th Int Conf on Learning Representations. Los Alamitos, CA: IEEE Computer Society, 2022: 1−24
[20]	Liu Qiang, Choi N, Han Tao. OnSlicing: Online end-to-end network slicing with reinforcement learning[C]//Proc of the 17th Int Conf on Emerging Networking Experiments and Technologies (CoNEXT’21). New York: ACM, 2021: 141–153
[21]	Villani C. Optimal Transport: Old and New[M]. Berlin: Springer, 2009
[22]	Alibaba Group. Alibaba Cluster Trace Program cluster-trace-v2017[EB/OL]. [2024-07-31]. https://github.com/alibaba/clusterdata/blob/7358bbaf40778d4bd0464a64a430812088b7b74e/cluster-trace-v2017/trace_201708.md
[23]	Ao Lixiang, Izhikevich L, Voelker G M, et al. Sprocket: A serverless video processing framework[C]//Proc of the ACM Symp on Cloud Computing (SoCC’18). New York: ACM, 2018: 263–274
[24]	Low Y, Bickson D, Gonzalez J, et al. Distributed GraphLab: A framework for machine learning and data mining in the cloud[C]//Proc of the VLDB Endowment. New York: ACM, 2012: 716–727
[25]	Hung C C, Ananthanarayanan G, Bodik P, et al. VideoEdge: Processing camera streams using hierarchical clusters[C]//Proc of 2018 IEEE/ACM Symp on Edge Computing. Los Alamitos, CA: IEEE Computer Society, 2018: 115−131

[1]	Fu Maozhong, Hu Haiyang, Li Zhongjin. Dynamic Resource Scheduling Method for GPU Cluster[J]. Journal of Computer Research and Development, 2023, 60(6): 1308-1321. DOI: 10.7544/issn1000-1239.202220149
[2]	Wang Ziyi, Hu Xiaoyu, Wang Xin, Zhang Xinggong, Cao Zhen, Zheng Kai, Cui Yong. Fairness Measurement and Algorithm Design of Network Transmission: A Case Study of Video Applications[J]. Journal of Computer Research and Development, 2023, 60(4): 810-827. DOI: 10.7544/issn1000-1239.202330022
[3]	Zhong Lujie, Wang Mu. Blockchain-Enpowered Cooperative Resource Allocation Scheme for Computing First Network[J]. Journal of Computer Research and Development, 2023, 60(4): 750-762. DOI: 10.7544/issn1000-1239.202330002
[4]	Fang Rongqiang, Wang Jing, Yao Zhicheng, Liu Chang, Zhang Weigong. Modeling Computational Feature of Multi-Layer Neural Network[J]. Journal of Computer Research and Development, 2019, 56(6): 1170-1181. DOI: 10.7544/issn1000-1239.2019.20190111
[5]	Xu Hongzhi, Li Renfa, Zeng Lining. Parallel Task Scheduling for Resource Consumption Minimization with Reliability Constraint[J]. Journal of Computer Research and Development, 2018, 55(11): 2569-2583. DOI: 10.7544/issn1000-1239.2018.20170893
[6]	Xu Ran, Wang Wendong, Gong Xiangyang, Que Xirong. Delay-Aware Resource Scheduling Optimization in Network Function Virtualization[J]. Journal of Computer Research and Development, 2018, 55(4): 738-747. DOI: 10.7544/issn1000-1239.2018.20170926
[7]	WeiWei, LiuYang, YangWeidong. A Fast Approximation Algorithm for the General Resource Placement Problem in Cloud Computing Platform[J]. Journal of Computer Research and Development, 2016, 53(3): 697-703. DOI: 10.7544/issn1000-1239.2016.20148323
[8]	Fan Pengyi, Wang Hui, Jiang Zhihong, and Li Pei. Measurement of Microblogging Network[J]. Journal of Computer Research and Development, 2012, 49(4): 691-699.
[9]	Xie Yingke, Wang Jiandong, Zhu Chao, Zhao Zili, Han Chengde. High Precision Timestamps in Network Measurement[J]. Journal of Computer Research and Development, 2010, 47(12).
[10]	Jiao Jian, Yao Shan, Li Xiaojian. Research on Network Bidirectional Topology Discovery Based on Measurer by Spreading[J]. Journal of Computer Research and Development, 2010, 47(5): 903-910.