Efficient Video Distribution Based on Collaboration of Heterogenous Computing Nodes

E Jinlong; He Lin

doi:10.7544/issn1000-1239.202330004

Journal of Computer Research and Development > 2023 > 60(4): 772-785. > DOI: 10.7544/issn1000-1239.202330004 CSTR: 32373.14.issn1000-1239.202330004

E Jinlong, He Lin. Efficient Video Distribution Based on Collaboration of Heterogenous Computing Nodes[J]. Journal of Computer Research and Development, 2023, 60(4): 772-785. DOI: 10.7544/issn1000-1239.202330004

Citation:

PDF (1736 KB)

Efficient Video Distribution Based on Collaboration of Heterogenous Computing Nodes

E Jinlong^{1, 2,},
He Lin^{3, 4, ,}

1.
School of Information, Renmin University of China, Beijing 100872
2.
School of Software, Tsinghua University, Beijing 100084
3.
Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing 100084
4.
Zhongguancun Laboratory, Beijing 100081

Funds: This work was supported by the National Natural Science Foundation of China (62102224), and Beijing Natural Science Foundation (4222026).

More Information

Author Bio:
E Jinlong: born in 1989. PhD, lecturer. Member of CCF. His main research interests include cloud/edge computing, Internet of things, and data analytics

He Lin: born in 1991. PhD, assistant research professor. Member of CCF. His main research interests include next-generation Internet, network measurement, and programmable networks
Received Date: January 02, 2023
Revised Date: February 18, 2023
Available Online: February 26, 2023

Graphical Abstract

Abstract

Abstract

Computing power network connects computing nodes through the network to break the limitation of single point of computing power, and it is rapidly developed and applied in more and more business fields in recent years. The popular live video broadcasting relies on the transmission and transcoding of a large number of video frames, and it is of great practical importance to explore computing power networks for efficient video distribution. Compared with the traditional large-scale data processing, video applications have higher requirements for transmission delay and bandwidth guarantee. However, the computing power of nodes provided by each cloud service varies, and the state of network links between nodes often varies. Therefore, it is a great challenge to realize low latency and high bandwidth video distribution by selecting nodes with the best combined transmission and transcoding performance. Therefore, we design an efficient video distribution scheme based on heterogeneous computing nodes, including planning video transmission paths and reasonably selecting transcoding nodes through reinforcement learning, using priority queuing scheduling for different video distribution tasks and adaptively adjusting node resources to reduce resource bursty competition, adopting a layered-log-synchronization fault tolerance mechanism to quickly restore data consistency after node failures, and finally deploying multi-cloud service distributed nodes to realize a complete video distribution system. A large number of live ultra-high definition video experiments show that the performance of this scheme is significantly improved compared with existing video distribution methods.
- video distribution,
- computing power network,
- collaboration of heterogenous nodes,
- reinforcement learning,
- multi-task scheduling

FullText(HTML)

References (40)

References

[1]	Tang Xiongyan, Cao Chang, Wang Youxiang, et al. Computing power network: The architecture of convergence of computing and networking towards 6G requirement[J]. China Communications, 2021, 18(2): 175−185 doi: 10.23919/JCC.2021.02.011
[2]	贾庆民,胡玉姣,张华宇,等. 确定性算力网络研究[J]. 通信学报,2022,43(10):55−64 Jia Qingmin, Hu Yujiao, Zhang Huayu, et al. Research on deterministic computing power network[J]. Journal of Communications, 2022, 43(10): 55−64 (in Chinese)
[3]	Subcommittee on Future Advanced Computing Ecosystems of the National Science and Technology Council (NSTC). Pioneering the future advanced computing ecosystem: A strategic plan[R]. Washington D. C. : NSTC, 2020
[4]	The European Commission. 2030 digital compass: The European way for the digital decade[R]. Brussels, Belgium: European Commission, 2021
[5]	国家发展改革委, 中央网信办, 工业和信息化部, 等. 全国一体化大数据中心协同创新体系算力枢纽实施方案[R]. 北京: 国家发展改革委, 2021 National Development and Reform Commission, Cyberspace Administration of China, Ministry of Industry and Information Technology, et al. Implementation plan of computing power hub of national integrated big data center collaborative innovation system[R]. Beijing: National Development and Reform Commission, 2021 (in Chinese)
[6]	中国联合网络通信集团有限公司. 中国联通算力网络白皮书[R]. 北京: 中国联合网络通信集团有限公司, 2019 China United Network Communication Group Co., Ltd. Computing power network whitepaper of China Unicom[R]. Beijing: China United Network Communication Group Co., Ltd. , 2019 (in Chinese)
[7]	中国移动通信集团有限公司. 算力网络技术白皮书[R]. 北京: 中国移动通信集团有限公司, 2021 China Mobile Communications Group Co., Ltd. Computing Force Network Technology Whitepaper[R]. Beijing: China Mobile Communications Group Co. , Ltd, 2021 (in Chinese)
[8]	中国电信集团有限公司. 云网融合2030技术白皮书[R]. 北京: 中国电信集团有限公司, 2020 China Telecommunications Group Co., Ltd. Cloud Network Integration 2030 Technology Whitepaper[R]. Beijing: China Telecommunications Group Co., Ltd, 2020 (in Chinese)
[9]	华为技术有限公司. 算力网络[R/OL]. 深圳: 华为技术有限公司, 2022 [2023-02-19].https://support.huawei.com/enterprise/zh/doc/EDOC1100249555/ec4b1f21 Huawei Technologies Co. , Ltd. Computing Power Network[R/OL]. Shenzhen: Huawei Technologies Co. , Ltd. , 2022 [2023-02-19].https://support.huawei.com/enterprise/zh/doc/EDOC1100249555/ec4b1f21 (in Chinese)
[10]	中国移动研究院. 算力网络视频应用白皮书[R]. 北京: 中国移动通信集团有限公司, 2022 China Mobile Research Institute. Video Applications of Computing Force Network Whitepaper[R]. Beijing: China Mobile Communications Group Co. , Ltd, 2022 (in Chinese)
[11]	国务院. “十四五”数字经济发展规划[R]. 北京: 国务院, 2021 The State Council. “14th Five Year Plan” for Digital Economy Development[R]. Beijing: The State Council, 2021 (in Chinese)
[12]	International Telecommunication Union Telecommunication (ITU-T). Y. 2501: Computing Power Network – Framework and Architecture[S]. Geneva, Switzerland: ITU-T, 2021
[13]	中国通信标准化协会. 算力网络总体技术要求[S]. 北京: 中国通信标准化协会, 2021 China Communications Standards Association. General Technical Requirements of Computing and Network Convergence[S]. Beijing: China Communications Standards Association, 2021 (in Chinese)
[14]	Król M, Mastorakis S, Oran D, et al. Compute first networking: Distributed computing meets ICN[C] //Proc of the 6th ACM Conf on Information-Centric Networking. New York: ACM Press, 2019: 67−77
[15]	Chen Huan, Tao Yu, Zhu Yi. NSACS-PS: A named service access control scheme based on proxy signature in named computing first networking[C] //Proc of 2021 4th Int Conf on Hot Information-Centric Networking (HotICN). Piscataway, NJ: IEEE, 2021: 81−85
[16]	Balakrishnan H, Banerjee S, Cidon I, et al. Revitalizing the public Internet by making it extensible[J]. ACM SIGCOMM Computer Communication Review, 2021, 51(2): 18−24 doi: 10.1145/3464994.3464998
[17]	中兴通讯技术（简讯）. 视频算力网络, 使能全场景视频应用[R/OL]. 深圳: 中兴通讯股份有限公司, 2022 [2023-02-19].https: //www.zte.com.cn/china/about/magazine/zte-technologies/2022/10-cn/_3/_.html ZTE Technology. Computing Power Network for Videos, Enabling Full-Scenario Video Applications[R/OL]. Shenzhen: ZTE Corporation, 2022 [2023-02-19].https://www.zte.com.cn/china/about/magazine/zte-technologies/2022/10-cn/_3/_.html (in Chinese)
[18]	Mao Hongzi, Netravali R, Alizadeh M. Neural adaptive video streaming with pensieve[C] //Proc of the Conf of the ACM Special Interest Group on Data Communication, New York: ACM, 2017: 197–210
[19]	Yeo H, Jung Y, Kim J, et al. Neural adaptive content-aware internet video delivery[C] //Proc of 13th USENIX Symp on Operating Systems Design and Implementation. Berkeley, CA: USENIX Association, 2018: 645–661
[20]	Fouladi S, Emmons J, Orbay E, et al. Salsify: Low-latency network video through tighter integration between a video codec and a transport protocol[C] //Proc of 15th USENIX Symp on Networked Systems Design and Implementation. Berkeley, CA: USENIX Association, 2018: 267–282
[21]	Kim J, Jung Y, Yeo H, et al. Neural-enhanced live streaming: Improving live video ingest via online learning[C] //Proc of the Annual conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2020: 107–125
[22]	Mukerjee M K, Naylor D, Jiang Junchen, et al. Practical, real-time centralized control for CDN-based live video delivery[C] //Proc of the Annual conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2015: 311–324
[23]	Han Bo, Qian Feng, Ji Lusheng, et al. MP-DASH: Adaptive video streaming over preference-aware multipath[C] //Proc of the 12th Int on Conf on Emerging Networking Experiments and Technologies. New York: ACM, 2016: 129–143
[24]	Zheng Zhilong, Ma Yunfei, Liu Yanmei, et al. XLINK: QoE-driven multi-path QUIC transport in large-scale video services[C] //Proc of the Annual Conf of the ACM Special Interest Group on Data Communication. New York: ACM, 2021: 418−432
[25]	Baig G, He Jian, Qureshi M A, et al. Jigsaw: Robust live 4K video streaming[C] //Proc of the 25th Annual Int Conf on Mobile Computing and Networking. New York: ACM, 2019: 1–16
[26]	He Jian, Qureshi M A, Qiu Lili, et al. Rubiks: Practical 360-degree streaming for smartphones[C] //Proc of the 16th Annual Int Conf on Mobile Systems, Applications, and Services. New York: ACM, 2018: 482–494
[27]	Sodagar I. The MPEG-DASH standard for multimedia streaming over the Internet[J]. IEEE Multimedia, 2011, 18(4): 62−67 doi: 10.1109/MMUL.2011.71
[28]	Cui Yong, Dai Ningwei, Lai Zeqi, et al. TailCutter: Wisely cutting tail latency in cloud CDNs under cost constraints[J]. IEEE/ACM Transactions on Networking, 2019, 27(4): 1612−1628 doi: 10.1109/TNET.2019.2926142
[29]	E Jinlong, Cui Yong, Ruan Mingkang, et al. HyCloud: Tweaking hybrid cloud storage services for cost-efficient filesystem hosting[C] //Proc of IEEE Int Conf on Computer Communications. Piscataway, NJ: IEEE, 2019: 2341−2349
[30]	Xiph.Org Foundation. Video test media [derf’s collection][R/OL]. Somerville, MA: Xiph.Org Foundation, 1994 [2023-02-19]. https://media.xiph.org/video/derf/
[31]	International Telecommunication Union Telecommunication (ITU-T). H. 264: Advanced Video Coding for Generic Audiovisual Services[S]. Geneva, Switzerland: ITU-T, 2021
[32]	Agarwal S, Dunagan J, Jain N, et al. Volley: Automated data placement for geo-distributed cloud services[C] //Proc of 20th USENIX Symp on Networked Systems Design and Implementation. Berkeley, CA: USENIX Association, 2010: 17−32
[33]	Pirotta M, Restelli M, Bascetta L. Adaptive step-size for policy gradient methods[C] //Proc of the 27th Conf on Neural Information Processing Systems. San Diego, CA: NIPS, 2013: 1394–1402
[34]	Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms[J]. arXiv preprint, arXiv: 1707.06347, 2017
[35]	McMahan H B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C] //Proc of 20th Int Conf on Artificial Intelligence and Statistics. AISTATS, 2017: 1273−1282
[36]	Ongaro D, Ousterhout J. In search of an understandable consensus algorithm[C] //Proc of Annual Technical Conf. Berkeley, CA: USENIX Association, 2014: 305−320
[37]	Merkle R C. Protocols for public key cryptosystems[C] //Proc of the IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 1980: 122−134
[38]	GitHub. 非拜占庭节点的分布式共识算法Raft的Python实现[CP/OL]. [2023 − 02-19]. https://github.com/hangsz/raft GitHub. Python implementation of non-Byzantine nodes’ distributed consensus algorithm Raft[CP/OL]. [2023-02-19]. https://github.com/hangsz/raft (in Chinese)
[39]	LaValle S M, Branicky M S, Lindemann S R. On the relationship between classical grid search and probabilistic roadmaps[J]. The International Journal of Robotics Research, 2004, 23(7-8): 673−692 doi: 10.1177/0278364904045481
[40]	Kingma D P, Ba J. Adam: A method for stochastic optimization[J]. arXiv preprint, arXiv: 1412.6980, 2017

[1]	Xue Zhihang, Xu Zheming, Lang Congyan, Feng Songhe, Wang Tao, Li Yidong. Text-to-Image Generation Method Based on Image-Text Semantic Consistency[J]. Journal of Computer Research and Development, 2023, 60(9): 2180-2190. DOI: 10.7544/issn1000-1239.202220416
[2]	Zhang Jing, Ju Jialiang, Ren Yonggong. Double-Generators Network for Data-Free Knowledge Distillation[J]. Journal of Computer Research and Development, 2023, 60(7): 1615-1627. DOI: 10.7544/issn1000-1239.202220024
[3]	Zhang Hao, Ma Jiayi, Fan Fan, Huang Jun, Ma Yong. Infrared and Visible Image Fusion Based on Multiclassification Adversarial Mechanism in Feature Space[J]. Journal of Computer Research and Development, 2023, 60(3): 690-704. DOI: 10.7544/issn1000-1239.202110639
[4]	Liu Guangrui, Zhang Weizhe, Li Xinjie. Data Contamination Defense Method for Intelligent Network Intrusion Detection Systems Based on Edge Examples[J]. Journal of Computer Research and Development, 2022, 59(10): 2348-2361. DOI: 10.7544/issn1000-1239.20220509
[5]	Guo Zhengshan, Zuo Jie, Duan Lei, Li Renhao, He Chengxin, Xiao Yingjie, Wang Peiyan. A Generative Adversarial Negative Sampling Method for Knowledge Hypergraph Link Prediction[J]. Journal of Computer Research and Development, 2022, 59(8): 1742-1756. DOI: 10.7544/issn1000-1239.20220074
[6]	Chen Dawei, Fu Anmin, Zhou Chunyi, Chen Zhenzhu. Federated Learning Backdoor Attack Scheme Based on Generative Adversarial Network[J]. Journal of Computer Research and Development, 2021, 58(11): 2364-2373. DOI: 10.7544/issn1000-1239.2021.20210659
[7]	Dai Hong, Sheng Lijie, Miao Qiguang. Adversarial Discriminative Domain Adaptation Algorithm with CapsNet[J]. Journal of Computer Research and Development, 2021, 58(9): 1997-2012. DOI: 10.7544/issn1000-1239.2021.20200569
[8]	Qian Yaguan, He Niannian, Guo Yankai, Wang Bin, Li Hui, Gu Zhaoquan, Zhang Xuhong, Wu Chunming. An Evasion Algorithm to Fool Fingerprint Detector for Deep Neural Networks[J]. Journal of Computer Research and Development, 2021, 58(5): 1106-1117. DOI: 10.7544/issn1000-1239.2021.20200903
[9]	Yu Haitao, Yang Xiaoshan, Xu Changsheng. Antagonistic Video Generation Method Based on Multimodal Input[J]. Journal of Computer Research and Development, 2020, 57(7): 1522-1530. DOI: 10.7544/issn1000-1239.2020.20190479
[10]	Tian Jiwei, Wang Jinsong, Shi Kai. Positive and Unlabeled Generative Adversarial Network on POI Positioning[J]. Journal of Computer Research and Development, 2019, 56(9): 1843-1850. DOI: 10.7544/issn1000-1239.2019.20180847