Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles

Tang Xiaolan; Liang Yuting; Chen Wenlong

doi:10.7544/issn1000-1239.202330885

Journal of Computer Research and Development > 2024 > 61(9): 2170-2184. > DOI: 10.7544/issn1000-1239.202330885 CSTR: 32373.14.issn1000-1239.202330885

Tang Xiaolan, Liang Yuting, Chen Wenlong. Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles[J]. Journal of Computer Research and Development, 2024, 61(9): 2170-2184. DOI: 10.7544/issn1000-1239.202330885

Citation:

PDF (2923 KB)

Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles

College of Information Engineering, Capital Normal University, Beijing 100048

Funds: This work was supported by the National Natural Science Foundation of China (61872252), the Beijing Outstanding Youth Talent Development Program (BPHR202203118), and the Research Project on “Artificial Intelligence Empowering Capital Education Reform and Development” at Capital Normal University (RGZNJY2023-YB-14).

More Information

Author Bio:
Tang Xiaolan: born in 1987. PhD, associate professor. Member of CCF. Her main research interests include Internet of vehicles, smart education, urban computing, and driving behavior analysis

Liang Yuting: born in 2000. Master candidate. Her main research interests include Internet of vehicles and federated learning

Chen Wenlong: born in 1976. PhD, professor. Member of CCF. His main research interests include network protocol, Internet architecture, high performance router, and wireless sensor networks
Received Date: October 31, 2023
Revised Date: May 14, 2024
Available Online: May 28, 2024

Graphical Abstract

Abstract

Abstract

The Internet of vehicles (IoV) plays an indispensable role in the construction of smart cities, where cars are not just a means of transportation but also a crucial medium for information collection and transmission in the era of big data. With the rapid growth in the volume of data collected from vehicles and the increased awareness of privacy protection, ensuring users’ data security and preventing data breaches in IoV have become an urgent issue to address. Federated learning, as a “data-does-not-move, model-moves” approach, offers a feasible method for protecting user privacy while achieving excellent performance. However, because of the differences of devices, regions and individual habits, data collected from multiple vehicle typically exhibit non-independent and identically distributed (non-IID) characteristics. Traditional federated learning algorithms have slow model convergence when processing non-IID data. In response to this challenge, we propose a multi-stage federated learning algorithm with non-IID data in IoV, named FedWO. In Stage 1, FedWO utilizes the federated averaging algorithm to expedite the global model in reaching a basic level of accuracy. In Stage 2, FedWO employs weighted federated learning, where the weight of a vehicle in the global model is calculated based on its data characteristics. This aggregation results in an improved global model. Moreover, we design a transmission control strategy to reduce communication overhead caused by model transmission. The Stage 3 involves personalized computation, where each vehicle employs its own data for personalized learning, fine-tuning the local model to obtain a model more aligned with local data. We conducted experimental evaluations using a driving behavior dataset. The results demonstrate that, compared with traditional methods, FedWO preserves data privacy while improving the accuracy of algorithms in non-IID data scenarios.
- Internet of vehicles (IoV),
- federated learning (FL),
- non-IID data,
- privacy preserving,
- transmission control

FullText(HTML)

References (36)

References

[1]	Chai Haoye, Leng Supeng, Chen Yijin, et al. A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in Internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(7): 3975−3986
[2]	Wang Ge, Xu Fangmin, Zhang Hengsheng, et al. Joint resource management for mobility supported federated learning in Internet of vehicles[J]. Future Generation Computer Systems, 2022, 129: 199−211 doi: 10.1016/j.future.2021.11.020
[3]	Ren Ju, Zhang Deyu, He Shiwen, et al. A survey on end-edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet[J]. ACM Computing Surveys, 2019, 52(6): 1−36
[4]	Zhang Wei, Li Xiang, Ma Hui, et al. Federated learning for machinery fault diagnosis with dynamic validation and self-supervision[J]. Knowledge-Based Systems, 2021, 213: 106679 doi: 10.1016/j.knosys.2020.106679
[5]	Cao Longbing. Non-IID recommender systems: A review and framework of recommendation paradigm shifting[J]. Engineering, 2016, 2(2): 212−224 doi: 10.1016/J.ENG.2016.02.013
[6]	Kaissis G A, Makowski M R, Rückert D, et al. Secure, privacy-preserving and federated machine learning in medical imaging[J]. Nature Machine Intelligence, 2020, 2(6): 305−311 doi: 10.1038/s42256-020-0186-1
[7]	Lu Yunlong, Huang Xiaohong, Dai Yueyue, et al. Blockchain and federated learning for privacy-preserved data sharing in industrial IoT[J]. IEEE Transactions on Industrial Informatics, 2019, 16(6): 4177−4186
[8]	Luo Bing, Li Xiang, Wang Shiqiang, et al. Cost-effective federated learning in mobile edge networks[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(12): 3606−3621 doi: 10.1109/JSAC.2021.3118436
[9]	He Chaoyang, Shah A D, Tang Zhenheng, et al. FedCV: A federated learning framework for diverse computer vision tasks[J]. arXiv preprint, arXiv: 2111.11066, 2021
[10]	莫慧凌,郑海峰,高敏,等. 基于联邦学习的多源异构数据融合算法[J]. 计算机研究与发展,2022,59(2):478−487 doi: 10.7544/issn1000-1239.20200668 Mo Huiling, Zheng Haifeng, Gao Min, et al. Multi-source heterogeneous data fusion based on federated learning[J]. Journal of Computer Research and Development, 2022, 59(2): 478−487 (in Chinese) doi: 10.7544/issn1000-1239.20200668
[11]	Kong Xiangjie, Wang Kailai, Hou Mingliang, et al. A federated learning-based license plate recognition scheme for 5G-enabled Internet of vehicles[J]. IEEE Transactions on Industrial Informatics, 2021, 17(12): 8523−8530 doi: 10.1109/TII.2021.3067324
[12]	Liang Feiyuan, Yang Qinglin, Liu Ruiqi, et al. Semi-synchronous federated learning protocol with dynamic aggregation in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2022, 71(5): 4677−4691 doi: 10.1109/TVT.2022.3148872
[13]	Lu Yunlong, Huang Xiaohong, Zhang Ke, et al. Blockchain empowered asynchronous federated learning for secure data sharing in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2020, 69(4): 4298−4311 doi: 10.1109/TVT.2020.2973651
[14]	Tang Xiaolan, Liang Yuting, Wang Guan, et al. Assisted driving system based on federated reinforcement learning[J]. Displays, 2023, 80: 102547
[15]	Parekh R, Patel N, Gupta R, et al. Gefl: Gradient encryption-aided privacy preserved federated learning for autonomous vehicles[J]. IEEE Access, 2023, 11: 1825−1839 doi: 10.1109/ACCESS.2023.3233983
[16]	Qu Zhiguo, Tang Yang, Muhammad G, et al. Privacy protection in intelligent vehicle networking: A novel federated learning algorithm based on information fusion[J]. Information Fusion, 2023, 98: 101824 doi: 10.1016/j.inffus.2023.101824
[17]	Li Zejun, Wu Hao, Lu Yunlong. Coalition based utility and efficiency optimization for multi-task federated learning in Internet of vehicles[J]. Future Generation Computer Systems, 2023, 140: 196−208 doi: 10.1016/j.future.2022.10.014
[18]	He Jingyi, Gong Biyao, Yang Jiadi, et al. ASCFL: Accurate and speedy semi-supervised clustering federated learning[J]. Tsinghua Science and Technology, 2023, 28(5): 1−15
[19]	Shu Jiangang, Yang Tingting, Liao Xinying, et al. Clustered federated multitask learning on non-IID data with enhanced privacy[J]. IEEE Internet of Things Journal, 2022, 10(4): 3453−3467
[20]	Tian Pu, Liao Weixian, Yu Wei, et al. WSCC: A weight-similarity-based client clustering approach for non-IID federated learning[J]. IEEE Internet of Things Journal, 2022, 9(20): 20243−20256 doi: 10.1109/JIOT.2022.3175149
[21]	Dong Fang, Ge Xinghua, Li Qinya, et al. PADP-FedMeta: A personalized and adaptive differentially private federated meta learning mechanism for AIoT[J]. Journal of Systems Architecture, 2023, 134: 102754 doi: 10.1016/j.sysarc.2022.102754
[22]	Yang Lei, Huang Jiaming, Lin Wanyu, et al. Personalized federated learning on non-IID data via group-based meta-learning[J]. ACM Transactions on Knowledge Discovery from Data, 2023, 17(4): 1−20
[23]	Li Wenzhu, Wang Shuang. Federated meta-learning for spatial-temporal prediction[J]. Neural Computing and Applications, 2022, 34(13): 10355−10374 doi: 10.1007/s00521-021-06861-3
[24]	Hu Kai, Li Yaogen, Zhang Shuai, et al. FedMMD: A federated weighting algorithm considering non-IID and local model deviation[J]. Expert Systems with Applications, 2024, 237: 121463 doi: 10.1016/j.eswa.2023.121463
[25]	Kim H, Kim Y, Park H. Reducing model cost based on the weights of each layer for federated learning clustering[C]//Proc of the 12th Int Conf on Ubiquitous and Future Networks (ICUFN). Piscataway, NJ: IEEE, 2021: 405−408
[26]	Zhang Wenyu, Wang Xiumin, Zhou Pan, et al. Client selection for federated learning with non-IID data in mobile edge computing[J]. IEEE Access, 2021, 9: 24462−24474 doi: 10.1109/ACCESS.2021.3056919
[27]	McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Artificial Intelligence and Statistics. New York: PMLR, 2017: 1273−1282
[28]	钟正仪,包卫东,王吉,等. 一种面向云边端系统的分层异构联邦学习方法[J]. 计算机研究与发展,2022,59(11):2408−2422 doi: 10.7544/issn1000-1239.20220458 Zhong Zhengyi, Bao Weidong, Wang Ji, et al. A hierarchically heterogeneous federated learning method for cloud-edge-end system[J]. Journal of Computer Research and Development, 2022, 59(11): 2408−2422(in Chinese) doi: 10.7544/issn1000-1239.20220458
[29]	Kairouz P, McMahan H B, Avent B, et al. Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1/2): 1−210
[30]	Li Yifei, Guo Yijia, Alazab M, et al. Joint optimal quantization and aggregation of federated learning scheme in VANETs[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(10): 19852−19863 doi: 10.1109/TITS.2022.3145823
[31]	Van Tilborg H C A, Jajodia S. Encyclopedia of Cryptography and Security[M]. Berlin: Springer, 2014
[32]	Sun Lichao, Qian Jianwei, Chen Xun. LDP-FL: Practical private aggregation in federated learning with local differential privacy[J]. arXiv preprint, arXiv: 2007.15789, 2020
[33]	Li Tian, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of the Machine Learning and Systems, 2020, 2: 429−450
[34]	Karimireddy S P, Kale S, Mohri M, et al. SCAFFOLD: Stochastic controlled averaging for federated learning[C]//Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 5132−5143
[35]	Wang Jianyu, Liu Qinghua, Liang Hao, et al. Tackling the objective inconsistency problem in heterogeneous federated optimization[J]. Advances in Neural Information Processing Systems, 2020, 33: 7611−7623
[36]	Google. StateFarm[EB/OL]. 2016[2023-02-01].https://www.kaggle.com/c/state-farm-distracted-driver-detection

[1]	Qian Zhongsheng, Huang Heng, Zhu Hui, Liu Jinping. Multi-Perspective Graph Contrastive Learning Recommendation Method with Layer Attention Mechanism[J]. Journal of Computer Research and Development, 2025, 62(1): 160-178. DOI: 10.7544/issn1000-1239.202330804
[2]	Zhang Jinyu, Ma Chenxi, Li Chao, Zhao Zhongying. Towards Lightweight Cross-Domain Sequential Recommendation via Tri-Branches Graph External Attention Network[J]. Journal of Computer Research and Development, 2024, 61(8): 1930-1944. DOI: 10.7544/issn1000-1239.202440197
[3]	Xie Jun, Wang Yuzhu, Chen Bo, Zhang Zehua, Liu Qin. Aspect-Based Sentiment Analysis Model with Bi-Guide Attention Network[J]. Journal of Computer Research and Development, 2022, 59(12): 2831-2843. DOI: 10.7544/issn1000-1239.20210708
[4]	Qian Zhongsheng, Yang Jiaxiu, Li Duanming, Ye Zulai. Event Recommendation Strategy Combining User Long-Short Term Interest and vent Influence[J]. Journal of Computer Research and Development, 2022, 59(12): 2803-2815. DOI: 10.7544/issn1000-1239.20210693
[5]	Sun Qian, Xue Leiqi, Gao Ling, Wang Hai, Wang Yuxiang. Selection of Network Defense Strategies Based on Stochastic Game and Tabu Search[J]. Journal of Computer Research and Development, 2020, 57(4): 767-777. DOI: 10.7544/issn1000-1239.2020.20190870
[6]	Xu Jinghang, Zuo Wanli, Liang Shining, Wang Ying. Causal Relation Extraction Based on Graph Attention Networks[J]. Journal of Computer Research and Development, 2020, 57(1): 159-174. DOI: 10.7544/issn1000-1239.2020.20190042
[7]	Sun Xiaowan, Wang Ying, Wang Xin, Sun Yudong. Aspect-Based Sentiment Analysis Model Based on Dual-Attention Networks[J]. Journal of Computer Research and Development, 2019, 56(11): 2384-2395. DOI: 10.7544/issn1000-1239.2019.20180823
[8]	Zhang Han, Guo Yuanbo, Li Tao. Domain Named Entity Recognition Combining GAN and BiLSTM-Attention-CRF[J]. Journal of Computer Research and Development, 2019, 56(9): 1851-1858. DOI: 10.7544/issn1000-1239.2019.20180733
[9]	Guo Chi, Wang Lina, Guan Yiping, Zhang Xiaoying. A Network Immunization Strategy Based on Dynamic Preference Scan[J]. Journal of Computer Research and Development, 2012, 49(4): 717-724.
[10]	Wang Bailing, Fang Binxing, Yun Xiaochun, Zhang Hongli, Chen Bo, Liu Yixuan. A New Friendly Worm Propagation Strategy Based on Diffusing Balance Tree[J]. Journal of Computer Research and Development, 2006, 43(9): 1593-1602.

Cited By

Cited by

Periodical cited type(6)

1.	韩宇捷，徐志杰，杨定裕，黄波，郭健美. CDES:数据驱动的云数据库效能评估方法. 计算机科学. 2024(06): 111-117 .
2.	刘传磊，张贺，杨贺. 地铁保护区智能化巡查系统开发及应用研究. 现代城市轨道交通. 2024(09): 23-30 .
3.	董文，张俊峰，刘俊，张雷. 国产数据库在能源数字化转型中的创新应用研究. 信息通信技术与政策. 2024(10): 68-74 .
4.	阎开. 计算机检测维修与数据恢复技术及应用研究. 信息记录材料. 2023(08): 89-91 .
5.	冯丽琴，冯花平. 基于人脸识别的可控化学习数据库系统设计. 数字通信世界. 2023(10): 69-71 .
6.	张惠芹，章小卫，杜坤，李江. 基于数字孪生的高校实验室高温设备智能化监管体系的探究. 实验室研究与探索. 2023(11): 249-252+282 .