Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles

Tang Xiaolan; Liang Yuting; Chen Wenlong

doi:10.7544/issn1000-1239.202330885

Journal of Computer Research and Development > 2024 > 61(9): 2170-2184. > DOI: 10.7544/issn1000-1239.202330885 CSTR: 32373.14.issn1000-1239.202330885

Tang Xiaolan, Liang Yuting, Chen Wenlong. Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles[J]. Journal of Computer Research and Development, 2024, 61(9): 2170-2184. DOI: 10.7544/issn1000-1239.202330885

Citation:

PDF (2923 KB)

Multi-Stage Federated Learning Mechanism with non-IID Data in Internet of Vehicles

College of Information Engineering, Capital Normal University, Beijing 100048

Funds: This work was supported by the National Natural Science Foundation of China (61872252), the Beijing Outstanding Youth Talent Development Program (BPHR202203118), and the Research Project on “Artificial Intelligence Empowering Capital Education Reform and Development” at Capital Normal University (RGZNJY2023-YB-14).

More Information

Author Bio:
Tang Xiaolan: born in 1987. PhD, associate professor. Member of CCF. Her main research interests include Internet of vehicles, smart education, urban computing, and driving behavior analysis

Liang Yuting: born in 2000. Master candidate. Her main research interests include Internet of vehicles and federated learning

Chen Wenlong: born in 1976. PhD, professor. Member of CCF. His main research interests include network protocol, Internet architecture, high performance router, and wireless sensor networks
Received Date: October 31, 2023
Revised Date: May 14, 2024
Available Online: May 28, 2024

Graphical Abstract

Abstract

Abstract

The Internet of vehicles (IoV) plays an indispensable role in the construction of smart cities, where cars are not just a means of transportation but also a crucial medium for information collection and transmission in the era of big data. With the rapid growth in the volume of data collected from vehicles and the increased awareness of privacy protection, ensuring users’ data security and preventing data breaches in IoV have become an urgent issue to address. Federated learning, as a “data-does-not-move, model-moves” approach, offers a feasible method for protecting user privacy while achieving excellent performance. However, because of the differences of devices, regions and individual habits, data collected from multiple vehicle typically exhibit non-independent and identically distributed (non-IID) characteristics. Traditional federated learning algorithms have slow model convergence when processing non-IID data. In response to this challenge, we propose a multi-stage federated learning algorithm with non-IID data in IoV, named FedWO. In Stage 1, FedWO utilizes the federated averaging algorithm to expedite the global model in reaching a basic level of accuracy. In Stage 2, FedWO employs weighted federated learning, where the weight of a vehicle in the global model is calculated based on its data characteristics. This aggregation results in an improved global model. Moreover, we design a transmission control strategy to reduce communication overhead caused by model transmission. The Stage 3 involves personalized computation, where each vehicle employs its own data for personalized learning, fine-tuning the local model to obtain a model more aligned with local data. We conducted experimental evaluations using a driving behavior dataset. The results demonstrate that, compared with traditional methods, FedWO preserves data privacy while improving the accuracy of algorithms in non-IID data scenarios.
- Internet of vehicles (IoV),
- federated learning (FL),
- non-IID data,
- privacy preserving,
- transmission control

FullText(HTML)

References (36)

References

[1]	Chai Haoye, Leng Supeng, Chen Yijin, et al. A hierarchical blockchain-enabled federated learning algorithm for knowledge sharing in Internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(7): 3975−3986
[2]	Wang Ge, Xu Fangmin, Zhang Hengsheng, et al. Joint resource management for mobility supported federated learning in Internet of vehicles[J]. Future Generation Computer Systems, 2022, 129: 199−211 doi: 10.1016/j.future.2021.11.020
[3]	Ren Ju, Zhang Deyu, He Shiwen, et al. A survey on end-edge-cloud orchestrated network computing paradigms: Transparent computing, mobile edge computing, fog computing, and cloudlet[J]. ACM Computing Surveys, 2019, 52(6): 1−36
[4]	Zhang Wei, Li Xiang, Ma Hui, et al. Federated learning for machinery fault diagnosis with dynamic validation and self-supervision[J]. Knowledge-Based Systems, 2021, 213: 106679 doi: 10.1016/j.knosys.2020.106679
[5]	Cao Longbing. Non-IID recommender systems: A review and framework of recommendation paradigm shifting[J]. Engineering, 2016, 2(2): 212−224 doi: 10.1016/J.ENG.2016.02.013
[6]	Kaissis G A, Makowski M R, Rückert D, et al. Secure, privacy-preserving and federated machine learning in medical imaging[J]. Nature Machine Intelligence, 2020, 2(6): 305−311 doi: 10.1038/s42256-020-0186-1
[7]	Lu Yunlong, Huang Xiaohong, Dai Yueyue, et al. Blockchain and federated learning for privacy-preserved data sharing in industrial IoT[J]. IEEE Transactions on Industrial Informatics, 2019, 16(6): 4177−4186
[8]	Luo Bing, Li Xiang, Wang Shiqiang, et al. Cost-effective federated learning in mobile edge networks[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(12): 3606−3621 doi: 10.1109/JSAC.2021.3118436
[9]	He Chaoyang, Shah A D, Tang Zhenheng, et al. FedCV: A federated learning framework for diverse computer vision tasks[J]. arXiv preprint, arXiv: 2111.11066, 2021
[10]	莫慧凌,郑海峰,高敏,等. 基于联邦学习的多源异构数据融合算法[J]. 计算机研究与发展,2022,59(2):478−487 doi: 10.7544/issn1000-1239.20200668 Mo Huiling, Zheng Haifeng, Gao Min, et al. Multi-source heterogeneous data fusion based on federated learning[J]. Journal of Computer Research and Development, 2022, 59(2): 478−487 (in Chinese) doi: 10.7544/issn1000-1239.20200668
[11]	Kong Xiangjie, Wang Kailai, Hou Mingliang, et al. A federated learning-based license plate recognition scheme for 5G-enabled Internet of vehicles[J]. IEEE Transactions on Industrial Informatics, 2021, 17(12): 8523−8530 doi: 10.1109/TII.2021.3067324
[12]	Liang Feiyuan, Yang Qinglin, Liu Ruiqi, et al. Semi-synchronous federated learning protocol with dynamic aggregation in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2022, 71(5): 4677−4691 doi: 10.1109/TVT.2022.3148872
[13]	Lu Yunlong, Huang Xiaohong, Zhang Ke, et al. Blockchain empowered asynchronous federated learning for secure data sharing in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2020, 69(4): 4298−4311 doi: 10.1109/TVT.2020.2973651
[14]	Tang Xiaolan, Liang Yuting, Wang Guan, et al. Assisted driving system based on federated reinforcement learning[J]. Displays, 2023, 80: 102547
[15]	Parekh R, Patel N, Gupta R, et al. Gefl: Gradient encryption-aided privacy preserved federated learning for autonomous vehicles[J]. IEEE Access, 2023, 11: 1825−1839 doi: 10.1109/ACCESS.2023.3233983
[16]	Qu Zhiguo, Tang Yang, Muhammad G, et al. Privacy protection in intelligent vehicle networking: A novel federated learning algorithm based on information fusion[J]. Information Fusion, 2023, 98: 101824 doi: 10.1016/j.inffus.2023.101824
[17]	Li Zejun, Wu Hao, Lu Yunlong. Coalition based utility and efficiency optimization for multi-task federated learning in Internet of vehicles[J]. Future Generation Computer Systems, 2023, 140: 196−208 doi: 10.1016/j.future.2022.10.014
[18]	He Jingyi, Gong Biyao, Yang Jiadi, et al. ASCFL: Accurate and speedy semi-supervised clustering federated learning[J]. Tsinghua Science and Technology, 2023, 28(5): 1−15
[19]	Shu Jiangang, Yang Tingting, Liao Xinying, et al. Clustered federated multitask learning on non-IID data with enhanced privacy[J]. IEEE Internet of Things Journal, 2022, 10(4): 3453−3467
[20]	Tian Pu, Liao Weixian, Yu Wei, et al. WSCC: A weight-similarity-based client clustering approach for non-IID federated learning[J]. IEEE Internet of Things Journal, 2022, 9(20): 20243−20256 doi: 10.1109/JIOT.2022.3175149
[21]	Dong Fang, Ge Xinghua, Li Qinya, et al. PADP-FedMeta: A personalized and adaptive differentially private federated meta learning mechanism for AIoT[J]. Journal of Systems Architecture, 2023, 134: 102754 doi: 10.1016/j.sysarc.2022.102754
[22]	Yang Lei, Huang Jiaming, Lin Wanyu, et al. Personalized federated learning on non-IID data via group-based meta-learning[J]. ACM Transactions on Knowledge Discovery from Data, 2023, 17(4): 1−20
[23]	Li Wenzhu, Wang Shuang. Federated meta-learning for spatial-temporal prediction[J]. Neural Computing and Applications, 2022, 34(13): 10355−10374 doi: 10.1007/s00521-021-06861-3
[24]	Hu Kai, Li Yaogen, Zhang Shuai, et al. FedMMD: A federated weighting algorithm considering non-IID and local model deviation[J]. Expert Systems with Applications, 2024, 237: 121463 doi: 10.1016/j.eswa.2023.121463
[25]	Kim H, Kim Y, Park H. Reducing model cost based on the weights of each layer for federated learning clustering[C]//Proc of the 12th Int Conf on Ubiquitous and Future Networks (ICUFN). Piscataway, NJ: IEEE, 2021: 405−408
[26]	Zhang Wenyu, Wang Xiumin, Zhou Pan, et al. Client selection for federated learning with non-IID data in mobile edge computing[J]. IEEE Access, 2021, 9: 24462−24474 doi: 10.1109/ACCESS.2021.3056919
[27]	McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Artificial Intelligence and Statistics. New York: PMLR, 2017: 1273−1282
[28]	钟正仪,包卫东,王吉,等. 一种面向云边端系统的分层异构联邦学习方法[J]. 计算机研究与发展,2022,59(11):2408−2422 doi: 10.7544/issn1000-1239.20220458 Zhong Zhengyi, Bao Weidong, Wang Ji, et al. A hierarchically heterogeneous federated learning method for cloud-edge-end system[J]. Journal of Computer Research and Development, 2022, 59(11): 2408−2422(in Chinese) doi: 10.7544/issn1000-1239.20220458
[29]	Kairouz P, McMahan H B, Avent B, et al. Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1/2): 1−210
[30]	Li Yifei, Guo Yijia, Alazab M, et al. Joint optimal quantization and aggregation of federated learning scheme in VANETs[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(10): 19852−19863 doi: 10.1109/TITS.2022.3145823
[31]	Van Tilborg H C A, Jajodia S. Encyclopedia of Cryptography and Security[M]. Berlin: Springer, 2014
[32]	Sun Lichao, Qian Jianwei, Chen Xun. LDP-FL: Practical private aggregation in federated learning with local differential privacy[J]. arXiv preprint, arXiv: 2007.15789, 2020
[33]	Li Tian, Sahu A K, Zaheer M, et al. Federated optimization in heterogeneous networks[J]. Proceedings of the Machine Learning and Systems, 2020, 2: 429−450
[34]	Karimireddy S P, Kale S, Mohri M, et al. SCAFFOLD: Stochastic controlled averaging for federated learning[C]//Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 5132−5143
[35]	Wang Jianyu, Liu Qinghua, Liang Hao, et al. Tackling the objective inconsistency problem in heterogeneous federated optimization[J]. Advances in Neural Information Processing Systems, 2020, 33: 7611−7623
[36]	Google. StateFarm[EB/OL]. 2016[2023-02-01].https://www.kaggle.com/c/state-farm-distracted-driver-detection

[1]	Zhang Qiang, Ye Ayong, Ye Guohua, Deng Huina, Chen Aimin. k-Anonymous Data Privacy Protection Mechanism Based on Optimal Clustering[J]. Journal of Computer Research and Development, 2022, 59(7): 1625-1635. DOI: 10.7544/issn1000-1239.20210117
[2]	Fu Yao, Li Qingdan, Zhang Zehui, Gao Tiegang. Data Integrity Verification Scheme for Privacy Protection and Fair Payment[J]. Journal of Computer Research and Development, 2022, 59(6): 1343-1355. DOI: 10.7544/issn1000-1239.20210023
[3]	Zhang Shaobo, Wang Guojun, Liu Qin, Liu Jianxun. Trajectory Privacy Protection Method Based on Multi-Anonymizer[J]. Journal of Computer Research and Development, 2019, 56(3): 576-584. DOI: 10.7544/issn1000-1239.2019.20180033
[4]	Wang Ziyu, Liu Jianwei, Zhang Zongyang, Yu Hui. Full Anonymous Blockchain Based on Aggregate Signature and Confidential Transaction[J]. Journal of Computer Research and Development, 2018, 55(10): 2185-2198. DOI: 10.7544/issn1000-1239.2018.20180430
[5]	Jiang Huowen, Zeng Guosun, Hu Kekun. A Graph-Clustering Anonymity Method Implemented by Genetic Algorithm for Privacy-Preserving[J]. Journal of Computer Research and Development, 2016, 53(10): 2354-2364. DOI: 10.7544/issn1000-1239.2016.20160435
[6]	Dai Hua, Yang Geng, Xiao Fu, Zhou Qiang, He Ruiliang. An Energy-Efficient and Privacy-Preserving Range Query Processing in Two-Tiered Wireless Sensor Networks[J]. Journal of Computer Research and Development, 2015, 52(4): 983-993. DOI: 10.7544/issn1000-1239.2015.20140066
[7]	Chen Wei, Xu Ruomei, Li Yuling. A Privacy-Preserving Integrity-Verification-Based Top-k Query Processing[J]. Journal of Computer Research and Development, 2014, 51(12): 2585-2592. DOI: 10.7544/issn1000-1239.2014.20140666
[8]	Dai Hua, Yang Geng, Qin Xiaolin, Liu Liang. Privacy-Preserving Top-k Query Processing in Two-Tiered Wireless Sensor Networks[J]. Journal of Computer Research and Development, 2013, 50(6): 1239-1252.
[9]	Xu Yong, Qin Xiaolin, Yang Yitao, Yang Zhongxue, Huang Can. A QI Weight-Aware Approach to Privacy Preserving Publishing Data Set[J]. Journal of Computer Research and Development, 2012, 49(5): 913-924.
[10]	Liu Yubao, Huang Zhilan, Ada Wai Chee Fu, Yin Jian. A Data Privacy Preservation Method Based on Lossy Decomposition[J]. Journal of Computer Research and Development, 2009, 46(7): 1217-1225.