Secure and Efficient Federated Learning for Continuous IoV Data Sharing

Le Junqing; Tan Zhouyong; Zhang Di; Liu Gao; Xiang Tao; Liao Xiaofeng

doi:10.7544/issn1000-1239.202330894

Journal of Computer Research and Development > 2024 > 61(9): 2199-2212. > DOI: 10.7544/issn1000-1239.202330894 CSTR: 32373.14.issn1000-1239.202330894

Le Junqing, Tan Zhouyong, Zhang Di, Liu Gao, Xiang Tao, Liao Xiaofeng. Secure and Efficient Federated Learning for Continuous IoV Data Sharing[J]. Journal of Computer Research and Development, 2024, 61(9): 2199-2212. DOI: 10.7544/issn1000-1239.202330894

Citation:

PDF (2636 KB)

Secure and Efficient Federated Learning for Continuous IoV Data Sharing

College of Computer Science, Chongqing University, Chongqing 400044
Key Laboratory of Dependable Service Computing in Cyber Physical Society (Chongqing University), Ministry of Education, Chongqing 400044

Funds: The work was supported by the National Key Research and Development Program of China (2022YFB3103500), the National Natural Science Foundation of China (61932006, 62202071, 62302072), the China Postdoctoral Science Foundation (2022M710518, 2022M710520), and the Natural Science Foundation of Chongqing (CSTB2022NSCQ-MSX0358, CSTB2022NSCQ-MSX1217).

More Information

Author Bio:
Le Junqing: born in 1991. PhD, research assistant. Member of CCF. His main research interests include privacy protection, federated learning, and information security

Tan Zhouyong: born in 1997. Master. His main research interests include privacy protection and federated learning

Zhang Di: born in 1993. PhD, research assistant. Her main research interests include privacy protection, blockchain, and cryptography

Liu Gao: born in 1991. PhD, research assistant. His main research interests include Internet of vehicles, blockchain, and privacy protection

Xiang Tao: born in 1980. PhD, professor, PhD supervisor. Member of CCF. His main research interests include privacy protection, information security, and machine learning

Liao Xiaofeng: born in 1964. PhD, professor, PhD supervisor. Member of CCF. His main research interests include neural network, privacy protection, and cryptography
Received Date: October 31, 2023
Revised Date: May 19, 2024
Available Online: June 12, 2024

Graphical Abstract

Abstract

Abstract

The combination of the Internet of vehicles (IoV) and artificial intelligence (AI) has driven the rapid development of autonomous vehicles. Sharing IoV data distributed across different vehicles for training AI models enables more efficient and reliable intelligent driving services. Autonomous vehicles can continuously gather real-time vehicle information, road images and videos among other IoV data, through onboard cameras and sensors. This data are then utilized to optimize and update intelligent traffic models, addressing issues where changes in IoV data result in decreased model accuracy. We propose an efficient and secure federated learning scheme (named as SEFL) for continuous data sharing in an IoV environment to address the problems related to inefficient data collection, catastrophic forgetting problems due to dynamic data updates and privacy leakage from model training parameters. In SEFL, to enable the automatic collection of training samples, each vehicle is based on the global model to only collect IoV data with lower recognition accuracy, and the output with the highest probability is used as the label for that sample. Since vehicle storage space is limited and new samples can overwrite old ones, the data on vehicles are dynamically changing, making traditional fine-tuning training methods prone to catastrophic forgetting. Thus, a dual-knowledge distillation-based training algorithm is proposed in SEFL to ensure that the model learns the knowledge of each sample, maintaining high accuracy. Besides, to prevent privacy leakage from the model parameters between vehicles and servers, an adaptive differential privacy strategy is proposed to achieve client-level privacy protection. Simultaneously, this strategy minimizes the negative impact of differential privacy noise on the accuracy of the global model. Finally, a security analysis and performance evaluation of SEFL scheme are conducted using the GTSRB dataset and vehicle identification dataset. The analysis and experimental results indicate that the proposed SEFL scheme can provide strong privacy protection and efficient data collection. Furthermore, SEFL outperforms existing federated learning-based algorithms in terms of model accuracy.
- Internet of vehicles,
- continuous sharing,
- federated learning,
- differential privacy,
- privacy protection

FullText(HTML)

References (38)

References

[1]	Yang Fangchun, Wang Shangguang, Li Jinglin, et al. An overview of Internet of vehicles[J]. China Communications, 2014, 11(10): 1−15
[2]	Muhammad K, Ullah A, Lloret J, et al. Deep learning for safe autonomous driving: Current challenges and future directions[J]. IEEE Transactions on Intelligent Transportation Systems, 2020, 22(7): 4316−4336
[3]	刘占文,赵祥模,李强,等. 基于图模型与卷积神经网络的交通标志识别方法[J]. 交通运输工程学报,2016,16(5):122−131 doi: 10.3969/j.issn.1671-1637.2016.05.014 Liu Zhanwen, Zhao Xiangmo, Li Qiang, et al. A traffic sign recognition method based on graph models and convolutional neural networks[J]. Journal of Transportation Engineering, 2016, 16(5): 122−131 (in Chinese) doi: 10.3969/j.issn.1671-1637.2016.05.014
[4]	Konečný J, McMahan H B, Ramage D, et al. Federated optimization: Distributed machine learning for on-device intelligence[J]. arXiv preprint, arXiv: 1610.02527, 2016
[5]	McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Int Conf on Artificial Intelligence and Statistics. Cambridge, MA: MIT, 2017: 1273−1282
[6]	Zhao Yue, Li Meng, Lai Liangzhen, et al. Federated learning with non-IID data[J]. arXiv preprint, arXiv: 1806.00582, 2018
[7]	Bonawitz K, Eichner H, Grieskamp W, et al. Towards federated learning at scale: System design[C/OL]//Proc of the 2nd Conf on Machine Learning and Systems. 2019[2024-03-12].https://proceedings.mlsys.org/paper_files/ paper/2019/file/7b770da633baf74895be22a8807f1a8f-Paper.pdf
[8]	刘飚,张方佼,王文鑫,等. 基于矩阵映射的拜占庭鲁棒联邦学习算法[J]. 计算机研究与发展,2021,58(11):2416−2429 doi: 10.7544/issn1000-1239.2021.20210633 Liu Biao, Zhang Fangjiao, Wang Wenxin, et al. Byzantine-robust federated learning algorithm based on matrix mapping[J]. Journal of Computer Research and Development, 2021, 58(11): 2416−2429 (in Chinese) doi: 10.7544/issn1000-1239.2021.20210633
[9]	Manias M D, Shami A. Making a case for federated learning in the Internet of vehicles and intelligent transportation systems[J]. IEEE Network, 2021, 35(3): 88−94 doi: 10.1109/MNET.011.2000552
[10]	Xing Ling, Zhao Pengcheng, Gao Jianping, et al. A survey of the social Internet of vehicles: Secure data issues, solutions, and federated learning[J]. IEEE Intelligent Transportation Systems Magazine, 2022, 15(2): 70−84
[11]	Xie Kan, Zhang Zhe, Li Bo, et al. Efficient federated learning with spike neural networks for traffic sign recognition[J]. IEEE Transactions on Vehicular Technology, 2022, 71(9): 9980−9992 doi: 10.1109/TVT.2022.3178808
[12]	Stergiou K D, Psannis K E, Vitsas V, et al. A federated learning approach for enhancing autonomous vehicles image recognition[C]//Proc of the 4th Int Conf on Computer Communication and the Internet. Berlin: Springer, 2022: 87−90
[13]	Liang Feiyuan, Yang Qinglin, Liu Ruiqi, et al. Semi-synchronous federated learning protocol with dynamic aggregation in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2022, 71(5): 4677−4691 doi: 10.1109/TVT.2022.3148872
[14]	Zhou Xiaokang, Liang Wei, She Jinhua, et al. Two-layer federated learning with heterogeneous model aggregation for 6G supported Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2021, 70(6): 5308−5317 doi: 10.1109/TVT.2021.3077893
[15]	Zhou Hongliang, Zheng Yifeng, Huang Hejiao, et al. Toward robust hierarchical federated learning in Internet of vehicles[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(5): 5600−5614 doi: 10.1109/TITS.2023.3243003
[16]	Li Zhizhon, Hoiem D. Learning without forgetting[J]. IEEE Transactions on Pattern Analysis and Aachine Intelligence, 2017, 40(12): 2935−2947
[17]	Kirkpatrick J, Pascanu R, Rabinowitz N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences, 2017, 114(13): 3521−3526 doi: 10.1073/pnas.1611835114
[18]	Rebuffi S A, Kolesnikov A, Sperl G, et al. iCaRL: Incremental classifier and representation learning[C]//Proc of the 30th IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2017: 2001−2010
[19]	Fredrikson M, Jha S, Ristenpart T. Model inversion attacks that exploit confidence information and basic countermeasures[C]//Proc of the 22nd ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2015: 1322−1333
[20]	Shokri R, Stronati M, Song Congzheng, et al. Membership inference attacks against machine learning models[C]//Proc of the 38th IEEE Symp on Security and Privacy. Piscataway, NJ: IEEE, 2017: 3−18
[21]	He Zecheng, Zhang Tianwei, Lee R B. Model inversion attacks against collaborative inference[C]//Proc of the 35th Annual Computer Security Applications Conf. New York: ACM, 2019: 148−162
[22]	周纯毅,陈大卫,王尚,等. 分布式深度学习隐私与安全攻击研究进展与挑战[J]. 计算机研究与发展,2021,58(5):927−943 doi: 10.7544/issn1000-1239.2021.20200966 Zhou Chunyi, Chen Dawei, Wang Shang, et al. Research progress and challenges in privacy and security attacks on distributed deep learning[J]. Journal of Computer Research and Development, 2021, 58(5): 927−943 (in Chinese) doi: 10.7544/issn1000-1239.2021.20200966
[23]	Bonawitz K, Ivanov V, Kreuter B, et al. Practical secure aggregation for privacy-preserving machine learning[C]//Proc of the 24th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2017: 1175−1191
[24]	Fang Chen, Guo Yuanbo, Wang Na, et al. Highly efficient federated learning with strong privacy preservation in cloud computing[J]. Computers & Security, 2020, 96: 101889
[25]	Lu Yunlong, Huang Xiaohong, Zhang Ke, et al. Blockchain empowered asynchronous federated learning for secure data sharing in Internet of vehicles[J]. IEEE Transactions on Vehicular Technology, 2020, 69(4): 4298−4311 doi: 10.1109/TVT.2020.2973651
[26]	Shokri R, Shmatikov V. Privacy-preserving deep learning[C]//Proc of the 22nd ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2015: 1310−1321
[27]	McMahan H B, Ramage D, Talwar K, et al. Learning differentially private recurrent language models[J]. arXiv preprint, arXiv: 1710.06963, 2017
[28]	Wei Kang, Li Jun, Ding Ming, et al. Federated learning with differential privacy: Algorithms and performance analysis[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 3454−3469 doi: 10.1109/TIFS.2020.2988575
[29]	Kairouz P, McMahan H B, Avent B, et al. Advances and open problems in federated learning[J]. Foundations and Trends® in Machine Learning, 2021, 14(1/2): 1−210
[30]	Dwork C, Roth A. The algorithmic foundations of differential privacy[J]. Foundations and Trends® in Theoretical Computer Science, 2014, 9(3/4): 211−407
[31]	Dwork C. Differential privacy: A survey of results[C]//Proc of the 5th Int Conf on Theory and Applications of Models of Computation. Berlin: Springer, 2008: 1−19
[32]	Abadi M, Chu A, Goodfellow I, et al. Deep learning with differential privacy[C]//Proc of the 23rd ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2016: 308−318
[33]	Amin K, Kulesza A, Munoz A, et al. Bounding user contributions: A bias-variance trade-off in differential privacy[C]//Proc of the 36th Int Conf on Machine Learning. New York: ACM, 2019: 263−271
[34]	Andrew G, Thakkar O, McMahan B, et al. Differentially private learning with adaptive clipping[J]. Advances in Neural Information Processing Systems, 2021, 34: 17455−17466
[35]	Le Junqing, Zhang Di, Lei Xinyu, et al. Privacy-preserving federated learning with malicious clients and honest-but-curious servers[J]. IEEE Transactions on Information Forensics and Security, 2023, 18: 4329−4344 doi: 10.1109/TIFS.2023.3295949
[36]	Dwork C, Lei Jing. Differential privacy and robust statistics[C]//Proc of the 41st Annual ACM Symp on Theory of Computing. New York: ACM, 2009: 371−380
[37]	Gou Jianping, Yu Baosheng, Maybank S J, et al. Knowledge distillation: A survey[J]. International Journal of Computer Vision, 2021, 129(6): 1789−1819 doi: 10.1007/s11263-021-01453-z
[38]	Fan Liyue, Li Xiong. An adaptive approach to real-time aggregate monitoring with differential privacy[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(9): 2094−2106 doi: 10.1109/TKDE.2013.96

[1]	Li Dongwen, Zhong Zhenyu, Sun Yufei, Shen Junyu, Ma Zizhi, Yu Chuanyue, Zhang Yuzhi. LingLong: A High-Quality Small-Scale Chinese Pre-trained Language Model[J]. Journal of Computer Research and Development, 2025, 62(3): 682-693. DOI: 10.7544/issn1000-1239.202330844
[2]	Jiang Yi, Yang Yong, Yin Jiali, Liu Xiaolei, Li Jiliang, Wang Wei, Tian Youliang, Wu Yingcai, Ji Shouling. A Survey on Security and Privacy Risks in Large Language Models[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440265
[3]	Zhang Naizhou, Cao Wei, Zhang Xiaojian, Li Shijun. Conversation Generation Based on Variational Attention Knowledge Selection and Pre-trained Language Model[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440551
[4]	Yi Xiaoyuan, Xie Xing. Unpacking the Ethical Value Alignment in Big Models[J]. Journal of Computer Research and Development, 2023, 60(9): 1926-1945. DOI: 10.7544/issn1000-1239.202330553
[5]	Feng Jun, Shi Yichen, Gao Yuhao, He Jingjing, Yu Zitong. Domain Adaptation for Face Anti-Spoofing Based on Dual Disentanglement and Liveness Feature Progressive Alignment[J]. Journal of Computer Research and Development, 2023, 60(8): 1727-1739. DOI: 10.7544/issn1000-1239.202330251
[6]	Zeng Weixin, Zhao Xiang, Tang Jiuyang, Tan Zhen, Wang Wei. Iterative Entity Alignment via Re-Ranking[J]. Journal of Computer Research and Development, 2020, 57(7): 1460-1471. DOI: 10.7544/issn1000-1239.2020.20190643
[7]	Shi Haihe, Zhou Weixing. Design and Implementation of Pairwise Sequence Alignment Algorithm Components Based on Dynamic Programming[J]. Journal of Computer Research and Development, 2019, 56(9): 1907-1917. DOI: 10.7544/issn1000-1239.2019.20180835
[8]	Jia Xibin, Jin Ya, Chen Juncheng. Domain Alignment Based on Multi-Viewpoint Domain-Shared Feature for Cross-Domain Sentiment Classification[J]. Journal of Computer Research and Development, 2018, 55(11): 2439-2451. DOI: 10.7544/issn1000-1239.2018.20170496
[9]	Wang Yuquan, Wen Lijie, Yan Zhiqiang. Alignment Based Conformance Checking Algorithm for BPMN 2.0 Model[J]. Journal of Computer Research and Development, 2017, 54(9): 1920-1930. DOI: 10.7544/issn1000-1239.2017.20160756
[10]	Zhuang Yan, Li Guoliang, Feng Jianhua. A Survey on Entity Alignment of Knowledge Base[J]. Journal of Computer Research and Development, 2016, 53(1): 165-192. DOI: 10.7544/issn1000-1239.2016.20150661