Detecting Free-Riding Attack in Federated Learning Based on Gradient Backtracking

Hong Zhen; Feng Wanglei; Wen Zhenyu; Wu Di; Li Taotao; Wu Yiming; Wang Cong; Ji Shouling

doi:10.7544/issn1000-1239.202330886

Journal of Computer Research and Development > 2024 > 61(9): 2185-2198. > DOI: 10.7544/issn1000-1239.202330886 CSTR: 32373.14.issn1000-1239.202330886

Hong Zhen, Feng Wanglei, Wen Zhenyu, Wu Di, Li Taotao, Wu Yiming, Wang Cong, Ji Shouling. Detecting Free-Riding Attack in Federated Learning Based on Gradient Backtracking[J]. Journal of Computer Research and Development, 2024, 61(9): 2185-2198. DOI: 10.7544/issn1000-1239.202330886

Citation:

PDF (1799 KB)

Detecting Free-Riding Attack in Federated Learning Based on Gradient Backtracking

Hong Zhen^{1, 2,},
Feng Wanglei¹,
Wen Zhenyu^{1, 2},
Wu Di^3,,
Li Taotao¹,
Wu Yiming^{1, 2, ,},
Wang Cong⁴,
Ji Shouling⁵

1.
College of Information Engineering, Zhejiang University of Technology, Hangzhou 310023
2.
Institute of Cyberspace Security, Zhejiang University of Technology, Hangzhou 310023
3.
College of Computer Science, University of St Andrews, St Andrews KY16 9AJ
4.
College of Control Science and Engineering, Zhejiang University, Hangzhou 310007
5.
College of Computer Science and Technology, Zhejiang University, Hangzhou 310007

Funds: This work was supported by the National Natural Science Foundation of China (62072408，62302454), the Natural Science Foundation of Zhejiang Province for Distinguished Young Scholars (LR24F020004), the Major Program of the Natural Science Foundation of Zhejiang Province (Youth Original Project) (LDQ24F020001), and the China Postdoctoral Science Foundation (2023M743403).

More Information

Author Bio:
Hong Zhen: born in 1983. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include Internet of things/information physical systems, intelligent systems security, big data analytics, and artificial intelligence

Feng Wanglei: born in 1997. Master candidate. His main research interests include federated learning and distributed machine learning

Wen Zhenyu: born in 1987. PhD, professor, PhD supervisor. Member of CCF. His main research interests include IoT, crowd sources, AI system, and cloud computing

Wu Di: born in 1993. PhD candidate. His main research interests include federated learning, distributed machine learning, edge computing, model compression, and Internet-of-Things

Li Taotao: born in 1996. PhD candidate. His main research interests include Web mining, information retrieval, machine learning

Wu Yiming: born in 1996. PhD, associate professor, master supervisor. Member of CCF. Her main research interests include data-driven security, black industry mining, and cybercrime research

Wang Cong: born in 1985. PhD, professor, PhD supervisor. Member of CCF. His main research interests include addressing security and privacy challenges in mobile, cloud computing, IoT, and machine learning and system

Ji Shouling: born in 1986. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include data-driven security and privacy, AI security, and big data mining and analytics
Received Date: October 31, 2023
Revised Date: May 19, 2024
Available Online: May 29, 2024

Graphical Abstract

Abstract

Abstract

With the development of the Internet of vehicles (IoV), the rapid growth of intelligent vehicles generates a massive amount of data. These data are invaluable for training intelligent IoV application models. Traditional model training requires the centralized collection of raw data through the cloud, consuming substantial communication resources and facing issues like privacy breaches and regulatory constraints. Federated learning (FL) offers a solution by using model transfer instead of data transfer to tackle these challenges. However, practical FL systems are confronted with the issue of malicious users attempting to deceive the server by uploading false local models, known as free-riding attacks. These attacks significantly undermine the fairness and effectiveness of FL. Current research assumes that free-riding attacks are limited to a small number of rational users. However, when there are multiple malicious free-riders, current research falls short in effectively detecting and defending against these attackers. To address this issue, we introduce a novel gradient backtracking based algorithm to identify free-riders. We introduce random testing rounds into standard FL and compare the similarity of user’s gradient between the testing round and the comparison round. It overcomes the challenge of ineffective defense in scenarios involving multiple malicious free-riders. Experimental results on the MNIST and CIFAR-10 datasets demonstrate that the proposed detection algorithm achieves outstanding performance in various free-riding attack scenarios.
- federated learning,
- Internet of vehicles,
- free-riding attack,
- gradient similarity,
- free-riding attack detection

FullText(HTML)

References (27)

References

[1]	况博裕,李雨泽,顾芳铭,等. 车联网安全研究综述:威胁、对策与未来展望[J]. 计算机研究与发展,2023,60(10):2304−2321 Kuang Boyu, Li Yuze, Gu Fangming, et al. Review of Internet of vehicle security research: Threats, countermeasures, and future prospects[J]. Journal of Computer Research and Development, 2023, 60(10): 2304−2321 (in Chinese)
[2]	郑笛,王俊,贲可荣. 扩展车联网应用中的海量传感器信息处理技术[J]. 计算机研究与发展,2013,50(S2):257−266 Zheng Di, Wang Jun, Ben Kerong. Information processing for massive sensorsin extended IOV applications[J]. Journal of Computer Research and Development, 2013, 50(S2): 257−266 (in Chinese)
[3]	Jung K, Lee J, Park K, et al. PRIVATA: Differentially private data market framework using negotiation-based pricing mechanism[C]//Proc of the 28th ACM Int Conf on Information and Knowledge Management. New York: ACM, 2019: 2897−2900
[4]	Sun Jingwei, Li Ang, Wang Binghui, et al. Soteria: Provable defense against privacy leakage in federated learning from representation perspective[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 9311−9319
[5]	董业,侯炜,陈小军,等. 基于秘密分享和梯度选择的高效安全联邦学习[J]. 计算机研究与发展,2020,57(10):2241−2250 Dong Ye, Hou Wei, Chen Xiaojun, et al. Efficient and secure federated learning based on secret sharing and gradients selection[J]. Journal of Computer Research and Development, 2020, 57(10): 2241−2250 (in Chinese)
[6]	Cheng Yong, Liu Yang, Chen Tianjian, et al. Federated learning for privacy-preserving AI[J]. Communications of the ACM, 2020, 63(12): 33−36 doi: 10.1145/3387107
[7]	Deng Yongheng, Lyu F, Ren Ju, et al. AUCTION: Automated and quality-aware client selection framework for efficient federated learning[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 33(8): 1996−2009
[8]	Chen Jinyin, Li Mingjun, Liu Tao, et al. Rethinking the defense against free-rider attack from the perspective of model weight evolving frequency[J]. arXiv preprint, arXiv: 2206.05406, 2022
[9]	Zhang Ning, Ma Qian, Chen Xu. Enabling long-term cooperation in cross-silo federated learning: A repeated game perspective[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 3910−3924 doi: 10.1109/TMC.2022.3148263
[10]	Lin Jierui, Du Min, Liu Jian. Free-riders in federated learning: Attacks and defenses[J]. arXiv preprint, arXiv: 1911.12560, 2019
[11]	Fraboni Y, Vidal R, Lorenzi M. Free-rider attacks on model aggregation in federated learning[C]//Proc of the 24th Int Conf on Artificial Intelligence and Statistics. Brookline, MA: Microtome Publishing, 2021: 1846−1854
[12]	Karimireddy S P, Guo Wenshuo, Jordan M I. Mechanisms that incentivize data sharing in federated learning[J]. arXiv preprint, arXiv: 2207.04557, 2022
[13]	Huang Hai, Zhang Borong, Sun Yinggang, et al. Delta-DAGMM: A free rider attack detection model in horizontal federated learning[J]. Security and Communication Networks, 2022, 2022(1): 310−322
[14]	Bernstein J, Zhao Jiawei, Azizzadenesheli K, et al. SignSGD with majority vote is communication efficient and fault tolerant[J]. arXiv preprint, arXiv: 1810.05291, 2018
[15]	Xu Xinyi, Lyu Lingjuan. A reputation mechanism is all you need: Collaborative fairness and adversarial robustness in federated learning[J]. arXiv preprint, arXiv: 2011.10464, 2020
[16]	Yin Dong, Chen Yudong, Kannan R, et al. Byzantine-robust distributed learning: Towards optimal statistical rates[C]//Proc of the 35th Int Conf on Machine Learning. New York: ACM, 2018: 5650−5659
[17]	Zong Bo, Song Qi, Min M R, et al. Deep autoencoding gaussian mixture model for unsupervised anomaly detection[C/OL]//Proc of the 6th Int Conf on Learning Representations. Brookline, MA: Microtome Publishing, 2018[2023-10-31].https://openreview.net/forum?id=BJJLHbb0-
[18]	McMahan B, Moore E, Ramage D, et al. Communication-efficient learning of deep networks from decentralized data[C]//Proc of the 20th Artificial Intelligence and Statistics. Brookline, MA: Microtome Publishing, 2017: 1273−1282
[19]	Wang Dong, Lu Huchuan, Bo Chunjuan. Visual tracking via weighted local cosine similarity[J]. IEEE Transactions on Cybernetics, 2014, 45(9): 1838−1850
[20]	Zhang J, Qiao Guanxiong, Lopotenco A, et al. Understanding stochastic optimization behavior at the layer update level[C]//Proc of the 36th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2022: 13109−13110
[21]	Makey G, Yavuz Ö, Kesim D K, et al. Breaking crosstalk limits to dynamic holography using orthogonality of high-dimensional random vectors[J]. Nature Photonics, 2019, 13(4): 251−256 doi: 10.1038/s41566-019-0393-7
[22]	LaValley M P. Logistic regression[J]. Circulation, 2008, 117(18): 2395−2399 doi: 10.1161/CIRCULATIONAHA.106.682658
[23]	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proc of IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2016: 770−778
[24]	Deng Li. The MNIST database of handwritten digit images for machine learning research[J]. IEEE Signal Processing Magazine, 2012, 29(6): 141−142 doi: 10.1109/MSP.2012.2211477
[25]	Krizhevsky A , Hinton G. Learning multiple layers of features from tiny images[D]. Toronto, Canada: Department of Computer Science, University of Toronto, 2009
[26]	He Chaoyang, Li Songze, So Jinhyun, et al. FedML: A research library and benchmark for federated machine learning[J]. arXiv preprint, arXiv: 2007.13518, 2020
[27]	Li Qinbin, Diao Yiqun, Chen Quan, et al. Federated learning on non-IID data silos: An experimental study[C]//Proc of the 38th Int Conf on Data Engineering. Piscataway, NJ: IEEE, 2022: 965−978

[1]	Zhang Huijuan, Huang Qinyang, Hu Shiyan, Yang Qing, Zhang Jingwei. Link Prediction Driven by High-Order Relations in Complete Graph[J]. Journal of Computer Research and Development, 2024, 61(7): 1825-1835. DOI: 10.7544/issn1000-1239.202221045
[2]	Ye Guixin, Zhang Yuxiang, Zhang Cheng, Zhao Jiaqi, Wang Huanting. Automatic Optimization Heuristics Method for OpenCL Program Based on Graph Neural Network[J]. Journal of Computer Research and Development, 2023, 60(5): 1121-1135. DOI: 10.7544/issn1000-1239.202110943
[3]	Xie Xiaojie, Liang Ying, Wang Zisen, Liu Zhengjun. Heterogeneous Network Node Classification Method Based on Graph Convolution[J]. Journal of Computer Research and Development, 2022, 59(7): 1470-1485. DOI: 10.7544/issn1000-1239.20210124
[4]	Wu Yao, Shen Derong, Kou Yue, Nie Tiezheng, Yu Ge. Heterogeneous Information Networks Embedding Based on Multiple Meta-Graph Fusion[J]. Journal of Computer Research and Development, 2020, 57(9): 1928-1938. DOI: 10.7544/issn1000-1239.2020.20190553
[5]	Chen Yiqi, Qian Tieyun, Li Wanli, Liang Yile. Exploiting Composite Relation Graph Convolution for Attributed Network Embedding[J]. Journal of Computer Research and Development, 2020, 57(8): 1674-1682. DOI: 10.7544/issn1000-1239.2020.20200206
[6]	Wang Yongxian, Zhang Lilun, Che Yonggang, Xu Chuanfu, Liu Wei, Cheng Xinghua. Heterogeneous Computing and Optimization on Tianhe-²,Supercomputer System for High-Order Accurate CFD Applications[J]. Journal of Computer Research and Development, 2015, 52(4): 833-842. DOI: 10.7544/issn1000-1239.2015.20131922
[7]	Yang Xinxin, Huang Shaobin. A Hierarchical Co-Clustering Algorithm for High-Order Heterogeneous Data[J]. Journal of Computer Research and Development, 2015, 52(1): 200-210. DOI: 10.7544/issn1000-1239.2015.20130493
[8]	Huang Mingkai, Liu Xianhua, Tan Mingxing, Xie Zichao, Cheng Xu. An Indirect Branch Prediction for Interpreters[J]. Journal of Computer Research and Development, 2015, 52(1): 66-82. DOI: 10.7544/issn1000-1239.2015.20130970
[9]	Jia Ning, Yang Chun, Tong Dong, and Wang Keyi. Correlated Software Prediction for Indirect Branch in Dynamic Translation Systems[J]. Journal of Computer Research and Development, 2014, 51(3): 661-671.
[10]	Yang Zhizhuo and Huang Heyan. WSD Method Based on Heterogeneous Relation Graph[J]. Journal of Computer Research and Development, 2013, 50(2): 437-444.

Cited By

Cited by

Periodical cited type(4)

1.	杨洁祎，董一鸿，钱江波 . 基于图神经网络的小样本学习方法研究进展. 计算机研究与发展. 2024(04): 856-876 . 本站查看
2.	秦志龙，邓琨，刘星妍. 基于元路径卷积的异构图神经网络算法. 电信科学. 2024(03): 89-103 .
3.	白宇康，陈彦敏，樊小超，孙睿军，李炜杰. 图神经网络和数值诱导正则化的数值推理方法. 智能系统学报. 2024(05): 1268-1276 .
4.	陈东洋，郭进利. 基于图注意力的高阶网络节点分类方法. 计算机应用研究. 2023(04): 1095-1100+1136 .