Security of Deep Neural Network Supply Chains: A Survey

Chen Ruoxi; Chen Jinyin; Zheng Haibin; Yang Xueyan; Ji Shouling; Chen Tieming

doi:10.7544/issn1000-1239.202440327

Journal of Computer Research and Development > 2025 > Accepted Manuscript > DOI: 10.7544/issn1000-1239.202440327 CSTR: 32373.14.issn1000-1239.202440327

Chen Ruoxi, Chen Jinyin, Zheng Haibin, Yang Xueyan, Ji Shouling, Chen Tieming. Security of Deep Neural Network Supply Chains: A Survey[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202440327

Citation:

PDF (1280 KB)

Security of Deep Neural Network Supply Chains: A Survey

1.
College of Information Engineering, Zhejiang University of Technology University, Hangzhou 310023
2.
College of Institute of Cyberspace Security, Zhejiang University of Technology University, Hangzhou 310023
3.
College of Computer Science and Technology, Zhejiang University, Hangzhou 310007
4.
College of Computer Science and Technology, Zhejiang University of Technology University, Hangzhou 310023

Funds: This work was supported by the Zhejiang Provincial Natural Science Foundation (LDQ23F020001), the National Natural Science Foundation of China (62072406, 62406286), the Key R&D Projects in Zhejiang Province (2022C01018), and the National Key R&D Projects of China (2018AAA0100801).

More Information

Author Bio:
Chen Ruoxi: born in 1998. PhD student. Student member of CCF. Her main research interests include deep learning and artificial intelligence security

Chen Jinyin: born in 1982. PhD. Professor. Member of CCF. Her main research interests include artificial intelligence security, graph data mining and evolutionary computing

Zheng Haibin: born in 1995. PhD. Member of CCF. His main research interests include deep learning, artificial intelligence security, and fairness algorithm

Yang Xueyan: born in 2003. Undergraduate student. Her main research interests include deep learning and artificial intelligence security

Ji Shouling: born in 1986. PhD. Senior member of CCF. ZJU 100-Young Professor in the College of Computer Science and Technology at Zhejiang University. His current research interests include Data-driven Security and Privacy, AI Security and Big Data Analytics

Chen Tieming: born in 1978. PhD. Professor. His main research interests include network and information security
Received Date: May 20, 2024
Revised Date: August 01, 2024
Accepted Date: January 08, 2025
Available Online: January 08, 2025

Graphical Abstract

Abstract

Abstract

Pre-trained models have mitigated the challenges posed by extensive training data and computational resources, and also give birth to the new paradigm of model development and application, which we refer to as model supply chain. In this framework, a pre-trained model is uploaded by its publisher and subsequently transferred, compressed, and deployed by secondary developers to meet various application needs. This emerging model supply chain introduces additional stages and multiple elements, inevitably leading to security concerns and privacy risks. Despite the widespread adoption of model supply chains, there is currently a lack of systematic review of security threats in them. To address this research gap, in this paper, we provide a comprehensive overview of the deep learning model supply chain, introducing its concept and fundamental structure. We conduct an in-depth analysis of vulnerabilities at various stages of the model’s lifecycle, including design, development, deployment, and usage. Furthermore, we compare and summarize prevalent attack methods, alongside introducing corresponding security protection strategies. To assist readers in effectively utilizing pre-trained models, we review and compare publicly available model repositories. Finally, we discuss potential future research avenues in areas such as security checks, real-time detection, and problem tracing. It aims to offer insights for safer and more reliable development and use of pre-training models. For the benefit of ongoing research, related papers and open-source codes of the methods discussed are accessible at https://github.com/Dipsy0830/DNN-supply-chain-survey.
- model supply chain,
- pre-trained models,
- security threats,
- security and privacy,
- deep neural networks,
- deep learning

FullText(HTML)

References (117)

References

[1]	Hugging Face. Hugging Face―The AI community building the future[EB/OL]. (2016-03-01)[2022-03-21]. https: //huggingface.co/
[2]	Yu Koh J. Model Zoo―Deep learning code and pretrained models[EB/OL]. (2018-06-14)[2023-03-15]. https://modelzoo.co/
[3]	Qiu Xipeng, Sun Tianxiang, Xu Yige, et al. Pre-trained models for natural language processing: A survey[J]. Science China Technological Sciences, 2020, 63(10): 1872−1897 doi: 10.1007/s11431-020-1647-3
[4]	Simonyan K, Zisserman A. Very deep convolutional networks for large-scale image recognition[J]. arXiv preprint, arXiv: 1409.1556, 2014
[5]	Baracaldo N, Oprea A. Machine learning security and privacy[J]. IEEE Security & Privacy, 2022, 20(5): 11−13
[6]	Liu Ximeng, Xie Lehui, Wang Yaopeng, et al. Privacy and security issues in deep learning: A survey[J]. IEEE Access, 2020, 9: 4566−4593
[7]	Xue Mingfu, Yuan Chengxiang, Wu Heyi, et al. Machine learning security: Threats, countermeasures, and evaluations[J]. IEEE Access, 2020, 8: 74720−74742 doi: 10.1109/ACCESS.2020.2987435
[8]	Kenton J D M W C, Toutanova L K. BERT: Pre-training of deep bidirectional transformers for language understanding[C]//Proc of the 17th NAACL-HLT. Stroudsburg, PA: ACL, 2019: 4171−4186
[9]	Dosovitskiy A, Beyer L, Kolesnikov A, et al. An image is worth 16x16 words: Transformers for image recognition at scale[J]. arXiv preprint, arXiv: 2010.11929, 2020
[10]	Radford A, Kim J W, Hallacy C, et al. Learning transferable visual models from natural language supervision[C]//Proc of the 18th Int Conf on Machine Learning. New York: PMLR, 2021: 8748−8763
[11]	Liu Haotian, Li Chunyuan, Wu Qingyang, et al. Visual instruction tuning[J]. arXiv preprint, arXiv: 2304.08485, 2023
[12]	Rombach R, Blattmann A, Lorenz D, et al. High-resolution image synthesis with latent diffusion models[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 10684−10695
[13]	Cruz Jr G V, Du Yunshu, Taylor M E. Pre-training neural networks with human demonstrations for deep reinforcement learning[J]. arXiv preprint, arXiv: 1709.04083, 2017
[14]	Jiang Wenxin, Synovic N, Sethi R, et al. An empirical study of artifacts and security risks in the pre-trained model supply chain[C]//Proc of the 1st ACM Workshop on Software Supply Chain Offensive Research and Ecosystem Defenses. New York: ACM, 2022: 105−114
[15]	TensorFlow. TensorFlow Hub[EB/OL]. (2018-03-01)[2023-03-18]. https://www.tensorflow.org/hub
[16]	NVIDIA. NVIDIA NGC: AI development catalog[EB/OL]. (2017-10-01) [2023-03-21]. https://catalog.ngc
[17]	Russakovsky O, Deng Jia, Su Hao, et al. Imagenet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115: 211−252 doi: 10.1007/s11263-015-0816-y
[18]	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proc of the 29th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2016: 770−778
[19]	He Kaiming, Girshick R, Dollár P. Rethinking imagenet pre-training[C]//Proc of the 32nd IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2019: 4918−4927
[20]	Jajal P, Jiang Wenxin, Tewari A, et al. Analysis of failures and risks in deep learning model converters: A case study in the ONNX ecosystem[J]. arXiv preprint, arXiv: 2303.17708, 2023
[21]	Choudhary T, Mishra V, Goswami A, et al. A comprehensive survey on model compression and acceleration[J]. Artificial Intelligence Review, 2020, 53: 5113−5155 doi: 10.1007/s10462-020-09816-7
[22]	Krishnamoorthi R. Quantizing deep convolutional networks for efficient inference: A whitepaper[J]. arXiv preprint, arXiv: 1806.08342, 2018
[23]	Dong Xin, Chen Shangyu, Pan S. Learning to prune deep neural networks via layer-wise optimal brain surgeon[J]. Advances in Neural Information Processing Systems, 2017, 30(1): 4857−4867
[24]	高晗,田育龙,许封元,等. 深度学习模型压缩与加速综述[J]. 软件学报,2021,32(1):68−92 Gao Han, Tian Yulong, Xu Fengyuan, et al. Survey of deep learning model compression and acceleration[J]. Journal of Software, 2021, 32(1): 68−92 (in Chinese)
[25]	Hinton G, Vinyals O, Dean J. Distilling the knowledge in a neural network[J]. arXiv preprint, arXiv: 1503.02531, 2015
[26]	Dube P, Bhattacharjee B, Huo Siyu, et al. Automatic labeling of data for transfer learning suggestion[C/OL]//Proc of the 32nd IEEE/CVF Conf on Computer Vision and Pattern Recognition. 2019[2023-05-06]. https://openreview.net/forum?id=rkxJgoRN_V
[27]	Hugging Face [EB/OL]. (2016-03-01)[2022-03-21]. https://huggingface.co/datasets/beans
[28]	Schelter S, Biessmann F, Januschowski T, et al. On challenges in machine learning model management[J/OL]. IEEE Data Engineering Bulletin. 2015[2023-05-02]. https://www.amazon.science/publications/on-challenges-in-machine-learning-model-management
[29]	Shen Lujia, Ji Shouling, Zhang Xuhong, et al. Backdoor pre-trained models can transfer to all[C]//Proc of the 28th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2021: 3141−3158
[30]	纪守领,杜天宇,李进锋,等. 机器学习模型安全与隐私研究综述[J]. 软件学报,2021,32(1):41−67 Ji Shouling, Du Tianyu, Li Jinfeng, et al. Security and privacy of machine learning models: A survey[J]. Journal of Software, 2021, 32(1): 41−67 (in Chinese)
[31]	Jiang Wenxin, Synovic N, Hyatt M, et al. An empirical study of pre-trained model reuse in the hugging face deep learning model registry[J]. arXiv preprint, arXiv: 2303.02552, 2023
[32]	Jiang Wenxin, Banna V, Vivek N, et al. Challenges and practices of deep learning model reengineering: A case study on computer vision[J]. arXiv preprint, arXiv: 2303.07476, 2023
[33]	Liu Zeyan, Li Fengjun, Li Zhu, et al. LoneNeuron: A highly-effective feature-domain neural trojan using invisible and polymorphic watermarks[C]//Proc of the 29th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2022: 2129−2143
[34]	Kurita K, Michel P, Neubig G. Weight poisoning attacks on pretrained models[C]//Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 2793−2806
[35]	Li Linyang, Song Demin, Li Xiaonan, et al. Backdoor attacks on pre-trained models by layerwise weight poisoning[C]//Proc of the 58th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 3023−3032
[36]	Zhang Xinyang, Zhang Zheng, Ji Shouling, et al. Trojaning language models for fun and profit[C]// Proc of the 6th IEEE European Symp on Security and Privacy (EuroS&P). Piscataway, NJ: IEEE, 2021: 179−197
[37]	Cai Xiangrui, Xu Haidong, Xu Sihan, et al. Badprompt: Backdoor attacks on continuous prompts[J]. Advances in Neural Information Processing Systems, 2022, 35(1): 37068−37080
[38]	Grill J B, Strub F, Altché F, et al. Bootstrap your own latent—A new approach to self-supervised learning[J]. Advances in Neural Information Processing Systems, 2020, 33(1): 21271−21284
[39]	Jia Jinyuan, Liu Yupei, Gong Zhengqiang. Badencoder: Backdoor attacks to pre-trained encoders in self-supervised learning[C]//Proc of the 43rd IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2022: 2043−2059
[40]	Liu Hongbin, Jia Jinyuan, Gong Zhengqiang. PoisonedEncoder: Poisoning the unlabeled pre-training data in contrastive learning[C]//Proc of the 31st USENIX Security Symp (USENIX Security 22). Berkeley, CA: USENIX Association, 2022: 3629−3645
[41]	Chen Ting, Kornblith S, Norouzi M, et al. A simple framework for contrastive learning of visual representations[C]//Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 1597−1607
[42]	Zhang Jinghuai, Liu Hongbin, Jia Jinyuan, et al. CorruptEncoder: Data poisoning based backdoor attacks to contrastive learning[J]. arXiv preprint, arXiv: 2211.08229, 2022
[43]	Ji Yujie, Zhang Xinyang, Ji Shouling, et al. Model-reuse attacks on deep learning systems[C]//Proc of the 25th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2018: 349−363
[44]	Chen Kangjie, Meng Yuxian, Sun Xiaofei, et al. Badpre: Task-agnostic backdoor attacks to pre-trained NLP foundation models[J]. arXiv preprint, arXiv: 2110.02467, 2021
[45]	Zhang Zhengyan, Xiao Guangxuan, Li Yongwei, et al. Red alarm for pre-trained models: Universal vulnerability to neuron-level backdoor attacks[J]. Machine Intelligence Research, 2023, 20(2): 180−193 doi: 10.1007/s11633-022-1377-5
[46]	Shen Lujia, Ji Shouling, Zhang Xuhong, et al. Backdoor pre-trained models can transfer to all[C]//Proc of the 28th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2021: 3141−3158
[47]	Lv Peizhuo, Yue Chang, Liang Ruigang, et al. A data-free backdoor injection approach in neural networks[C]//Proc of the 32nd USENIX Security Symp (USENIX Security 23). Berkeley, CA: USENIX Association, 2023: 2671−2688
[48]	Mei Kai, Li Zheng, Wang Zhenting, et al. NOTABLE: Transferable backdoor attacks against prompt-based NLP models[C]//Proc of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2023: 15551−15565
[49]	Du Wei, Li Peixuan, Li Boqun, et al. UOR: Universal backdoor attacks on pre-trained language models[J]. arXiv preprint, arXiv: 2305.09574, 2023
[50]	Wang Hao, Guo Shangwei, He Jialiang, et al. Model supply chain poisoning: backdooring pre-trained models via embedding indistinguishability[J]. arXiv preprint, arXiv: 2401.15883, 2024
[51]	Huang Yujun, Zhuo Terry Yue, Xu Qiongkai, et al. Training-free lexical backdoor attacks on language models[C]//Proc of the 32nd ACM Web Conf 2023. New York: ACM, 2023: 2198−2208
[52]	Wang Yue, Wang Weishi, Joty S, et al. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation[C]//Proc of the 36th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 8696−8708
[53]	Li Yanzhou, Liu Shangqing, Chen Kangjie, et al. Multi-target backdoor attacks for code pre-trained models[J]. arXiv preprint, arXiv: 2306.08350, 2023
[54]	Wang Bolun, Yao Yuanshun , Viswanath B, et al. With great training comes great vulnerability: Practical attacks against transfer learning[C]//Proc of the 27th USENIX Security Symp (USENIX Security 18). Berkeley, CA: USENIX Association, 2018: 1281−1297
[55]	Yao Yuanshun, Li Huiying, Zheng Haitao, et al. Latent backdoor attacks on deep neural networks[C]//Proc of the 26th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2019: 2041−2055
[56]	Wang Shuo, Nepal S, Rudolph C, et al. Backdoor attacks against transfer learning with pre-trained deep learning models[J]. IEEE Transactions on Services Computing, 2020, 15(3): 1526−1539
[57]	Li Zhizhong, Hoiem D. Learning without forgetting[J]. IEEE Transactions on Pattern Analysis And Machine Intelligence, 2017, 40(12): 2935−2947
[58]	Jiang Wenbo, Zhang Tianwei, Qiu H, et al. Incremental learning, incremental backdoor threats[J]. IEEE Transactions on Dependable and Secure Computing, 2022, 21(2): 559−572
[59]	Ban Yuanhao, Dong Yinpeng. Pre-trained adversarial perturbations[J]. Advances in Neural Information Processing Systems, 2022, 35(1): 1196−1209
[60]	Lester B, Al-Rfou R, Constant N. The power of scale for parameter-efficient prompt tuning[C]//Proc of the 26th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 3045−3059
[61]	Du Wei, Zhao Yichun, Li Boqun, et al. Ppt: Backdoor attacks on pre-trained models via poisoned prompt tuning[C]//Proc of the 31st Int Joint Conf on Artificial Intelligence (IJCAI−22). San Francisco, CA: Morgan Kaufmann: 2022: 680−686
[62]	Chung H W, Hou Le, Longpre S, et al. Scaling instruction-finetuned language models[J]. Journal of Machine Learning Research, 2024, 25(70): 1−53
[63]	Xu Jiashu, Ma Mingyu, Wang Fei, et al. Instructions as backdoors: Backdoor vulnerabilities of instruction tuning for large language models[C]//Proc of the 20th Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers). Stroudsburg, PA: ACL, 2024: 3111−3126
[64]	Chen Yufei, Shen Chao, Wang Cong, et al. Teacher model fingerprinting attacks against transfer learning[C]//Proc of the 31st USENIX Security Symp (USENIX Security 22). Berkeley, CA: USENIX Association, 2022: 3593−3610
[65]	Tian Yulong, Suya F, Suri A, et al. Manipulating transfer learning for property inference[C]//Proc of the 36th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 15975−15984
[66]	Song Congzheng, Raghunathan A. Information leakage in embedding models[C]//Proc of the 27th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2020: 377−390
[67]	Carlini N, Tramer F, Wallace E, et al. Extracting training data from large language models[C]//Proc of the 30th USENIX Security Symp (USENIX Security 21). Berkeley, CA: USENIX Association, 2021: 2633−2650
[68]	Liu Hongbin, Jia Jinyuan, Qu Wenjie, et al. EncoderMI: Membership inference against pre-trained encoders in contrastive learning[C]//Proc of the 28th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2021: 2081−2095
[69]	Liu Yupei, Jia Jinyuan, Liu Hongbin, et al. StolenEncoder: Stealing pre-trained encoders in self-supervised learning[C]//Proc of the 29th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2022: 2115−2128
[70]	Sha Zeyang, He Xinlei, Yu Ning, et al. Can't steal? Cont-steal! Contrastive stealing attacks against image encoders[C]//Proc of the 40th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 16373−16383
[71]	Hong S, Panaitescu-Liess M A, Kaya Y, et al. Qu-anti-zation: Exploiting quantization artifacts for achieving adversarial outcomes[J]. Advances in Neural Information Processing Systems, 2021, 34(1): 9303−9316
[72]	Ma Hua, Qiu Huming, Gao Yansong, et al. Quantization backdoors to deep learning commercial frameworks[J]. IEEE Transactions on Dependable and Secure Computing, 2023, 21(3): 1155−1172
[73]	Tian Yulong, Suya F, Xu Fengyan, et al. Stealthy backdoors as compression artifacts[J]. IEEE Transactions on Information Forensics and Security, 2022, 17: 1372−1387 doi: 10.1109/TIFS.2022.3160359
[74]	Phan H, Shi Cong, Xie Yi, et al. RIBAC: Towards robust and imperceptible backdoor attack against compact DNN[C]//Proc of the 17th European Conf on Computer Vision 2022. Berlin: Springer, 2022: 708−724
[75]	Ge Yunjie, Wang Qian, Zheng Baolin, et al. Anti-distillation backdoor attacks: Backdoors can really survive in knowledge distillation[C]//Proc of the 29th ACM Int Conf on Multimedia. New York: ACM, 2021: 826−834
[76]	Qi Xiangyu, Xie Tinghao, Pan Ruizhe, et al. Towards practical deployment-stage backdoor attack on deep neural networks[C]//Proc of the 39th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2022: 13347−13357
[77]	Zou A, Wang Zifan, Kolter J Z, et al. Universal and transferable adversarial attacks on aligned language models[J]. arXiv preprint, arXiv: 2307.15043, 2023
[78]	Wei A,Haghtalab N,Steinhardt J. Jailbroken:How does llm safety training fail?[J]. Advances in Neural Information Processing Systems,2024,36(1):80079-80110(英文名
[79]	Shen Xinyue, Chen Zeyuan, Backes M, et al. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models[C]// Proc of the 31st on ACM SIGSAC Conf on Computer and Communications Security. 2024: 1671−1685
[80]	Zanella-Béguelin S, Wutschitz L, Tople S, et al. Analyzing information leakage of updates to natural language models[C]//Proc of the 27th ACM SIGSAC Conf on Computer and Communications Security. New York: ACM, 2020: 363−375
[81]	Panchendrarajan R, Bhoi S. Dataset reconstruction attack against language models[C/OL]//Proc of the 30th Int Joint Conf on Artificial Intelligence. 2021[2021-07-01]. http://cdap.sliit.lk/bitstream/123456789/2045/1/DatasetReconstructionAttackagainstLanguageModels.pdf
[82]	Lukas N, Salem A, Sim R, et al. Analyzing leakage of personally identifiable information in language models[C]//Proc of the 44th IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2023: 346−363
[83]	Keskar N S, McCann B, Xiong Caiming, et al. The thieves on sesame street are polyglots-extracting multilingual models from monolingual APIs[C]//Proc of the 25th Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2020: 6203−6207
[84]	Xu Qiongkai, He Xuanli, Lyu Lingjuan, et al. Student surpasses teacher: imitation attack for black-box NLP APIs[C]//Proc of the 29th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2022: 2849−2860
[85]	Li Zongjie, Wang Chaozheng, Ma Pingchuan, et al. On extracting specialized code abilities from large language models: A feasibility study[C]//Proc of the 46th IEEE/ACM Int Conf on Software Engineering. 2024: 1−13
[86]	Feng Shiwei, Tao Guanhong, Cheng Siyuan, et al. Detecting backdoors in pre-trained encoders[C]//Proc of the 40th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2023: 16352−16362
[87]	Zhu Biru, Qin Yujia, Cui Ganqu, et al. Moderate-fitting as a natural backdoor defender for pre-trained language models[J]. Advances in Neural Information Processing Systems, 2022, 35(1): 1086−1099
[88]	Zhang Xiaoyu, Jin Yulin, Wang Tao, et al. Purifier: Plug-and-play backdoor mitigation for pre-trained models via anomaly activation suppression[C]//Proc of the 30th ACM Int Conf on Multimedia. New York: ACM, 2022: 4291−4299
[89]	Shan S, Bhagoji A N, Zheng Haitao, et al. Poison forensics: Traceback of data poisoning attacks in neural networks[C]//Proc of the 31st USENIX Security Symp (USENIX Security 22). Berkeley, CA: USENIX Association, 2022: 3575−3592
[90]	Cheng Siyuan, Tao Guanhong, Liu Yingqi, et al. BEAGLE: Forensics of deep learning backdoor attack for better defense[J]. arXiv preprint, arXiv: 2301.06241, 2023
[91]	Liu Kang, Dolan-Gavitt B, Garg S. Fine-pruning: Defending against backdooring attacks on deep neural networks[C]//Proc of the 21st Int Symp on Research in Attacks, Intrusions, and Defenses. Berlin: Springer, 2018: 273−294
[92]	Li Yige, Lyu Xixiang, Koren N, et al. Neural attention distillation: Erasing backdoor triggers from deep neural networks[J]. arXiv preprint, arXiv: 2101.05930, 2021
[93]	Papernot N, McDaniel P, Wu Xi, et al. Distillation as a defense to adversarial perturbations against deep neural networks[C]// Proc of the 37th IEEE Symp on Security and Privacy (SP). Piscataway, NJ: IEEE, 2016: 582−597
[94]	Khalid F, Ali H, Tariq H, et al. Qusecnets: Quantization-based defense mechanism for securing deep neural network against adversarial attacks[C]//Proc of the 25th IEEE Int Symp on On-Line Testing and Robust System Design (IOLTS). Piscataway, NJ: IEEE, 2019: 182−187
[95]	Dong Xinshuai, Luu A T, Lin Min, et al. How should pre-trained language models be fine-tuned towards adversarial robustness?[J]. Advances in Neural Information Processing Systems, 2021, 34(1): 4356−4369
[96]	Jiang Lan, Zhou Hao, Lin Yankai, et al. ROSE: Robust selective fine-tuning for pre-trained language models[C]//Proc of the 27th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA, ACL: 2022: 2886−2897
[97]	Liu Hongbin, Reiter M K, Gong Neil Zhengqiang. Mudjacking: Patching backdoor vulnerabilities in foundation models[J]. arXiv preprint, arXiv: 2402.14977, 2024
[98]	Zhang Zhiyuan, Lyu Lingjuan, Ma Xingjun, et al. Fine-mixing: Mitigating backdoors in fine-tuned language models[C]/Proc of the 27th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA, ACL: 2022: 355−372
[99]	Zhang Zhiyuan, Chen Deli, Zhou Hao, et al. Diffusion theory as a scalpel: Detecting and purifying poisonous dimensions in pre-trained language models caused by backdoor or bias[C]//Proc of the 61st Association for Computational Linguistics. Stroudsburg, PA: ACL, 2023: 2495−2517
[100]	Sheng Lijun, Liang Jian, He Ran, et al. Adaptguard: Defending against universal attacks for model adaptation[C]//Proc of the 39th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 19093−19103
[101]	Sheng Lijun, Liang Jian, He Ran, et al. Can we trust the unlabeled target data? Towards backdoor attack and defense on model adaptation[J]. arXiv preprint, arXiv: 2401.06030, 2024
[102]	Ahmed S, Al Arafat A, Rizve M N, et al. SSDA: Secure source-free domain adaptation[C]//Proc of the 39th IEEE/CVF Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2023: 19180−19190
[103]	Zhang Ziqi, Li Yuanchun, Wang Jindong, et al. ReMoS: Reducing defect inheritance in transfer learning via relevant model slicing[C]//Proc of the 44th Int Conf on Software Engineering. Piscataway, NJ: IEEE, 2022: 1856−1868
[104]	Jajal P, Jiang Wenxin, Tewari A, et al. Analysis of failures and risks in deep learning model converters: A case study in the ONNX Ecosystem[J]. arXiv preprint, arXiv: 2303.17708, 2023
[105]	Zhu Jie, Wang Leye, Han Xiao. Safety and Performance, Why Not Both? Bi-Objective Optimized Model Compression against Heterogeneous Attacks Toward AI Software Deployment[J]. IEEE Transactions on Software Engineering, 2024, 50(3): 376−390 doi: 10.1109/TSE.2023.3348515
[106]	Tian Yongqiang, Zhang Wuqi, Wen Ming, et al. Finding deviated behaviors of the compressed dnn models for image classifications[J]. ACM Transactions on Software Engineering and Methodology, 2023, 32(5): 1−32
[107]	Xiao Dongwei, Liu Zhibo, Yuan Yuanyuan, et al. Metamorphic testing of deep learning compilers[J]. ACM on Measurement and Analysis of Computing Systems, 2022, 6(1): 1−28
[108]	Liu Jiawei, Lin Jinkun, Ruffy F, et al. Nnsmith: Generating diverse and valid test cases for deep learning compilers[C]//Proc of the 28th ACM Int Conf on Architectural Support for Programming Languages and Operating Systems, Volume 2. New York: ACM, 2023: 530−543
[109]	Ma Haoyang, Shen Qingchao, Tian Yongqiang, et al. Fuzzing deep learning compilers with HirGen[C]//Proc of the 32nd ACM SIGSOFT Int Symp on Software Testing and Analysis. New York: ACM, 2023: 248−260
[110]	Zhou Xin, Lu Jinzhu, Gui Tao, et al. TextFusion: Privacy-preserving pre-trained model inference via token fusion[C]//Proc of the 27th Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA, ACL: 2022: 8360−8371
[111]	Zhou Xin, Lu Yi, Ma Ruotian, et al. TextObfuscator: Making pre-trained language model a privacy protector via obfuscating word representations[C]//Proc of the 61st Association for Computational Linguistics. Stroudsburg, PA: ACL, 2023: 5459−5473
[112]	Modelscope. ModelScope: bring the notion of Model-as-a-Service to life[EB/OL]. (2022-11-03)[2024-05-01]https://github.com/modelscope/modelscope
[113]	PaddlePaddle. PaddleHub―An open-source deep learning platform originated from industrial practice[EB/OL]. (2019-07-08)[2023-03-15]https://www.paddlepaddle.org.cn/hub
[114]	Pytorch. PyTorch hub. [EB/OL]. (2019-06-01)[2023-05-01]https://pytorch.org/hub/
[115]	ONNX. ONNX model zoo[EB/OL]. (2017-04-20)[2023-06-01]https://github.com/onnx/models
[116]	Debenedetti E, Severi G, Carlini N, et al. Privacy side channels in machine learning systems[C]//Proc of the 33rd USENIX Security Symp (USENIX Security 24). Berkeley, CA: USENIX Association, 2024: 6861−6848
[117]	Tan Qi, Li Qi, Zhao Yi, et al. Defending against data reconstruction attacks in federated learning: an information theory approach[J]. arXiv preprint, arXiv: 2403.01268, 2024

[1]	Wu Huanhuan, Xie Ruilin, Qiao Yuanxin, Chen Xiang, Cui Zhanqi. Optimizing Deep Neural Network Based on Interpretability Analysis[J]. Journal of Computer Research and Development, 2024, 61(1): 209-220. DOI: 10.7544/issn1000-1239.202220803
[2]	Li Qian, Lin Chenhao, Yang Yulong, Shen Chao, Fang Liming. Adversarial Attacks and Defenses Against Deep Learning Under the Cloud-Edge-Terminal Scenes[J]. Journal of Computer Research and Development, 2022, 59(10): 2109-2129. DOI: 10.7544/issn1000-1239.20220665
[3]	Shen Zhengchen, Zhang Qianli, Zhang Chaofan, Tang Xiangyu, Wang Jilong. Location Privacy Attack Based on Deep Learning[J]. Journal of Computer Research and Development, 2022, 59(2): 390-402. DOI: 10.7544/issn1000-1239.20200843
[4]	Gu Mianxue, Sun Hongyu, Han Dan, Yang Su, Cao Wanying, Guo Zhen, Cao Chunjie, Wang Wenjie, Zhang Yuqing. Software Security Vulnerability Mining Based on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(10): 2140-2162. DOI: 10.7544/issn1000-1239.2021.20210620
[5]	Wang Huijiao, Cong Peng, Jiang Hua, Wei Yongzhuang. Security Analysis of SIMON32/64 Based on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(5): 1056-1064. DOI: 10.7544/issn1000-1239.2021.20200900
[6]	Zhou Chunyi, Chen Dawei, Wang Shang, Fu Anmin, Gao Yansong. Research and Challenge of Distributed Deep Learning Privacy and Security Attack[J]. Journal of Computer Research and Development, 2021, 58(5): 927-943. DOI: 10.7544/issn1000-1239.2021.20200966
[7]	Chen Jinyin, Chen Yipeng, Chen Yiming, Zheng Haibin, Ji Shouling, Shi Jie, Cheng Yao. Fairness Research on Deep Learning[J]. Journal of Computer Research and Development, 2021, 58(2): 264-280. DOI: 10.7544/issn1000-1239.2021.20200758
[8]	Wang Ruiqin, Wu Zongda, Jiang Yunliang, Lou Jungang. An Integrated Recommendation Model Based on Two-stage Deep Learning[J]. Journal of Computer Research and Development, 2019, 56(8): 1661-1669. DOI: 10.7544/issn1000-1239.2019.20190178
[9]	Zhou Yucong, Liu Yi, Wang Rui. Training Deep Neural Networks for Image Applications with Noisy Labels by Complementary Learning[J]. Journal of Computer Research and Development, 2017, 54(12): 2649-2659. DOI: 10.7544/issn1000-1239.2017.20170637
[10]	Zhang Lei, Zhang Yi. Big Data Analysis by Infinite Deep Neural Networks[J]. Journal of Computer Research and Development, 2016, 53(1): 68-79. DOI: 10.7544/issn1000-1239.2016.20150663