ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (10): 2135-2150.doi: 10.7544/issn1000-1239.2019.20190415

Special Issue: 2019密码学与智能安全研究专题

Previous Articles     Next Articles

Security and Privacy Risks in Artificial Intelligence Systems

Chen Yufei1,2, Shen Chao1,2, Wang Qian3, Li Qi4, Wang Cong5, Ji Shouling6,7, Li Kang8, Guan Xiaohong1,2   

  1. 1(Key Laboratory for Intelligent Networks and Network Security(Xi’an Jiaotong University), Ministry of Education, Xi’an 710049);2(Faculty of Electronic and Information Engineering, Xi’an Jiaotong University, Xi’an 710049);3(School of Cyber Science and Engineering, Wuhan University, Wuhan 430072);4(Institute for Network Sciences and Cyberspace, Tsinghua University, Beijing 100084);5(Department of Computer Science, City University of Hong Kong, Hong Kong 999077);6(Institute of Cyberspace Research, Zhejiang University, Hangzhou 310027);7(College of Computer Science and Technology, Zhejiang University, Hangzhou 310027);8(Department of Computer Science, University of Georgia, Athens, Georgia, the United States 30602)
  • Online:2019-10-16

Abstract: Human society is witnessing a wave of artificial intelligence (AI) driven by deep learning techniques, bringing a technological revolution for human production and life. In some specific fields, AI has achieved or even surpassed human-level performance. However, most previous machine learning theories have not considered the open and even adversarial environments, and the security and privacy issues are gradually rising. Besides of insecure code implementations, biased models, adversarial examples, sensor spoofing can also lead to security risks which are hard to be discovered by traditional security analysis tools. This paper reviews previous works on AI system security and privacy, revealing potential security and privacy risks. Firstly, we introduce a threat model of AI systems, including attack surfaces, attack capabilities and attack goals. Secondly, we analyze security risks and counter measures in terms of four critical components in AI systems: data input (sensor), data preprocessing, machine learning model and output. Finally, we discuss future research trends on the security of AI systems. The aim of this paper is to arise the attention of the computer security society and the AI society on security and privacy of AI systems, and so that they can work together to unlock AI’s potential to build a bright future.

Key words: intelligent system security, system security, data processing, artificial intelligence (AI), deep learning

CLC Number: