ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2019, Vol. 56 ›› Issue (10): 2049-2070.doi: 10.7544/issn1000-1239.2019.20190437

所属专题: 2019密码学与智能安全研究专题

• 综述 • 上一篇    下一篇



  1. (信息安全国家重点实验室(中国科学院信息工程研究所) 北京 100195) (中国科学院信息工程研究所 北京 100195) (中国科学院大学网络空间安全学院 北京 101408) (
  • 出版日期: 2019-10-16
  • 基金资助: 
    国家重点研发计划项目(2016QY04W0805);国家自然科学基金项目(U1836211, 61728209);中国科学院青年创新促进会;北京市科技新星计划;北京市自然科学基金项目(JQ18011);国家前沿科技创新项目(YJKYYQ20170070)

Privacy and Security Issues in Machine Learning Systems: A Survey

He Yingzhe, Hu Xingbo, He Jinwen, Meng Guozhu, Chen Kai   

  1. (State Key Laboratory of Information Security (Institute of Information Engineering, Chinese Academy of Sciences), Beijing 100195) (Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100195) (School of Cyber Security, University of Chinese Academy of Sciences, Beijing 101408)
  • Online: 2019-10-16

摘要: 人工智能已经渗透到生活的各个角落,给人类带来了极大的便利.尤其是近年来,随着机器学习中深度学习这一分支的蓬勃发展,生活中的相关应用越来越多.不幸的是,机器学习系统也面临着许多安全隐患,而机器学习系统的普及更进一步放大了这些风险.为了揭示这些安全隐患并实现一个强大的机器学习系统,对主流的深度学习系统进行了调查.首先设计了一个剖析深度学习系统的分析模型,并界定了调查范围.调查的深度学习系统跨越了4个领域——图像分类、音频语音识别、恶意软件检测和自然语言处理,提取了对应4种类型的安全隐患,并从复杂性、攻击成功率和破坏等多个维度对其进行了表征和度量.随后,调研了针对深度学习系统的防御技术及其特点.最后通过对这些系统的观察,提出了构建健壮的深度学习系统的建议.

关键词: 机器学习安全, 深度学习安全, 攻防竞赛, 对抗攻击, 成员推理攻击, 隐私保护

Abstract: Artificial intelligence has penetrated into every corners of our life and brought humans great convenience. Especially in recent years, with the vigorous development of the deep learning branch in machine learning, there are more and more related applications in our life. Unfortunately, machine learning systems are suffering from many security hazards. Even worse, the popularity of machine learning systems further magnifies these hazards. In order to unveil these security hazards and assist in implementing a robust machine learning system, we conduct a comprehensive investigation of the mainstream deep learning systems. In the beginning of the study, we devise an analytical model for dissecting deep learning systems, and define our survey scope. Our surveyed deep learning systems span across four fields-image classification, audio speech recognition, malware detection, and natural language processing. We distill four types of security hazards and manifest them in multiple dimensions such as complexity, attack success rate, and damage. Furthermore, we survey defensive techniques for deep learning systems as well as their characteristics. Finally, through the observation of these systems, we propose the practical proposals of constructing robust deep learning system.

Key words: machine learning security, deep learning security, attack and defense race, adversarial attack, membership inference attack, privacy-preserving