ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (10): 2071-2096.doi: 10.7544/issn1000-1239.2019.20190540

Special Issue: 2019密码学与智能安全研究专题

Previous Articles     Next Articles

Survey on Techniques, Applications and Security of Machine Learning Interpretability

Ji Shouling1, Li Jinfeng1, Du Tianyu1, Li Bo2   

  1. 1(Institute of Cyberspace Research and College of Computer Science and Technology, Zhejiang University, Hangzhou 310027);2(Department of Computer Science, University of Illinois at Urbana-Champaign, Urbana-Champaign, IL, USA 61822)
  • Online:2019-10-16

Abstract: While machine learning has achieved great success in various domains, the lack of interpretability has limited its widespread applications in real-world tasks, especially security-critical tasks. To overcome this crucial weakness, intensive research on improving the interpretability of machine learning models has emerged, and a plethora of interpretation methods have been proposed to help end users understand its inner working mechanism. However, the research on model interpretation is still in its infancy, and there are a large amount of scientific issues to be resolved. Furthermore, different researchers have different perspectives on solving the interpretation problem and give different definitions for interpretability, and the proposed interpretation methods also have different emphasis. Till now, the research community still lacks a comprehensive understanding of interpretability as well as a scientific guide for the research on model interpretation. In this survey, we review the explanatory problems in machine learning, and make a systematic summary and scientific classification of the existing research works. At the same time, we discuss the potential applications of interpretation related technologies, analyze the relationship between interpretability and the security of interpretable machine learning, and discuss the current research challenges and potential future research directions, aiming at providing necessary help for future researchers to facilitate the research and application of model interpretability.

Key words: machine learning, interpretability, interpretation method, interpretable machine learning, security

CLC Number: