Optimizing Deep Neural Network Based on Interpretability Analysis

Wu Huanhuan; Xie Ruilin; Qiao Yuanxin; Chen Xiang; Cui Zhanqi

doi:10.7544/issn1000-1239.202220803

Journal of Computer Research and Development > 2024 > 61(1): 209-220. > DOI: 10.7544/issn1000-1239.202220803

Wu Huanhuan, Xie Ruilin, Qiao Yuanxin, Chen Xiang, Cui Zhanqi. Optimizing Deep Neural Network Based on Interpretability Analysis[J]. Journal of Computer Research and Development, 2024, 61(1): 209-220. DOI: 10.7544/issn1000-1239.202220803

Citation:

PDF (1205 KB)

Optimizing Deep Neural Network Based on Interpretability Analysis

1.
School of Computer Science, Beijing Information Science and Technology University, Beijing 100101
2.
School of Information Science and Technology, Nantong University, Nantong, Jiangsu 226019

Funds: This work was supported by the Jiangsu Provincial Frontier Leading Technology Fundamental Research Project (BK202002001), the National Natural Science Foundation of China (61702041), and the Beijing Information Science and Technology University "Qin-Xin Talent" Cultivation Project (QXTCP C201906).

More Information

Author Bio:
Wu Huanhuan: born in 1995. Master. Student member of CCF. Her main research interests include intelligent software engineering and trustworthy artificial intelligence

Xie Ruilin: born in 1996. Master candidate. Student member of CCF. His main research interests include intelligent software engineering and trustworthy artificial intelligence

Qiao Yuanxin: born in 1999. Master candidate. Student member of CCF. His main research interests include intelligent software engineering and trustworthy artificial intelligence

Chen Xiang: born in 1980. PhD, associate professor, master supervisor. Senior member of CCF. His main research interests include software testing, software maintenance, empirical software engineering, and mining software repository

Cui Zhanqi: born in 1984. PhD, professor, master supervisor. Senior member of CCF. His main research interests include intelligent software engineering and trustworthy artificial intelligence
Received Date: September 13, 2022
Revised Date: February 21, 2023
Available Online: November 27, 2023

Graphical Abstract

Abstract

Abstract

In recent years, deep neural networks (DNN) have been widely used in many fields, even replacing human to make decisions in some safety-critical systems, such as autonomous driving and smart healthcare, which requires higher reliability of DNN. It is difficult to understand the internal prediction mechanism and debug because of the complex multi-layer nonlinear network structure of DNN. The existing DNN debugging work mainly improves the performance by adjusting the parameters and augments the training set to optimize DNN. However, it is difficult to control the modification range of adjusting parameters directly, and probably make the model lose the ability of fitting the training set. And unguided augmentation of training set will dramatically increase training costs. To address this problem, a DNN optimization method OptDIA (optimizing DNN based on interpretability analysis) is proposed. Interpretability analysis is conducted on the training process and the decision-making behavior of DNN. According to the interpretability analysis results, the original training data is split into different partitions to evaluate their influences on decision-making results of DNN. After that, the partitions of original training data are transformed with different probabilities to generate new training data, which are used to retrain DNN to improve the performance of the model. The experiments on nine DNNs trained by three datasets shows that OptDIA can improve the accuracy of DNNs by 0.39% to 2.15% and F1-score of DNNs by 0.11% to 2.03%.
- deep neural network,
- DNN optimization,
- DNN defect,
- DNN performance,
- interpretability analysis

FullText(HTML)

References (44)

References

[1]	Esteva A, Chou K, Yeung S, et al. Deep learning-enabled medical computer vision[J]. NPJ Digital Medicine, 2021, 4(1): 1−9 doi: 10.1038/s41746-020-00373-5
[2]	Malik M, Malik M K, Mehmood K, et al. Automatic speech recognition: A survey[J]. Multimedia Tools and Applications, 2021, 80(6): 9411−9457 doi: 10.1007/s11042-020-10073-7
[3]	郑海斌,陈晋音,章燕,等. 面向自然语言处理的对抗攻防与鲁棒性分析综述[J]. 计算机研究与发展,2021,58(8):1727−1750 doi: 10.7544/issn1000-1239.2021.20210304 Zheng Haibin, Chen Jinyin, Zhang Yan, et al. Survey of adversarial attack, defense and robustness analysis for natural language processing[J]. Journal of Computer Research and Development, 2021, 58(8): 1727−1750 (in Chinese) doi: 10.7544/issn1000-1239.2021.20210304
[4]	Fang Peng, Wang Zecong, Zhang Xiangyun. Vehicle automatic driving system based on embedded and machine learning[C]//Proc of Int Conf on Computer Vision, Image and Deep Learning. Piscataway, NJ: IEEE, 2020: 281−284
[5]	Bakator M, Radosav D. Deep learning and medical diagnosis: A review of literature[J]. Multimodal Technologies and Interaction, 2018, 2(3): 47 doi: 10.3390/mti2030047
[6]	Ghenescu V, Mihaescu R E, Carata S V, et al. Face detection and recognition based on general purpose DNN object detector[C]//Proc of Int Symp on Electronics and Telecommunications. Piscataway, NJ: IEEE, 2018: 1−4
[7]	Tulbure A A, Tulbure A A, Dulf E H. A review on modern defect detection models using DCNNs-Deep convolutional neural networks[J]. Journal of Advanced Research, 2022, 35: 33−48 doi: 10.1016/j.jare.2021.03.015
[8]	界面新闻. 新能源汽车评论[EB/OL]. 2022 [2023-02-16].https://www.jiemian.com/article/7530592.html Interface News. Review of new energy vehicles[EB/OL]. 2022 [2023-02-16].https://www.jiemian.com/article/7530592.html (in Chinese)
[9]	Goues C L, Nguyen T V, Forrest S, et al. GenProg: A generic method for automatic software repair[J]. IEEE Transactions on Software Engineering, 2011, 38(1): 54−72
[10]	Baudry B, Fleurey F, Traon Y L. Improving test suites for efficient fault localization[C]//Proc of the 28th Int Conf on Software Engineering. New York: ACM, 2006: 82−91
[11]	Zeller A. Yesterday, my program worked. Today, it does not. Why?[J]. ACM SIGSOFT Software Engineering Notes, 1999, 24(6): 253−267 doi: 10.1145/318774.318946
[12]	Zhang Xiangyu, Gupta N, Gupta R. Locating faults through automated predicate switching[C]//Proc of the 28th Int Conf on Software Engineering. New York: ACM, 2006: 272−281
[13]	Ma Lei, Xu Felix Juefei, Zhang Fuyuan, et al. Deepgauge: Multi-granularity testing criteria for deep learning systems[C]//Proc of the 33rd ACM/IEEE Int Conf on Automated Software Engineering. New York: ACM, 2018: 120−131
[14]	Ma Shiqing, Liu Yingqi, Lee W C, et al. MODE: Automated neural network model debugging via state differential analysis and input selection[C]//Proc of the 26th ACM Joint Meeting on European Software Engineering Conf and Symp on the Foundations of Software Engineering. New York: ACM, 2018: 175−186
[15]	王赞,闫明,刘爽,等. 深度神经网络测试研究综述[J]. 软件学报,2020,31(5):1255−1275 Wang Zan, Yan Ming, Liu Shuang, et al. A survey on testing of deep neural network. [J] Journal of Software, 2020, 31(5): 1255−1275 (in Chinese)
[16]	Pei Kexin, Cao Yinzhi, Yang Junfen, et al. Deepxplore: Automated whitebox testing of deep learning systems[C]//Proc of the 26th Symp on Operating Systems Principles. New York: ACM, 2017: 1−18
[17]	Zhang Mengshi, Zhang Yuqun, Zhang Lingming, et al. DeepRoad: GAN-based metamorphic testing and input validation framework for autonomous driving systems[C]//Proc of the 33rd IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2018: 132−142
[18]	Zhang Hao, Chan W K. Apricot: A weight-adaptation approach to fixing deep learning models[C]//Proc of the 34th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2019: 376−387
[19]	Selvaraju R R, Cogswell M, Das A, et al. Grad-CAM: Visual explanations from deep networks via gradient-based localization[C]//Proc of the IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2017: 618−626
[20]	Simonyan K, Vedaldi A, Zisserman A. Deep inside convolutional networks: Visualising image classification models and saliency maps[J]. arXiv preprint, arXiv: 1312. 6034, 2013
[21]	Du Mengnan, Liu Ninghao, Song Qinghao, et al. Towards explanation of DNN-based prediction with guided feature inversion[C]//Proc of the 24th ACM SIGKDD Int Conf on Knowledge Discovery & Data Mining. New York: ACM, 2018: 1358−1367
[22]	Zhou Bolei, Khosla A, Lapedriza A, et al. Learning deep features for discriminative localization[C]//Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 2921−2929
[23]	CIFAR official website. Dataset[EB/OL]. [2023-02-16]. http://www.cs.toronto.edu/~kriz/cifar.html
[24]	GitHub. Fashion-MNIST[EB/OL]. 2017[2023-02-16].https://github.com/zalandoresearch/fashion-mnist
[25]	Mikołajczyk A, Grochowski M. Data augmentation for improving deep learning in image classification problem[C]//Proc of 2018 Int Interdisciplinary PhD Workshop (IIPhDW). Piscataway, NJ: IEEE, 2018: 117−122
[26]	Taylor L, Nitschke G. Improving deep learning with generic data augmentation[C]// Proc of IEEE Symp Series on Computational Intelligence. Piscataway, NJ: IEEE, 2018: 1542−1547
[27]	Kang Guoliang, Dong Xuanyi, Zheng Liang, et al. Patchshuffle regularization[J]. arXiv preprint, arXiv: 1707. 07103, 2017
[28]	Zhong Zhun, Zheng Liang, Kang Guoliang, et al. Random erasing data augmentation[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 13001−13008
[29]	Zajac M, Zołna K, Rostamzadeh N, et al. Adversarial framing for image and video classification[C]//Proc of the AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 10077−10078
[30]	Johnson J, Alahi A, Li Feifei. Perceptual losses for real-time style transfer and super-resolution[C]//Proc of the 14th European Conf on Computer Vision. Berlin: Springer, 2016: 694−711
[31]	He Kaiming, Zhang Xiangyu, Ren Shaoqing, et al. Deep residual learning for image recognition[C]//Proc of the IEEE Conf on Computer Vision and Pattern Recognition. Piscataway, NJ: IEEE, 2016: 770−778
[32]	GitHub. Grad-CAM[EB/OL]. 2022[2023-02-16].https://github.com/jacobgil/pytorch-grad-cam
[33]	Chen Lei, Chen Jianhui, Hajimirsadeghi H, et al. Adapting Grad-CAM for embedding networks[C]//Proc of the IEEE/CVF Winter Conf on Applications of Computer Vision. Piscataway, NJ: IEEE, 2020: 2794−2803
[34]	GitHub. Random-Erasing[EB/OL]. 2019[2023-02-16].https://github.com/zhunzhong07/Random-Erasing
[35]	GitHub. Torchvision[EB/OL]. 2022[2023-02-16].https://github.com/pytorch/vision
[36]	Tian Yuchi, Ray B. Automatically diagnosing and repairing error handling bugs in C[C]//Proc of the 11th Joint Meeting on Foundations of Software Engineering. New York: ACM, 2017: 752−762
[37]	Nguyen H D T, Qi Dawei, Roychoudhury A, et al. Semfix: Program repair via semantic analysis[C]//Proc of the 35th Int Conf on Software Engineering. Piscataway, NJ: IEEE, 2013: 772−781
[38]	Jeffrey D, Feng Min, Gupta N, et al. BugFix: A learning-based tool to assist developers in fixing bugs[C]//Proc of the 17th IEEE Int Conf on Program Comprehension. Piscataway, NJ: IEEE, 2009: 70−79
[39]	Gao Xiang, Saha R K, Prasad M R, et al. Fuzz testing based data augmentation to improve robustness of deep neural networks[C]//Proc of the 42nd IEEE/ACM Int Conf on Software Engineering. Piscataway, NJ: IEEE, 2020: 1147−1158
[40]	Liu Meixi, Hong Weijiang, Pan Weiyu, et al. Styx: A data-oriented mutation framework to improve the robustness of DNN[C]//Proc of the 35th IEEE/ACM Int Conf on Automated Software Engineering. Piscataway, NJ: IEEE, 2020: 1260−1261
[41]	Zhang Hao, Chan W K. Plum: Exploration and prioritization of model repair strategies for fixing deep learning models[C]//Proc of the 8th Int Conf on Dependable Systems and Their Applications. Piscataway, NJ: IEEE, 2021: 140−151
[42]	Wu Huanhuan, Li Zheng, Cui Zhanqi, et al. GenMuNN: A mutation-based approach to repair deep neural network models[J]. International Journal of Modeling, Simulation, and Scientific Computing, 2022, 13(2): 2341008 doi: 10.1142/S1793962323410088
[43]	Zhang Xiaoyu, Zhai Juan, Ma Shiqing, et al. AUTOTRAINER: An automatic DNN training problem detection and repair system[C]//Proc of the 43rd IEEE/ACM Int Conf on Software Engineering. Piscataway, NJ: IEEE, 2021: 359−371
[44]	Sun Bing, Sun Jun, Pham L H, et al. Causality-based neural network repair[C]//Proc of the 44th Int Conf on Software Engineering. New York: ACM, 2022: 338−349

[1]	Cao Yiran, Zhu Youwen, He Xingyu, Zhang Yue. Utility-Optimized Local Differential Privacy Set-Valued Data Frequency Estimation Mechanism[J]. Journal of Computer Research and Development, 2022, 59(10): 2261-2274. DOI: 10.7544/issn1000-1239.20220504
[2]	Hong Jinxin, Wu Yingjie, Cai Jianping, Sun Lan. Differentially Private High-Dimensional Binary Data Publication via Attribute Segmentation[J]. Journal of Computer Research and Development, 2022, 59(1): 182-196. DOI: 10.7544/issn1000-1239.20200701
[3]	Wu Wanqing, Zhao Yongxin, Wang Qiao, Di Chaofan. A Safe Storage and Release Method of Trajectory Data Satisfying Differential Privacy[J]. Journal of Computer Research and Development, 2021, 58(11): 2430-2443. DOI: 10.7544/issn1000-1239.2021.20210589
[4]	Zhang Yuxuan, Wei Jianghong, Li Ji, Liu Wenfen, Hu Xuexian. Graph Degree Histogram Publication Method with Node-Differential Privacy[J]. Journal of Computer Research and Development, 2019, 56(3): 508-520. DOI: 10.7544/issn1000-1239.2019.20170886
[5]	Zhu Weijun, You Qingguang, Yang Weidong, Zhou Qinglei. Trajectory Privacy Preserving Based on Statistical Differential Privacy[J]. Journal of Computer Research and Development, 2017, 54(12): 2825-2832. DOI: 10.7544/issn1000-1239.2017.20160647
[6]	Wu Yingjie, Zhang Liqun, Kang Jian, Wang Yilei. An Algorithm for Differential Privacy Streaming Data Adaptive Publication[J]. Journal of Computer Research and Development, 2017, 54(12): 2805-2817. DOI: 10.7544/issn1000-1239.2017.20160555
[7]	Wang Liang, Wang Weiping, Meng Dan. Privacy Preserving Data Publishing via Weighted Bayesian Networks[J]. Journal of Computer Research and Development, 2016, 53(10): 2343-2353. DOI: 10.7544/issn1000-1239.2016.20160465
[8]	Lu Guoqing, Zhang Xiaojian, Ding Liping, Li Yanfeng, Liao Xin. Frequent Sequential Pattern Mining under Differential Privacy[J]. Journal of Computer Research and Development, 2015, 52(12): 2789-2801. DOI: 10.7544/issn1000-1239.2015.20140516
[9]	Ouyang Jia, Yin Jian, Liu Shaopeng, Liu Yubao. An Effective Differential Privacy Transaction Data Publication Strategy[J]. Journal of Computer Research and Development, 2014, 51(10): 2195-2205. DOI: 10.7544/issn1000-1239.2014.20130824
[10]	Ni Weiwei, Chen Geng, Chong Zhihong, Wu Yingjie. Privacy-Preserving Data Publication for Clustering[J]. Journal of Computer Research and Development, 2012, 49(5): 1095-1104.

Cited By

Cited by

Periodical cited type(5)

1.	张涵，于航，周继威，白云开，赵路坦. 面向隐私计算的可信执行环境综述. 计算机应用. 2025(02): 467-481 .
2.	付裕，林璟锵，冯登国. 虚拟化与密码技术应用:现状与未来. 密码学报(中英文). 2024(01): 3-21 .
3.	徐传康，李忠月，刘天宇，种统洪，杨发雪. 基于可信执行环境的汽车域控系统安全研究. 汽车实用技术. 2024(15): 18-25+73 .
4.	徐文嘉，岑孟杰，陈亮. 隐私保护下单细胞RNA测序数据细胞分类研究. 医学信息学杂志. 2024(10): 86-89 .
5.	孙钰，熊高剑，刘潇，李燕. 基于可信执行环境的安全推理研究进展. 信息网络安全. 2024(12): 1799-1818 .