ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2019, Vol. 56 ›› Issue (12): 2589-2599.doi: 10.7544/issn1000-1239.2019.20180741

• 人工智能 • 上一篇    下一篇

基于层间模型知识迁移的深度堆叠最小二乘分类器

冯伟1,杭文龙1,梁爽2,刘学军1,王辉1   

  1. 1(南京工业大学计算机科学与技术学院 南京 211816);2(南京邮电大学地理与生物信息学院 南京 210023) (fengwei@njtech.edu.cn)
  • 出版日期: 2019-12-01
  • 基金资助: 
    国家自然科学基金项目(61802177);江苏省高校自然科学基金研究面上项目(18KJB520020);南京工业大学引进人才启动项目(3827401749);江苏省重点研发计划项目(BE2015697)

Deep Stack Least Square Classifier with Inter-Layer Model Knowledge Transfer

Feng Wei1, Hang Wenlong1, Liang Shuang2, Liu Xuejun1, Wang Hui1   

  1. 1(School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816);2(School of Geographic and Biologic Information, Nanjing University of Posts and Telecommunications, Nanjing 210023)
  • Online: 2019-12-01

摘要: 经典的最小二乘分类器(least square classifier, LSC)由于其简洁、有效性已早被广泛应用于图像识别、语音识别等领域.然而,利用原始数据特征构建的最小二乘分类器,其泛化性能往往较差.为解决上述问题,提出了基于深度堆叠泛化和迁移学习机制的深度迁移最小二乘分类器(deep transfer least square classifier, DTLSC).首先,基于堆叠泛化原理,利用LSC模型作为基本堆叠单元构建深度堆叠网络,避免了传统深度网络中需要求解非凸优化的问题,提升模型分类性能的同时提高了网络计算效率.其次,基于迁移学习机制,利用前层单元中的模型知识辅助当前层的模型构建,尽可能保持了层间模型的一致性,提升了模型泛化性能.在此基础上,引入自适应迁移策略,有选择地利用前层模型知识,缓解了利用前层不相关模型知识而导致的负迁移效应.在人造数据集及真实数据集上,实验结果验证了所提DTLSC算法的有效性.

关键词: 最小二乘分类器, 泛化性能, 深度学习, 堆叠泛化, 迁移学习

Abstract: The traditional least square classifier (LSC) has been widely used in image recognition, speech recognition and other fields due to its simplicity and effectiveness. However, the traditional LSC may suffer from the weak generalization capacity when taking the natural data in their raw form as the input. In order to overcome this problem, a deep transfer least square classifier (DTLSC) is proposed on the basis of the stack generalization philosophy and the transfer learning mechanism. Firstly, following the stack generalization philosophy, DTLSC adopts LSC as the basic stacking unit to construct a deep stacking network, which avoids solving the non-convex optimization problem existing in traditional deep networks. Thus, the classification performance and the computational efficiency of the proposed network are improved. Secondly, transfer learning mechanism is used to leverage the model knowledge of the previous layers to help construction the model of the current layer such that the consistency of the inter-layer model is guaranteed. Thus, the generalization performance of the proposed DTLSC is further improved. In addition, the adaptive transfer learning strategy is introduced to selectively use the model knowledge of the previous layers, which alleviates the negative transfer effect by rejecting the uncorrelated model knowledge of the previous layer. Experimental results on synthetic datasets and real world datasets show the effectiveness of the proposed DTLSC.

Key words: least squares classifier(LSC), generalization capacity, deep learning, stack generalization, transfer learning

中图分类号: