ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2019, Vol. 56 ›› Issue (12): 2589-2599.doi: 10.7544/issn1000-1239.2019.20180741

Previous Articles     Next Articles

Deep Stack Least Square Classifier with Inter-Layer Model Knowledge Transfer

Feng Wei1, Hang Wenlong1, Liang Shuang2, Liu Xuejun1, Wang Hui1   

  1. 1(School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816);2(School of Geographic and Biologic Information, Nanjing University of Posts and Telecommunications, Nanjing 210023)
  • Online:2019-12-01

Abstract: The traditional least square classifier (LSC) has been widely used in image recognition, speech recognition and other fields due to its simplicity and effectiveness. However, the traditional LSC may suffer from the weak generalization capacity when taking the natural data in their raw form as the input. In order to overcome this problem, a deep transfer least square classifier (DTLSC) is proposed on the basis of the stack generalization philosophy and the transfer learning mechanism. Firstly, following the stack generalization philosophy, DTLSC adopts LSC as the basic stacking unit to construct a deep stacking network, which avoids solving the non-convex optimization problem existing in traditional deep networks. Thus, the classification performance and the computational efficiency of the proposed network are improved. Secondly, transfer learning mechanism is used to leverage the model knowledge of the previous layers to help construction the model of the current layer such that the consistency of the inter-layer model is guaranteed. Thus, the generalization performance of the proposed DTLSC is further improved. In addition, the adaptive transfer learning strategy is introduced to selectively use the model knowledge of the previous layers, which alleviates the negative transfer effect by rejecting the uncorrelated model knowledge of the previous layer. Experimental results on synthetic datasets and real world datasets show the effectiveness of the proposed DTLSC.

Key words: least squares classifier(LSC), generalization capacity, deep learning, stack generalization, transfer learning

CLC Number: