ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2018, Vol. 55 ›› Issue (1): 179-187.doi: 10.7544/issn1000-1239.2018.20160784

• 人工智能 • 上一篇    下一篇

基于稀疏自学习卷积神经网络的句子分类模型

高云龙1,2,左万利1,2,王英1,2,王鑫2,3   

  1. 1(吉林大学计算机科学与技术学院 长春 130012);2(符号计算与知识工程教育部重点实验室(吉林大学) 长春 130012);3(长春工程学院计算机技术与工程学院 长春 130012) (1458299660@qq.com)
  • 出版日期: 2018-01-01
  • 基金资助: 
    国家自然科学基金项目(60903098,60973040,61602057);国家自然科学基金青年科学基金项目(61300148);吉林省科技厅优秀青年人才基金项目(20170520059JH);吉林省教育厅青年基金项目(2016311);吉林大学研究生创新基金项目(2016184)

Sentence Classification Model Based on Sparse and Self-Taught Convolutional Neural Networks

Gao Yunlong1,2, Zuo Wanli1,2, Wang Ying1,2, Wang Xin2,3   

  1. 1(College of Computer Science and Technology, Jilin University, Changchun 130012);2(Key Laboratory of Symbolic Computation and Knowledge Engineering(Jilin University), Ministry of Education, Changchun 130012);3(College of Computer Technology and Engineering, Changchun Institute of Technology, Changchun 130012)
  • Online: 2018-01-01

摘要: 句子分类模型的建立对于自然语言理解的研究有着十分重要的意义.基于卷积神经网络(convolutional neural networks, CNN)提取数据特征的特点,提出基于稀疏自学习卷积神经网络(sparse and self-taught CNN, SCNN)的句子分类模型.首先,在卷积层排除人为约定的特征map输入,自学习前一层输入的特征矩阵的有效组合,动态捕获句子范围内各个特征的有效关联;然后,在训练过程中利用L1范数增加稀疏性约束,降低模型复杂度;最后,在采样层利用K-Max Pooling选择句子中最大特征的序列,并保留特征之间的相对次序.SCNN可以处理变长的句子输入,模型的建立不依赖于句法、分析树等语言学特征,从而适用于任何一种语言.通过对语料库进行句子分类实验,验证了所提出模型有较好的分类效果.

关键词: CNN, 稀疏, 自学习, 分类, L1范数

Abstract: The study and establishment of sentence classification model have an important impact on the study of nature language processing and understanding. In this paper, we propose a sentence classification model named SCNN based on sparse and self-taught convolutional neural networks in extracting characteristics of the features from data in the CNN model. Firstly, in this method, the convolutional layer itself studies the effective combinations from the feature matrices of the previous layers in order to dynamically learn the relationships of data features in the scope of the sentence, eliminating the user-defined feature-map input of the convolutional layers. Secondly, during the unsupervised training process, using L1-norm to increase sparse constraints, the complexity of the proposed model can be effectively decreased, on the contrary, the accuracy of SCNN model can be effectively increased. Finally, by employing K-Max Pooling in the feature extraction layer, the maximal feature sequence can be selected, and relative orders among features can be effectively preserved. SCNN can cope with sentence with variant length, and furthermore, the model can apply to any language due to its independence to any linguistic features like syntax and parse trees. Experiments on the standard corpus dataset show that the proposed model is effective for the task of the sentence classification.

Key words: convolutional neural networks (CNN), sparse, self-taught, classification, L1-norm

中图分类号: