基于层间模型知识迁移的深度堆叠最小二乘分类器

冯伟; 杭文龙; 梁爽; 刘学军; 王辉

doi:10.7544/issn1000-1239.2019.20180741

基于层间模型知识迁移的深度堆叠最小二乘分类器

¹(南京工业大学计算机科学与技术学院南京 211816)
²(南京邮电大学地理与生物信息学院南京 210023) (fengwei@njtech.edu.cn)

基金项目: 国家自然科学基金项目(61802177);江苏省高校自然科学基金研究面上项目(18KJB520020);南京工业大学引进人才启动项目(3827401749);江苏省重点研发计划项目(BE2015697)

详细信息

中图分类号: TP391; TP18
计量
- 文章访问数: 1137
- HTML全文浏览量: 1
- PDF下载量: 295
出版历程
- 发布日期: 2019-11-30

Deep Stack Least Square Classifier with Inter-Layer Model Knowledge Transfer

¹(School of Computer Science and Technology, Nanjing Tech University, Nanjing 211816)
²(School of Geographic and Biologic Information, Nanjing University of Posts and Telecommunications, Nanjing 210023)

摘要

摘要: 经典的最小二乘分类器(least square classifier, LSC)由于其简洁、有效性已早被广泛应用于图像识别、语音识别等领域.然而，利用原始数据特征构建的最小二乘分类器，其泛化性能往往较差.为解决上述问题，提出了基于深度堆叠泛化和迁移学习机制的深度迁移最小二乘分类器(deep transfer least square classifier, DTLSC).首先，基于堆叠泛化原理，利用LSC模型作为基本堆叠单元构建深度堆叠网络，避免了传统深度网络中需要求解非凸优化的问题，提升模型分类性能的同时提高了网络计算效率.其次，基于迁移学习机制，利用前层单元中的模型知识辅助当前层的模型构建，尽可能保持了层间模型的一致性，提升了模型泛化性能.在此基础上，引入自适应迁移策略，有选择地利用前层模型知识，缓解了利用前层不相关模型知识而导致的负迁移效应.在人造数据集及真实数据集上，实验结果验证了所提DTLSC算法的有效性.
- 最小二乘分类器 /
- 泛化性能 /
- 深度学习 /
- 堆叠泛化 /
- 迁移学习
Abstract: The traditional least square classifier (LSC) has been widely used in image recognition, speech recognition and other fields due to its simplicity and effectiveness. However, the traditional LSC may suffer from the weak generalization capacity when taking the natural data in their raw form as the input. In order to overcome this problem, a deep transfer least square classifier (DTLSC) is proposed on the basis of the stack generalization philosophy and the transfer learning mechanism. Firstly, following the stack generalization philosophy, DTLSC adopts LSC as the basic stacking unit to construct a deep stacking network, which avoids solving the non-convex optimization problem existing in traditional deep networks. Thus, the classification performance and the computational efficiency of the proposed network are improved. Secondly, transfer learning mechanism is used to leverage the model knowledge of the previous layers to help construction the model of the current layer such that the consistency of the inter-layer model is guaranteed. Thus, the generalization performance of the proposed DTLSC is further improved. In addition, the adaptive transfer learning strategy is introduced to selectively use the model knowledge of the previous layers, which alleviates the negative transfer effect by rejecting the uncorrelated model knowledge of the previous layer. Experimental results on synthetic datasets and real world datasets show the effectiveness of the proposed DTLSC.
- least squares classifier(LSC) /
- generalization capacity /
- deep learning /
- stack generalization /
- transfer learning

HTML全文

参考文献(0)

施引文献(25)

期刊类型引用(9)

1.	张恩，王梦涛，郑东，禹勇，黄昱晨. 低带宽场景下防合谋多方隐私集合并集协议. 通信学报. 2025(01): 210-221 . 百度学术
2.	李子贤，刘文杰. 求解最小公倍数问题的量子安全多方计算协议. 计算机学报. 2024(06): 1393-1412 . 百度学术
3.	逯绍锋，胡玉龙，逯跃锋. 保护隐私的集合相似性度量协同计算协议. 计算机技术与发展. 2023(01): 137-143 . 百度学术
4.	魏立斐，王勤，张蕾，陈聪聪，陈玉娇，宁建廷. 半可信云服务器辅助的高效隐私交集计算协议. 软件学报. 2023(02): 932-944 . 百度学术
5.	家珠亮，赵雪玲，李顺东. 分布式多重集众数及重数的保密计算. 密码学报. 2023(01): 102-117 . 百度学术
6.	魏立斐，刘纪海，张蕾，宁建廷. 双云辅助的超阈值多方隐私集合交集计算协议. 软件学报. 2023(11): 5442-5456 . 百度学术
7.	赵宗渠，王书静，汤永利，霍亚超，杨丽. 基于理想格的两方隐私集合交集协议. 计算机应用研究. 2023(12): 3795-3799 . 百度学术
8.	徐琳，魏晓超，蔡国鹏，王皓，郑志华. 一个高效的安全两方近似模式匹配协议. 计算机研究与发展. 2022(08): 1819-1830 . 本站查看
9.	魏立斐，刘纪海，张蕾，王勤，贺崇德. 面向隐私保护的集合交集计算综述. 计算机研究与发展. 2022(08): 1782-1799 . 本站查看