面向增量分类的多示例学习

魏秀参; 徐书林; 安鹏; 杨健

doi:10.7544/issn1000-1239.20220071

面向增量分类的多示例学习

魏秀参^1,2,3,4,5,
徐书林^1,3,4,
安鹏⁶,
杨健^1,3,4

¹(南京理工大学计算机科学与工程学院南京 210094)
²(综合业务网理论及关键技术国家重点实验室(西安电子科技大学) 西安 710071)
³(高维信息智能感知与系统教育部重点实验室(南京理工大学) 南京 210094)
⁴(社会安全图像与视频理解江苏省重点实验室(南京理工大学) 南京 210094)
⁵(计算机软件新技术国家重点实验室(南京大学) 南京 210023)
⁶(中国海洋石油集团有限公司信息技术中心北京 100010) (weixs@njust.edu.cn)

基金项目: 国家重点研发计划青年科学家项目(2021YFA1001100)；江苏省基础研究计划(自然科学基金)项目(BK20210340)；中国人工智能学会-华为MindSpore学术奖励基金；中央高校基本科研业务费专项资金(30920041111)；北京智源人工智能研究院悟道科研基金

详细信息

中图分类号: TP391
计量
- 文章访问数: 290
- HTML全文浏览量: 2
- PDF下载量: 132
出版历程
- 发布日期: 2022-07-31

Multi-Instance Learning with Incremental Classes

Wei Xiushen^1,2,3,4,5,
Xu Shulin^1,3,4,
An Peng⁶,
Yang Jian^1,3,4

¹(School of Computer Science and Engineering, Nanjing University of Science and Technology, Nanjing 210094)
²(State Key Laboratory of Integrated Services Networks(Xidian University), Xi’an 710071)
³(Key Laboratory of Intelligent Perception and Systems for High-Dimensional Information (Nanjing University of Science and Technology), Ministry of Education, Nanjing 210094)
⁴(Jiangsu Key Laboratory of Image and Video Understanding for Social Security (Nanjing University of Science and Technology), Nanjing 210094)
⁵(State Key Laboratory for Novel Software Technology (Nanjing University), Nanjing 210023)
⁶(Information Technology Center, China National Offshore Oil Corporation, Beijing 100010)

Funds: This work was supported by the National Key Research and Development Program of China (2021YFA1001100), the Natural Science Foundation of Jiangsu Province of China (BK20210340), the CAAI-Huawei MindSpore Open Fund, the Fundamental Research Funds for the Central Universities (30920041111), and the Wudao Research Funding of Beijing Academy of Artificial Intelligence (BAAI).

摘要

摘要: 近年来多示例学习(multi-instance learning, MIL)被广泛应用于复杂数据问题中，但现有的多示例学习算法往往在封闭静态环境中工作良好，其所处理的类别数量也恒定不变.然而在现实应用当中，常会有新的类别不断地加入到系统当中，例如科学的发展中不断出现新的议题、社交媒体中不断出现新的话题.由于存储限制或保密协议等原因，旧数据可能随着时间的发展变得不可见，这使得直接学习新的类别时模型会忘记曾经学过的知识.增量学习则被用于解决上述问题.因此，在多示例学习设定下进行增量数据挖掘十分有意义，然而目前针对多示例学习下的增量数据挖掘的工作十分稀少.提出一个基于注意力机制和原型分类器映射的多示例增量数据挖掘方法，通过注意力机制选择性地将多示例包的示例汇合为统一的特征表示，然后为每个类别生成类别原型表示并存储下来.类别原型通过原型分类器映射模块得到无偏鲁棒的类别分类器，并通过上一个增量阶段生成的分类器的预测结果对新增量阶段生成的分类器的预测结果进行知识蒸馏，使得模型能够在多示例学习下以极低的存储很好地保留模型的旧知识.实验结果表明：提出的方法能够有效地进行面向增量分类的多示例学习.
- 多示例学习 /
- 增量学习 /
- 注意力机制 /
- 知识蒸馏 /
- 原型
Abstract: In recent years, multi-instance learning (MIL) has been widely used in complicated data problems, but the existing MIL methods often study a fixed number of categories in a closed environment. However, in real applications, novel categories are constantly added to the system, such as the continuous emergence of new topics in the development of science or social media. Due to storage restrictions or confidentiality agreements, old data may become invisible over time, which makes the model forget the previously learned knowledge when directly learning new categories. Incremental learning is often used to deal with the aforementioned problems. The mining of multi-instance learning with incremental classes is very meaningful, but the current works on this is rare to be focused. We propose a novel multi-instance incremental data mining method based on both attention mechanism and prototype classifier mapping. Through the attention mechanism, the MIL bags are selectively merged into unified feature representations, which will be used to generate the corresponding storable category prototypes. Through the prototype classifier mapping, each category prototype is mapped into an unbiased and robust classifier. The prediction results of the classifier generated in the previous incremental stage are used to perform knowledge distillation on the prediction results of the classifier generated in novel incremental stages, so that the model can retain the old knowledge with very low storage under MIL. Experimental results on benchmarks of three different tasks show that our proposed method have achieved effective performance in MIL with incremental classes.
- multi-instance learning /
- incremental learning /
- attention mechanism /
- knowledge distillation /
- prototype

HTML全文

参考文献(0)

施引文献(16)

期刊类型引用(4)

1.	钱罗雄，陈梅，马学艳，张弛，张锦宏. 自适应张量奇异值收缩的多视角聚类. 计算机研究与发展. 2025(03): 733-750 . 本站查看
2.	赵兴旺，侯哲栋，姚凯旋，梁吉业. 基于注意力机制的两阶段融合多视图图聚类. 清华大学学报(自然科学版). 2024(01): 1-12 . 百度学术
3.	赵振廷，赵旭俊. 多样性约束和高阶信息挖掘的多视图聚类. 计算机应用研究. 2024(08): 2309-2314 . 百度学术
4.	李顺勇，许晓丽. 基于信息熵加权的多视图子空间聚类算法. 陕西科技大学学报. 2023(02): 207-214 . 百度学术