Modeling of Few-Shot Relation Extraction Based on Adaptive Self-Training

Chen Honghui; Zheng Jianming; Cai Fei; Han Yi

doi:10.7544/issn1000-1239.202220216

Journal of Computer Research and Development > 2023 > 60(7): 1581-1591. > DOI: 10.7544/issn1000-1239.202220216

Chen Honghui, Zheng Jianming, Cai Fei, Han Yi. Modeling of Few-Shot Relation Extraction Based on Adaptive Self-Training[J]. Journal of Computer Research and Development, 2023, 60(7): 1581-1591. DOI: 10.7544/issn1000-1239.202220216

Citation:

PDF (1552 KB)

Modeling of Few-Shot Relation Extraction Based on Adaptive Self-Training

1.
College of System Engineering, National University of Defense Technology, Changsha 410073
2.
College of Meteorology and Oceanography, National University of Defense Technology, Changsha 410073

Funds: This work was supported by the Postgraduate Scientific Research Innovation Project of Hunan Province （CX20190034, CX20210068）.

More Information

Author Bio:
Chen Honghui: born in 1969. PhD, professor, PhD supervisor. His main research interests include recommendation systems, information retrieval and natural language processing

Zheng Jianming: born in 1993. PhD candidate. His main research interests include event representation and few-shot learning

Cai Fei: born in 1984. PhD, associate professor. His main research interests include recommend system, information retrieval, and query formulation

Han Yi: born in 1993. PhD, lecturer. His main research interests include event representation and few-shot learning
Received Date: March 13, 2022
Revised Date: September 15, 2022
Available Online: April 13, 2023

Graphical Abstract

Abstract

Abstract

Relation extraction (RE) is a basic task in natural language processing, which supports plenty of downstream tasks, e.g., dialogue generation, machine reading comprehension, etc. In real life, due to the continuously emerging new relation labels, the speed and cost of human annotation cannot catch up with the data quantity that the training of the traditional supervised RE models demands. Facing this practical challenge, the neural snowball model proposes a bootstrapping method that transfers the RE knowledge from limited labeled instances to iteratively annotate unlabeled data as to increase the amount of labeled data, thereby improving the classification performance of the model. However, the fixed threshold selection and the equally treated unlabeled data make the neural snowball model vulnerable to noise data. To solve these two defects, an adaptive self-training relation extraction (Ada-SRE) model is proposed. In specific, for the fixed-threshold issue, Ada-SRE proposes an adaptive threshold module by the meta learning of threshold, which can provide an appropriate threshold for each relation category. For the equally-treated issue, Ada-SRE designs a gradient-feedback strategy to weight each selected example, avoiding the interference of noise data. The experimental results show that compared with the neural snowball model, Ada-SRE has a better relation extraction ability.
- self-training,
- relation extraction,
- gradient feedback,
- few-shot learning,
- meta learning

FullText(HTML)

References (42)

References

[1]	Wang Hailin, Lu Guoming, Yin Jin, et al. Relation extraction: A brief survey on deep neural network based methods [C] //Proc of the 4th Int Conf on Software Engineering and Information Management. New York: ACM, 2021: 220−228
[2]	夏维,王珊蕾,尹子都,等. 基于互信息的知识图谱实体关联关系建模与补全[J]. 计算机科学与探索,2018,12(7):1064−1074 doi: 10.3778/j.issn.1673-9418.1709092 Xia Wei, Wang Shanlei, Yin Zidu, et al. Mutual information based modeling and completion of correlations in knowledge graphs[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(7): 1064−1074 (in Chinese) doi: 10.3778/j.issn.1673-9418.1709092
[3]	张芳容,杨青. 知识库问答系统中实体关系抽取方法研究[J]. 计算机工程与应用,2020,56(11):219−224 doi: 10.3778/j.issn.1002-8331.1904-0151 Zhang Fangrong, Yang Qing. Research on entity relation extraction method in knowledge-based question answering[J]. Computer Engineering and Applications, 2020, 56(11): 219−224 (in Chinese) doi: 10.3778/j.issn.1002-8331.1904-0151
[4]	周孟佳. 面向对话文本的关系抽取研究[D]. 武汉: 武汉大学, 2021 Zhou Mengjia. Research on relation extraction for dialogue text [D]. Wuhan: Wuhan University, 2021 (in Chinese)
[5]	Despina C, Grigorios T. Improving distantly-supervised relation extraction through BERT-based label and instance embeddings[J]. IEEE Access, 2021, 9: 62574−62582 doi: 10.1109/ACCESS.2021.3073428
[6]	Garg S, Galstyan A, Steeg G, et al. Kernelized hashcode representations for relation extraction [C] //Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 6431−6440
[7]	Gormley M, Yu Mo, Dredze M. Improved relation extraction with feature-rich compositional embedding models [C] //Proc of the 2015 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 1774−1784
[8]	Zeng Daojian, Liu Kang, Lai Siwei, et al. Relation classification via convolutional deep neural network [C] //Proc of the 25th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2014: 2335−2344
[9]	Han Xu, Zhu Hao, Yu Pengfei, et al. FewRel: A large-scale supervised few-shot relation classification dataset with state-of-the-art evaluation [C] //Proc of the 2018 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 4803−4809
[10]	Vrandecic D, Krötzsch M. Wikidata: A free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78−85 doi: 10.1145/2629489
[11]	Munkhdalai T, Yu Hong. Meta networks [C] //Proc of the 34th Int Conf on Machine Learning. New York: PMLR, 2017: 2554−2563
[12]	Satorras V, Estrach J. Few-shot learning with graph neural networks [C/OL] //Proc of the 6th Int Conf on Learning Representations. 2018[2022-08-24].https://openreview.net/forum?id=BJj6qGbRW
[13]	Snell J, Swersky K, Zemel R. Prototypical networks for few-shot learning [C] //Proc of Annual Conf on Neural Information Processing Systems 2017. New York: PMLR, 2017: 4077−4087
[14]	Mishra N, Rohaninejad M, Chen Xi, et al. A simple neural attentive meta-learner [C/OL] //Proc of the 6th Int Conf on Learning Representations. 2018[2022-08-24].https://openreview.net/forum?id=B1DmUzWAW
[15]	Gao Tianyu, Han Xu, Xie Ruobing, et al. Neural snowball for few-shot relation learning [C] //Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 7772−7779
[16]	Finn C, Abbeel P, Levine S. Model-agnostic meta-learning for fast adaptation of deep networks [C] //Proc of the 34th Int Conf on Machine Learning. New York: PMLR, 2017: 1126−1135
[17]	Ghosh D, Muresan S. Relation classification using entity sequence kernels [C] //Proc of the 24th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2012: 391−400
[18]	Leeuwenberg A, Buzmakov A, Toussaint Y, et al. Exploring pattern structures of syntactic trees for relation extraction [C] // Proc of the 13th Int Conf on Formal Concept Analysis. Berlin: Springer, 2015: 153−168
[19]	Cho C, Choi Y. Dependency tree positional encoding method for relation extraction [C] // Proc of the 36th ACM/SIGAPP Symp on Applied Computing. New York: ACM, 2021: 1012−1020
[20]	Shi Yong, Xiao Yang, Quan Pei, et al. Distant supervision relation extraction via adaptive dependency-path and additional knowledge graph supervision[J]. Neural Networks, 2021, 134: 42−53 doi: 10.1016/j.neunet.2020.10.012
[21]	Reichartz F, Korte H, Paass G. Composite kernels for relation extraction [C] //Proc of the 47th Annual Meeting of the Association for Computational Linguistics and the 4th Int Joint Conf on Natural Language Processing. Stroudsburg, PA: ACL, 2009: 365−368
[22]	Bhamare B, Prabhu J. A supervised scheme for aspect extraction in sentiment analysis using the hybrid feature set of word dependency relations and lemmas[J]. PeerJ Computer Science, 2021, 7: e347 doi: 10.7717/peerj-cs.347
[23]	Ravi S, Larochelle H. Optimization as a model for few-shot learning [C/OL] //Proc of the 5th Int Conf on Learning Representations. 2017[2022-08-24].https://openreview.net/forum?id=rJY0-Kcll
[24]	Yu Yang, Wang Guohua, Ren Haopeng, et al. Incorporating bidirection-interactive information and semantic features for relational facts extraction (student abstract) [C] //Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 15947−15948
[25]	Wen Haixu, Zhu Xinhua, Zhang Lanfang, et al. A gated piecewise CNN with entity-aware enhancement for distantly supervised relation extraction[J]. Information Processing & Management, 2020, 57(6): 102373−102373
[26]	Niu Weicai, Chen Quan, Zhang Weiwen, et al. GCN2-NAA: Two-stage graph convolutional networks with node-aware attention for joint entity and relation extraction [C] //Proc of the 13th Int Conf on Machine Learning and Computing. New York: ACM, 2021: 542−549
[27]	Peng Yifan, Rios A, Kavuluru R, et al. Chemical-protein relation extraction with ensembles of SVM, CNN, and RNN models [J]. arXiv preprint, arXiv: 1802.01255, 2018
[28]	Prabhudesai M, Lal S, Patil D, et al. Disentangling 3D prototypical networks for few-shot concept learning [C/OL] //Proc of the 9th Int Conf on Learning Representations. 2021 [2022-08-24].https://openreview.net/forum?id=-Lr-u0b42he
[29]	Sung F, Yang Yongxin, Zhang Li, et al. Learning to compare: Relation network for few-shot learning [C] //Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 1199−1208
[30]	Obamuyide A, Vlachos A. Model-agnostic meta-learning for relation classification with limited supervision [C] //Proc of the 57th Conf of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 5873−5879
[31]	Santoro A, Bartunov S, Botvinick M, et al. Meta-learning with memory-augmented neural networks [C] //Proc of the 33rd Int Conf on Machine Learning. New York: JMLR, 2016: 1842−1850
[32]	George V, Morar V, Yang Weiwei, et al. Learning without gradient descent encoded by the dynamics of a neurobiological model [J]. arXiv preprint, arXiv: 2103.08878, 2021
[33]	Huang Posen, Wang Chenglong, Singh R, et al. Natural language to structured query generation via meta-learning [C] //Proc of the 2018 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2018: 732−738
[34]	Gao Tianyu, Han Xu, Liu Zhiyuan, et al. Hybrid attention-based prototypical networks for noisy few-shot relation classification [C] //Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 6407−6414
[35]	Ye Zhixiu, Ling Zhenhua. Multi-level matching and aggregation network for few-shot relation classification [C] //Proc of the 57th Conf of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 2872−2881
[36]	Soares L, FitzGerald N, Ling J, et al. Matching the blanks: Distributional similarity for relation learning [C] //Proc of the 57th Conf of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 2895−2905
[37]	Qu Meng, Gao Tianyu, Xhonneux L, et al. Few-shot relation extraction via Bayesian meta-learning on relation graphs [C] //Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 7867−7876
[38]	Stoica G, Platanios E, Póczos B. Re-TACRED: Addressing shortcomings of the TACRED dataset [C] //Proc of the 35th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 13843−13850
[39]	Batista D, Martins B, Silva M. Semi-supervised bootstrapping of relationship extractors with distributional semantics [C] //Proc of the 2015 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 499−504
[40]	Agichtein E, Gravano L. Snowball: Extracting relations from large plain-text collections [C] //Proc of the 5th ACM Conf on Digital Libraries. New York: ACM, 2000: 85−94
[41]	Hu Xuming, Zhang Chenwei, Ma Fukun, et al. Semi-supervised relation extraction via incremental meta self-training [C] //Proc of the 2021 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2021: 487−496
[42]	Li Wanli, Qian Tieyun, Chen Xu, et al. Exploit a multi-head reference graph for semi-supervised relation extraction [C] //Proc of the 31st Int Joint Conf on Neural Networks. Piscataway, NJ: IEEE, 2021: 1−7

[1]	Wang Jihong, Zhao Shuqing, Luo Minnan, Liu Huan, Zhao Xiang, Zheng Qinghua. Robust Few-Label Misinformation Detection Based on Information Bottleneck Theory[J]. Journal of Computer Research and Development, 2024, 61(7): 1629-1642. DOI: 10.7544/issn1000-1239.202330506
[2]	Yang Jieyi, Dong Yihong, Qian Jiangbo. Research Progress of Few-Shot Learning Methods Based on Graph Neural Networks[J]. Journal of Computer Research and Development, 2024, 61(4): 856-876. DOI: 10.7544/issn1000-1239.202220933
[3]	Gao Yujia, Wang Pengfei, Liu Liang, Ma Huadong. Personalized Federated Learning Method Based on Attention-Enhanced Meta-Learning Network[J]. Journal of Computer Research and Development, 2024, 61(1): 196-208. DOI: 10.7544/issn1000-1239.202220922
[4]	Lu Shaoshuai, Chen Long, Lu Guangyue, Guan Ziyu, Xie Fei. Weakly-Supervised Contrastive Learning Framework for Few-Shot Sentiment Classification Tasks[J]. Journal of Computer Research and Development, 2022, 59(9): 2003-2014. DOI: 10.7544/issn1000-1239.20210699
[5]	Ren Jiarui, Zhang Haiyan, Zhu Menghan, Ma Bo. Embedding Learning Algorithm for Heterogeneous Network Based on Meta-Graph Convolution[J]. Journal of Computer Research and Development, 2022, 59(8): 1683-1693. DOI: 10.7544/issn1000-1239.20220063
[6]	Wang Hang, Tian Shengzhao, Tang Qing, Chen Duanbing. Few-Shot Image Classification Based on Multi-Scale Label Propagation[J]. Journal of Computer Research and Development, 2022, 59(7): 1486-1495. DOI: 10.7544/issn1000-1239.20210376
[7]	Zhang Lingling, Chen Yiwei, Wu Wenjun, Wei Bifan, Luo Xuan, Chang Xiaojun, Liu Jun. Interpretable Few-Shot Learning with Contrastive Constraint[J]. Journal of Computer Research and Development, 2021, 58(12): 2573-2584. DOI: 10.7544/issn1000-1239.2021.20210999
[8]	Cheng Daning, Zhang Hanping, Xia Fen, Li Shigang, Yuan Liang, Zhang Yunquan. AccSMBO: Using Hyperparameters Gradient and Meta-Learning to Accelerate SMBO[J]. Journal of Computer Research and Development, 2020, 57(12): 2596-2609. DOI: 10.7544/issn1000-1239.2020.20190670
[9]	Dong Ye, Hou Wei, Chen Xiaojun, Zeng Shuai. Efficient and Secure Federated Learning Based on Secret Sharing and Gradients Selection[J]. Journal of Computer Research and Development, 2020, 57(10): 2241-2250. DOI: 10.7544/issn1000-1239.2020.20200463
[10]	Liu Huan, Zheng Qinghua, Luo Minnan, Zhao Hongke, Xiao Yang, Lü Yanzhang. Cross-Domain Adversarial Learning for Zero-Shot Classification[J]. Journal of Computer Research and Development, 2019, 56(12): 2521-2535. DOI: 10.7544/issn1000-1239.2019.20190614