Graph Convolution-Enhanced Multi-Channel Decoding Joint Entity and Relation Extraction Model

Qiao Yongpeng; Yu Yaxin; Liu Shuyue; Wang Ziteng; Xia Zifang; Qiao Jiaqi

doi:10.7544/issn1000-1239.202110767

Journal of Computer Research and Development > 2023 > 60(1): 153-166. > DOI: 10.7544/issn1000-1239.202110767

Qiao Yongpeng, Yu Yaxin, Liu Shuyue, Wang Ziteng, Xia Zifang, Qiao Jiaqi. Graph Convolution-Enhanced Multi-Channel Decoding Joint Entity and Relation Extraction Model[J]. Journal of Computer Research and Development, 2023, 60(1): 153-166. DOI: 10.7544/issn1000-1239.202110767

Citation:

PDF (1645 KB)

Graph Convolution-Enhanced Multi-Channel Decoding Joint Entity and Relation Extraction Model

School of Computer Science and Engineering, Northeastern University, Shenyang 110169
Key Laboratory of Intelligent Computing in Medical Image (Northeastern University), Ministry of Education, Shenyang 110169

Funds: This work was supported by the National Natural Science Foundation of China (61871106，61973059) and the National Key Research and Development Program of China (2016YFC0101500).

More Information

Received Date: July 14, 2021
Revised Date: March 21, 2022
Available Online: February 10, 2023

Graphical Abstract

Abstract

Abstract

Extracting relational triplets from unstructured natural language texts are the most critical step in building a large-scale knowledge graph, but existing researches still have the following problems: 1) Existing models ignore the problem of relation overlapping caused by multiple triplets sharing the same entity in text; 2) The current joint extraction model based on encoder-decoder does not fully consider the dependency relationship among words in the text; 3) The excessively long sequence of triplets leads to the accumulation and propagation of errors, which affects the precision and efficiency of relation extraction in entity. Based on this, a graph convolution-enhanced multi-channel decoding joint entity and relation extraction model (GMCD-JERE) is proposed. First, the BiLSTM is introduced as a model encoder to strengthen the two-way feature fusion of words in the text; second, the dependency relationship between the words in the sentence is merged through the graph convolution multi-hop mechanism to improve the accuracy of relation classification; third, through multi-channel decoding mechanism, the model solves the problem of relation overlapping, and alleviates the effect of error accumulation and propagation at the same time; fourth, the experiment selects the current three mainstream models for performance verification, and the results on the NYT (New York times) dataset show that the accuracy rate, recall rate, and F1 are increased by 4.3%, 5.1% and 4.8%. Also, the extraction order starting with the relation is verified in the WebNLG (Web natural language generation) dataset.
- relation extraction,
- encoder-decoder,
- multi-channel decoding,
- relation overlapping,
- graph convolution neural network

FullText(HTML)

References (43)

References

[1]	李冬梅,张扬,李东远,等. 实体关系抽取方法研究综述[J]. 计算机研究与发展,2020,57(7):1424−1448 doi: 10.7544/issn1000-1239.2020.20190358 Li Dongmei, Zhang Yang, Li Dongyuan, et al. Review of entity relation extraction methods[J]. Journal of Computer Research and Development, 2020, 57(7): 1424−1448 (in Chinese) doi: 10.7544/issn1000-1239.2020.20190358
[2]	Zeng Daojian, Liu Kang, Lai Siwei, et al. Relation classification via convolutional deep neural network[C] //Proc of the 25th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2014: 2335−2344
[3]	Xu Kun, Feng Yansong, Huang Songfang, et al. Semantic relation classification via convolutional neural networks with simple negative sampling[C] //Proc of the 2015 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 536−540
[4]	Chan S Y, Roth D. Exploiting syntactico-semantic structures for relation extraction[C] //Proc of the 49th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2011: 551−560
[5]	Li Qi, Ji Heng. Incremental joint extraction of entity mentions and relations[C] //Proc of the 52nd Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2014: 402−412
[6]	Miwa M, Bansal M. End-to-end relation extraction using LSTMs on sequences and tree structures[C] //Proc of the 54th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2016: 1105−1116
[7]	曹明宇,杨志豪,罗凌,等. 基于神经网络的药物实体与关系联合抽取[J]. 计算机研究与发展,2019,56(7):1432−1440 doi: 10.7544/issn1000-1239.2019.20180714 Cao Mingyu, Yang Zhihao, Luo Ling, et al. Joint drug entities and relations extraction based on neural networks[J]. Journal of Computer Research and Development, 2019, 56(7): 1432−1440 (in Chinese) doi: 10.7544/issn1000-1239.2019.20180714
[8]	Zhang Meishan, Zhang Yue, Fu Guohong. End-to-end neural relation extraction with global optimization[C] //Proc of the 2017 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2017: 1730−1740
[9]	Gupta P, Schtze H, Andrassy B. Table filling multi-task recurrent neural network for joint entity and relation extraction[C] //Proc of the 26th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2016: 2537−2547
[10]	Zheng Suncong, Wang Feng, Bao Hongyun, et al. Joint extraction of entities and relations based on a novel tagging scheme[C] //Proc of the 55th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2017: 1227−1236
[11]	Zeng Xiangrong, Zeng Daojian, He Shizhu, et al. Extracting relational facts by an end-to-end neural model with copy mechanism[C] //Proc of the 56th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2018: 506−514
[12]	Peng Nanyun, Hoifung P, Chris Q, et al. Cross-sentence n-ary relation extraction with graph LSTMs[J]. Transactions of the ACL, 2017, 5: 101–115
[13]	Sahu K S, Christopoulou F, Miwa M, et al. Inter-sentence relation extraction with document-level graph convolutional neural network[C] //Proc of the 57th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2019: 4309−4316
[14]	Dai Dai, Xiao Xinyan, Lu Yajuan, et al. Joint extraction of entities and overlapping relations using position-attentive sequence labeling[C] //Proc of the 33rd AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2019: 6300−6308
[15]	Nayak T, Ng H T. Effective modeling of encoder-decoder architecture for joint entity and relation extraction[C] //Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 8528−8535
[16]	甘丽新,万常选,刘德喜,等. 基于句法语义特征的中文实体关系抽取[J]. 计算机研究与发展,2016,53(2):284−302 doi: 10.7544/issn1000-1239.2016.20150842 Gan Lixin, Wan Changxuan, Liu Dexi, et al. Chinese named entity relation extraction based on syntactic and semantic features[J]. Journal of Computer Research and Development, 2016, 53(2): 284−302 (in Chinese) doi: 10.7544/issn1000-1239.2016.20150842
[17]	田驰远,陈德华,王梅,等. 基于依存句法分析的病例报告结构化处理方法[J]. 计算机研究与发展,2016,52(12):2669−2680 doi: 10.7544/issn1000-1239.2016.20160611 Tian Chiyuan, Chen Dehua, Wang Mei, et al. Structured processing for pathological reports based on dependency parsing[J]. Journal of Computer Research and Development, 2016, 52(12): 2669−2680 (in Chinese) doi: 10.7544/issn1000-1239.2016.20160611
[18]	刘克彬,李芳,刘磊,等. 基于核函数中文关系自动抽取系统的实现[J]. 计算机研究与发展,2007,44(8):1406−1411 doi: 10.1360/crad20070818 Liu Kebin, Li Fang, Liu Lei, et al. Implement of a kernel-based chinese relation extraction system[J]. Journal of Computer Research and Development, 2007, 44(8): 1406−1411 (in Chinese) doi: 10.1360/crad20070818
[19]	Zelenko D, Aone C, Richardella A. Kernel methods for relation extraction[C] //Proc of the 2002 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2002: 71−78
[20]	Hendrickx I, Kim N S, Kozareva Z, et al. Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals[C] //Proc of the 5th Int Workshop on Semantic Evaluation. Stroudsburg, PA: ACL, 2010: 33−38
[21]	Xu Yan, Mou Lili, Li Ge, et al. Classifying relations via long short term memory networks along shortest dependency paths[C] //Proc of the 2015 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2015: 1785−1794
[22]	Florian R, Jing Hongyan, Kambhatla N, et al. Factorizing complex models: A case study in mention detection[C] //Proc of the 44th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2006: 473−480
[23]	Florian R, Pitrelli J, Roukos S. et al. Improving mention detection robustness to noisy input[C] //Proc of the 2010 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2010: 335−345
[24]	Zhao Shubin, Grishman R. Extracting relations with integrated information using kernel methods[C] //Proc of the 43rd Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2005: 419−426
[25]	Jiang Jing, Zhai Chengxiang. A systematic exploration of the feature space for relation extraction[C] //Proc of the 2007 Conf of the North American Chapter of the ACL. Stroudsburg, PA: ACL, 2007: 113−120
[26]	Sun A, Grishman R, Sekine S. Semi-supervised relation extraction with large-scale word clustering[C] //Proc of the 49th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2011: 521−529
[27]	Plank B, Moschitti A. Embedding semantic similarity in tree kernels for domain adaptation of relation extraction[C] //Proc of the 51st Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2013: 1498−1507
[28]	Mintz M, Bills S, Snow R, et al. Distant supervision for relation extraction without labeled data[C] //Proc of the Conf of the 47th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2009: 1003−1011
[29]	Cai Rui, Zhang Xiaodong, Wang Houfang. Bidirectional recurrent convolutional neural network for relation classification[C] //Proc of the 54th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2016: 756−765
[30]	Christopoulou F, Miwa M, Ananiadou S. A walk-based model on entity graphs for relation extraction[C] //Proc of the 56th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2018: 81−88
[31]	Qin Pengda, Xu Weiran, Wang Yang William. Robust distant supervision relation extraction via deep reinforcement learning[C] //Proc of the 56th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2018: 2137−2147
[32]	Yu Xiaofeng, Lam W. Jointly identifying entities and extracting relations in encyclopedia text via a graphical model approach[C] //Proc of the 2010 Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2010: 1399−1407
[33]	Miwa M, Sasaki Y. Modeling joint entity and relation extraction with table representation[C] //Proc of the 2014 Conf on EMNLP. Stroudsburg, PA: ACL, 2014: 1858−1869
[34]	Ren Xiang, Wu Zeqiu, He Wenqi, et al. CoType: Joint extraction of typed entities and relations with knowledge bases[C] //Proc of the 26th Int Conf on World Wide Web. New York: ACM, 2017: 1015−1024
[35]	黄培馨,赵翔,方阳,等. 融合对抗训练的端到端知识三元组联合抽取[J]. 计算机研究与发展,2019,56(12):2536−2548 doi: 10.7544/issn1000-1239.2019.20190640 Huang Peixin, Zhao Xiang, Fang Yang, et al. End-to-end knowledge triplet extraction combined with adversarial training[J]. Journal of Computer Research and Development, 2019, 56(12): 2536−2548 (in Chinese) doi: 10.7544/issn1000-1239.2019.20190640
[36]	Fu T J, Li P, Ma W Y. GraphRel: Modeling text as relational graphs for joint entity and relation extraction[C] //Proc of the 57th Annual Meeting of the ACL. Stroudsburg, PA: ACL, 2019: 1409−1418
[37]	Zeng Daojian, Zhang Haoran, Liu Qianying. CopyMTL: Copy mechanism for joint extraction of entities and relations with multi-task learning[C] //Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020: 9507−9514
[38]	Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling[C] //Proc of the 2017 Conf on EMNLP. Stroudsburg, PA: ACL, 2017: 1506−1515
[39]	Liu Bang, Zhang Ting, Niu Di, et al. Matching long text documents via graph convolutional networks[J]. arXiv preprint, arXiv: 1802.07459, 2018
[40]	Cetoli A, Bragaglia S, O’Harney A, et al. Graph convolutional networks for named entity recognition[C] //Proc of the 16th Int Workshop on Treebanks and Linguistic Theories. Stroudsburg, PA: ACL, 2018: 37−45
[41]	Zhang Yuhao, Qi Peng, Christopher D M. Graph convolution over pruned dependency trees improves relation extraction[C] //Proc of the 2018 Conf on EMNLP. Stroudsburg, PA: ACL, 2018: 2205−2215
[42]	Luan Yi, Wadden D, He Luheng, et al. A general framework for information extraction using dynamic span graphs[C] //Proc of the 2019 Conf of the North American Chapter of the ACL. Stroudsburg, PA: ACL, 2019: 3036−3046
[43]	Qian Yujie, Santus E, Jin Zhijing, et al. GraphIE: A graph-based framework for information extraction[C] //Proc of the 2019 Conf of the North American Chapter of the ACL. Stroudsburg, PA: ACL, 2019: 751−761

[1]	Zhu Chuanzhen, Wang Xuan, Zheng Qiang. Element Separation and Holistic Attention Based Graph Convolutional Network Framework[J]. Journal of Computer Research and Development, 2024, 61(8): 2008-2019. DOI: 10.7544/issn1000-1239.202440143
[2]	Shang Jing, Wu Zhihui, Xiao Zhiwen, Zhang Yifei. Graph4Cache: A Graph Neural Network Model for Cache Prefetching[J]. Journal of Computer Research and Development, 2024, 61(8): 1945-1956. DOI: 10.7544/issn1000-1239.202440190
[3]	Liu Linlan, Feng Zhenxing, Shu Jian. Dynamic Network Link Prediction Based on Sequential Graph Convolution[J]. Journal of Computer Research and Development, 2024, 61(2): 518-528. DOI: 10.7544/issn1000-1239.202220776
[4]	Xie Kunpeng, Yi Dezhi, Liu Yiqing, Liu Hang, He Xinyu, Gong Cheng, Lu Ye. SAF-CNN：A Sparse Acceleration Framework of Convolutional Neural Network forEmbedded FPGAs[J]. Journal of Computer Research and Development, 2023, 60(5): 1053-1072. DOI: 10.7544/issn1000-1239.202220735
[5]	Ren Jiarui, Zhang Haiyan, Zhu Menghan, Ma Bo. Embedding Learning Algorithm for Heterogeneous Network Based on Meta-Graph Convolution[J]. Journal of Computer Research and Development, 2022, 59(8): 1683-1693. DOI: 10.7544/issn1000-1239.20220063
[6]	Xie Xiaojie, Liang Ying, Wang Zisen, Liu Zhengjun. Heterogeneous Network Node Classification Method Based on Graph Convolution[J]. Journal of Computer Research and Development, 2022, 59(7): 1470-1485. DOI: 10.7544/issn1000-1239.20210124
[7]	Ma Shuai, Liu Jianwei, Zuo Xin. Survey on Graph Neural Network[J]. Journal of Computer Research and Development, 2022, 59(1): 47-80. DOI: 10.7544/issn1000-1239.20201055
[8]	Wang Lei, Xiong Yuning, Li Yunpeng, Liu Yuanyuan. A Collaborative Recommendation Model Based on Enhanced Graph Convolutional Neural Network[J]. Journal of Computer Research and Development, 2021, 58(9): 1987-1996. DOI: 10.7544/issn1000-1239.2021.20200617
[9]	Li Teng, Qiao Wei, Zhang Jiawei, Gao Yiyang, Wang Shenao, Shen Yulong, Ma Jianfeng. Privacy-Preserving Network Attack Provenance Based on Graph Convolutional Neural Network[J]. Journal of Computer Research and Development, 2021, 58(5): 1006-1020. DOI: 10.7544/issn1000-1239.2021.20200942
[10]	Chen Yiqi, Qian Tieyun, Li Wanli, Liang Yile. Exploiting Composite Relation Graph Convolution for Attributed Network Embedding[J]. Journal of Computer Research and Development, 2020, 57(8): 1674-1682. DOI: 10.7544/issn1000-1239.2020.20200206

Cited By

Cited by

Periodical cited type(5)

1.	曾伟，奚雪峰，崔志明. 基于并联残差膨胀卷积网络的短文本实体关系联合抽取. 现代电子技术. 2025(02): 169-178 .
2.	董永峰，白佳明，王利琴，王旭. 融合先验知识和字形特征的中文命名实体识别. 计算机应用. 2024(03): 702-708 .
3.	林海香，白万胜，赵正祥，胡娜娜，李冬，陆人杰. 面向高速铁路道岔运维文本的知识抽取方法. 铁道科学与工程学报. 2024(07): 2569-2580 .
4.	魏杰，杨悦欣，王长浩. 基于分区过滤网络的中文简历实体关系联合抽取研究. 软件工程. 2024(12): 45-51 .
5.	郝志刚，刘冲，秦丽. 基于中文字邻接图的食品抽检公告实体及关系联合抽取. 农业工程学报. 2023(14): 283-292 .