Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement

Zhang Wenhan; Liu Xiaoming; Yang Guan; Liu Jie

doi:10.7544/issn1000-1239.202220413

Journal of Computer Research and Development > 2023 > 60(12): 2864-2876. > DOI: 10.7544/issn1000-1239.202220413 CSTR: 32373.14.issn1000-1239.202220413

Zhang Wenhan, Liu Xiaoming, Yang Guan, Liu Jie. Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement[J]. Journal of Computer Research and Development, 2023, 60(12): 2864-2876. DOI: 10.7544/issn1000-1239.202220413

Citation:

PDF (1971 KB)

Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement

Zhang Wenhan^1,,
Liu Xiaoming^{1, 4, ,},
Yang Guan^{1, 2},
Liu Jie^{3, 4}

1.
School of Computer Science, Zhongyuan University of Technology, Zhengzhou 450007
2.
Henan provincial Key Laboratory on Public Opinion Intelligent Analysis (Zhongyuan University of Technology), Zhengzhou 450007
3.
School of Information Science, North China University of Technology, Beijing 100144
4.
China Language Intelligence Research Center of the National Language Commission (Capital Normal University), Beijing 102206

Funds: This work was supported by the National Key Research and Development Program of China (2020AAA0109700), the National Natural Science Foundation of China (62076167), and the Key Scientific Research Projects of Colleges and Universities in Henan Province (23A520022).

More Information

Author Bio:
Zhang Wenhan: born in 1999. Master candidate. His main research interests include natural language processing and machine learning

Liu Xiaoming: born in 1979. PhD, lecturer, master supervisor. Member of CCF. His main research interests include natural language processing, Chinese information processing, and machine learning

Yang Guan: born in 1974. PhD, associate professor, master supervisor. His main research interests include image processing and machine learning

Liu Jie: born in 1973. PhD, professor, PhD supervisor. His main research interests include natural language processing, knowledge engineering, and knowledge graph
Received Date: May 20, 2022
Revised Date: February 19, 2023
Available Online: September 19, 2023

Graphical Abstract

Abstract

Abstract

Cross-domain named entity recognition aims to alleviate the problem of insufficient annotation data in the target domain. Most existing methods, which exploit the feature representation or model parameter sharing to achieve cross-domain transfer of entity recognition capabilities and can only partially utilize structured knowledge entailed in text sequences. To address this, we propose a multi-level structured semantic knowledge enhanced cross-domain named entity recognition MSKE-CDNER, which could facilitate the transfer of entity recognition capabilities by aligning the structured knowledge representations embedded in the source and target domains from multiple levels. First, MSKE-CDNER uses the structural feature representation layer to achieve structured semantic knowledge representations of texts from different fields’ structured alignment. And then, these structured semantic representations are aligned at the corresponding layers by a latent alignment module to obtain cross-domain invariant knowledge. Finally, this cross-domain consistent structured knowledge is fused with domain-specific knowledge to enhance the generalization capability of the model. Experiments on five datasets and a specific cross-domain named entity recognition dataset have shown that the average performance of MSKE-CDNER improved by 0.43% and 1.47% compared with the current models. All of these indicate that exploiting text sequences’ structured semantic knowledge representation could effectively enhance entity recognition in the target domain.
- cross-domain named entity recognition,
- cross-domain transfer,
- structured alignment,
- structured knowledge,
- domain invariant knowledge

FullText(HTML)

References (42)

References

[1]	Liu Zihan, Xu Yan, Yu Tiezheng, et al. CrossNER: Evaluating cross-domain named entity recognition[C] //Proc of the 21st AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 13452−13460
[2]	张雪松,郭瑞强,黄德根. 基于依存关系的命名实体识别[J]. 中文信息学报,2021,35(6):63−73 Zhang Xuesong, Guo Ruiqiang, Huang Degen. Named entity recognition based on dependency[J]. Journal of Chinese Information Processing, 2021, 35(6): 63−73 (in Chinese)
[3]	邓依依,邬昌兴,魏永丰,等. 基于深度学习的命名实体识别综述[J]. 中文信息学报,2021,35(9):30−45 Deng Yiyi, Wu Changxing, Wei Yongfeng, et al. A survey on named entity recognition based on deep learing[J]. Journal of Chinese Information Processing, 2021, 35(9): 30−45 (in Chinese)
[4]	Jia Chen, Liang Xiaobo, Zhang Yue. Cross-domain NER using cross-domain language modeling[C] //Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 2464−2474
[5]	Wang Zhenghui, Qu Yanru, Chen Liheng, et al. Label-Aware double transfer learning for cross-specialty medical named entity recognition[C] //Proc of the 2018 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2018: 1−15
[6]	Jia Chen, Zhang Yue. Multi-cell compositional LSTM for NER domain adaptation[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 5906−5917
[7]	Bill L Y, Lu Wei. Neural adaptation layers for cross-domain named entity recognition[C] //Proc of the 2018 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2018: 2012−2022
[8]	Rodriguez J D, Caldwell A, Liu A. Transfer learning for entity recognition of novel classes[C] //Proc of the 27th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2018: 1974−1985
[9]	Titouan V, Courty N, Tavenard R, et al. Optimal transport for structured data with application on graphs[C] //Proc of the 36th Int Conf on Machine Learning. New York: PMLR, 2019: 6275−6284
[10]	Kriege N M, Fey M, Fisseler D, et al. Recognizing cuneiform signs using graph based methods[C] //Proc of the Int Workshop on Cost-Sensitive Learning. New York: PMLR, 2018: 31−44
[11]	Ktena S I, Parisot S, Ferrante E, et al. Distance metric learning using graph convolutional networks: Application to functional brain networks[C] //Proc of Int Conf on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer , 2017: 469−477
[12]	Yanardag P, Vishwanathan S. Deep graph kernels[C] //Proc of the 21st ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1365−1374
[13]	Yang Qiang, Zheng Vincent W, Li Bin, et al. Transfer learning by reusing structured knowledge[J]. AI Magazine, 2011, 32(2): 95−106 doi: 10.1609/aimag.v32i2.2335
[14]	Alvarez-Melis D, Jaakkola T, Jegelka S. Structured optimal transport[C] //Proc of the 21st Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2018: 1771−1780
[15]	Chen Liqun, Gan Zhe, Cheng Yu, et al. Graph optimal transport for cross-domain alignment[C] //Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 1542−1553
[16]	张晗,郭渊博,李涛. 结合GAN与BiLSTM-Attention-CRF的领域命名实体识别[J]. 计算机研究与发展,2019,56(9):1851−1858 Zhang Han, Guo Yuanbo, Li Tao. Domain named entity recognition combining GAN and BiLSTM-Attention-CRF[J]. Journal of Computer Research and Development, 2019, 56(9): 1851−1858 (in Chinese)
[17]	Li Jing, Shang Shuo, Shao Ling. Metaner: Named entity recognition with meta-learning[C] //Proc of the Web Conf 2020. New York: ACM, 2020: 429−440
[18]	Chen Shuguang, Aguilar G, Neves L. Data augmentation for cross-domain named entity recognition[C] //Proc of the 2021 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2021: 5346−5356
[19]	Hao Zhifeng, Lv D, Li Zijian, et al. Semi-supervised disentangled framework for transferable named entity recognition[J]. Neural Networks, 2021, 135: 127−138 doi: 10.1016/j.neunet.2020.11.017
[20]	Nozza D, Manchanda P, Fersini E, et al. Learning to adapt with word embeddings: Domain adaptation of named entity recognition systems[J/OL]. Information Processing & Management, 2021[2023-01-16]. https://www.sciencedirect.com/science/article/abs/pii/S0306457321000455
[21]	李鑫,李哲民,魏居辉,等. 基于特征分离的跨域自适应学习模型[J]. 计算机研究与发展,2022,59(1):105−117 Li Xin, Li Zhemin, Wei Juhui, et al. Cross-domain adaptive learning model based on feature separation[J]. Journal of Computer Research and Development, 2022, 59(1): 105−117 (in Chinese)
[22]	Gu Shuhao, Feng Yang, Liu Qun. Improving domain adaptation translation with domain invariant and specific information[C] //Proc of the 2019 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2019: 3081−3091
[23]	Dong Jianfeng, Long Zhongzi, Mao Xiaofeng, et al. Multi-level alignment network for domain adaptive cross-modal retrieval[J]. Neurocomputing, 2021, 440: 207−219 doi: 10.1016/j.neucom.2021.01.114
[24]	Zhang Tao, Xia Congying, Yu P S, et al. PDALN: Progressive domain adaptation over a pre-trained model for low-resource cross-domain named entity recognition[C] //Proc of the 2021 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2021: 5441−5451
[25]	Swarup S, Ray S R. Cross-domain knowledge transfer using structured representations[C] //Proc of the 21st National Conf on Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conf. Menlo Park, CA: AAAI, 2006: 506−511
[26]	Lee J, Kim H, Lee J, et al. Transfer learning for deep learning on graph-structured data[C] //Proc of the 31st AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2017: 2154−2160
[27]	Zheng Junhao, Chen Haibin, Ma Qianlin. Cross-domain named entity recognition via graph matching[C] //Proc of Findings of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2022: 2670−2680
[28]	Courty N, Flamary R, Tuia D, et al. Optimal transport for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(9): 1853−1865 doi: 10.1109/TPAMI.2016.2615921
[29]	Peyré G, Cuturi M, Solomon J. Gromov-Wasserstein averaging of kernel and distance matrices[C] //Proc of the 33rd Int Conf on Machine Learning. New York: PMLR, 2016: 2664−2672
[30]	Ma X, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C] //Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016: 1064−1074
[31]	尉桢楷,程梦,周夏冰,等. 基于类卷积交互式注意力机制的属性抽取研究[J]. 计算机研究与发展,2020,57(11):2456−2466 Wei Zhenkai, Cheng Meng, Zhou Xiabing, et al. Convolutional interactive attention mechanism for aspect extraction[J]. Journal of Computer Research and Development, 2020, 57(11): 2456−2466 (in Chinese)
[32]	Sang E F T K, De Meulder F. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition[C] //Proc of the 7th Conf on Natural Language Learning at HLT-NAACL 2003. Stroudsburg, PA: ACL, 2003: 142−147
[33]	Lu Di, Neves L, Carvalho V, et al. Visual attention model for name tagging in multimodal social media[C] //Proc of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2018: 1990−1999
[34]	Derczynski L, Bontcheva K, Roberts I. Broad Twitter corpus: A diverse named entity recognition resource[C] //Proc of the 26th Int Conf on Computational Linguistics: Technical Papers. Stroudsburg, PA: ACL, 2016: 1169−1179
[35]	Nédellec C, Bossy R, Kim J, et al. Overview of BioNLP shared task 2013[C] //Proc of the BioNLP Shared Task 2013 Workshop. Stroudsburg, PA: ACL, 2013: 1−7
[36]	Yang Jie, Zhang Yue. NCRF++: An open-source neural sequence labeling toolkit[C] //Proc of Association for Computational Linguistics 2018 System Demonstrations. Stroudsburg, PA: ACL, 2018: 74−79
[37]	Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C] //Proc of the 2014 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2014: 1532−1543
[38]	Chiu B, Crichton G, Korhonen A, et al. How to train good word embeddings for biomedical NLP[C] //Proc of the 15th Workshop on Biomedical Natural Language. Stroudsburg, PA: ACL, 2016: 166−174
[39]	Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[C] //Proc of the 2016 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2016: 260−270
[40]	Liu Zihan, Winata G I, Xu Peng, et al. Coach: A coarse-to-fine approach for cross-domain slot filling[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 19−25
[41]	Zhou Tianyi, Zhang Hao, Jin Di, et al. Dual adversarial neural transfer for low-resource named entity recognition[C] //Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 3461−3471
[42]	Baziotis C, Haddow B, Birch A. Language model prior for low-resource neural machine translation[C] //Proc of the 2020 Conf on Empirical Methods in Natural Language Processing(EMNLP). Stroudsburg, PA: ACL, 2020: 7622−7634

[1]	Yang Lihua, Dong Yong, Wu Huijun, Tan Zhipeng, Wang Fang, Lu Kai. Survey of Log-Structured File Systems in Mobile Devices[J]. Journal of Computer Research and Development, 2025, 62(1): 58-74. DOI: 10.7544/issn1000-1239.202330789
[2]	Chen Huimin, Jin Sichen, Lin Wei, Zhu Zeyu, Tong Lingbo, Liu Yipeng, Ye Yining, Jiang Weihan, Liu Zhiyuan, Sun Maosong, Jin Jianbin. Quantitative Analysis on the Communication of COVID-19 Related Social Media Rumors[J]. Journal of Computer Research and Development, 2021, 58(7): 1366-1384. DOI: 10.7544/issn1000-1239.2021.20200818
[3]	Guo Hongyi, Liu Gongshen, Su Bo, Meng Kui. Collaborative Filtering Recommendation Algorithm Combining Community Structure and Interest Clusters[J]. Journal of Computer Research and Development, 2016, 53(8): 1664-1672. DOI: 10.7544/issn1000-1239.2016.20160175
[4]	Wang Di, Zhao Tianlei, Tang Yuxing, Dou Qiang. A Communication Feature-Oriented 3D NoC Architecture Design[J]. Journal of Computer Research and Development, 2014, 51(9): 1971-1979. DOI: 10.7544/issn1000-1239.2014.20130131
[5]	Chen Ping, Xing Xiao, Xin Zhi, Wang Yi, Mao Bing, and Xie Li. Protecting Programs Based on Randomizing the Encapsulated Structure[J]. Journal of Computer Research and Development, 2011, 48(12): 2227-2234.
[6]	Li Shaofang, Hu Shanli, Shi Chunyi. An Anytime Coalition Structure Generation Based on the Grouping Idea of Cardinality Structure[J]. Journal of Computer Research and Development, 2011, 48(11): 2047-2054.
[7]	Liu Jinglei, Zhang Wei, Liu Zhaowei, and Sun Xuejiao. Properties and Application of Coalition Structure Graph[J]. Journal of Computer Research and Development, 2011, 48(4): 602-609.
[8]	Su Shexiong, Hu Shanli, Zheng Shengfu, Lin Chaofeng, and Luo Jianbin. An Anytime Coalition Structure Generation Algorithm Based on Cardinality Structure[J]. Journal of Computer Research and Development, 2008, 45(10): 1756.
[9]	Cao Yafei, Wang Dawei, and Li Sikun. A Novel System-Level Communication Synthesis Methodology Containing Crossbar Bus and Shared Bus[J]. Journal of Computer Research and Development, 2008, 45(8): 1439-1445.
[10]	Zheng Zhirong, Cai Yi, and Shen Changxiang. Research on an Application Class Communication Security Model on Operating System Security Framework[J]. Journal of Computer Research and Development, 2005, 42(2): 322-328.

Cited By

Cited by

Periodical cited type(5)

1.	何业锋，刘闪闪，刘妍，权家辉，田哲铭，杨梦玫，李智. 支持虚拟车辆辅助假名更新的混合区位置隐私保护方案. 计算机应用研究. 2024(01): 272-276 .
2.	况博裕，李雨泽，顾芳铭，苏铓，付安民. 车联网安全研究综述：威胁、对策与未来展望. 计算机研究与发展. 2023(10): 2304-2321 . 本站查看
3.	王佳星，周武源，李甜甜. 人工智能发展态势的文献计量分析与研究. 小型微型计算机系统. 2023(11): 2424-2433 .
4.	张迪，曹利，李原帅. 车联网环境下基于多策略访问树的安全访问控制算法. 计算机应用研究. 2023(11): 3394-3401 .
5.	邓雨康，张磊，李晶. 车联网隐私保护研究综述. 计算机应用研究. 2022(10): 2891-2906 .