Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement

Zhang Wenhan; Liu Xiaoming; Yang Guan; Liu Jie

doi:10.7544/issn1000-1239.202220413

Journal of Computer Research and Development > 2023 > 60(12): 2864-2876. > DOI: 10.7544/issn1000-1239.202220413 CSTR: 32373.14.issn1000-1239.202220413

Zhang Wenhan, Liu Xiaoming, Yang Guan, Liu Jie. Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement[J]. Journal of Computer Research and Development, 2023, 60(12): 2864-2876. DOI: 10.7544/issn1000-1239.202220413

Citation:

PDF (1971 KB)

Cross-Domain Named Entity Recognition of Multi-Level Structured Semantic Knowledge Enhancement

Zhang Wenhan^1,,
Liu Xiaoming^{1, 4, ,},
Yang Guan^{1, 2},
Liu Jie^{3, 4}

1.
School of Computer Science, Zhongyuan University of Technology, Zhengzhou 450007
2.
Henan provincial Key Laboratory on Public Opinion Intelligent Analysis (Zhongyuan University of Technology), Zhengzhou 450007
3.
School of Information Science, North China University of Technology, Beijing 100144
4.
China Language Intelligence Research Center of the National Language Commission (Capital Normal University), Beijing 102206

Funds: This work was supported by the National Key Research and Development Program of China (2020AAA0109700), the National Natural Science Foundation of China (62076167), and the Key Scientific Research Projects of Colleges and Universities in Henan Province (23A520022).

More Information

Author Bio:
Zhang Wenhan: born in 1999. Master candidate. His main research interests include natural language processing and machine learning

Liu Xiaoming: born in 1979. PhD, lecturer, master supervisor. Member of CCF. His main research interests include natural language processing, Chinese information processing, and machine learning

Yang Guan: born in 1974. PhD, associate professor, master supervisor. His main research interests include image processing and machine learning

Liu Jie: born in 1973. PhD, professor, PhD supervisor. His main research interests include natural language processing, knowledge engineering, and knowledge graph
Received Date: May 20, 2022
Revised Date: February 19, 2023
Available Online: September 19, 2023

Graphical Abstract

Abstract

Abstract

Cross-domain named entity recognition aims to alleviate the problem of insufficient annotation data in the target domain. Most existing methods, which exploit the feature representation or model parameter sharing to achieve cross-domain transfer of entity recognition capabilities and can only partially utilize structured knowledge entailed in text sequences. To address this, we propose a multi-level structured semantic knowledge enhanced cross-domain named entity recognition MSKE-CDNER, which could facilitate the transfer of entity recognition capabilities by aligning the structured knowledge representations embedded in the source and target domains from multiple levels. First, MSKE-CDNER uses the structural feature representation layer to achieve structured semantic knowledge representations of texts from different fields’ structured alignment. And then, these structured semantic representations are aligned at the corresponding layers by a latent alignment module to obtain cross-domain invariant knowledge. Finally, this cross-domain consistent structured knowledge is fused with domain-specific knowledge to enhance the generalization capability of the model. Experiments on five datasets and a specific cross-domain named entity recognition dataset have shown that the average performance of MSKE-CDNER improved by 0.43% and 1.47% compared with the current models. All of these indicate that exploiting text sequences’ structured semantic knowledge representation could effectively enhance entity recognition in the target domain.
- cross-domain named entity recognition,
- cross-domain transfer,
- structured alignment,
- structured knowledge,
- domain invariant knowledge

FullText(HTML)

References (42)

References

[1]	Liu Zihan, Xu Yan, Yu Tiezheng, et al. CrossNER: Evaluating cross-domain named entity recognition[C] //Proc of the 21st AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2021: 13452−13460
[2]	张雪松,郭瑞强,黄德根. 基于依存关系的命名实体识别[J]. 中文信息学报,2021,35(6):63−73 Zhang Xuesong, Guo Ruiqiang, Huang Degen. Named entity recognition based on dependency[J]. Journal of Chinese Information Processing, 2021, 35(6): 63−73 (in Chinese)
[3]	邓依依,邬昌兴,魏永丰,等. 基于深度学习的命名实体识别综述[J]. 中文信息学报,2021,35(9):30−45 Deng Yiyi, Wu Changxing, Wei Yongfeng, et al. A survey on named entity recognition based on deep learing[J]. Journal of Chinese Information Processing, 2021, 35(9): 30−45 (in Chinese)
[4]	Jia Chen, Liang Xiaobo, Zhang Yue. Cross-domain NER using cross-domain language modeling[C] //Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 2464−2474
[5]	Wang Zhenghui, Qu Yanru, Chen Liheng, et al. Label-Aware double transfer learning for cross-specialty medical named entity recognition[C] //Proc of the 2018 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2018: 1−15
[6]	Jia Chen, Zhang Yue. Multi-cell compositional LSTM for NER domain adaptation[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 5906−5917
[7]	Bill L Y, Lu Wei. Neural adaptation layers for cross-domain named entity recognition[C] //Proc of the 2018 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2018: 2012−2022
[8]	Rodriguez J D, Caldwell A, Liu A. Transfer learning for entity recognition of novel classes[C] //Proc of the 27th Int Conf on Computational Linguistics. Stroudsburg, PA: ACL, 2018: 1974−1985
[9]	Titouan V, Courty N, Tavenard R, et al. Optimal transport for structured data with application on graphs[C] //Proc of the 36th Int Conf on Machine Learning. New York: PMLR, 2019: 6275−6284
[10]	Kriege N M, Fey M, Fisseler D, et al. Recognizing cuneiform signs using graph based methods[C] //Proc of the Int Workshop on Cost-Sensitive Learning. New York: PMLR, 2018: 31−44
[11]	Ktena S I, Parisot S, Ferrante E, et al. Distance metric learning using graph convolutional networks: Application to functional brain networks[C] //Proc of Int Conf on Medical Image Computing and Computer-Assisted Intervention. Cham: Springer , 2017: 469−477
[12]	Yanardag P, Vishwanathan S. Deep graph kernels[C] //Proc of the 21st ACM SIGKDD Int Conf on Knowledge Discovery and Data Mining. New York: ACM, 2015: 1365−1374
[13]	Yang Qiang, Zheng Vincent W, Li Bin, et al. Transfer learning by reusing structured knowledge[J]. AI Magazine, 2011, 32(2): 95−106 doi: 10.1609/aimag.v32i2.2335
[14]	Alvarez-Melis D, Jaakkola T, Jegelka S. Structured optimal transport[C] //Proc of the 21st Int Conf on Artificial Intelligence and Statistics. New York: PMLR, 2018: 1771−1780
[15]	Chen Liqun, Gan Zhe, Cheng Yu, et al. Graph optimal transport for cross-domain alignment[C] //Proc of the 37th Int Conf on Machine Learning. New York: PMLR, 2020: 1542−1553
[16]	张晗,郭渊博,李涛. 结合GAN与BiLSTM-Attention-CRF的领域命名实体识别[J]. 计算机研究与发展,2019,56(9):1851−1858 Zhang Han, Guo Yuanbo, Li Tao. Domain named entity recognition combining GAN and BiLSTM-Attention-CRF[J]. Journal of Computer Research and Development, 2019, 56(9): 1851−1858 (in Chinese)
[17]	Li Jing, Shang Shuo, Shao Ling. Metaner: Named entity recognition with meta-learning[C] //Proc of the Web Conf 2020. New York: ACM, 2020: 429−440
[18]	Chen Shuguang, Aguilar G, Neves L. Data augmentation for cross-domain named entity recognition[C] //Proc of the 2021 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2021: 5346−5356
[19]	Hao Zhifeng, Lv D, Li Zijian, et al. Semi-supervised disentangled framework for transferable named entity recognition[J]. Neural Networks, 2021, 135: 127−138 doi: 10.1016/j.neunet.2020.11.017
[20]	Nozza D, Manchanda P, Fersini E, et al. Learning to adapt with word embeddings: Domain adaptation of named entity recognition systems[J/OL]. Information Processing & Management, 2021[2023-01-16]. https://www.sciencedirect.com/science/article/abs/pii/S0306457321000455
[21]	李鑫,李哲民,魏居辉,等. 基于特征分离的跨域自适应学习模型[J]. 计算机研究与发展,2022,59(1):105−117 Li Xin, Li Zhemin, Wei Juhui, et al. Cross-domain adaptive learning model based on feature separation[J]. Journal of Computer Research and Development, 2022, 59(1): 105−117 (in Chinese)
[22]	Gu Shuhao, Feng Yang, Liu Qun. Improving domain adaptation translation with domain invariant and specific information[C] //Proc of the 2019 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2019: 3081−3091
[23]	Dong Jianfeng, Long Zhongzi, Mao Xiaofeng, et al. Multi-level alignment network for domain adaptive cross-modal retrieval[J]. Neurocomputing, 2021, 440: 207−219 doi: 10.1016/j.neucom.2021.01.114
[24]	Zhang Tao, Xia Congying, Yu P S, et al. PDALN: Progressive domain adaptation over a pre-trained model for low-resource cross-domain named entity recognition[C] //Proc of the 2021 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2021: 5441−5451
[25]	Swarup S, Ray S R. Cross-domain knowledge transfer using structured representations[C] //Proc of the 21st National Conf on Artificial Intelligence and the 18th Innovative Applications of Artificial Intelligence Conf. Menlo Park, CA: AAAI, 2006: 506−511
[26]	Lee J, Kim H, Lee J, et al. Transfer learning for deep learning on graph-structured data[C] //Proc of the 31st AAAI Conf on Artificial Intelligence. Menlo Park, CA: AAAI, 2017: 2154−2160
[27]	Zheng Junhao, Chen Haibin, Ma Qianlin. Cross-domain named entity recognition via graph matching[C] //Proc of Findings of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2022: 2670−2680
[28]	Courty N, Flamary R, Tuia D, et al. Optimal transport for domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(9): 1853−1865 doi: 10.1109/TPAMI.2016.2615921
[29]	Peyré G, Cuturi M, Solomon J. Gromov-Wasserstein averaging of kernel and distance matrices[C] //Proc of the 33rd Int Conf on Machine Learning. New York: PMLR, 2016: 2664−2672
[30]	Ma X, Hovy E. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF[C] //Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016: 1064−1074
[31]	尉桢楷,程梦,周夏冰,等. 基于类卷积交互式注意力机制的属性抽取研究[J]. 计算机研究与发展,2020,57(11):2456−2466 Wei Zhenkai, Cheng Meng, Zhou Xiabing, et al. Convolutional interactive attention mechanism for aspect extraction[J]. Journal of Computer Research and Development, 2020, 57(11): 2456−2466 (in Chinese)
[32]	Sang E F T K, De Meulder F. Introduction to the CoNLL-2003 shared task: Language-independent named entity recognition[C] //Proc of the 7th Conf on Natural Language Learning at HLT-NAACL 2003. Stroudsburg, PA: ACL, 2003: 142−147
[33]	Lu Di, Neves L, Carvalho V, et al. Visual attention model for name tagging in multimodal social media[C] //Proc of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2018: 1990−1999
[34]	Derczynski L, Bontcheva K, Roberts I. Broad Twitter corpus: A diverse named entity recognition resource[C] //Proc of the 26th Int Conf on Computational Linguistics: Technical Papers. Stroudsburg, PA: ACL, 2016: 1169−1179
[35]	Nédellec C, Bossy R, Kim J, et al. Overview of BioNLP shared task 2013[C] //Proc of the BioNLP Shared Task 2013 Workshop. Stroudsburg, PA: ACL, 2013: 1−7
[36]	Yang Jie, Zhang Yue. NCRF++: An open-source neural sequence labeling toolkit[C] //Proc of Association for Computational Linguistics 2018 System Demonstrations. Stroudsburg, PA: ACL, 2018: 74−79
[37]	Pennington J, Socher R, Manning C D. Glove: Global vectors for word representation[C] //Proc of the 2014 Conf on Empirical Methods in Natural Language Processing (EMNLP). Stroudsburg, PA: ACL, 2014: 1532−1543
[38]	Chiu B, Crichton G, Korhonen A, et al. How to train good word embeddings for biomedical NLP[C] //Proc of the 15th Workshop on Biomedical Natural Language. Stroudsburg, PA: ACL, 2016: 166−174
[39]	Lample G, Ballesteros M, Subramanian S, et al. Neural architectures for named entity recognition[C] //Proc of the 2016 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2016: 260−270
[40]	Liu Zihan, Winata G I, Xu Peng, et al. Coach: A coarse-to-fine approach for cross-domain slot filling[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 19−25
[41]	Zhou Tianyi, Zhang Hao, Jin Di, et al. Dual adversarial neural transfer for low-resource named entity recognition[C] //Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 3461−3471
[42]	Baziotis C, Haddow B, Birch A. Language model prior for low-resource neural machine translation[C] //Proc of the 2020 Conf on Empirical Methods in Natural Language Processing(EMNLP). Stroudsburg, PA: ACL, 2020: 7622−7634

[1]	Du Yuyue, Sun Ya’nan, Liu Wei. Petri Nets Based Recognition of Model Deviation Domains and Model Repair[J]. Journal of Computer Research and Development, 2016, 53(8): 1766-1780. DOI: 10.7544/issn1000-1239.2016.20160099
[2]	Zhu Jun, Guo Changguo, Wu Quanyuan. A Web Services Interaction Behavior-Environment Model Based on Generalized Stochastic Petri Nets[J]. Journal of Computer Research and Development, 2012, 49(11): 2450-2463.
[3]	Sun Cong, Tang Liyong, Chen Zhong, Ma Jianfeng. Secure Information Flow in Java by Optimized Reachability Analysis of Weighted Pushdown System[J]. Journal of Computer Research and Development, 2012, 49(5): 901-912.
[4]	Zhou Hang, Huang Zhiqiu, Zhu Yi, Xia Liang, Liu Linyuan. Real-Time Systems Contact Checking and Resolution Based on Time Petri Net[J]. Journal of Computer Research and Development, 2012, 49(2): 413-420.
[5]	Men Peng and Duan Zhenhua. Extension of Model Checking Tool of Colored Petri Nets and Its Applications in Web Service Composition[J]. Journal of Computer Research and Development, 2009, 46(8): 1294-1303.
[6]	Zhao Mingfeng, Song Wen, Yang Yixian. Confusion Detection Based on Petri-Net[J]. Journal of Computer Research and Development, 2008, 45(10): 1631-1637.
[7]	Cui Huanqing and Wu Zhehui. Structural Properties of Parallel Program's Petri Net Model[J]. Journal of Computer Research and Development, 2007, 44(12): 2130-2135.
[8]	Tang Da, Li Ye. Model Analysis of Supply Chain System Based on Color Stochastic Petri Net[J]. Journal of Computer Research and Development, 2007, 44(10): 1782-1789.
[9]	Lao Songyang, Huang Guanglian, Alan F. Smeaton, Gareth J. F. Jones, Hyowon Lee. A Query Description Model of Soccer Video Based on BSU Composite Petri-Net[J]. Journal of Computer Research and Development, 2006, 43(1): 159-168.
[10]	Li Botao and Luo Junzhou. Modeling and Analysis of Non-Repudiation Protocols by Using Petri Nets[J]. Journal of Computer Research and Development, 2005, 42(9): 1571-1577.

Cited By

Cited by

Periodical cited type(22)

1.	黄蔚亮，李锦煊，余志文，蔡亚永，刘元. 确定性网络：架构、关键技术和应用. 重庆邮电大学学报(自然科学版). 2025(01): 1-16 .
2.	姜旭艳，全巍，付文文，张小亮，孙志刚. OpenPlanner：一个开源的时间敏感网络流量规划器. 计算机研究与发展. 2025(05): 1307-1329 . 本站查看
3.	齐玉玲，黄涛，张军贤，贾焱鑫，徐龙，熊伟，朱海龙，彭开来. 基于时间敏感网络的列车通信网络研究及应用. 城市轨道交通研究. 2024(05): 184-189 .
4.	何倩，郭雅楠，赵宝康，潘琪，王勇. 无等待与时隙映射复用结合的时间触发流调度方法. 通信学报. 2024(08): 192-204 .
5.	郭若彤，许方敏，张恒升，赵成林. 基于循环排队转发的时间触发流量路由与调度优化方法. 微电子学与计算机. 2024(10): 55-63 .
6.	薛强，吴梦，杨世标，屠礼彪，李伟，廖江. 嵌入式人工智能技术在IP网络的创新应用. 邮电设计技术. 2024(10): 66-72 .
7.	张浩，郭偶凡，周飞飞，马涛，何迎利，姚苏滨. 基于分段帧复制和消除的时间敏感网络动态冗余机制研究. 计算机科学. 2024(S2): 750-756 .
8.	罗峰，周杰，王子通，张晓先，孙志鹏. 基于多域冗余的车载时间敏感网络时间同步增强方法. 系统工程与电子技术. 2024(12): 4259-4268 .
9.	王雪荣，唐政治，李银川，齐美玉，朱建波，张亮. 基于优化决策树的时延敏感流智能感知调度. 电信科学. 2023(04): 120-132 .
10.	陆以勤，谢文静，王海瀚，陈卓星，程喆，潘伟锵，覃健诚. 面向时间敏感网络的安全感知调度方法. 华南理工大学学报(自然科学版). 2023(05): 1-12 .
11.	朱渊，胡馨予，吴思远，黄蓉. 基于OMNeT++的5G-TSN调度算法综述. 西安邮电大学学报. 2023(01): 9-18 .
12.	王家兴，杨思锦，庄雷，宋玉，阳鑫宇. 时间敏感网络中多目标在线混合流量调度算法. 计算机科学. 2023(07): 286-292 .
13.	李维，梁巍，周策. 基于ACO算法的SDN网络流量调度优化研究. 自动化与仪器仪表. 2023(07): 42-46 .
14.	吴昭祥，李文凯，袁亚洲，刘志新. 时间敏感网络中基于抢占式通道模型的资源调度算法研究. 移动通信. 2023(08): 67-73+97 .
15.	刘美鹭，刘留，王凯，韩紫杰. 真空管高速飞行列车通信业务建模. 移动通信. 2023(08): 98-106 .
16.	彭紫梅，寿国础，郭梦杰，刘雅琼，胡怡红. 时间敏感网络中的冗余机制研究综述. 电信科学. 2023(08): 29-42 .
17.	胡文学，孙雷，王健全，朱渊，毕紫航. 基于网络演算的时间敏感网络时延上界分析模型研究. 自动化学报. 2023(11): 2297-2310 .
18.	王新蕾，周敏，张涛. 时间敏感网络流量调度算法研究综述. 电讯技术. 2023(11): 1830-1838 .
19.	段晓东，刘鹏，陆璐，孙滔，李志强. 确定性网络技术综述. 电信科学. 2023(11): 1-12 .
20.	陆以勤，熊欣，王猛，覃健诚，潘伟锵. TSN中基于链路负载均衡的AVB流量带宽分配方法. 华南理工大学学报(自然科学版). 2023(11): 1-9 .
21.	周阳，陈鸿龙，张雷. 时间敏感网络中的动态路由与调度联合优化算法. 物联网学报. 2023(04): 52-62 .
22.	裴金川，胡宇翔，田乐，胡涛，李子勇. 联合路由规划的时间敏感网络流量调度方法. 通信学报. 2022(12): 54-65 .