Processing math: 0%
  • 中国精品科技期刊
  • CCF推荐A类中文期刊
  • 计算领域高质量科技期刊T1类
高级检索

基于知识增强的文本隐喻识别图编码方法

黄河燕, 刘啸, 刘茜

黄河燕, 刘啸, 刘茜. 基于知识增强的文本隐喻识别图编码方法[J]. 计算机研究与发展, 2023, 60(1): 140-152. DOI: 10.7544/issn1000-1239.202110927
引用本文: 黄河燕, 刘啸, 刘茜. 基于知识增强的文本隐喻识别图编码方法[J]. 计算机研究与发展, 2023, 60(1): 140-152. DOI: 10.7544/issn1000-1239.202110927
Huang Heyan, Liu Xiao, Liu Qian. Knowledge-Enhanced Graph Encoding Method for Metaphor Detection in Text[J]. Journal of Computer Research and Development, 2023, 60(1): 140-152. DOI: 10.7544/issn1000-1239.202110927
Citation: Huang Heyan, Liu Xiao, Liu Qian. Knowledge-Enhanced Graph Encoding Method for Metaphor Detection in Text[J]. Journal of Computer Research and Development, 2023, 60(1): 140-152. DOI: 10.7544/issn1000-1239.202110927
黄河燕, 刘啸, 刘茜. 基于知识增强的文本隐喻识别图编码方法[J]. 计算机研究与发展, 2023, 60(1): 140-152. CSTR: 32373.14.issn1000-1239.202110927
引用本文: 黄河燕, 刘啸, 刘茜. 基于知识增强的文本隐喻识别图编码方法[J]. 计算机研究与发展, 2023, 60(1): 140-152. CSTR: 32373.14.issn1000-1239.202110927
Huang Heyan, Liu Xiao, Liu Qian. Knowledge-Enhanced Graph Encoding Method for Metaphor Detection in Text[J]. Journal of Computer Research and Development, 2023, 60(1): 140-152. CSTR: 32373.14.issn1000-1239.202110927
Citation: Huang Heyan, Liu Xiao, Liu Qian. Knowledge-Enhanced Graph Encoding Method for Metaphor Detection in Text[J]. Journal of Computer Research and Development, 2023, 60(1): 140-152. CSTR: 32373.14.issn1000-1239.202110927

基于知识增强的文本隐喻识别图编码方法

基金项目: 国家重点研发计划项目(2018YFB1005100);国家自然科学基金项目(61732005)
详细信息
    作者简介:

    黄河燕: 1963年生.博士,教授,博士生导师.CCF会员.主要研究方向为自然语言处理、信息抽取与信息检索

    刘啸: 1994年生.博士研究生.主要研究方向为自然语言处理、信息抽取与事件抽取

    刘茜: 1991年生.博士.主要研究方向为自然语言处理、信息检索与语义表示学习

  • 中图分类号: TP391

Knowledge-Enhanced Graph Encoding Method for Metaphor Detection in Text

Funds: This work was supported by the National Key Research and Development Program of China (2018YFB1005100) and the National Natural Science Foundation of China (61732005).
  • 摘要:

    隐喻识别是自然语言处理中语义理解的重要任务之一,目标为识别某一概念在使用时是否借用了其他概念的属性和特点. 由于单纯的神经网络方法受到数据集规模和标注稀疏性问题的制约,近年来,隐喻识别研究者开始探索如何利用其他任务中的知识和粗粒度句法知识结合神经网络模型,获得更有效的特征向量进行文本序列编码和建模.然而,现有方法忽略了词义项知识和细粒度句法知识,造成了外部知识利用率低的问题,难以建模复杂语境.针对上述问题,提出一种基于知识增强的图编码方法(knowledge-enhanced graph encoding method,KEG)来进行文本中的隐喻识别. 该方法分为3个部分:在文本编码层,利用词义项知识训练语义向量,与预训练模型产生的上下文向量结合,增强语义表示;在图网络层,利用细粒度句法知识构建信息图,进而计算细粒度上下文,结合图循环神经网络进行迭代式状态传递,获得表示词的节点向量和表示句子的全局向量,实现对复杂语境的高效建模;在解码层,按照序列标注架构,采用条件随机场对序列标签进行解码.实验结果表明,该方法的性能在4个国际公开数据集上均获得有效提升.

    Abstract:

    Metaphor recognition is one of the essential tasks of semantic understanding in natural language processing, aiming to identify whether one concept is viewed in terms of the properties and characteristics of the other. Since pure neural network methods are restricted by the scale of datasets and the sparsity of human annotations, recent researchers working on metaphor recognition explore how to combine the knowledge in other tasks and coarse-grained syntactic knowledge with neural network models, obtaining more effective feature vectors for sequence coding and modeling in text. However, the existing methods ignore the word sense knowledge and fine-grained syntactic knowledge, resulting in the problem of low utilization of external knowledge and the difficulty to model complex context. Aiming at the above issues, a knowledge-enhanced graph encoding method (KEG) for metaphor detection in text is proposed. This method consists of three parts. In the encoding layer, the sense vector is trained using the word sense knowledge, combined with the context vector generated by the pre-training model to enhance the semantic representation. In the graph layer, the information graph is constructed using fine-grained syntactic knowledge, and then the fine-grained context is calculated. The layer is combined with the graph recurrent neural network, whose state transition is carried out iteratively to obtain the node vector and the global vector representing the word and the sentence, respectively, to realize the efficient modeling of the complex context. In the decoding layer, conditional random fields are used to decode the sequence tags following the sequence labeling architecture. Experimental results show that this method effectively improves the performance on four international public datasets.

  • 图  1   KEG整体网络架构图

    Figure  1.   Overall network framework of KEG

    图  2   文本编码层结构

    Figure  2.   Structure of the encoding layer

    图  3   图循环神经网络层结构

    Figure  3.   Layer structure of the graph recurrent neural network

    表  1   包含隐喻词汇的文本样例

    Table  1   Examples of Sentences with Metaphor Words

    编号文本样例
    1It went to his head rather.
    2How to kill a process?
    3I invested myself fully in this relationship.
    4They forged contacts with those left behind in Germany.
    注:黑体单词表示隐喻现象.
    下载: 导出CSV

    表  2   数据集统计信息

    Table  2   Statistics of Datasets

    数据集单词总数句子总数隐喻总数隐喻占比/%平均单词数
    VUA-VERB23984716189655428.3614.82
    VUA-ALLPOS239847161891502615.8514.82
    TroFi1059493737162743.5428.35
    MOH-X517864731548.688.00
    下载: 导出CSV

    表  3   本文实验中使用的超参数

    Table  3   Hyper-Parameters Used in Our Experiment

    超参数取值
    {{\boldsymbol{e}}^{\rm{c}}}的向量维度1024
    {{\boldsymbol{e}}^{\rm{s}}}的向量维度200
    {\boldsymbol{k}}的向量维度256
    {\boldsymbol{m}}{\boldsymbol{h}}{\boldsymbol{g}}的向量维度512
    状态传递次数K3
    批大小(batch size)16
    BERT模型dropout率0.5
    模型其他部分dropout率0.4
    BERT模型学习率1E−5
    模型其他部分学习率1E−3
    BERT模型{L_2}权重衰减1E−5
    模型其他部分{L_2}权重衰减1E−3
    Warmup迭代轮次5
    最大迭代轮次50
    梯度裁剪4.0
    下载: 导出CSV

    表  4   KEG与SOTA方法对比的实验结果

    Table  4   Experimental Results of KEG and SOTA Methods %

    组别对比模型VUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    第1组RNN_ELMo[7]68.271.369.781.471.673.672.693.170.771.671.174.679.173.575.677.2
    RNN_BERT[20]66.771.569.080.771.571.971.792.970.367.168.773.475.181.878.278.1
    RNN_HG[6]69.372.370.882.171.876.374.093.667.477.872.274.979.779.879.879.7
    RNN_MHCA[6]66.375.270.581.873.075.774.393.868.676.872.475.277.583.180.079.8
    第2组Go Figure![9]73.282.377.572.174.873.4
    DeepMet-S[10]76.278.377.286.273.873.273.590.5
    第3组GCN_BERT[11]72.370.570.772.079.879.479.379.4
    MWE[11]73.871.872.873.580.080.480.280.5
    本文KEG80.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    下载: 导出CSV

    表  5   对比不同状态传递次数的实验结果

    Table  5   Experimental Results With Different State Transition Times %

    状态传递次数KVUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    167.372.669.882.571.873.372.591.772.670.471.574.677.879.878.878.1
    274.675.775.184.273.375.774.592.474.175.674.875.381.380.580.980.8
    380.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    480.977.179.086.275.176.075.592.875.877.576.676.382.181.881.981.2
    580.877.278.986.674.876.175.493.176.177.876.975.381.482.181.781.6
    下载: 导出CSV

    表  6   词向量和上下文向量构建方法的实验结果

    Table  6   Experimental Results of Constructing Word Vectors and Context Vectors %

    对比模型VUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    KEG-word2vec74.273.273.782.373.574.774.190.674.173.573.874.478.576.677.578.6
    KEG-GloVe74.673.273.982.673.674.774.190.874.073.873.974.278.476.877.679.1
    KEG-ELMo76.475.575.984.874.276.175.191.774.675.875.275.680.480.380.380.7
    KEG80.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    下载: 导出CSV

    表  7   KEG有无词级别义项知识的实验结果

    Table  7   Experimental Results of KEG With and Without Lexicon Word Sense Knowledge %

    对比模型VUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    KEG-NOS68.272.970.582.772.673.172.892.173.271.572.375.479.478.278.878.8
    KEG80.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    下载: 导出CSV

    表  8   KEG有无细粒度句法知识的实验结果

    Table  8   Experimental Results of KEG With and Without Fine-Grained Syntactic Knowledge %

    对比模型VUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    KEG-UCG67.571.169.380.670.371.370.889.472.570.171.374.077.576.977.277.1
    KEG-CSK71.270.770.981.372.071.471.790.273.170.771.974.778.278.378.278.3
    KEG80.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    下载: 导出CSV

    表  9   2种解码方式的实验结果

    Table  9   Experimental Results of Two Decoding Methods %

    对比模型VUA-VERBVUA-ALLPOSTroFiMOH-X
    PRF1AccPRF1AccPRF1AccPRF1Acc
    KEG-C79.577.378.486.574.275.674.992.275.876.376.075.982.081.281.681.5
    KEG80.577.478.986.774.676.575.592.875.677.876.776.282.181.681.881.6
    下载: 导出CSV
  • [1] Shutova E. Models of metaphor in NLP[C] //Proc of the 48th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2010: 688−697
    [2]

    Jang H, Moon S, Jo Y, et al. Metaphor detection in discourse[C] //Proc of the 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue. Stroudsburg, PA: ACL, 2015: 384−392

    [3]

    Klebanov B, Leong C, Gutiérrez E, et al. Semantic classifications for detection of verb metaphors[C] //Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016: 101−106

    [4]

    Bulat L, Clark S, Shutova E. Modelling metaphor with attribute-based semantics[C] //Proc of the 15th Conf of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2017: 523−528

    [5] Shutova E, Kiela D, Maillard J. Black holes and white rabbits: Metaphor identification with visual features[C] //Proc of the 2016 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2016: 160−170
    [6]

    Mao Rui, Lin Chenghua, Guerin F. End-to-end sequential metaphor identification inspired by linguistic theories[C] //Proc of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2019: 3888−3898

    [7] Gao Ge, Choi E, Choi Y, et al. Neural metaphor detection in context[C] //Proc of the 2018 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 607−613
    [8] Rei M, Bulat L, Kiela D, et al. Grasping the finer point: A supervised similarity network for metaphor detection[C] //Proc of the 2017 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2017: 1537−1546
    [9] Chen Xianyang, Leong C, Flor M, et al. Go Figure! Multi-task transformer-based architecture for metaphor detection using idioms: ETS team in 2020 metaphor shared task[C] //Proc of the 2nd Workshop on Figurative Language Processing, Fig-Lang@ACL 2020. Stroudsburg, PA: ACL, 2020: 235−243
    [10]

    Su Chuandong, Fukumoto F, Huang Xiaoxi, et al. DeepMet: A reading comprehension paradigm for token-level metaphor detection[C] //Proc of the 2nd Workshop on Figurative Language Processing, Fig-Lang@ACL 2020. Stroudsburg, PA: ACL, 2020: 30−39

    [11]

    Rohanian O, Rei M, Taslimipoor S, et al. Verbal multiword expressions for identification of metaphor[C] //Proc of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2020: 2890−2895

    [12]

    Leong C, Klebanov B, Shutova E. A report on the 2018 VUA metaphor detection shared task[C] //Proc of the 1st Workshop on Figurative Language Processing, Fig-Lang@NAACL-HLT 2018. Stroudsburg, PA: ACL, 2018: 56−66

    [13]

    Birke J, Sarkar A. A clustering approach for nearly unsupervised recognition of nonliteral language[C] //Proc of the 11th Conf of the European Chapter of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2017: 523−528

    [14]

    Dunn J. Measuring metaphoricity[C] //Proc of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2014: 745−751

    [15]

    Gutiérrez E, Shutova E, Marghetis T, et al. Literal and metaphorical senses in compositional distributional semantic models[C] //Proc of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2016: 183−193

    [16]

    Mohammad S, Shutova E, Turney P. Metaphor as a medium for emotion: An empirical study[C] //Proc of the 5th Joint Conf on Lexical and Computational Semantics, *SEM@ACL 2016. Stroudsburg, PA: ACL, 2016: 23−33

    [17]

    Tsvetkov Y, Boytsov L, Gershman A, et al. Metaphor detection with cross-lingual model transfer[C] //Proc of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2014: 248−258

    [18]

    Wilks Y. A preferential, pattern-seeking, semantics for natural language inference[J]. Artificial Intelligence, 1975, 6(1): 53−74 doi: 10.1016/0004-3702(75)90016-8

    [19]

    Pragglejaz Group. MIP: A method for identifying metaphorically used words in discourse[J]. Metaphor and Symbol, 2007, 22(1): 1−39 doi: 10.1080/10926480709336752

    [20] Devlin J, Chang M, Lee K, et al. BERT: Pre-training of deep bidirectional transformers for language understanding[C] //Proc of the 2019 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2019: 4171−4186
    [21] Vaswani A, Shazzer N, Parmar N, et al. Attention is all you need[C] //Proc of the Advances in Neural Information Processing Systems 30: Annual Conf on Neural Information Processing Systems 2017. San Diego, CA: NIPS, 2017: 5998−6008
    [22]

    Liu Yinhan, Ott M, Goyal N, et al. RoBERTa: A robustly optimized BERT pretraining approach [J]. arXiv preprint, arXiv: 1907.11692, 2019

    [23]

    Leong C, Klebanov B, Hamill C, et al. A report on the 2020 VUA and TOEFL metaphor detection shared task[C] //Proc of the 2nd Workshop on Figurative Language Processing, Fig-Lang@ACL 2020. Stroudsburg, PA: ACL, 2020: 18−29

    [24]

    Kipf T, Welling M. Semi-supervised classification with graph convolutional networks[C/OL] //Proc of the 5th Int Conf on Learning Representations. La Jolla, CA: ICLR, 2017 [2018-05-20]. https://openreview.net/pdf?id=SJU4ayYgl

    [25]

    Steen G, Dorst A, Herrmann J, et al. A method for linguistic metaphor identification: From MIP to MIPVU[M] //Converging Evidence in Language and Communication Research, Volume 14. Amsterdam: John Benjamins Publishing Company, 2010: 25−42

    [26] Klebanov B, Leong C, Flor M. A corpus of non-native written english annotated for metaphor[C] //Proc of the 2018 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2018: 86−91
    [27]

    Levin L, Mitamura T, MacWhinney B, et al. Resources for the detection of conventionalized metaphors in four languages[C] //Proc of the 9th Int Conf on Language Resources and Evaluation. Paris: LREC, 2014: 498−501

    [28]

    Mohler M, Brunson M, Rink B, et al. Introducing the LCC metaphor datasets[C] //Proc of the 10th Int Conf on Language Resources and Evaluation. Paris: LREC, 2016: 4221−4227

    [29]

    Parde N, Nielsen R. A corpus of metaphor novelty scores for syntactically-related word pairs[C] //Proc of the 11th Int Conf on Language Resources and Evaluation. Paris: LREC, 2018: 1535−1540

    [30]

    Tredici M, Bel N. Assessing the potential of metaphoricity of verbs using corpus data[C] //Proc of the 10th Int Conf on Language Resources and Evaluation. Paris: LREC, 2016: 4573−4577

    [31] Chen Xinxiong, Liu Zhiyuan, Sun Maosong. A unified model for word sense representation and disambiguation[C] //Proc of the 2014 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2014: 1025−1035
    [32] Mikolov T, Sutskever I, Chen Kai, et al. Distributed representations of words and phrases and their compositionality[C] //Proc of the Advances in Neural Information Processing Systems 26: Annual Conf on Neural Information Processing Systems 2013. San Diego, CA: NIPS, 2013: 3111−3119
    [33] Marcheggiani D, Titov I. Encoding sentences with graph convolutional networks for semantic role labeling[C] //Proc of the 2017 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2017: 1506−1515
    [34] Liu Xiao, Luo Zhunchen, Huang Heyan. Jointly multiple events extraction via attention-based graph information aggregation[C] //Proc of the 2018 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2018: 1247−1256
    [35]

    Zhang Yue, Liu Qi, Song Linfeng. Sentence-state LSTM for text representation[C] //Proc of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2018: 317−327

    [36]

    Song Linfeng, Zhang Yue, Wang Zhiguo, et al. A graph-to-sequence model for AMR-to-text generation[C] //Proc of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg, PA: ACL, 2018: 1616−1626

    [37] Cho K, Merrienboer B, Gülçehre Ç, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C] //Proc of the 2014 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2014: 1724−1734
    [38]

    Lafferty J, McCallum A, Pereira F. Conditional random fields: Probabilistic models for segmenting and labeling sequence data[C] //Proc of the 18th Int Conf on Machine Learning. San Diego, CA: ICML, 2001: 282−289

    [39]

    Linguistic Data Consortium. BLLIP 1987-89 WSJ Corpus Release 1[DB/OL]. [2020-12-28]. https://catalog.ldc.upenn.edu/LDC2000T43

    [40]

    Loshchilov I, Hutter F. Decoupled weight decay regularization[C/OL] //Proc of the 7th Int Conf on Learning Representations. La Jolla, CA: ICLR, 2019 [2020-01-10]. https://openreview.net/pdf?id=Bkg6RiCqY7

    [41] Pennington J, Socher R, Manning C. GloVe: Global vectors for word representation[C] //Proc of the 2014 Conf on Empirical Methods in Natural Language Processing. Stroudsburg, PA: ACL, 2014: 1532−1543
    [42] Peters M, Neumann M, Iyyer M, et al. Deep contextualized word representations[C] //Proc of the 2018 Conf of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg, PA: ACL, 2018: 2227−2237
  • 期刊类型引用(12)

    1. 郭锋,刘飞洋,刘鸽. LoongArch处理器存储管理技术研究与实现. 电脑编程技巧与维护. 2025(03): 82-84 . 百度学术
    2. 邢世远,张见齐,王焕东,吴学智,吴瑞阳. 片间互连总线协议层关键技术研究. 高技术通讯. 2025(02): 113-123 . 百度学术
    3. 舒燕君,郑翔宇,徐成华,黄沛,王永琪,周凡,张展,左德承. 面向LoongArch边界检查访存指令的GCC优化. 计算机研究与发展. 2025(05): 1136-1150 . 本站查看
    4. 孙东华,刘亚冬. 基于龙芯3A5000的全国产化EtherCAT主站控制器设计. 现代信息科技. 2025(08): 16-19+24 . 百度学术
    5. 谢汶兵,田雪,漆锋滨,武成岗,王俊,罗巧玲. 二进制翻译技术综述. 软件学报. 2024(06): 2687-2723 . 百度学术
    6. 谢君,陈汉云,袁璐,张梦娇,王增,石锐. 基于信创的医院自助服务系统建设探索. 中国数字医学. 2024(07): 41-45+69 . 百度学术
    7. 谭弘泽,王剑. 基于动态压缩的高存储效率末级分支目标缓冲. 高技术通讯. 2024(07): 671-680 . 百度学术
    8. 刘登峰,李东亚,柴志雷,周浩杰,丁海峰. 基于QEMU的SIMD指令替换浮点指令框架. 湖南大学学报(自然科学版). 2024(08): 70-77 . 百度学术
    9. 陈国良,汤晓宇,尤帅,姚小良,梅超君,林时俊,刘尚东,吴少刚,孙雅薇,王汝传,季一木. 基于国产处理器的智能大数据一体机架构及应用研究. 南京邮电大学学报(自然科学版). 2024(04): 1-16 . 百度学术
    10. 贾金成,朱家鑫,唐震,王志鹏,王伟. 映射字典导向的64位ARM到RISC-V汇编翻译. 小型微型计算机系统. 2024(08): 2041-2048 . 百度学术
    11. 游英杰,刘宣佑,唐文武,张统兵,王岩. 基于CPU的PCIe驱动及与DSP和FPGA的交互设计. 火控雷达技术. 2024(04): 88-93+123 . 百度学术
    12. 王一泠,吴琦,安军社. 支持MIPS架构的轻量型开源鸿蒙系统移植. 计算机工程. 2023(12): 25-34+45 . 百度学术

    其他类型引用(23)

图(3)  /  表(9)
计量
  • 文章访问数:  291
  • HTML全文浏览量:  39
  • PDF下载量:  126
  • 被引次数: 35
出版历程
  • 收稿日期:  2021-09-08
  • 修回日期:  2022-04-14
  • 网络出版日期:  2023-02-10
  • 刊出日期:  2022-12-31

目录

    /

    返回文章
    返回