高级检索
    林萌, 戴程威, 郭涛. 基于多模态大语言模型的攻击性模因解释生成方法[J]. 计算机研究与发展. DOI: 10.7544/issn1000-1239.202330960
    引用本文: 林萌, 戴程威, 郭涛. 基于多模态大语言模型的攻击性模因解释生成方法[J]. 计算机研究与发展. DOI: 10.7544/issn1000-1239.202330960
    Lin Meng, Dai Chengwei, Guo Tao. A Method for Generating Explanations of Offensive Memes Based on Multimodal Large Language Models[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202330960
    Citation: Lin Meng, Dai Chengwei, Guo Tao. A Method for Generating Explanations of Offensive Memes Based on Multimodal Large Language Models[J]. Journal of Computer Research and Development. DOI: 10.7544/issn1000-1239.202330960

    基于多模态大语言模型的攻击性模因解释生成方法

    A Method for Generating Explanations of Offensive Memes Based on Multimodal Large Language Models

    • 摘要: 随着5G的发展,攻击性言论逐渐以多模态的方式在社交网络上广泛传播. 因此,攻击性模因的检测与解释生成对于提高内容审核效果、维护和谐健康的舆论场环境有着重要的作用. 现有的攻击性模因解释生成研究只关注于攻击对象和攻击内容,忽略了模因包含的社会背景知识和隐喻表达手法,无法全面、准确地解释攻击性模因的含义,大大限制了解释的应用范围. 为了应对这一挑战,提出一种基于多模态大模型的攻击性模因解释生成方法,通过增强攻击目标、攻击内容和隐喻识别等多种指令数据,利用其微调多模态大模型,以提升大模型对攻击性模因的解释生成能力. 实验结果证实,该方法生成的解释具有3点优势:一是相比基线模型在BERTScore评估指标上提高了19%;二是解释中包含了攻击性隐喻表达的相关背景知识;三是在处理未见的模因数据时也表现出良好的泛化性能.

       

      Abstract: With the advancement of 5G technology, offensive speech has increasingly proliferated across social networks in the form of multimodal memes. Consequently, the detection and interpretive generation of offensive memes play a crucial role in enhancing content moderation effectiveness and maintaining a harmonious and healthy public discourse environment. Existing studies on the interpretive generation of offensive memes focus solely on the targets and content of offense, neglecting the societal background knowledge and metaphorical expressions embedded in memes. This oversight limits the ability to comprehensively and accurately interpret the meaning of offensive memes, thus constraining the applicability of interpretations. To address this challenge, we propose a method based on multimodal large language model for generating interpretations of offensive memes. By augmenting elements such as offense targets, the content of the offense, and metaphor recognition into the instruction tuning data, we can effectively improve the multimodal large model’s proficiency in interpretively generating offensive memes through instruction tuning. The experimental outcomes validate three key strengths of our method: first, it achieves a notable 19% enhancement in the BERTScore evaluation metric over baseline models; second, it incorporates comprehensive background knowledge pertinent to offensive metaphorical expressions within its interpretations; third, it exhibits strong generalization capabilities when handling previously unseen meme data.

       

    /

    返回文章
    返回