语义增强的多模态虚假新闻检测

亓鹏; 曹娟; 盛强

doi:10.7544/issn1000-1239.2021.20200804

语义增强的多模态虚假新闻检测

(中国科学院智能信息处理重点实验室(中国科学院计算技术研究所) 北京 100190) (中国科学院计算技术研究所北京 100190) (中国科学院大学北京 100049) (qipeng@ict.ac.cn)

基金项目: 国家自然科学基金重点项目(U1703261)

详细信息

中图分类号: TP391
计量
- 文章访问数: 1451
- HTML全文浏览量: 30
- PDF下载量: 828
出版历程
- 发布日期: 2021-06-30

Semantics-Enhanced Multi-Modal Fake News Detection

(Key Laboratory of Intelligent Information Processing of Chinese Academy of Science (Institute of Computing Technology, Chinese Academy of Sciences), Beijing 100190) (Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190) (University of Chinese Academy of Sciences, Beijing 100049)

Funds: This work was supported by the Key Program of the National Natural Science Foundation of China (U1703261).

摘要

摘要: 近年来社交媒体逐渐成为人们获取新闻信息的主要渠道，但其在给人们带来方便的同时也促进了虚假新闻的传播.在社交媒体的富媒体化趋势下，虚假新闻逐渐由单一的文本形式向多模态形式转变，因此多模态虚假新闻检测正在受到越来越多的关注.现有的多模态虚假新闻检测方法大多依赖于和数据集高度相关的表现层面特征，对新闻的语义层面特征建模不足，难以理解文本和视觉实体的深层语义，在新数据上的泛化能力受限.提出了一种语义增强的多模态虚假新闻检测方法，通过利用预训练语言模型中隐含的事实知识以及显式的视觉实体提取，更好地理解多模态新闻的深层语义.提取不同语义层次的视觉特征，在此基础上采用文本引导的注意力机制建模图文之间的语义交互，从而更好地融合多模态异构特征.在基于微博新闻的真实数据集上的实验结果表明:该方法能够有效提高多模态虚假新闻检测的性能.
- 社交媒体 /
- 虚假新闻检测 /
- 多模态 /
- 知识融合 /
- 注意力机制
Abstract: In recent years, social media has become the main access where people acquire the latest news. However, the convenience and openness of social media have also facilitated the proliferation of fake news. With the development of multimedia technology, fake news on social media has been evolving from text-only posts to multimedia posts containing images or videos. Therefore, multi-modal fake news detection is attracting more and more attention. Existing methods for multi-modal fake news detection mostly focus on capturing appearance-level features that are highly dependent on the dataset distribution but insufficiently exploit the semantics-level features. Thus, the methods often fail to understand the deep semantics of textual and visual entities in the fake news, which indeed limits the generalizability of models in real applications. To tackle this problem, this paper proposes a semantics-enhanced multi-modal model for fake news detection, which better models the underlying semantics of multi-modal news by implicitly utilizing the factual knowledge in the pre-trained language model and explicitly extracting the visual entities. Furthermore, the proposed method extracts visual features of different semantic levels and models the semantic interaction between the textual and visual features by the text-guided attention mechanism, which better fuses the multi-modal heterogeneous features. Extensive experiments on the Weibo dataset strongly evidence that our method outperforms the state of the art significantly.
- social media /
- fake news detection /
- multi-modal /
- knowledge fusion /
- attention mechanism

HTML全文

参考文献(0)

施引文献(29)

期刊类型引用(11)

1.	袁子轩，张峰，许岗，魏光辉，石永强. 融合MAML和TGAT的机会网络动态链路预测模型. 小型微型计算机系统. 2024(12): 2957-2963 . 百度学术
2.	曹志威，樊志杰，王青杨，韩伟力，李欣. 一种降噪自编码器的复杂网络链路预测算法. 小型微型计算机系统. 2023(03): 665-672 . 百度学术
3.	刘林峰，于子兴，祝贺. 基于门控循环单元的移动社会网络链路预测方法. 计算机研究与发展. 2023(03): 705-716 . 本站查看
4.	王曙燕，巩婧怡. 融合节点标签与强弱关系的链路预测算法. 计算机工程与应用. 2022(18): 71-77 . 百度学术
5.	张瑾，朱桂祥，王宇琛，郑烁佳，陈镜潞. 基于异质图表达学习的跨境电商推荐模型. 电子与信息学报. 2022(11): 4008-4017 . 百度学术
6.	唐明虎. 基于多种信息组合模式的非负矩阵分解链路预测模型. 计算机应用研究. 2021(05): 1393-1397+1408 . 百度学术
7.	顾秋阳，吴宝，池仁勇. 基于高阶路径相似度的复杂网络链路预测方法. 通信学报. 2021(07): 61-69 . 百度学术
8.	许爽，李淼磊. 基于子图特征的科学家合作网络链路预测. 大连民族大学学报. 2020(01): 51-63 . 百度学术
9.	张尚田，陈光，邱天. 基于融合特征的LSTM评分预测. 计算机与现代化. 2020(03): 49-53+59 . 百度学术
10.	顾秋阳，琚春华，吴功兴. 基于子图演化与改进蚁群优化算法的社交网络链路预测方法. 通信学报. 2020(12): 21-35 . 百度学术
11.	李琦，王智强，梁吉业. 基于PU学习的链接预测方法. 模式识别与人工智能. 2019(09): 793-799 . 百度学术