高级检索

    语义对齐与自适应回放增强的动态多模态知识图谱持续补全

    Continual Completion for Dynamic Multimodal Knowledge Graphs via Semantic Alignment and Adaptive Replay Enhancement

    • 摘要: 面向知识增强人工智能中动态知识更新与长期知识保持的需求,动态多模态知识图谱持续补全随图谱快照演化增量推断缺失事实、保持历史知识稳定,为动态知识更新提供基础支撑。现有持续补全方法多关注单一模态,难以充分利用文本与视觉语义;直接引入多模态信息又易导致跨快照语义非预期偏移,进而加剧灾难性遗忘。为此,提出语义对齐与自适应回放增强的动态多模态知识图谱持续补全方法SAREC。利用大语言模型生成实体语义描述,并结合视觉-语言预训练表征构建多模态差异感知融合机制,提升实体表示的完整性与鲁棒性;进而通过多模态时序语义对齐,对实体表征的跨快照演化进行约束,保持内在语义一致性;最后,提出时序感知自适应回放机制,依据快照时序动态分配回放预算并选择性回放历史知识,缓解灾难性遗忘。实验结果表明,SAREC在多种知识演化模式下取得优于基线方法的性能,为开放环境下的持续知识注入与自适应知识增强提供了有效支撑。

       

      Abstract: To meet the demand for dynamic knowledge updating and long-term knowledge retention in knowledge-enhanced artificial intelligence, dynamic multimodal knowledge graph continual completion infers missing facts incrementally as graph snapshots evolve while keeping historical knowledge stable. It thus provides fundamental support for dynamic knowledge updates. Existing methods predominantly focus on single-modal structural information, failing to fully exploit textual and visual semantics; directly incorporating multimodal information may introduce unexpected semantic shifts across snapshots, thereby exacerbating catastrophic forgetting. To this end, this paper proposes SAREC, a semantic alignment and adaptive replay enhanced method for dynamic multimodal knowledge graph continual completion. SAREC leverages large language models to generate entity semantic descriptions and combines them with vision-language pretrained representations to construct a multimodal discrepancy-aware fusion mechanism, enhancing the completeness and robustness of entity representations. A multimodal temporal semantic alignment module is further introduced to constrain cross-snapshot representation evolution and maintain semantic consistency. Finally, a temporal-aware adaptive replay mechanism is proposed, which dynamically allocates the replay budget according to snapshot recency and selectively replays historical knowledge to alleviate catastrophic forgetting. Experiments demonstrate that SAREC achieves superior performance under various knowledge evolution modes, providing effective support for continual knowledge injection and adaptive knowledge enhancement in open dynamic environments.

       

    /

    返回文章
    返回