高级检索

    基于大语言模型的双视角多级跨模态推荐

    Dual-Perspective Multi-Level Cross-Modal Recommendation Based on Large Language Models

    • 摘要: 多模态推荐系统旨在提供更为精准和个性化的推荐服务. 然而,现有研究仍存在以下问题:1)特征失真. 由于输入的嵌入均由小型预训练语言模型和深层卷积神经网络等模型进行处理,导致得到的特征表示不准确. 2)编码视角单一. 目前模型的多模态编码层只考虑在单一的记忆或扩展视角进行编码,造成信息缺失. 3)多模态对齐效果欠佳. 不同模态嵌入分布在不同空间中,需将其映射至同一空间以实现对齐. 而现有方法通过简单的行为信息乘积无法捕捉模态之间的复杂关系,导致多种模态无法精确对齐. 基于上述问题,提出了一个新颖的模型DPRec. 该模型同时考虑了记忆与扩展的双视角编码,并引入超图进行多级精准跨模态对齐. 所提模型在3个真实数据集上进行了扩展实验,实验结果验证了所提模型的有效性.

       

      Abstract: Multimodal recommendation systems aim to deliver more accurate and personalized recommendations by leveraging diverse data modalities such as text and images. Despite their potential, existing approaches face several critical challenges. 1) Feature distortion. The input embeddings are processed by relatively small pre-trained language models and deep convolutional neural networks, which may lead to inaccurate and incomplete feature representations. 2) Single encoding perspective. The multimodal encoding layers of current models only consider encoding from a single perspective that is either memory-based or expansion-based, which limits representational capacity and leads to information loss. 3) Poor multimodal alignment. Embeddings from different modalities are distributed in distinct and heterogeneous semantic spaces, and need to be effectively mapped into a shared semantic space to achieve accurate alignment. However, existing methods typically rely on simple operations such as direct product of behavioral information, which cannot adequately capture the complex and high-order relationships among modalities. As a result, precise alignment across multiple modalities is difficult to achieve. To address these issues, we propose a novel model named DPRec. This model considers encoding from both memory and expansion perspectives and introduces hypergraphs for multi-level precise cross-modal alignment. The model is evaluated through extensive experiments on three real-world datasets. Experimental results confirm the effectiveness of DPRec, demonstrating its superiority over state-of-the-art approaches in improving recommendation accuracy.

       

    /

    返回文章
    返回