高级检索

    基于大语言模型的双视角多级跨模态推荐

    Dual-Perspective Multi-Level Cross-Modal Recommendation Based on Large Language Models

    • 摘要: 多模态推荐系统旨在提供更为精准和个性化的推荐服务.然而,现有研究仍存在以下问题:1)特征失真:由于输入的嵌入均由小型预训练语言模型和深层卷积神经网络等模型进行处理,导致得到的特征表示不准确.2)编码视角单一:目前模型的多模态编码层只考虑在单一的记忆或扩展视角进行编码,造成信息缺失.3)多模态对齐效果欠佳:不同模态嵌入分布在不同空间中,需将其映射至同一空间以实现对齐,而现有方法通过简单的行为信息乘积无法捕捉模态之间的复杂关系,导致多种模态无法精确对齐.基于上述问题,提出了一个新颖的模型DPRec.该模型同时考虑了记忆与扩展的双视角编码,并引入超图进行多级精准跨模态对齐.所提模型在三个真实数据集上进行了扩展实验,实验结果验证了所提模型的有效性.

       

      Abstract: Multimodal recommendation systems aim to provide more accurate and personalized recommendation services. However, existing research still faces the following issues: 1) Feature distortion: The input embeddings are processed by small pre-trained language models and deep convolutional neural networks, resulting in inaccurate feature representations. 2) Single encoding perspective: The multimodal encoding layers of current models only consider encoding from a single perspective of memory or expansion, leading to information loss. 3) Poor multimodal alignment: Embeddings from different modalities are distributed in different spaces and need to be mapped to the same space for alignment. However, existing methods, which rely on simple behavioral information multiplication, fail to capture the complex relationships between modalities, preventing precise alignment. To address these issues, a novel model called DPRec is proposed. This model considers encoding from both memory and expansion perspectives and introduces hypergraphs for multi-level precise cross-modal alignment. The proposed model has been tested on three real-world datasets, and the experimental results have validated its effectiveness.

       

    /

    返回文章
    返回