高级检索
    张璐, 曹峰, 梁新彦, 钱宇华. 基于关联特征传播的跨模态检索[J]. 计算机研究与发展, 2022, 59(9): 1993-2002. DOI: 10.7544/issn1000-1239.20210475
    引用本文: 张璐, 曹峰, 梁新彦, 钱宇华. 基于关联特征传播的跨模态检索[J]. 计算机研究与发展, 2022, 59(9): 1993-2002. DOI: 10.7544/issn1000-1239.20210475
    Zhang Lu, Cao Feng, Liang Xinyan, Qian Yuhua. Cross-Modal Retrieval with Correlation Feature Propagation[J]. Journal of Computer Research and Development, 2022, 59(9): 1993-2002. DOI: 10.7544/issn1000-1239.20210475
    Citation: Zhang Lu, Cao Feng, Liang Xinyan, Qian Yuhua. Cross-Modal Retrieval with Correlation Feature Propagation[J]. Journal of Computer Research and Development, 2022, 59(9): 1993-2002. DOI: 10.7544/issn1000-1239.20210475

    基于关联特征传播的跨模态检索

    Cross-Modal Retrieval with Correlation Feature Propagation

    • 摘要: 深度学习的快速发展和关联学习的深度研究,使得跨模态检索的性能有了很大提升.跨模态检索研究面临的挑战是:不同模态的数据在高层语义上具有关联关系,但在底层特征上存在异构鸿沟.现有方法主要通过单个相关性约束将不同模态的特征映射到具有一定相关性的特征空间中来解决底层特征上的异构鸿沟问题.然而,表征学习表明,不同层次的特征在帮助模型最终性能的提升上都会起作用.所以,现有方法学习到的单一特征空间的关联性是弱的,即该特征空间可能不是最优的检索空间.为解决该问题,提出了基于关联特征传播的跨模态检索模型,其基本思想是强化深度网络各层之间的关联性,即前一层具有一定关联的特征经过非线性变化传到后一层,有利于找到使2种模态关联性更强的特征空间.通过在Wikipedia,Pascal数据集上的大量实验验证得到,该方法提升了平均精度均值.

       

      Abstract: With the rapid development of deep learning and the deep research of correlation learning, the performance of cross-modal retrieval has been greatly improved. The challenge of cross-modal retrieval research is that different modal data are related in high-level semantics, but there is a heterogeneous gap in low-level features. The existing methods mainly map the features of different modalities to feature space with certain correlation by single correlation constraint to solve the heterogeneous gap problem of the low-level features. However, representation learning shows that different layers of features can help improve the final performance of the model. Therefore, the correlation of the single feature space learned by existing methods is weak, namely the feature space may not be the optimal retrieval space. In order to solve this problem, we propose the modal of cross-modal retrieval with correlation feature propagation. Its basic idea is to strengthen the correlation between the layers of the deep network, namely the characteristics of the former layer with certain correlation are transmitted to the latter layer through nonlinear changes, which is more conducive to find the feature space that makes the two modalities more correlated. A lot of experiments on Wikipedia, Pascal data sets show that this method can improve mean average precision.

       

    /

    返回文章
    返回