高级检索

    一种残差置乱上下文信息的场景图生成方法

    Scene Graph Generation Based on Shuffle Residual Context Information

    • 摘要: 场景图在视觉理解中有着很重要的作用.现有的场景图生成方法对于主语、宾语以及主宾语间的视觉关系进行研究.但是,人类通过空间关系上下文、语义上下文和目标之间的互动信息来进行关系的理解和推理.为了获得更好的全局上下文表示,同时减少数据集偏差的影响,提出了一个新的场景图生成框架RSSQ(residual shuffle sequence model).该框架由目标解码、残差置乱和位置嵌入3部分构成.残差置乱模块由随机置乱和残差连接的双向LSTM的基本结构叠加而成,利用迭代方式实现随机打乱双向LSTM的隐藏状态以减少数据集偏差影响,利用残差连接提取共享的全局上下文信息.在位置嵌入模块中,通过对目标的相对位置和面积比例的编码则可以增强目标对之间的空间关系.在数据集Visual Genome的3个不同层次子任务的实验中,证明了提出的RSSQ方法因全局上下文改善和空间关系增强,在Recall@50和Recall@100指标评价下,相对于现有方法能生成更好的场景图.

       

      Abstract: Scene graphs play an important role in visual understanding. Existing scene graph generation methods focus on the research of the subjects, the objects as well as the predicates between them. However, human being abstracts the relationships using spatial relation context, semantic context and interaction between scene objects for better understanding and reasoning as whole. In order to obtain the better global context representation and reduce the impact of dataset bias, we propose a new framework of scene graph generation, called as residual shuffle sequence model (RSSQ). Our method is made up of object decoding, residual shuffle and position embedding modules. Residual shuffle module is stacked with two basic structures including the random shuffle operation and the residual bidirectional LSTM. We implement the random shuffle on the hidden state of bidirectional LSTM by the process of iterative operation to reduce the impact of dataset bias, and extract the shared global context information by the residual connection structure. To strengthen the spatial relationship between pair-wise objects, the encoding is achieved using the relative position and area ratio of objects in position embedding module. The experimental results of three sub-tasks of different difficulty performed on Visual Genome dataset, demonstrate that the poposed method can generate better scene graphs under Recall@50 and Recall@100 settings due to better global context and spatial information.

       

    /

    返回文章
    返回