ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2018, Vol. 55 ›› Issue (12): 2600-2610.doi: 10.7544/issn1000-1239.2018.20180606

Special Issue: 2018碎片化知识融合与应用专题

Previous Articles     Next Articles

Question Answering Algorithm on Image Fragmentation Information Based on Deep Neural Network

Wang Yilei, Zhuo Yifan, Wu Yingjie, Chen Mingqin   

  1. (College of Mathematics and Computer Science, Fuzhou University, Fuzhou 350108)
  • Online:2018-12-01

Abstract: Many fragmentation information is highly dispersed in different data sources, such as text, image, video and Web. They are characterized by structural disorder and content one-sided. Current researches implement the extraction, expression and understanding of multi-modal fragmentation information by constructing visual question answering (VQA) system. The VQA task is required to provide the correct answer to a given problem with a corresponding image. The aim of this paper is to design a complete framework and algorithm for image fragmentation information question answering under the basic background of visual question answering task. The main research includes image feature extraction, question text feature extraction, multi-modal feature fusion and answer reasoning. Deep neural network is constructed to extract features for representing images and problem information. Attention mechanism and variational inference method are combined to fusion two modal features of image and problem and reason answers. Experiment results show that the model can effectively extract and understand multi-modal fragmentation information, and improve the accuracy of VQA.

Key words: artificial intelligence, fragmented information, neural network, deep learning, visual question answering (VQA)

CLC Number: