ISSN 1000-1239 CN 11-1777/TP

Journal of Computer Research and Development ›› 2020, Vol. 57 ›› Issue (5): 1037-1045.doi: 10.7544/issn1000-1239.2020.20190474

Previous Articles     Next Articles

Multi-Modal Knowledge-Aware Attention Network for Question Answering

Zhang Yingying1,2, Qian Shengsheng2, Fang Quan2, Xu Changsheng1,2   

  1. 1( University of Chinese Academy of Sciences, Beijing 100049);2( National Laboratory of Pattern Recognition (Institute of Automation, Chinese Academy of Sciences), Beijing 100190)
  • Online:2020-05-01
  • Supported by: 
    This work was supported by the National Natural Science Foundation of China (Y8F7011M61).

Abstract: With the popularity of the Internet, more people choose to search online to find the solutions when they feel sick. With the emergence of reliable medical question answering websites, e.g. Chunyu Doctor, XYWY, patients can communicate with the doctor one-one at home. However, existing question answering methods focus on word-level interaction or semantics, but rarely notice the hidden rationale with doctors’ commonsense, while in the real scenes, doctors need to acquire plenty of domain knowledge to give advice to the patients. This paper proposes a novel multi-modal knowledge-aware attention network (MKAN) to effectively exploit multi-modal knowledge graph for medical question answering. The incorporation of multi-modal information can provide more fine-grained information. This information shows how entities in the medical graph are related. Our model first generates multi-modal entity representation with a translation-based method, and then defines question-answer interactions as the paths in the multi-modal knowledge graph that connect the entities in the question and answer. Furthermore, to discriminate the importance of paths, we propose an attention network. We build a large-scale multi-modal medical knowledge graph based on Symptom-in-Chinese, as well as one real-world medical question answering datasets based on Chunyu Doctor website. Extensive experiments strongly evidence that our proposed model obtains significant performance compared with state-of-the arts.

Key words: multi-modal knowledge graph, medical question answering system, attention, information retrieval, deep learning

CLC Number: