ISSN 1000-1239 CN 11-1777/TP

• 图形图像 •

### 基于语义分割的红外和可见光图像融合

1. 1(武汉工程大学计算机科学与工程学院 武汉 430205);2(智能机器人湖北省重点实验室(武汉工程大学) 武汉 430205);3(武汉大学电子信息学院 武汉 430072) (zhouhuabing@gmail.com)
• 出版日期: 2021-02-01
• 基金资助:
国家自然科学基金项目(61771353,61773295,62072350,41501505)；湖北省技术创新工程项目(2019AAA045)

### Infrared and Visible Image Fusion Based on Semantic Segmentation

Zhou Huabing1,2, Hou Jilei1,2, Wu Wei1,2, Zhang Yanduo1,2, Wu Yuntao1,2, Ma Jiayi3

1. 1(College of Computer Science and Engineering, Wuhan Institute of Technology, Wuhan 430205);2(Hubei Key Laboratory of Intelligent Robot (Wuhan Institute of Technology), Wuhan 430205);3(Electronic Information School, Wuhan University, Wuhan 430072)
• Online: 2021-02-01
• Supported by:
This work was supported by the National Natural Science Foundation of China (61771353, 61773295, 62072350, 41501505) and the Hubei Technology Innovation Project (2019AAA045).

Abstract: Infrared images can distinguish targets from their backgrounds due to the difference in thermal radiation even in poor lighting conditions. By contrast, visible images can represent texture details with high spatial resolution. Meanwhile, both of infrared and visible images preserve corresponding semantic information. Therefore, infrared and visible image fusion should keep both radiation information of the infrared image and texture details of the visible image; additionally, it needs to reserve the semantic information of both. Semantic segmentation can transform the source images into the masks with semantic information. In this paper, an infrared and visible image fusion method is proposed based on semantic segmentation. It can overcome the shortcomings that the existing fusion methods are not specific to different regions. Considering the specific information for different regions of infrared and visible images, we design two loss functions for different regions to improve the quality of fused image under the framework of generative adversarial network. Firstly, we gain the masks of the infrared images with semantic information by semantic segmentation; then we use the masks to divide the infrared and visible images into infrared target area, infrared background area, visible target area, and visible background area. Secondly, we employ different methods to fuse the target and background area, respectively. Finally, we combine the two regions to obtain the final fused image. The experiment shows that the proposed method outperforms state-of-the-art, where our results have higher contrast in the target area and richer texture details in the background area.