A Multi-Branch Defogging Network Based on Fog Concentration Classification and Dark and Bright Channel Priors

Zhang Qidong; Chi Jing; Chen Yuyan; Zhang Caiming

doi:10.7544/issn1000-1239.202220812

Journal of Computer Research and Development > 2024 > 61(3): 762-779. > DOI: 10.7544/issn1000-1239.202220812

Zhang Qidong, Chi Jing, Chen Yuyan, Zhang Caiming. A Multi-Branch Defogging Network Based on Fog Concentration Classification and Dark and Bright Channel Priors[J]. Journal of Computer Research and Development, 2024, 61(3): 762-779. DOI: 10.7544/issn1000-1239.202220812

Citation:

PDF (5890 KB)

A Multi-Branch Defogging Network Based on Fog Concentration Classification and Dark and Bright Channel Priors

Zhang Qidong^{1, 2,},
Chi Jing^{1, 2, ,},
Chen Yuyan^{1, 2},
Zhang Caiming^{2, 3}

1.
College of Computer Science and Technology, Shandong University of Finance and Economics, Jinan 250014
2.
Shandong Provincial Key Laboratory of Digital Media Technology (Shandong University of Finance and Economics), Jinan 250014
3.
School of Software, Shandong University, Jinan 250101

Funds: This work was supported by the Science and Technology Plan for Young Talents in Colleges and Universities of Shandong Province (2020KJN007), the Plan of Scientific Research Leader Studio in Colleges and Universities of Jinan City (2021GXRC092), the Key Program of the National Natural Science Foundation of China (U1909210), and the Key Research and Development Program of Shandong Province (2019GSF109112, 2021SFGC0102).

More Information

Author Bio:
Zhang Qidong: born in 1997. Master. His main research interest includes image processing

Chi Jing: born in 1980. PhD, professor, PhD supervisor. Her main research interests include computer animation, digital image processing, and information visualization

Chen Yuyan: born in 1997. Master candidate. Her main research interest includes image processing

Zhang Caiming: born in 1955. PhD, professor, PhD supervisor. Senior member of CCF. His main research interests include data mining, computer graphics, and information visualization
Received Date: September 13, 2022
Revised Date: April 27, 2023
Available Online: November 29, 2023

Graphical Abstract

Abstract

Abstract

In the field of image defogging, it is difficult for most defogging models to maintain a balance between accuracy and efficiency. Specifically, high-precision models are often accompanied by complex network structures, and simple network structures often lead to low-quality results. To address the problem, we propose a multi-branch defogging network based on fog concentration classification and dark and bright channel priors. The model uses the defogging networks with different complexity to handle the images with different fog concentrations, which significantly raises the computational efficiency under ensuring the defogging precision. The model is composed of a lightweight foggy image classifier and a multi-branch defogging network. The classifier divides the foggy images into light, medium and dense foggy images and outputs the fog concentration labels. The multi-branch network contains three branches with the same structure but different widths that process three types of fog images separately. We propose a new fog concentration classification method and a new fog concentration classification loss function. The function combines the dark channel characteristics and defogging difficulty of the foggy image with the defogging precision and computational efficiency of the model, so as to obtain a reasonable fog concentration classification, and consequently achieve a good balance of defogging quality and computing power requirements. We propose a new dark channel prior loss function and a new bright channel prior loss function to constrain the multi-branch defogging network, which effectively enhances the defogging precision. Extensive experiments show that the model is beneficial to get better defogging effect with lower network parameters and complexity.
- image defogging,
- fog concentration classification,
- dark channel prior,
- bright channel prior,
- CNN

FullText(HTML)

References (37)

References

[1]	Li Pengyue, Tian Jiandong, Tang Yandong, et al. Deep retinex network for single image dehazing[J]. IEEE Transactions on Image Processing, 2020, 30(99): 1100−1115
[2]	曹绪民,刘春晓. 基于亮度对比度增强与饱和度补偿的快速图像去雾算法[J]. 计算机辅助设计与图形学学报,2018,30(10):1925−1934 doi: 10.3724/SP.J.1089.2018.17000 Cao Xumin, Liu Chunxiao. Fast image defogging algorithm based on luminance contrast enhancement and saturation compensation[J]. Journal of Computer-Aided Design and Computer Graphics, 2018, 30(10): 1925−1934(in Chinese) doi: 10.3724/SP.J.1089.2018.17000
[3]	杨燕,陈高科,周杰. 基于高斯权重衰减的迭代优化去雾算法[J]. 自动化学报,2019,45(4):819−828 Yang Yan, Chen Gaoke, Zhou Jie. Iterative optimization defogging algorithm using Gaussian weight decay[J]. Acta Automatica Sinica, 2019, 45(4): 819−828(in Chinese)
[4]	胡伟,袁国栋,董朝,等. 基于暗通道优先的单幅图像去雾新方法[J]. 计算机研究与发展,2010,47(12):2132−2140 Hu Wei, Yuan Guodong, Dong Zhao, et al. Improved single image dehazing using dark channel prior[J]. Journal of Computer Research and Development, 2010, 47(12): 2132−2140(in Chinese)
[5]	范新南,冶舒悦,史朋飞,等. 改进大气散射模型实现的图像去雾算法[J]. 计算机辅助设计与图形学学报,2019,31(7):1148−1155 doi: 10.3724/SP.J.1089.2019.17458 Fan Xinnan, Ye Shuyue, Shi Pengfei, et al. An image dehazing algorithm based on improved atmospheric scattering model[J]. Journal of Computer-Aided Design and Computer Graphics, 2019, 31(7): 1148−1155(in Chinese) doi: 10.3724/SP.J.1089.2019.17458
[6]	He Kaiming, Sun Jian, Tang Xiaoou, et al. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12): 2341−2353 doi: 10.1109/TPAMI.2010.168
[7]	Xu Yueshu, Guo Xiaoqiang, Wang Haiying, et al. Single image haze removal using light and dark channel prior[C]// Proc of the 5th IEEE Int Conf on Communications in China. Piscataway, NJ: IEEE, 2016: 323−328
[8]	Cai Bolun, Xu Xiangmin, Jia Kui, et al. DehazeNet: An end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11): 5187−5198 doi: 10.1109/TIP.2016.2598681
[9]	Ren Wenqi, Liu Si, Zhang Hua, et al. Single image dehazing via multi-scale convolutional neural networks[C]// Proc of the 14th European Conf on Computer Vision. Berlin: Springer, 2016: 154−169
[10]	Zhang He, Patel V M. Densely connected pyramid dehazing network[C]// Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 3194−3203
[11]	Li Boyi , Peng Xiulian , Wang Zhangyang , et al. AOD-Net: All-in-one dehazing network[C]// Proc of the 16th IEEE Int Conf on Computer Vision. Piscataway, NJ: IEEE, 2017: 4770−4778
[12]	Qin Xu, Wang Zhilin, Bai Yuanchao, et al. FFA-Net: Feature fusion attention network for single image dehazing[C/OL]// Proc of the 34th AAAI Conf on Artificial Intelligence. Palo Alto, CA: AAAI, 2020[2022-08-20].https://doi.org/10.1609/aaai.v34i07.6865
[13]	Liu Xiaohong, Ma Yongrui, Shi Zhihao, et al. GridDehazeNet: Attention-based multi-scale network for image dehazing[C]// Proc of the 17th IEEE/CVF Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2019: 7314−7323
[14]	Dong Hang, Pan Jinshan, Xiang Lei, et al. Multi-scale boosted dehazing network with dense feature fusion[C] //Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2020: 2157−2167
[15]	Wu Haiyan, Qu Yanyun, Lin Shaohui, et al. Contrastive learning for compact single image dehazing[C]// Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 10551−10560
[16]	Yang Yang, Wang Chaoyue, Liu Risheng, et al. Self-augmented unpaired image dehazing via density and depth decomposition[C]//Proc of the 35th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2022: 2037−2046
[17]	Engin D, Genç A, Ekenel H K. Cycle-dehaze: Enhanced cycleGAN for single image dehazing[C] //Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition Workshops. Los Alamitos, CA: IEEE Computer Society, 2018: 825−833
[18]	Chen Dongdong, He Mingming, Fan Qingnan, et al. Gated context aggregation network for image dehazing and deraining[C]// Proc of the 19th IEEE Winter Conf on Applications of Computer Vision. Piscataway, NJ: IEEE, 2019: 1375−1383
[19]	Shao Yuanjie , Li Lerenhan , Ren Wenqi , et al. Domain adaptation for image dehazing[C]// Proc of the 33rd IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2020: 2808−2817
[20]	Asery R, Sunkaria R K, Sharma L D, et al. Fog detection using GLCM based features and SVM[C]// Proc of the 3rd Conf on Advances in Signal Processing. Piscataway, NJ: IEEE, 2016: 72−76
[21]	Wan Jinjin, Qiu Zhenan, Gao Haifeng, et al. Classification of fog situations based on Gaussian mixture model[C]// Proc of the 36th Chinese Control Conf. Piscataway, NJ: IEEE, 2017: 10902−10906
[22]	Zhang Yuanyuan, Sun Guangmin, Ren Qian, et al. Foggy images classification based on features extraction and SVM[C]// Proc of the 3rd Int Conf on Software Engineering and Computer Science. Berlin: Springer, 2013: 142−145
[23]	吴琪,汪小武,章军,等. 利用序数关系实现相对学习的雾图像能见度检测算法[J]. 计算机辅助设计与图形学学报,2020,32(12):1938−1947 doi: 10.3724/SP.J.1089.2020.18250 Wu Qi, Wang Xiaowu, Zhang Jun, et al. Ordinal relation and relative learning for foggy image visibility detection[J]. Journal of Computer-Aided Design and Computer Graphics, 2020, 32(12): 1938−1947(in Chinese) doi: 10.3724/SP.J.1089.2020.18250
[24]	Ronneberger O, Fischer P, Brox T. U-Net: Convolutional networks for biomedical image segmentation[C]// Proc of the 18th Conf on Medical Image Computing and Computer-Assisted Intervention. Berlin: Springer, 2015: 234−241
[25]	Goodfellow I J, Pouget-Abadie J, Mirza M, et al. Generative adversarial nets[C] //Proc of the 27th Int Conf on Neural Information Proc Systems. Cambridge, MA: MIT, 2014: 2672−2680
[26]	Zhu Junyan, Park T, Isola P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C] //Proc of the 16th IEEE Conf on Computer Vision. Los Alamitos, CA: IEEE Computer Society, 2017: 2242−2251
[27]	汪航,田晟兆,唐青,等. 基于多尺度标签传播的小样本图像分类[J]. 计算机研究与发展,2022,59(7):1486−1495 Wang Hang, Tian Shengzhao, Tang Qing, et al. Few-shot image classification based on multi-scale label propagation[J]. Journal of Computer Research and Development, 2022, 59(7): 1486−1495(in Chinese)
[28]	张强,杨吉斌,张雄伟,等. CS-Softmax:一种基于余弦相似性的Softmax损失函数[J]. 计算机研究与发展,2022,59(4):936−949 Zhang Qiang, Yang Jibin, Zhang Xiongwei, et al. CS-Softmax: A cosine similarity-based Softmax loss function[J]. Journal of Computer Research and Development, 2022, 59(4): 936−949(in Chinese)
[29]	Li Boyi, Ren Wenqi, Fu Dengpan, et al. Benchmarking single image dehazing and beyond[J]. IEEE Transactions on Image Processing, 2018, 28(1): 492−505
[30]	Hu Jie, Shen Li, Sun Gang, et al. Squeeze-and-excitation networks[C]// Proc of the 31st IEEE Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2018: 7132−7141
[31]	Lin Min, Chen Qiang, Yan Shuicheng. Network in network[C/OL]// Proc of the 2nd IEEE Int Conf on Learning Representations. 2014[2022-08-20].https://openreview.net/forum?id=ylE6yojDR5yqX
[32]	Kong Xiangtao, Zhao Hengyuan, Qiao Yu, et al. ClassSR: A general framework to accelerate super-resolution networks by data characteristic[C]// Proc of the 34th IEEE/CVF Conf on Computer Vision and Pattern Recognition. Los Alamitos, CA: IEEE Computer Society, 2021: 12016−12025
[33]	Song Yuda, He Zhuqing, Qian Hui, et al. Vision transformers for single image dehazing[J]. arXiv preprint, arXiv: 2204. 03883, 2022
[34]	Ancuti C O, Ancuti C, Timofte R, et al. O-HAZE: A dehazing benchmark with real hazy and haze-free outdoor images[C] //Proc of the 31st IEEE/CVF Conf on Computer Vision and Pattern Recognition Workshops. Los Alamitos, CA: IEEE Computer Society, 2018: 754−762
[35]	Silberman N, Hoiem D, Kohli P, et al. Indoor segmentation and support inference from RGBD images[C]// Proc of the 12th European conf on Computer Vision. Berlin: Springer, 2012: 746−760
[36]	Scharstein D, Szeliski R. High-accuracy stereo depth maps using structured light[C]// Proc of the 13th IEEE Computer Society Conf on Computer Vision and Pattern Recognition. Los Alamitos , CA: IEEE Computer Society, 2003: 195−202
[37]	Kingma D P, Ba J. Adam: A method for stochastic optimization[C/OL]// Proc of the 3rd Int Conf on Learning Representations. 2015[2022-08-20]. https://openreview.net/forum?id=8gmWwjFyLj

[1]	Xie Kunpeng, Lu Ye, Jin Zongming, Liu Yiqing, Gong Cheng, Chen Xinwei, Li Tao. FAQ-CNN: A Flexible Acceleration Framework for Quantized Convolutional Neural Networks on Embedded FPGAs[J]. Journal of Computer Research and Development, 2022, 59(7): 1409-1427. DOI: 10.7544/issn1000-1239.20210142
[2]	Liu Yanxiao, Wu Ping, Sun Qindong. Secret Image Sharing Schemes Based on Region Convolution Neural Network[J]. Journal of Computer Research and Development, 2021, 58(5): 1065-1074. DOI: 10.7544/issn1000-1239.2021.20200898
[3]	Cheng Yan, Yao Leibo, Zhang Guanghe, Tang Tianwei, Xiang Guoxiong, Chen Haomai, Feng Yue, Cai Zhuang. Text Sentiment Orientation Analysis of Multi-Channels CNN and BiGRU Based on Attention Mechanism[J]. Journal of Computer Research and Development, 2020, 57(12): 2583-2595. DOI: 10.7544/issn1000-1239.2020.20190854
[4]	Shi Leyi, Zhu Hongqiang, Liu Yihao, Liu Jia. Intrusion Detection of Industrial Control System Based on Correlation Information Entropy and CNN-BiLSTM[J]. Journal of Computer Research and Development, 2019, 56(11): 2330-2338. DOI: 10.7544/issn1000-1239.2019.20190376
[5]	Shi Wenhao, Meng Jun, Zhang Peng, Liu Chanjuan. Prediction of miRNA-lncRNA Interaction by Combining CNN and Bi-LSTM[J]. Journal of Computer Research and Development, 2019, 56(8): 1652-1660. DOI: 10.7544/issn1000-1239.2019.20190128
[6]	Cheng Xiaoyang, Zhan Yongzhao, Mao Qirong, Zhan Zhicai. Video Semantic Analysis Based on Topographic Sparse Pre-Training CNN[J]. Journal of Computer Research and Development, 2018, 55(12): 2703-2714. DOI: 10.7544/issn1000-1239.2018.20170579
[7]	Zhong Zhiquan, Yuan Jin, Tang Xiaoying. Left-vs-Right Eye Discrimination Based on Convolutional Neural Network[J]. Journal of Computer Research and Development, 2018, 55(8): 1667-1673. DOI: 10.7544/issn1000-1239.2018.20180215
[8]	Chen Ke, Liang Bin, Ke Wende, Xu Bo, Zeng Guochao. Chinese Micro-Blog Sentiment Analysis Based on Multi-Channels Convolutional Neural Networks[J]. Journal of Computer Research and Development, 2018, 55(5): 945-957. DOI: 10.7544/issn1000-1239.2018.20170049
[9]	Du Yumeng, Zhang Weinan, Liu Ting. Topic Augmented Convolutional Neural Network for User Interest Recognition[J]. Journal of Computer Research and Development, 2018, 55(1): 188-197. DOI: 10.7544/issn1000-1239.2018.20160892
[10]	Hu Wei, Yuan Guodong, Dong Zhao, Shu Xueming. Improved Single Image Dehazing Using Dark Channel Prior[J]. Journal of Computer Research and Development, 2010, 47(12).