红外技术, 2020, 42 (7): 660, 网络出版: 2020-08-18   

基于卷积神经网络的红外与可见光图像融合

Infrared and Visible Image Fusion Based on Convolutional Neural Network
作者单位
1 昆明理工大学信息工程与自动化学院,云南 昆明 650500
2 昆明北方红外技术股份有限公司,云南昆明 650500
摘要
非下采样剪切波变换(NSST)域中低频子带的融合需要人工给定融合模式,因此未能充分捕获源图像的空间连续性和轮廓细节信息。针对上述问题,提出了基于深度卷积神经网络的红外与可见光图像融合算法。首先,使用孪生双通道卷积神经网络学习NSST 域低频子带的特征来输出衡量子带空间细节信息的特征图。然后,根据高斯滤波处理的特征图设计了基于局部相似性的测量函数来自适应地调整NSST 域低频子带的融合模式。最后,根据NSST 域高频子带的方差、局部区域能量以及可见度特征来自适应地设置脉冲耦合神经网络参数完成NSST 域高频子带的融合。实验结果表明:该算法QAB/F 指标略弱于对比算法,但SF、SP、SSIM 以及VIFF 指标分别提高了约50.42%、14.25%、7.91%以及61.67%,有效地解决了低频子带融合模式给定的问题,同时又克服了手动设置PCNN 参数的缺陷。
Abstract
The fusion of the low-frequency subband in the non-subsampled shearlet transform (NSST) domain requires artificially obtained fusion modes; thus, the spatial continuity and contour detail information of the source image are not adequately captured. An infrared and visible image fusion algorithm based on a convolutional neural network is proposed to solve this problem. First, the Siamese convolutional neural network is used to learn the characteristics of the low-frequency subband in the NSST domain and output a feature map that measures the spatial detail information of the subbands. Then, on the basis of the feature map obtained by Gaussian filter processing, a local-similarity-based measurement function is designed to adaptively adjust the fusion mode of the low-frequency subband in the NSST domain. Finally, on the basis of the variance of the high-frequency subband in the NSST domain, the local region energy, and the visibility characteristics, the pulse-coupled neural network (PCNN) parameters are adaptively set to complete the fusion of the high-frequency subband in the NSST domain. Experimental results show that the QAB/F index of the algorithm is slightly lower than that of the comparison algorithm. However, the spatial frequency, SP, structural similarity, and visual information fidelity for fusion are improved by approximately 50.42%, 14.25%, 7.91%, and 61.67%, respectively, which indicates that the method effectively solves the low-frequency subband fusion mode. It also eliminates the need to manually set the PCNN parameters to solve this problem.

董安勇, 杜庆治, 苏斌, 赵文博, 于闻. 基于卷积神经网络的红外与可见光图像融合[J]. 红外技术, 2020, 42(7): 660. DONG Anyong, DU Qingzhi, SU Bin, ZHAO Wenbo, YU Wen. Infrared and Visible Image Fusion Based on Convolutional Neural Network[J]. Infrared Technology, 2020, 42(7): 660.

本文已被 7 篇论文引用
被引统计数据来源于中国光学期刊网
引用该论文: TXT   |   EndNote

相关论文

加载中...

关于本站 Cookie 的使用提示

中国光学期刊网使用基于 cookie 的技术来更好地为您提供各项服务,点击此处了解我们的隐私策略。 如您需继续使用本网站,请您授权我们使用本地 cookie 来保存部分信息。
全站搜索
您最值得信赖的光电行业旗舰网络服务平台!