红外技术, 2020, 42 (7): 660, 网络出版: 2020-08-18
基于卷积神经网络的红外与可见光图像融合
Infrared and Visible Image Fusion Based on Convolutional Neural Network
摘要
非下采样剪切波变换(NSST)域中低频子带的融合需要人工给定融合模式,因此未能充分捕获源图像的空间连续性和轮廓细节信息。针对上述问题,提出了基于深度卷积神经网络的红外与可见光图像融合算法。首先,使用孪生双通道卷积神经网络学习NSST 域低频子带的特征来输出衡量子带空间细节信息的特征图。然后,根据高斯滤波处理的特征图设计了基于局部相似性的测量函数来自适应地调整NSST 域低频子带的融合模式。最后,根据NSST 域高频子带的方差、局部区域能量以及可见度特征来自适应地设置脉冲耦合神经网络参数完成NSST 域高频子带的融合。实验结果表明:该算法QAB/F 指标略弱于对比算法,但SF、SP、SSIM 以及VIFF 指标分别提高了约50.42%、14.25%、7.91%以及61.67%,有效地解决了低频子带融合模式给定的问题,同时又克服了手动设置PCNN 参数的缺陷。
Abstract
The fusion of the low-frequency subband in the non-subsampled shearlet transform (NSST) domain requires artificially obtained fusion modes; thus, the spatial continuity and contour detail information of the source image are not adequately captured. An infrared and visible image fusion algorithm based on a convolutional neural network is proposed to solve this problem. First, the Siamese convolutional neural network is used to learn the characteristics of the low-frequency subband in the NSST domain and output a feature map that measures the spatial detail information of the subbands. Then, on the basis of the feature map obtained by Gaussian filter processing, a local-similarity-based measurement function is designed to adaptively adjust the fusion mode of the low-frequency subband in the NSST domain. Finally, on the basis of the variance of the high-frequency subband in the NSST domain, the local region energy, and the visibility characteristics, the pulse-coupled neural network (PCNN) parameters are adaptively set to complete the fusion of the high-frequency subband in the NSST domain. Experimental results show that the QAB/F index of the algorithm is slightly lower than that of the comparison algorithm. However, the spatial frequency, SP, structural similarity, and visual information fidelity for fusion are improved by approximately 50.42%, 14.25%, 7.91%, and 61.67%, respectively, which indicates that the method effectively solves the low-frequency subband fusion mode. It also eliminates the need to manually set the PCNN parameters to solve this problem.
董安勇, 杜庆治, 苏斌, 赵文博, 于闻. 基于卷积神经网络的红外与可见光图像融合[J]. 红外技术, 2020, 42(7): 660. DONG Anyong, DU Qingzhi, SU Bin, ZHAO Wenbo, YU Wen. Infrared and Visible Image Fusion Based on Convolutional Neural Network[J]. Infrared Technology, 2020, 42(7): 660.