Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN

doi:10.11947/j.AGCS.2021.20200589

Abstract

Abstract: In order to solve the problem that the parameters of pulse-coupled neural network can't be adjusted adaptively in pan-sharpening image fusion, a non-subsampled shearlet transform remote sensing image fusion method based on the combination of parametric-adaptive pulse coupled neural network model and energy-attributing fusion strategy is proposed. First, the high and low frequency coefficients are obtained by extracting the Y luminance component of the multispectral image YUV color space transform and transforming it with the panchromatic image. Then, aiming at the low-frequency sub-band coefficients are fused by the EA method, the high-frequency sub-band coefficients are obtained by the PA-PCNN model to determine the optimal PCNN model, and then the high-frequency sub-band coefficients are fused; finally, the fusion image is obtained by inverse transformation of NSST and YUV. In this paper, six objective quality indexes, such as spatial frequency, relative dimensionless global error, ERGAS, correlation coefficient, visual information fidelity for fusion, gradient-based fusion performance and structural similarity index, are selected to evaluate the spectral and spatial detail information of the fused images, compared with SE, DGIF, COF and PA-PCNN fusion methods, the proposed method is validated by using multiple sets of high-and low-resolution panchromatic and multispectral remote sensing images, the results show that this method is generally superior to the traditional fusion method of panchromatic and multispectral remote sensing images in objective evaluation and visual perception.

Key words: image fusion, non-subsampled shearlet transform, parameter-adaptive pulse-coupled neural network, panchromatic image, multispectral image

CLC Number:

TP751
P237

CHENG Feifei, FU Zhitao, HUANG Liang, CHEN Pengdi, HUANG Kun. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1380-1389.

References

[1] GHAMISI P, RASTI B, YOKOYA N, et al. Multisource and multitemporal data fusion in remote sensing:a comprehensive review of the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(1):6-39.
[2] 黄波, 赵涌泉. 多源卫星遥感影像时空融合研究的现状及展望[J]. 测绘学报, 2017, 46(10):1492-1499. DOI:10.11947/j.AGCS.2017.20170376. HUANG Bo, ZHAO Yongquan. Research status and prospect of spatiotemporal fusion of multi-source satellite remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1492-1499.DOI:10.11947/j.AGCS.2017.20170376.
[3] Yun ZHANG. 智能摄影测量和图像处理在高分辨率光学遥感影像处理中的应用:CRC-AGIP实验室的案例[J]. 测绘学报, 2018, 47(6):722-729. DOI:10.11947/j.AGCS.2018.20180011. ZHANG Yun. Smart photogrammetric and remote sensing image processing for optical very high resolution images-examples from the CRC-AGIP lab at UNB[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):722-729.DOI:10.11947/j.AGCS.2018.20180011.
[4] FANG Faming, LI Fang, SHEN Chaomin, et al. A variational approach for pan-sharpening[J]. IEEE Transactions on Image Processing, 2013, 22(7):2822-2834.
[5] 王密, 何鲁晓, 程宇峰, 等. 自适应高斯滤波与SFIM模型相结合的全色多光谱影像融合方法[J]. 测绘学报, 2018, 47(1):82-90. DOI:10.11947/j.AGCS.2018.20170421. WANG Mi, HE Luxiao, CHENG Yufeng, et al. Panchromatic and multi-spectral fusion method combined with adaptive Gaussian filter and SFIM model[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1):82-90.DOI:10.11947/j.AGCS.2018.20170421.
[6] 王仲妮, 余先川, 张立保. 一种基于整数小波变换的遥感影像融合算法[J]. 光电子·激光, 2008, 19(11):1542-1545. WANG Zhongni, YU Xianchuan, ZHANG Libao. A remote sensing image fusion algorithm based on integer wavelet transform[J]. Journal of Optoelectronics·Laser, 2008, 19(11):1542-1545.
[7] 陈应霞, 陈艳, 刘丛. 遥感影像融合AIHS转换与粒子群优化算法[J]. 测绘学报, 2019, 48(10):1296-1304. DOI:10.11947/j.AGCS.2019.20180509 CHEN Yingxia, CHEN Yan, LIU Cong. Joint AIHS and particle swarm optimization for Pan-sharpening[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10):1296-1304.DOI:10.11947/j.AGCS.2019.20180509。
[8] ZHANG Yun, HONG Gang. An IHS and wavelet integrated approach to improve pan-sharpening visual quality of natural colour IKONOS and QuickBird images[J]. Information Fusion, 2005, 6(3):225-234.
[9] DELLEJI T, KALLEL A, BEN HAMIDA A. Multispectral image adaptive pansharpening based on wavelet transformation and NMDB approaches[J]. International Journal of Remote Sensing, 2014, 35(19):7069-7098.
[10] 李晖晖, 郭雷, 刘坤. 基于曲波变换的遥感图像融合研究[J]. 光电子激光, 2008, 19(3):400-403, 411. LI Huihui, GUO Lei, LIU Kun. Remote sensing image fusion based on curvelet transform[J]. Journal of OptoelectronicsLaser, 2008, 19(3):400-403, 411.
[11] 赵学军, 雷书彧, 滕尚志. 粒子群优化Contourlet变换的遥感影像融合算法[J]. 北京邮电大学学报, 2015, 38(2):118-121. ZHAO Xuejun, LEI Shuyu, TENG Shangzhi. Particle swarm optimization contourlet transform of remote sensing image fusion algorithm research[J]. Journal of Beijing University of Posts and Telecommunications, 2015, 38(2):118-121.
[12] 孙岩, 赵春晖, 江凌. 基于NSCT和PCA变换域的遥感图像融合算法[J]. 沈阳工业大学学报, 2011, 33(3):308-314. SUN Yan, ZHAO Chunhui, JIANG Ling. Remote sensing image fusion algorithm based on NSCT and PCA transform-domain[J]. Journal of Shenyang University of Technology, 2011, 33(3):308-314.
[13] 吴一全, 陶飞翔. 改进投影梯度NMF的NSST域多光谱与全色图像融合[J]. 光学学报, 2015, 35(4):139-148. WU Yiquan, TAO Feixiang. Multispectral and panchromatic image fusion based on improved projected gradient NMF in NSST domain[J]. Acta Optica Sinica, 2015, 35(4):139-148.
[14] SULAIMAN A G, ELASHMAWI W H, EL-TAWEL G S. A robust pan-sharpening scheme for improving resolution of satellite images in the domain of the nonsubsampled shearlet transform[J]. Sensing and Imaging, 2019, 21(1):1-27.
[15] 吴一全, 王志来. 混沌蜂群优化的NSST域多光谱与全色图像融合[J]. 遥感学报, 2017, 21(4):549-557. WU Yiquan, WANG Zhilai. Multispectral and panchromatic image fusion using chaotic Bee Colony optimization in NSST domain[J]. Journal of Remote Sensing, 2017, 21(4):549-557.
[16] 焦姣, 吴玲达. 形态学滤波和改进PCNN的NSST域多光谱与全色图像融合[J]. 中国图象图形学报, 2019, 24(3):435-446. JIAO Jiao, WU Lingda. Fusion of multispectral and panchromatic images via morphological filter and improved PCNN in NSST domain[J]. Journal of Image and Graphics, 2019, 24(3):435-446.
[17] CONNAH D, DREW M S, FINLAYSON G D. Spectral edge image fusion:Theory and applications[C]//Proceedings of the 13th European Conference on Computer Vision (ECCV)Zurich, SWITZERLAND:ECCV,2014:65-80.
[18] JIAN Lihua, YANG Xiaomin, WU Wei, et al. Pansharpening using a guided image filter based on dual-scale detail extraction[J]. Journal of Ambient Intelligence and Humanized Computing, 2018:1-15.
[19] TAN Wei, XIANG Pei, ZHANG Jiajia, et al. Remote sensing image fusion via boundary measured dual-channel PCNN in multi-scale morphological gradient domain[J]. IEEE Access, 2020, 8:42540-42549.
[20] YIN Ming, LIU Xiaoning, LIU Yu, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1):49-64.
[21] ECKHORN R, REITBOCK H J, ARNDT M, et al. A neural network for feature linking via synchronous activity:results from cat visual cortex and from simulations[J]. Models of Brain Function. 1989:255-272.
[22] WANG Zhaobin, MA Yide, CHENG Feiyan, et al. Review of pulse-coupled neural networks[J]. Image & Vision Computing, 2010, 28(1):5-13.
[23] CHEN Yuli, PARK S K, MA Yide, et al. A new automatic parameter setting method of a simplified PCNN for image segmentation[J]. IEEE Transactions on Neural Networks, 2011, 22(6):880-892.
[24] 王昭, 杜庆治, 董安勇, 等. 基于非下采样剪切波变换的红外与可见光图像融合[J]. 光电子·激光, 2020, 31(10):1062-1073. WANG Zhao, DU Qingzhi, DONG Anyong, et al. Infrared and visible image fusion based on nonsubsampled shearlet transform[J]. Journal of Optoelectronics·Laser, 2020, 31(10):1062-1073.
[25] LI Jiayuan, HU Qingwu, AI Mingyao. Multispectral and panchromatic image fusion based on spatial consistency[J]. International Journal of Remote Sensing, 2018, 39(4):1017-1041.
[26] TAN Wei, TIWARI P, PANDEY H M, et al. Multimodal medical image fusion algorithm in the era of big data[J]. Neural Computing and Applications, 2020(3):1-21.