Remote sensing image fusion method combining improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network

doi:10.11947/j.AGCS.2023.20220541

Abstract

Abstract: The fusion of backscattering information of SAR images and spectral information of optical images is one of the important means to improve the accuracy of land cover classification, and multi-scale transform is an effective fusion method. However, the fusion rules of multi-scale transform method are usually designed based on local feature information and pulse coupled neural network models, and there are some problems such as limited ability to extract structural and detailed information, complex parameter settings of pulse coupled neural network and poor spatial correlation. To this end, a remote sensing image fusion method based on improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network (ULPCNN) is proposed in this paper. This method combines the component substitution method and the multi-scale transform method. Firstly, the multi-spectral image is transformed by IHS to obtain the intensity component I, and then the intensity component I and SAR image are decomposed by non-subsampled shearlet transform (NSST) to obtain high and low frequency sub-bands.Secondly, a fusion rule combining weighted local energy and weighted sum of eight-neighborhood-based modified Laplacian is used for low frequency sub-bands, a fusion rule of the parameter adaptive dual-channel ULPCNN method is used for the high frequency sub-bands, the multi-scale morphological gradient of the high-frequency sub-band is used as the link strength, and the adaptive representation of other parameters is realized according to the OTSU threshold and image strength. Finally, the NSST inverse transform and the IHS inverse transform are performed in turn to obtain the fusion image, and the random forest classifier is selected to classify the fusion image for land cover. The experimental results show that the proposed method has the overall best performance in eleven fusion evaluation indexes and land cover classification accuracy compared with 13 other methods. The overall accuracy and Kappa coefficient of land cover classification improved by 8.350% and 0.107, respectively, in area 1, and by 6.896% and 0.091, respectively, in area 2 compared with those of the original multi-spectral images.

Key words: remote sensing image fusion, parameter adaptive dual-channel ULPCNN, non-subsampled shearlet transform, improved Laplacian energy

CLC Number:

P237

GONG Xunqiang, HOU Zhaoyang, LÜ Kaiyun, LU Tieding, XIA Yuanping, LI Weijun. Remote sensing image fusion method combining improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(11): 1892-1905.

References

[1] 李树涛, 李聪妤, 康旭东. 多源遥感图像融合发展现状与未来展望[J]. 遥感学报, 2021, 25(1):148-166. LI Shutao, LI Congyu, KANG Xudong. Development status and future prospects of multi-source remote sensing image fusion[J]. National Remote Sensing Bulletin, 2021, 25(1):148-166.
[2] 赵诣, 蒋弥. 极化SAR参数优化与光学波谱相结合的面向对象土地覆盖分类[J]. 测绘学报, 2019, 48(5):609-617. DOI:10.11947/j.AGCS.2019.20170746. ZHAO Yi, JIANG Mi. Integration of SAR polarimetric parameters and multi-spectral data for object-based land cover classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):609-617. DOI:10.11947/j.AGCS.2019.20170746.
[3] XU Zhe, ZHU Jinbiao, GENG Jie, et al. Triplet attention feature fusion network for SAR and optical image land cover classification[C]//Proceedings of 2021 IEEE International Geoscience and Remote Sensing Symposium. Brussels:IEEE, 2021:4256-4259.
[4] KULKARNI S C, REGE P P. Pixel level fusion techniques for SAR and optical images:a review[J]. Information Fusion, 2020, 59:13-29.
[5] 陈应霞, 陈艳, 刘丛. 遥感影像融合AIHS转换与粒子群优化算法[J]. 测绘学报, 2019, 48(10):1296-1304. DOI:10.11947/j.AGCS.2019.20180509. CHEN Yingxia, CHEN Yan, LIU Cong. Joint AIHS and particle swarm optimization for pan-sharpening[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10):1296-1304. DOI:10.11947/j.AGCS.2019.20180509.
[6] BATUR E, MAKTAV D. Assessment of surface water quality by using satellite images fusion based on PCA method in the Lake Gala, Turkey[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2983-2989.
[7] 童莹萍, 全英汇, 冯伟, 等. 基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法[J]. 系统工程与电子技术, 2022, 44(7):2074-2083. TONG Yingping, QUAN Yinghui, FENG Wei, et al. Multi-source remote sensing image fusion method based on spatial-spectrum information collaboration and Gram-Schmidt transform[J]. Systems Engineering and Electronics, 2022, 44(7):2074-2083.
[8] WU Yuanyuan, FENG Siling, LIN Cong, et al. A three stages detail injection network for remote sensing images pansharpening[J]. Remote Sensing, 2022, 14(5):1077.
[9] DANIEL E. Optimum wavelet-based homomorphic medical image fusion using hybrid genetic-grey wolf optimization algorithm[J]. IEEE Sensors Journal, 2018, 18(16):6804-6811.
[10] ARIF M, WANG Guojun. Fast curvelet transform through genetic algorithm for multimodal medical image fusion[J]. Soft Computing, 2020, 24(3):1815-1836.
[11] AISHWARYA N, BENNILA THANGAMMAL C. Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary[J]. Infrared Physics & Technology, 2018, 93:300-309.
[12] WANG Zeyu, LI Xiongfei, DUAN Haoran, et al. Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform[J]. Expert Systems with Applications, 2021, 171:114574.
[13] 侯昭阳, 吕开云, 龚循强, 等. 一种结合低级视觉特征和PAPCNN的NSST域遥感影像融合方法[J]. 武汉大学学报(信息科学版), 2023, 48(6):960-969. HOU Zhaoyang, LÜ Kaiyun, GONG Xunqiang, et al. Remote sensing image fusion based on low-level visual features and PAPCNN in NSST domain[J]. Geomatics and Information Science of Wuhan University, 2023, 48(6):960-969.
[14] ZHANG Yi, JIN Mingming, HUANG Gang. Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map[J]. Biomedical Signal Processing and Control, 2022, 74:103535.
[15] ZHANG Yu, BAI Xiangzhi, WANG Tao. Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure[J]. Information Fusion, 2017, 35:81-101.
[16] YIN Ming, LIU Xiaoning, LIU Yu, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1):49-64.
[17] HOU Zhaoyang, LÜ Kaiyun, GONG Xunqiang, et al. A remote sensing image fusion method combining low-level visual features and parameter-adaptive dual-channel pulse-coupled neural network[J]. Remote Sensing, 2023, 15(2):344.
[18] LIU Yu, LIU Shuping, WANG Zengfu. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24:147-164.
[19] LIU Yu, WANG Zengfu. Simultaneous image fusion and denoising with adaptive sparse representation[J]. IET Image Processing, 2015, 9(5):347-357.
[20] LIU Yu, CHEN Xun, WARD R K, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12):1882-1886.
[21] LIU Yu, CHEN Xun, WARD R K, et al. Medical image fusion via convolutional sparsity based morphological component analysis[J]. IEEE Signal Processing Letters, 2019, 26(3):485-489.
[22] SHREYAMSHA KUMAR B K. Image fusion based on pixel significance using cross bilateral filter[J]. Signal, Image and Video Processing, 2015, 9(5):1193-1204.
[23] JIAN Lihua, YANG Xiaomin, ZHOU Zhili, et al. Multi-scale image fusion through rolling guidance filter[J]. Future Generation Computer Systems, 2018, 83:310-325.
[24] MA Jiayi, CHEN Chen, LI Chang, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31:100-109.
[25] 成飞飞, 付志涛, 黄亮, 等. 结合自适应PCNN的非下采样剪切波遥感影像融合[J]. 测绘学报, 2021, 50(10):1380-1389. DOI:10.11947/j.AGCS.2021.20200589. CHENG Feifei, FU Zhitao, HUANG Liang, et al. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10):1380-1389.DOI:10.11947/j.AGCS.2021.20200589.
[26] TAN Wei, TIWARI P, PANDEY H M, et al. Multimodal medical image fusion algorithm in the era of big data[J/OL].[2022-09-15]. https://doi.org/10.1007/s00521-020-05173-2.
[27] PANIGRAHY C, SEAL A, MAHATO N K. Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion[J]. Optics and Lasers in Engineering, 2020, 133:106141.
[28] TALUKDAR S, SINGHA P, MAHATO S, et al. Land-use land-cover classification by machine learning classifiers for satellite observations-a review[J]. Remote Sensing, 2020, 12(7):1135.
[29] MA Lei, LI Manchun, MA Xiaoxue, et al. A review of supervised object-based land-cover image classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130:277-293.
[30] BELGIU M, DRAGUT L. Random forest in remote sensing:a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114:24-31.