结合改进Laplacian能量和参数自适应双通道ULPCNN的遥感影像融合方法

doi:10.11947/j.AGCS.2023.20220541

测绘学报 ›› 2023, Vol. 52 ›› Issue (11): 1892-1905.doi: 10.11947/j.AGCS.2023.20220541

结合改进Laplacian能量和参数自适应双通道ULPCNN的遥感影像融合方法

龚循强^1,2, 侯昭阳^1,2, 吕开云^1,2, 鲁铁定^1,2, 夏元平^1,2, 李威俊³

1. 东华理工大学自然资源部环鄱阳湖区域矿山环境监测与治理重点实验室, 江西南昌 330013;
2. 东华理工大学测绘与空间信息工程学院, 江西南昌 330013;
3. 江西省地质局第六地质大队, 江西鹰潭 335000

收稿日期:2022-09-15 修回日期:2023-03-20 发布日期:2023-12-15
通讯作者: 侯昭阳 E-mail:zyhou@ecut.edu.cn
作者简介:龚循强(1988-),男,博士,副教授,研究方向为卫星遥感影像智能处理。E-mail:xqgong1988@ecut.edu.cn
基金资助:
国家自然科学基金(42101457;42061077;42174055);自然资源部环鄱阳湖区域矿山环境监测与治理重点实验室开放基金(MEMI-2021-2022-13);江西生态文明建设制度研究中心项目(JXST2104)

Remote sensing image fusion method combining improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network

GONG Xunqiang^1,2, HOU Zhaoyang^1,2, LÜ Kaiyun^1,2, LU Tieding^1,2, XIA Yuanping^1,2, LI Weijun³

1. Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake of Ministry of Natural Resources, East China University of Technology, Nanchang 330013, China;
2. School of Surveying and Geoinformation Engineering, East China University of Technology, Nanchang 330013, China;
3. The Sixth Geological Brigade, Jiangxi Bureau of Geology, Yingtan 335000, China

Received:2022-09-15 Revised:2023-03-20 Published:2023-12-15
Supported by:
The National Natural Science Foundation of China (Nos. 42101457;42061077;42174055);The Open Fund of Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake of Ministry of Natural Resources (No. MEMI-2021-2022-13);The Project of Research Center for Ecological Civilization Construction System of Jiangxi Province (No. JXST2104)

摘要/Abstract

摘要： 融合SAR影像的后向散射信息和光学影像的光谱信息是提高土地覆盖分类精度的重要手段之一,其中多尺度变换是一种有效的融合方法。然而,多尺度变换方法的融合规则通常根据局部特征信息和脉冲耦合神经网络模型进行设计,存在结构信息和细节信息提取能力有限,以及脉冲耦合神经网络参数设置复杂和空间相关性差等问题。为此,本文提出一种结合改进Laplacian能量和参数自适应双通道单位连接脉冲耦合神经网络(ULPCNN)的遥感影像融合方法。该方法混合成分替换方法和多尺度变换方法,首先对多光谱影像进行IHS变换得到亮度分量I,将亮度分量I与SAR影像通过非下采样剪切波变换(NSST)分解得到高低频子带。然后对低频子带采用结合加权局部能量和八邻域修正拉普拉斯加权和的融合规则,同时对高频子带采用参数自适应双通道ULPCNN的融合规则,将高频子带的多尺度形态梯度作为链接强度,并根据OTSU阈值和影像强度来实现其他参数的自适应表示。最后依次进行NSST重建和IHS逆变换得到融合影像,并选择随机森林分类器对融合影像进行土地覆盖分类。试验结果表明,本文方法相较于13种其他方法在11个融合评价指标和土地覆盖分类精度上总体表现最佳,土地覆盖分类的总体精度和Kappa系数在区域1中比原多光谱影像分别提高了8.350%和0.107,在区域2中比原多光谱影像分别提高了6.896%和0.091。

关键词: 遥感影像融合, 参数自适应双通道ULPCNN, 非下采样剪切波变换, 改进Laplacian能量

Abstract: The fusion of backscattering information of SAR images and spectral information of optical images is one of the important means to improve the accuracy of land cover classification, and multi-scale transform is an effective fusion method. However, the fusion rules of multi-scale transform method are usually designed based on local feature information and pulse coupled neural network models, and there are some problems such as limited ability to extract structural and detailed information, complex parameter settings of pulse coupled neural network and poor spatial correlation. To this end, a remote sensing image fusion method based on improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network (ULPCNN) is proposed in this paper. This method combines the component substitution method and the multi-scale transform method. Firstly, the multi-spectral image is transformed by IHS to obtain the intensity component I, and then the intensity component I and SAR image are decomposed by non-subsampled shearlet transform (NSST) to obtain high and low frequency sub-bands.Secondly, a fusion rule combining weighted local energy and weighted sum of eight-neighborhood-based modified Laplacian is used for low frequency sub-bands, a fusion rule of the parameter adaptive dual-channel ULPCNN method is used for the high frequency sub-bands, the multi-scale morphological gradient of the high-frequency sub-band is used as the link strength, and the adaptive representation of other parameters is realized according to the OTSU threshold and image strength. Finally, the NSST inverse transform and the IHS inverse transform are performed in turn to obtain the fusion image, and the random forest classifier is selected to classify the fusion image for land cover. The experimental results show that the proposed method has the overall best performance in eleven fusion evaluation indexes and land cover classification accuracy compared with 13 other methods. The overall accuracy and Kappa coefficient of land cover classification improved by 8.350% and 0.107, respectively, in area 1, and by 6.896% and 0.091, respectively, in area 2 compared with those of the original multi-spectral images.

Key words: remote sensing image fusion, parameter adaptive dual-channel ULPCNN, non-subsampled shearlet transform, improved Laplacian energy

中图分类号:

P237

龚循强, 侯昭阳, 吕开云, 鲁铁定, 夏元平, 李威俊. 结合改进Laplacian能量和参数自适应双通道ULPCNN的遥感影像融合方法[J]. 测绘学报, 2023, 52(11): 1892-1905.

GONG Xunqiang, HOU Zhaoyang, LÜ Kaiyun, LU Tieding, XIA Yuanping, LI Weijun. Remote sensing image fusion method combining improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(11): 1892-1905.

参考文献

[1] 李树涛, 李聪妤, 康旭东. 多源遥感图像融合发展现状与未来展望[J]. 遥感学报, 2021, 25(1):148-166. LI Shutao, LI Congyu, KANG Xudong. Development status and future prospects of multi-source remote sensing image fusion[J]. National Remote Sensing Bulletin, 2021, 25(1):148-166.
[2] 赵诣, 蒋弥. 极化SAR参数优化与光学波谱相结合的面向对象土地覆盖分类[J]. 测绘学报, 2019, 48(5):609-617. DOI:10.11947/j.AGCS.2019.20170746. ZHAO Yi, JIANG Mi. Integration of SAR polarimetric parameters and multi-spectral data for object-based land cover classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):609-617. DOI:10.11947/j.AGCS.2019.20170746.
[3] XU Zhe, ZHU Jinbiao, GENG Jie, et al. Triplet attention feature fusion network for SAR and optical image land cover classification[C]//Proceedings of 2021 IEEE International Geoscience and Remote Sensing Symposium. Brussels:IEEE, 2021:4256-4259.
[4] KULKARNI S C, REGE P P. Pixel level fusion techniques for SAR and optical images:a review[J]. Information Fusion, 2020, 59:13-29.
[5] 陈应霞, 陈艳, 刘丛. 遥感影像融合AIHS转换与粒子群优化算法[J]. 测绘学报, 2019, 48(10):1296-1304. DOI:10.11947/j.AGCS.2019.20180509. CHEN Yingxia, CHEN Yan, LIU Cong. Joint AIHS and particle swarm optimization for pan-sharpening[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(10):1296-1304. DOI:10.11947/j.AGCS.2019.20180509.
[6] BATUR E, MAKTAV D. Assessment of surface water quality by using satellite images fusion based on PCA method in the Lake Gala, Turkey[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2983-2989.
[7] 童莹萍, 全英汇, 冯伟, 等. 基于空谱信息协同与Gram-Schmidt变换的多源遥感图像融合方法[J]. 系统工程与电子技术, 2022, 44(7):2074-2083. TONG Yingping, QUAN Yinghui, FENG Wei, et al. Multi-source remote sensing image fusion method based on spatial-spectrum information collaboration and Gram-Schmidt transform[J]. Systems Engineering and Electronics, 2022, 44(7):2074-2083.
[8] WU Yuanyuan, FENG Siling, LIN Cong, et al. A three stages detail injection network for remote sensing images pansharpening[J]. Remote Sensing, 2022, 14(5):1077.
[9] DANIEL E. Optimum wavelet-based homomorphic medical image fusion using hybrid genetic-grey wolf optimization algorithm[J]. IEEE Sensors Journal, 2018, 18(16):6804-6811.
[10] ARIF M, WANG Guojun. Fast curvelet transform through genetic algorithm for multimodal medical image fusion[J]. Soft Computing, 2020, 24(3):1815-1836.
[11] AISHWARYA N, BENNILA THANGAMMAL C. Visible and infrared image fusion using DTCWT and adaptive combined clustered dictionary[J]. Infrared Physics & Technology, 2018, 93:300-309.
[12] WANG Zeyu, LI Xiongfei, DUAN Haoran, et al. Medical image fusion based on convolutional neural networks and non-subsampled contourlet transform[J]. Expert Systems with Applications, 2021, 171:114574.
[13] 侯昭阳, 吕开云, 龚循强, 等. 一种结合低级视觉特征和PAPCNN的NSST域遥感影像融合方法[J]. 武汉大学学报(信息科学版), 2023, 48(6):960-969. HOU Zhaoyang, LÜ Kaiyun, GONG Xunqiang, et al. Remote sensing image fusion based on low-level visual features and PAPCNN in NSST domain[J]. Geomatics and Information Science of Wuhan University, 2023, 48(6):960-969.
[14] ZHANG Yi, JIN Mingming, HUANG Gang. Medical image fusion based on improved multi-scale morphology gradient-weighted local energy and visual saliency map[J]. Biomedical Signal Processing and Control, 2022, 74:103535.
[15] ZHANG Yu, BAI Xiangzhi, WANG Tao. Boundary finding based multi-focus image fusion through multi-scale morphological focus-measure[J]. Information Fusion, 2017, 35:81-101.
[16] YIN Ming, LIU Xiaoning, LIU Yu, et al. Medical image fusion with parameter-adaptive pulse coupled neural network in nonsubsampled shearlet transform domain[J]. IEEE Transactions on Instrumentation and Measurement, 2019, 68(1):49-64.
[17] HOU Zhaoyang, LÜ Kaiyun, GONG Xunqiang, et al. A remote sensing image fusion method combining low-level visual features and parameter-adaptive dual-channel pulse-coupled neural network[J]. Remote Sensing, 2023, 15(2):344.
[18] LIU Yu, LIU Shuping, WANG Zengfu. A general framework for image fusion based on multi-scale transform and sparse representation[J]. Information Fusion, 2015, 24:147-164.
[19] LIU Yu, WANG Zengfu. Simultaneous image fusion and denoising with adaptive sparse representation[J]. IET Image Processing, 2015, 9(5):347-357.
[20] LIU Yu, CHEN Xun, WARD R K, et al. Image fusion with convolutional sparse representation[J]. IEEE Signal Processing Letters, 2016, 23(12):1882-1886.
[21] LIU Yu, CHEN Xun, WARD R K, et al. Medical image fusion via convolutional sparsity based morphological component analysis[J]. IEEE Signal Processing Letters, 2019, 26(3):485-489.
[22] SHREYAMSHA KUMAR B K. Image fusion based on pixel significance using cross bilateral filter[J]. Signal, Image and Video Processing, 2015, 9(5):1193-1204.
[23] JIAN Lihua, YANG Xiaomin, ZHOU Zhili, et al. Multi-scale image fusion through rolling guidance filter[J]. Future Generation Computer Systems, 2018, 83:310-325.
[24] MA Jiayi, CHEN Chen, LI Chang, et al. Infrared and visible image fusion via gradient transfer and total variation minimization[J]. Information Fusion, 2016, 31:100-109.
[25] 成飞飞, 付志涛, 黄亮, 等. 结合自适应PCNN的非下采样剪切波遥感影像融合[J]. 测绘学报, 2021, 50(10):1380-1389. DOI:10.11947/j.AGCS.2021.20200589. CHENG Feifei, FU Zhitao, HUANG Liang, et al. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10):1380-1389.DOI:10.11947/j.AGCS.2021.20200589.
[26] TAN Wei, TIWARI P, PANDEY H M, et al. Multimodal medical image fusion algorithm in the era of big data[J/OL].[2022-09-15]. https://doi.org/10.1007/s00521-020-05173-2.
[27] PANIGRAHY C, SEAL A, MAHATO N K. Fractal dimension based parameter adaptive dual channel PCNN for multi-focus image fusion[J]. Optics and Lasers in Engineering, 2020, 133:106141.
[28] TALUKDAR S, SINGHA P, MAHATO S, et al. Land-use land-cover classification by machine learning classifiers for satellite observations-a review[J]. Remote Sensing, 2020, 12(7):1135.
[29] MA Lei, LI Manchun, MA Xiaoxue, et al. A review of supervised object-based land-cover image classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 130:277-293.
[30] BELGIU M, DRAGUT L. Random forest in remote sensing:a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114:24-31.

结合改进Laplacian能量和参数自适应双通道ULPCNN的遥感影像融合方法

Remote sensing image fusion method combining improved Laplacian energy and parameter adaptive dual-channel unit-linking pulse coupled neural network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价