Remote sensing image fusion combining energy attribute and guided filter

doi:10.11947/j.AGCS.2023.20220470

Abstract

Abstract: A pan-sharpening method based on energy attribute (EA) and guided filter is presented to solve the problem of spatial and spectral information distortion in multispectral and panchromatic image fusion. First, the intensity-hue-saturation (IHS) transformation is applied to the multispectral image to extract the intensity component. The high and low-frequency components are obtained from the intensity component and the panchromatic image using the mean filter and difference operator. The guided filter enhances the high-frequency information. The decision map is obtained using the rule of maximum pixel value, and the high-frequency image is fused with the decision map by the rule of the weighted average pixel. The EA strategy fuses low-frequency components. The fusion image is obtained by combining the new low-frequency component with the high-frequency component instead of the original intensity component and inverting the IHS transformation. In this paper, many experiments are carried out on SPOT-6, WorldView-2, and Pléiades NEO remote sensing images, and the results are compared quantitatively and qualitatively with four advanced methods. The spectral angle mapping, relative dimensionless global error in synthesis, relative average spectral error, root mean square error, universal image quality index, and peak signal-to-noise ratio values of the proposed methods were improved by 77.13%, 10.78%, 9.57%, 12.20%, 1.35%, and 0.39% compared with the sub-optimal values, respectively. The experimental results show that this method can fully incorporate the spatial information of panchromatic images while maintaining the spectral information of multispectral images and achieve optimal fusion results in visual perception, quantitative indicators, and time efficiency.

Key words: image fusion, energy attribute, guided filter, mean filter, panchromatic image, multispe-ctral image

CLC Number:

P237

SONG Jiawen, ZHU Daming, FU Zhitao, CHEN Sijing. Remote sensing image fusion combining energy attribute and guided filter[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(12): 2154-2163.

References

[1] VIVONE G, DALLA MURA M, GARZELLI A, et al. A new benchmark based on recent advances in multispectral pansharpening: revisiting pansharpening with classical and emerging pansharpening methods[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9(1): 53-81.
[2] BRUZZONE L, BOVOLO F. A novel framework for the design of change-detection systems for very-high-resolution remote sensing images[J]. Proceedings of the IEEE, 2013, 101(3): 609-630.
[3] PEREIRA O J R, MELFI A J, MONTES C R. Image fusion of Sentinel-2 and CBERS-4 satellites for mapping soil cover in the Wetlands of Pantanal[J]. International Journal of Image and Data Fusion, 2017, 8(2): 148-172.
[4] KUMAR U, MILESI C, NEMANI R R, et al. Multi-sensor multi-resolution image fusion for improved vegetation and urban area classification[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015, XL-7/W4: 51-58.
[5] LIU Pengfei, XIAO Liang, LI Tao. A variational pan-sharpening method based on spatial fractional-order geometry and spectral-spatial low-rank priors[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(3): 1788-1802.
[6] GUO Penghao, ZHUANG Peixian, GUO Yecai. Bayesian pan-sharpening with multiorder gradient-based deep network constraints[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 950-962.
[7] XIAO Jinliang, HUANG Tingzhu, DENG Liangjian, et al. A new context-aware details injection fidelity with adaptive coefficients estimation for variational pansharpening[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-15.
[8] 王密, 何鲁晓, 程宇峰, 等. 自适应高斯滤波与SFIM模型相结合的全色多光谱影像融合方法[J]. 测绘学报, 2018, 47(1): 82-90. DOI: 10.11947/j.AGCS.2018.20170421. WANG Mi, HE Luxiao, CHENG Yufeng, et al. Panchromatic and multi-spectral fusion method combined with adaptive Gaussian filter and SFIM model[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(1): 82-90. DOI: 10.11947/j.AGCS.2018.20170421.
[9] ZHANG Yun. Smart photogrammetric and remote sensing image processing for very high resolution optical images—examples from the CRC-AGIP lab at UNB[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2): 17-26.
[10] JIAN Lihua, YANG Xiaomin, WU Wei, et al. Pansharpening using a guided image filter based on dual-scale detail extraction[J]. Journal of Ambient Intelligence and Humanized Computing, 2018: 1-15.
[11] LI Qilei, YANG Xiaomin, WU Wei, et al. Pansharpening multispectral remote-sensing images with guided filter for monitoring impact of human behavior on environment[J]. Concurrency and Computation: Practice and Experience, 2021, 33(4): e5074.
[12] QIU Xiaohua, LI Min, ZHANG Liqiong, et al. Guided filter-based multi-focus image fusion through focus region detection[J]. Signal Processing: Image Communication, 2019, 72: 35-46.
[13] MA Jinlei, ZHOU Zhiqiang, WANG Bo, et al. Multi-focus image fusion using boosted random walks-based algorithm with two-scale focus maps[J]. Neurocomputing, 2019, 335: 9-20.
[14] HE Kaiming, SUN Jian, TANG Xiaoou. Guided image filtering[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(6): 1397-1409.
[15] TAN Wei, THITØN W, XIANG Pei, et al. Multi-modal brain image fusion based on multi-level edge-preserving filtering[J]. Biomedical Signal Processing and Control, 2021, 64: 102280.
[16] OTAZU X, GONZALEZ-AUDICANA M, FORS O, et al. Introduction of sensor spectral response into image fusion methods. Application to wavelet-based methods[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(10): 2376-2385.
[17] 成飞飞, 付志涛, 黄亮, 等. 结合自适应PCNN的非下采样剪切波遥感影像融合[J]. 测绘学报, 2021, 50(10): 1380-1389. DOI: 10.11947/j.AGCS.2021.20200589. CHENG Feifei, FU Zhitao, HUANG Liang, et al. Non-subsampled shearlet transform remote sensing image fusion combined with parameter-adaptive PCNN[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(10): 1380-1389. DOI: 10.11947/j.AGCS.2021.20200589.
[18] LI Xiaosong, ZHOU Fuqiang, TAN Haishu, et al. Multimodal medical image fusion based on joint bilateral filter and local gradient energy[J]. Information Sciences, 2021, 569: 302-325.
[19] YUHAS R H, GOETZ A, BOARDMAN J. Discrimination among semi-arid landscape endmembers using the spectral angle mapper(SAM) algorithm[J]. Summaries of the Third Annual JPL Airborne Geoscience Workshop,1992,1: 147-149.
[20] MENG Xiangchao, XIONG Yiming, SHAO Feng, et al. A large-scale benchmark data set for evaluating pansharpening performance: overview and implementation[J]. IEEE Geoscience and Remote Sensing Magazine, 2021, 9(1): 18-52.
[21] YANG Yong, WAN Weiguo, HUANG Shuying, et al. Remote sensing image fusion based on adaptive IHS and multiscale guided filter[J]. IEEE Access, 2016, 4: 4573-4582.
[22] YANG Yong, TONG Song, HUANG Shuying, et al. Multi-focus image fusion based on NSCT and focused area detection[J]. IEEE Sensors Journal, 2014,15(5):2824-2838.
[23] WANG Z, BOVIK A C. A universal image quality index[J]. IEEE Signal Processing Letters, 2002, 9(3): 81-84.
[24] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.