Thick cloud removal of hyperspectral images by fusing with multispectral images

doi:10.11947/j.AGCS.2022.20220017

Abstract

Abstract: The cloud contamination issue poses a significant obstacle to the application of hyperspectral images. Existing cloud removal methods usually use temporally close images from the same sensors as cloudy images to provide auxiliary information. Unfortunately, the coarse temporal resolution of hyperspectral images (GF-5 and EO-1 hyperspectral images) may result in great land cover changes. The finer temporal resolution of multispectral images (Landsat 8 OLI images) allows to provide auxiliary information temporally closer to the hyperspectral cloudy images, thus reducing the effect uncertainty caused by land cover changes. To deal with the large spectral differences between auxiliary multispectral bands and cloud-contaminated hyperspectral bands, this paper applied the spatial-spectral-based random forest (SSRF) method to use multispectral images (Landsat 8 OLI images) for cloud removal of hyperspectral images, namely, the SSRF_M method. Benefiting from the strong nonlinear fitting ability, the proposed SSRF_M method can use simultaneously the effective information from multiple bands of the auxiliary multispectral image for cloud removal of each hyperspectral band. In this paper, the GF-5 and EO-1 hyperspectral images were used for cloud simulation experiments. The visual and quantitative evaluation results show that compared with the strategy using homologous auxiliary images, the proposed SSRF_M method can reconstruct the information under clouds more accurately.

Key words: hyperspectral images, thick cloud removal, GF-5, Landsat 8, EO-1, spatial-spectral random forest

CLC Number:

P227

WANG Lanxing, WANG Qunming, TONG Xiaohua. Thick cloud removal of hyperspectral images by fusing with multispectral images[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4): 612-621.

References

[1] 李敏,朱国康,张学武,等. 基于多孔径映射的高光谱异常检测算法[J]. 测绘学报,2016,45(10):1222-1230. DOI:10.11947/J.AGCS.2016.20160119. LI Min,ZHU Guokang,ZHANG Xuewu,et al. An anomaly detector based on multi-aperture mapping for hyperspectral data[J]. Acta Geodaetica et Cartographica Sinica,2016,45(10):1222-1230.DOI:10.11947/J.AGCS.2016.20160119.
[2] LU Xiaoqiang,WANG Yulong,YUAN Yuan. Graph-regularized low-rank representation for destriping of hyperspectral images[J]. IEEE Transactions on Geoscience and Remote Sensing,2013,51(7):4009-4018.
[3] KRUSE F A,BOARDMAN J W,HUNTINGTON J F. Comparison of airborne hyperspectral data and EO-1 hyperion for mineral mapping[J]. IEEE Transactions on Geoscience and Remote Sensing,2003,41(6):1388-1400.
[4] EISMANN M T, STOCKER A D, NASRABADI N M. Automated hyperspectral cueing for civilian search and rescue[J]. Proceedings of the IEEE,2009,97(6):1031-1055.
[5] GUILLEMOT C, LE MEUR O. Image inpainting: overview and recent advances[J]. IEEE Signal Processing Magazine, 2014, 31(1): 127-144.
[6] 刘冰,左溪冰,谭熊,等. 高光谱影像分类的深度少样例学习方法[J]. 测绘学报,2020,49(10):1331-1342. DOI: 10.11947/J.AGCS.2020.20190486. LIU Bing,ZUO Xibing,TAN Xiong,et al. A deep few-shot learning algorithm for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica,2020,49(10):1331-1342. DOI:10.11947/J.AGCS.2020.20190486.
[7] SIRAVENHA A C, SOUSA D, BISPO A, et al. Evaluating inpainting methods to the satellite images clouds and shadows removing[C]//Proceedings of 2011 Signal Processing, Image Processing and Pattern Recognition.Jeju Island,Korea:SPPR,2011.
[8] CHENG Qing,SHEN Huanfeng,ZHANG Liangpei,et al. Missing information reconstruction for single remote sensing images using structure-preserving global optimization[J]. IEEE Signal Processing Letters,2017,24(8):1163-1167.
[9] LORENZI L, MELGANI F, MERCIER G. Missing-area reconstruction in multispectral images under a compressive sensing perspective[J]. IEEE Transactions on Geoscience and Remote Sensing,2013,51(7):3998-4008.
[10] LIN C H, TSAI P H, LAI Kanghua, et al. Cloud removal from multitemporal satellite images using information cloning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(1): 232-241.
[11] LIN C,LAI K,CHEN Z,et al. Patch-based information reconstruction of cloud-contaminated multitemporal images[J]. IEEE Transactions on Geoscience and Remote Sensing,2014,52(1):163-174.
[12] CHEN Feng, TANG Lina, QIU Quanyi. Exploitation of CBERS-02B as auxiliary data in recovering the Landsat7 ETM+ SLC-off image[C]//Proceedings of the 18th International Conference on Geoinformatics. Beijing,China: IEEE, 2010.
[13] CHENG Qing,SHEN Huanfeng,ZHANG Liangpei,et al. Cloud removal for remotely sensed images by similar pixel replacement guided with a spatio-temporal MRF model[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2014,92:54-68.
[14] CHEN Bin, HUANG Bo, CHEN Lifan, et al. Spatially and temporally weighted regression: a novel method to produce continuous cloud-free landsat imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(1): 27-37.
[15] 梁栋, 孔颉, 胡根生, 等. 基于支持向量机的遥感影像厚云及云阴影去除[J]. 测绘学报, 2012, 41(2): 225-231, 238. LIANG Dong, KONG Jie, HU Gensheng, et al. The removal of thick cloud and cloud shadow of remote sensing image based on support vector machine[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(2): 225-231, 238.
[16] GAO Guoming,GU Yanfeng. Multitemporal Landsat missing data recovery based on tempo-spectral angle model[J]. IEEE Transactions on Geoscience and Remote Sensing,2017,55(7):3656-3668.
[17] ZHU Xiaolin,GAO Feng,LIU Desheng,et al. A modified neighborhood similar pixel interpolator approach for removing thick clouds in Landsat images[J]. IEEE Geoscience and Remote Sensing Letters,2012,9(3):521-525.
[18] LI Xinghua,SHEN Huanfeng,ZHANG Liangpei,et al. Sparse-based reconstruction of missing information in remote sensing images from spectral/temporal complementary information[J]. ISPRS Journal of Photogrammetry and Remote Sensing,2015,106:1-15.
[19] LI Xinghua,SHEN Huanfeng,LI Huifang,et al. Patch matching-based multitemporal group sparse representation for the missing information reconstruction of remote-sensing images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2016,9(8):3629-3641.
[20] CAO Ruyin, CHEN Yang, CHEN Jin, et al. Thick cloud removal in Landsat images based on autoregression of Landsat time-series data[J]. Remote Sensing of Environment, 2020, 249: 112001.
[21] TANG Zhipeng, ADHIKARI H, PELLIKKA P K E, et al. A method for predicting large-area missing observations in Landsat time series using spectral-temporal metrics[J]. International Journal of Applied Earth Observation and Geoinformation, 2021, 99: 102319.
[22] TAN Kun, WANG Xue, NIU Chao, et al. Vicarious calibration for the AHSI instrument of Gaofen-5 with reference to the CRCS Dunhuang test site[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(4): 3409-3419.
[23] USGS.EO-1 user guide.[2022-02-05].https://www.usgs.gov/media/files/eo-1-user-guide-v-23.
[24] SHEN Huanfeng, LI Xinghua, CHENG Qing, et al. Missing information reconstruction of remote sensing data: a technical review[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3): 61-85.
[25] WANG Qunming,TANG Yijie,TONG Xiaohua,et al. Virtual image pair-based spatio-temporal fusion[J]. Remote Sensing of Environment,2020,249:112009.
[26] YAN L, ROY D P. Conterminous united states crop field size quantification from multi-temporal Landsat data[J]. Remote Sensing of Environment, 2016, 172: 67-86.
[27] ZHU Zhe, WOODCOCK C E. Continuous change detection and classification of land cover using all available Landsat data[J]. Remote Sensing of Environment, 2014, 144: 152-171.
[28] SHENG Yongwei,SONG Chunqiao,WANG Jida,et al. Representative lake water extent mapping at continental scales using multi-temporal Landsat-8 imagery[J]. Remote Sensing of Environment,2016,185:129-141.
[29] SHEN Huanfeng, HUANG Liwen, ZHANG Liangpei, et al. Long-term and fine-scale satellite monitoring of the urban heat island effect by the fusion of multi-temporal and multi-sensor remote sensed data: a 26-year case study of the city of Wuhan in China[J]. Remote Sensing of Environment, 2016, 172: 109-125.
[30] LIU Zhaoyan,TANG Lingli,LI Chuanrong,et al. T-S fuzzy remote sensing monitoring model of snail distribution by Landsat 8 and Sentinel 2 data. Journal of Geodesy and Geoinformation Science,2020,3(4):118-125.
[31] USGS. Landsat 8 data users handbook.[2022-02-05].https://www.usgs.gov/media/files/landsat-8-data-users- handbook.
[32] WANG Qunming,WANG Lanxing,Zhu Xiaolin,et al. Remote sensing image gap filling based on spatial-spectral random forest[J]. Science of Remote Sensing,2022, 5: 100048.
[33] BREIMAN L. Random forests[J]. Machine Learning,2001,45:5-32.
[34] WANG Zhou, BOVIK A C. A universal image quality index[J]. IEEE Signal Processing Letters, 2002, 9(3): 81-84.