Data-driven multi-source remote sensing data fusion: progress and challenges

doi:10.11947/j.AGCS.2022.20220171

Abstract

Abstract: Multi-source remote sensing data fusion is an important technology to generate seamless observation data of large scene with a high temporal-spatial-spectral resolution, which breaks through the limitation of single sensor observation and realize the complementary utilization of multi-platform and multi-mode observation data. With the improvement of artificial intelligence theory and technology, data-driven multi-source remote sensing data fusion has been widely favored by researchers. However, the inherent low physical interpretability and weak generalization ability of data-driven algorithms have impeded its further development in multi-source remote sensing data fusion. Therefore, this paper systematically summarizes the researches of homogeneous remote sensing data fusion, heterogeneous remote sensing data fusion and point-surface fusion through three sections, and analyzes the trend of each fusion problem. Finally, this paper discusses the challenges faced by data-driven fusion algorithm, and points out some feasible future directions of multi-source remote sensing data fusion, which provides some suggestions for researchers in this field.

Key words: remote sensing, multi-source fusion, information fusion, data-driven, model-driven, deep learning

CLC Number:

P227

ZHANG Liangpei, HE Jiang, YANG Qianqian, XIAO Yi, YUAN Qiangqiang. Data-driven multi-source remote sensing data fusion: progress and challenges[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1317-1337.

References

[1] 杨景辉.遥感影像像素级融合通用模型及其并行计算方法[J].测绘学报, 2015, 44(8):943. DOI:10.11947/j.AGCS.2015.20150059. YANG Jinghui. The generalized model and parallel computing methods for pixel-level remote sensing image fusion[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(8):943. DOI:10.11947/j.AGCS.2015.20150059.
[2] 童庆禧,张兵,张立福.中国高光谱遥感的前沿进展[J].遥感学报, 2016, 20(5):689-707. TONG Qingxi, ZHANG Bing, ZHANG Lifu. Current progress of hyperspectral remote sensing in China[J]. Journal of Remote Sensing, 2016, 20(5):689-707.
[3] 孙锐,荣媛,苏红波,等. MODIS和HJ-1CCD数据时空融合重构NDVI时间序列[J].遥感学报, 2016, 20(3):361-373. SUN Rui, RONG Yuan, SU Hongbo, et al. NDVI time-series reconstruction based on MODIS and HJ-1 CCD data spatial-temporal fusion[J]. Journal of Remote Sensing, 2016, 20(3):361-373.
[4] LERTRATTANAPANICH S, BOSE N K. High resolution image formation from low resolution frames using Delaunay triangulation[J]. IEEE Transactions on Image Processing, 2002, 11(12):1427-1441.
[5] IRANI M, PELEG S. Improving resolution by image registration[J]. CVGIP:Graphical Models and Image Processing, 1991, 53(3):231-239.
[6] STARK H, OSKOUI P. High-resolution image recovery from image-plane arrays, using convex projections[J]. Journal of the Optical Society of America A, 1989, 6(11):1715-1726.
[7] FARSIU S, ROBINSON M D, ELAD M, et al. Fast and robust multiframe super resolution[J]. IEEE Transactions on Image Processing, 2004, 13(10):1327-1344.
[8] YANG Jianchao, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
[9] SCHOWENGERDT R A. Reconstruction of multispatial, multispectral image data using spatial frequency content[J]. Photogrammetric Engineering and Remote Sensing, 1980, 46(10):1325-1334.
[10] HALLADA W A, COX S. Image sharpening for mixed spatial and spectral resolution satellite systems[C]//Proceedings of the 17th International Symposium on Remote Sensing of Environment. Ann Arbor:IOP, 1983.
[11] CLICHE G, BONN F, TEILLET P. Integration of the SPOT panchromatic channel into its multispectral mode for image sharpness enhancement[J]. Hotogrammetric Engineering&Remote Sensing, 1985, 51(3):311-316.
[12] SHEN Huanfeng, WU Penghai, LIU Yaolin, et al. A spatial and temporal reflectance fusion model considering sensor observation differences[J]. International Journal of Remote Sensing, 2013, 34(12):4367-4383.
[13] WU Mingquan, NIU Zheng, WANG Changyao, et al. Use of MODIS and Landsat time series data to generate high-resolution temporal synthetic Landsat data using a spatial and temporal reflectance fusion model[J]. Journal of Applied Remote Sensing, 2012, 6(1):063507.
[14] LI Dacheng, LI Yanrong, YANG Wenfu, et al. An enhanced single-pair learning-based reflectance fusion algorithm with spatiotemporally extended training samples[J]. Remote Sensing, 2018, 10(8):1207.
[15] XUE Jie, LEUNG Y, FUNG T. A Bayesian data fusion approach to spatio-temporal fusion of remotely sensed images[J]. Remote Sensing, 2017, 9(12):1310.
[16] YANG J W, JIANG L M, LEMMETYINEN J, et al. Improving snow depth estimation by coupling HUT-optimized effective snow grain size parameters with the random forest approach[J]. Remote Sensing of Environment, 2021, 264:112630.
[17] WANG Bin, YUAN Qiangqiang, YANG Qianqian, et al. Estimate hourly PM_2.5 concentrations from Himawari-8 TOA reflectance directly using geo-intelligent long short-term memory network[J]. Environmental Pollution, 2021, 271:116327.
[18] MIN Min, BAI Chen, GUO Jianping, et al. Estimating summertime precipitation from Himawari-8 and global forecast system based on machine learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5):2557-2570.
[19] 黄波,赵涌泉.多源卫星遥感影像时空融合研究的现状及展望[J].测绘学报, 2017, 46(10):1492-1499. DOI:10.11947/j.AGCS.2017.20170376. HUANG Bo, ZHAO Yongquan. Research status and prospect of spatiotemporal fusion of multi-source satellite remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1492-1499. DOI:10.11947/j.AGCS.2017.20170376.
[20] 张良培,沈焕锋.遥感数据融合的进展与前瞻[J].遥感学报, 2016, 20(5):1050-1061. ZHANG Liangpei, SHEN Huanfeng. Progress and future of remote sensing data fusion[J]. Journal of Remote Sensing, 2016, 20(5):1050-1061.
[21] TSAI R. Multiframe image restoration and registration[J]. Advance Computer Visual and Image Processing, 1984, 1:317-339.
[22] TIAN Yapeng, ZHANG Yulun, FU Yun, et al. TDAN:temporally-deformable alignment network for video super-resolution[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle, WA, USA:IEEE, 2020:3360-3369.
[23] SALVETTI F, MAZZIA V, KHALIQ A, et al. Multi-image super resolution of remotely sensed images using residual attention deep neural networks[J]. Remote Sensing, 2020, 12(14):2207.
[24] SONG Huihui, XU Wenjie, LIU Dong, et al. Multi-stage feature fusion network for video super-resolution[J]. IEEE Transactions on Image Processing, 2021, 30:2923-2934.
[25] SHI Wenzhe, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA:IEEE, 2016:1874-1883.
[26] TAKEDA H, FARSIU S, MILANFAR P. Kernel regression for image processing and reconstruction[J]. IEEE Transactions on Image Processing, 2007, 16(2):349-366.
[27] XIAO Yi, SU Xin, YUAN Qiangqiang, et al. Satellite video super-resolution via multiscale deformable convolution alignment and temporal grouping projection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-19.
[28] YI Peng, WANG Zhongyuan, JIANG Kui, et al. Progressive fusion video super-resolution network via exploiting non-local spatio-temporal correlations[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, Korea:IEEE, 2019:3106-3115.
[29] WANG Zhijun, ZIOU D, ARMENAKIS C, et al. A comparative analysis of image fusion methods[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(6):1391-1402.
[30] ALPARONE L, AIAZZI B, BARONTI S, et al. Remote sensing image fusion[M]. Boca Raton, FL:CRC Press, 2015.
[31] THOMAS C, RANCHIN T, WALD L, et al. Synthesis of multispectral images to high spatial resolution:a critical review of fusion methods based on remote sensing physics[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(5):1301-1312.
[32] AIAZZI B, ALPARONE L, BARONTI S, et al. Twenty-five years of pansharpening:a critical review and new developments[M]//CHEN C H. Signal and Image Processing for Remote Sensing. 2nd ed. Boca Raton, FL:CRC Press, 2012:533-548.
[33] VIVONE G, ALPARONE L, CHANUSSOT J, et al. A critical comparison among pansharpening algorithms[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2565-2586.
[34] LONCAN L, DE ALMEIDA L B, BIOUCAS-DIAS J M, et al. Hyperspectral pansharpening:a review[J]. IEEE Geoscience and Remote Sensing Magazine, 2015, 3(3):27-46.
[35] HASANLOU M, SARADJIAN M R. Quality assessment of pan-sharpening methods in high-resolution satellite images using radiometric and geometric index[J]. Arabian Journal of Geosciences, 2016, 9(1):45.
[36] JIANG Dong, ZHUANG Dafang, HUANG Yaohuan, et al. Survey of multispectral image fusion techniques in remote sensing applications[M]//ZHENG Yufeng. Image Fusion and Its Applications.[S.l.]:IntechOpen, 2011:1-23.
[37] TU Teming, CHENG Wenchun, CHANG C P, et al. Best tradeoff for high-resolution image fusion to preserve spatial details and minimize color distortion[J]. IEEE Geoscience and Remote Sensing Letters, 2007, 4(2):302-306.
[38] KLONUS S, EHLERS M. Performance of evaluation methods in image fusion[C]//Proceedings of the 12th International Conference on Information Fusion. Seattle, WA, USA:IEEE, 2009:1409-1416.
[39] GARZELLI A, NENCINI F, CAPOBIANCO L. Optimal MMSE pan sharpening of very high resolution multispectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(1):228-236.
[40] CHOI J, YU K, KIM Y. A new adaptive component-substitution-based satellite image fusion by using partial replacement[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(1):295-309.
[41] OTAZU X, GONZÁLEZ-AUDÍCANA M, FORS O, et al. Introduction of sensor spectral response into image fusion methods application to wavelet-based methods[J]. IEEE Transactions on Geoscience and Remote Sensing, 2005, 43(10):2376-2385.
[42] AMOLINS K, ZHANG Yun, DARE P. Wavelet based image fusion techniques-an introduction, review and comparison[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2007, 62(4):249-263.
[43] AIAZZI B, ALPARONE L, BARONTI S, et al. MTF-tailored multiscale fusion of high-resolution MS and Pan imagery[J]. Photogrammetric Engineering&Remote Sensing, 2006, 72(5):591-596.
[44] VIVONE G, RESTAINO R, CHANUSSOT J. Full scale regression-based injection coefficients for panchromatic sharpening[J]. IEEE Transactions on Image Processing, 2018, 27(7):3418-3431.
[45] BALLESTER C, CASELLES V, IGUAL L, et al. A variational model for P+XS image fusion[J]. International Journal of Computer Vision, 2006, 69(1):43-58.
[46] FU Xueyang, LIN Zihuang, HUANG Yue, et al. A variational pan-sharpening with local gradient constraints[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA, USA:IEEE, 2019:10265-10274.
[47] ZHANG Liangpei, SHEN Huanfeng, GONG Wei, et al. Adjustable model-based fusion method for multispectral and panchromatic images[J]. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 2012, 42(6):1693-1704.
[48] 孟祥超,沈焕锋,张洪艳,等.基于梯度一致性约束的多光谱/全色影像最大后验融合方法[J].光谱学与光谱分析, 2014, 34(5):1332-1337. MENG Xiangchao, SHEN Huanfeng, ZHANG Hongyan, et al. Maximum a posteriori fusion method based on gradient consistency constraint for multispectral/panchromatic remote sensing images[J]. Spectroscopy and Spectral Analysis, 2014, 34(5):1332-1337.
[49] JIANG Cheng, ZHANG Hongyan, SHEN Huanfeng, et al. A practical compressed sensing-based pan-sharpening method[J]. IEEE Geoscience and Remote Sensing Letters, 2012, 9(4):629-633.
[50] LI Shutao, YANG Bin. A new pan-sharpening method using a compressed sensing technique[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011, 49(2):738-746.
[51] KIM Y, LEE C, HAN D, et al. Improved additive-wavelet image fusion[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(2):263-267.
[52] VIVONE G, ALPARONE L, GARZELLI A, et al. Fast reproducible pansharpening based on instrument and acquisition modeling:AWLP revisited[J]. Remote Sensing, 2019, 11(19):2315.
[53] JAVAN F D, SAMADZADEGAN F, MEHRAVAR S, et al. A review of image fusion techniques for pan-sharpening of high-resolution satellite imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:101-117.
[54] CHENG Ming, WANG Cheng, LI J. Sparse representation based pansharpening using trained dictionary[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(1):293-297.
[55] ZHU Xiaoxiang, GROHNFELDT C, BAMLER R. Exploiting joint sparsity for pansharpening:the J-SparseFI algorithm[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(5):2664-2681.
[56] HUANG Wei, XIAO Liang, WEI Zhihui, et al. A new pan-sharpening method with deep neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5):1037-1041.
[57] MASI G, COZZOLINO D, VERDOLIVA L, et al. Pansharpening by convolutional neural networks[J]. Remote Sensing, 2016, 8(7):594.
[58] RAO Yizhou, HE Lin, ZHU Jiawei. A residual convolutional neural network for pan-shaprening[C]//Proceedings of 2017 International Workshop on Remote Sensing with Intelligent Processing (RSIP). Shanghai, China:IEEE, 2017:1-4.
[59] WEI Yancong, YUAN Qiangqiang, SHEN Huanfeng, et al. Boosting the accuracy of multispectral image pansharpening by learning a deep residual network[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(10):1795-1799.
[60] YANG Junfeng, FU Xueyang, HU Yuwen, et al. PanNet:a deep network architecture for pan-sharpening[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:5449-5457.
[61] LIU Xiangyu, WANG Yunhong, LIU Qingjie. Psgan:a generative adversarial network for remote sensing image pan-sharpening[C]//Proceedings of the 25th IEEE International Conference on Image Processing (ICIP). Athens, GREECE:IEEE, 2018:873-877.
[62] SHEN Huanfeng, JIANG Menghui, LI Jie, et al. Spatial-spectral fusion by combining deep learning and variational model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(8):6169-6181.
[63] JIANG Menghui, SHEN Huanfeng, LI Jie, et al. A differential information residual convolutional neural network for pansharpening[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163:257-271.
[64] XING Yinghui, YANG Shuyuan, FENG Zhixi, et al. Dual-collaborative fusion model for multispectral and panchromatic image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 60:5400215.
[65] ZHOU Man, FU Xueyang, HUANG Jie, et al. Effective pan-sharpening with transformer and invertible neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:5406815.
[66] CETIN M, MUSAOGLU N. Merging hyperspectral and panchromatic image data:qualitative and quantitative analysis[J]. International Journal of Remote Sensing, 2009, 30(7):1779-1804.
[67] DIAN Renwei, LI Shutao, GUO Anjing, et al. Deep hyperspectral image sharpening[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(11):5345-5355.
[68] LICCIARDI G A, KHAN M M, CHANUSSOT J, et al. Fusion of hyperspectral and panchromatic images using multiresolution analysis and nonlinear PCA band reduction[J]. EURASIP Journal on Advances in Signal Processing, 2012, 2012(1):207.
[69] GOMEZ R B, JAZAERI A, KAFATOS M. Wavelet-based hyperspectral and multispectral image fusion[C]//Proceedings of 2001 SPIE 4383, Geo-Spatial Image and Data Exploitation II. Orlando, FL, USA:SPIE, 2001:36-42.
[70] CHEN Zhao, PU Hanye, WANG Bin, et al. Fusion of hyperspectral and multispectral images:a novel framework based on generalization of pan-sharpening methods[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(8):1418-1422.
[71] WEI Qi, BIOUCAS-DIAS J, DOBIGEON N, et al. Hyperspectral and multispectral image fusion based on a sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(7):3658-3668.
[72] YOKOYA N, GROHNFELDT C, CHANUSSOT J. Hyperspectral and multispectral data fusion:a comparative review of the recent literature[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(2):29-56.
[73] PALSSON F, SVEINSSON J R, ULFARSSON M O, et al. Model-based fusion of multi-and hyperspectral images using PCA and wavelets[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(5):2652-2663.
[74] HU Jinfan, HUANG Tingzhu, DENG Liangjian, et al. Hyperspectral image super-resolution via deep spatiospectral attention convolutional neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021(99):1-15.
[75] XIE Qi, ZHOU Minghao, ZHAO Qian, et al. MHF-Net:an interpretable deep network for multispectral and hyperspectral image fusion[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 44(3):1457-1473.
[76] MEI Shaohui, YUAN Xin, JI Jingyu, et al. Hyperspectral image spatial super-resolution via 3D full convolutional neural network[J]. Remote Sensing, 2017, 9(11):1139.
[77] LI Kaiyan, XIE Weiying, DU Qian, et al. DDLPS:detail-based deep laplacian pansharpening for hyperspectral imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10):8011-8025.
[78] WU Penghai, SHEN Huanfeng, ZHANG Liangpei, et al. Integrated fusion of multi-scale polar-orbiting and geostationary satellite observations for the mapping of high spatial and temporal resolution land surface temperature[J]. Remote Sensing of Environment, 2015, 156:169-181.
[79] WU Mingquan, HUANG Wenjiang, NIU Zheng, et al. Generating daily synthetic Landsat imagery by combining Landsat and MODIS data[J]. Sensors, 2015, 15(9):24002-24025.
[80] LIU Xun, DENG Chenwei, CHANUSSOT J, et al. StfNet:a two-stream convolutional neural network for spatiotemporal image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9):6552-6564.
[81] SHEN Huanfeng, MENG Xiangchao, ZHANG Liangpei. An integrated framework for the spatio-temporal-spectral fusion of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12):7135-7148.
[82] GAO Feng, MASEK J, SCHWALLER M, et al. On the blending of the Landsat and MODIS surface reflectance:predicting daily Landsat surface reflectance[J]. IEEE Transactions on Geoscience and Remote Sensing, 2006, 44(8):2207-2218.
[83] HILKER T, WULDER M A, COOPS N C, et al. A new data fusion model for high spatial-and temporal-resolution mapping of forest disturbance based on Landsat and MODIS[J]. Remote Sensing of Environment, 2009, 113(8):1613-1627.
[84] ZHU Xiaolin, CHEN Jin, GAO Feng, et al. An enhanced spatial and temporal adaptive reflectance fusion model for complex heterogeneous regions[J]. Remote Sensing of Environment, 2010, 114(11):2610-2623.
[85] WANG Qunming, ZHANG Yihang, ONOJEGHUO A O, et al. Enhancing spatio-temporal fusion of MODIS and Landsat data by incorporating 250 m MODIS data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(9):4116-4123.
[86] HUANG Bo, WANG Juan, SONG Huihui, et al. Generating high spatiotemporal resolution land surface temperature for urban heat island monitoring[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 10(5):1011-1015.
[87] XIA Haiping, CHEN Yunhao, LI Ying, et al. Combining kernel-driven and fusion-based methods to generate daily high-spatial-resolution land surface temperatures[J]. Remote Sensing of Environment, 2019, 224:259-274.
[88] CHENG Qing, LIU Huiqing, SHEN Huanfeng, et al. A spatial and temporal nonlocal filter-based data fusion method[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8):4476-4488.
[89] HAZAYMEH K, HASSAN Q K. Spatiotemporal image-fusion model for enhancing the temporal resolution of Landsat-8 surface reflectance images using MODIS images[J]. Journal of Applied Remote Sensing, 2015, 9(1):096095.
[90] ZHUKOV B, OERTEL D, LANZL F, et al. Unmixing-based multisensor multiresolution image fusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 1999, 37(3):1212-1226.
[91] ZURITA-MILLA R, CLEVERS J G P W, SCHAEPMAN M E. Unmixing-based Landsat TM and MERIS FR data fusion[J]. IEEE Geoscience and Remote Sensing Letters, 2008, 5(3):453-457.
[92] MASELLI F, REMBOLD F. Integration of LAC and GAC NDVI data to improve vegetation monitoring in semi-arid environments[J]. International Journal of Remote Sensing, 2002, 23(12):2475-2488.
[93] ZHANG Wei, LI Ainong, JIN Huaan, et al. An enhanced spatial and temporal data fusion model for fusing Landsat and MODIS surface reflectance to generate high temporal Landsat-like data[J]. Remote Sensing, 2013, 5(10):5346-5368.
[94] HUANG Bo, ZHANG Hankui. Spatio-temporal reflectance fusion via unmixing:accounting for both phenological and land-cover changes[J]. International Journal of Remote Sensing, 2014, 35(16):6213-6233.
[95] XU Yong, HUANG Bo, XU Yuyue, et al. Spatial and temporal image fusion via regularized spatial unmixing[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(6):1362-1366.
[96] MIZUOCHI H, HIYAMA T, OHTA T, et al. Development and evaluation of a lookup-table-based approach to data fusion for seasonal wetlands monitoring:an integrated use of AMSR series, MODIS, and Landsat[J]. Remote Sensing of Environment, 2017, 199:370-388.
[97] HUANG Bo, SONG Huihui. Spatiotemporal reflectance fusion via sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2012, 50(10):3707-3716.
[98] SONG Huihui, HUANG Bo. Spatiotemporal satellite image fusion through one-pair image learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(4):1883-1896.
[99] ZHAO Chongyue, GAO Xinbo, EMERY W J, et al. An integrated spatio-spectral-temporal sparse representation method for fusing remote-sensing images with different resolutions[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(6):3358-3370.
[100] LIU Xun, DENG Chenwei, WANG Shuigen, et al. Fast and accurate spatiotemporal fusion based upon extreme learning machine[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(12):2039-2043.
[101] TAN Zhenyu, YUE Peng, DI Liping, et al. Deriving high spatiotemporal remote sensing images using deep convolutional network[J]. Remote Sensing, 2018, 10(7):1066.
[102] LI Aihua, BO Yanchen, ZHU Yuxin, et al. Blending multi-resolution satellite sea surface temperature (SST) products using Bayesian maximum entropy method[J]. Remote Sensing of Environment, 2013, 135:52-63.
[103] HUANG Bo, ZHANG Hankui, SONG Huihui, et al. Unified fusion of remote-sensing imagery:generating simultaneously high-resolution synthetic spatial-temporal-spectral earth observations[J]. Remote Sensing Letters, 2013, 4(6):561-569.
[104] ECKARDT R, BERGER C, THIEL C, et al. Removal of optically thick clouds from multi-spectral satellite images using multi-frequency SAR data[J]. Remote Sensing, 2013, 5(6):2973-3006.
[105] HUANG Bo, LI Ying, HAN Xiaoyu, et al. Cloud removal from optical satellite imagery with SAR imagery using sparse representation[J]. IEEE Geoscience and Remote Sensing Letters, 2015, 12(5):1046-1050.
[106] BERMUDEZ J D, HAPP P N, OLIVEIRA D A B, et al. SAR to optical image synthesis for cloud removal with generative adversarial networks[C]//Proceedings of 2018 ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Karlsruhe, Germany:ISPRS, 2018.
[107] GAO Jianhao, YUAN Qiangqiang, LI Jie, et al. Cloud removal with fusion of high resolution optical and SAR images using generative adversarial networks[J]. Remote Sensing, 2020, 12(1):191.
[108] TURNES J N, CASTRO J D B, TORRES D L, et al. Atrous cGAN for SAR to optical image translation[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 19:4003905.
[109] HE Wei, YOKOYA N. Multi-temporal sentinel-1 and-2 data fusion for optical image simulation[J]. ISPRS International Journal of Geo-Information, 2018, 7(10):389.
[110] BERMUDEZ J D, HAPP P N, FEITOSA R Q, et al. Synthesis of multispectral optical images from SAR/optical multitemporal data using conditional generative adversarial networks[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(8):1220-1224.
[111] XIA Yu, ZHANG Hongyan, ZHANG Liangpei, et al. Cloud removal of optical remote sensing imagery with multitemporal SAR-optical data using X-mtgan[C]//Proceedings of 2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama, Japan:IEEE, 2019:3396-3399.
[112] SCHMITT M, HUGHES L H, ZHU X X. The SEN1-2 dataset for deep learning in SAR-optical data fusion[C]//Proceedings of 2018 ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Karlsruhe, Germany:ISPRS, 2018.
[113] LI Wenbo, LI Ying, CHAN J C W. Thick cloud removal with optical and SAR imagery via convolutional-mapping-deconvolutional network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2865-2879.
[114] MERANER A, EBEL P, ZHU Xiaoxiang, et al. Cloud removal in Sentinel-2 imagery using a deep residual neural network and SAR-optical data fusion[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166:333-346.
[115] LAURIN G V, LIESENBERG V, CHEN Qi, et al. Optical and SAR sensor synergies for forest and land cover mapping in a tropical site in West Africa[J]. International Journal of Applied Earth Observation and Geoinformation, 2013, 21:7-16.
[116] HONG Gang, ZHANG Aining, ZHOU Fuqun, et al. Integration of optical and synthetic aperture radar (SAR) images to differentiate grassland and alfalfa in Prairie area[J]. International Journal of Applied Earth Observation and Geoinformation, 2014, 28:12-19.
[117] INGLADA J, VINCENT A, ARIAS M, et al. Improved early crop type identification by joint use of high temporal resolution SAR and optical image time series[J]. Remote Sensing, 2016, 8(5):362.
[118] SUKAWATTANAVIJIT C, CHEN Jie, ZHANG Hongsheng. GA-SVM algorithm for improving land-cover classification using SAR and optical remote sensing data[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(3):284-288.
[119] IRWIN K, BEAULNE D, BRAUN A, et al. Fusion of SAR, optical imagery and airborne LiDAR for surface water detection[J]. Remote Sensing, 2017, 9(9):890.
[120] SHAO Zhenfeng, ZHANG Linjing, WANG Lei. Stacked sparse autoencoder modeling using the synergy of airborne LiDAR and satellite optical and SAR data to map forest above-ground biomass[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(12):5569-5582.
[121] LIU Shengjie, QI Zhixin, LI Xia, et al. Integration of convolutional neural networks and object-based post-classification refinement for land use and land cover mapping with optical and SAR data[J]. Remote Sensing, 2019, 11(6):690.
[122] ZHANG Puzhao, BAN Yifang, NASCETTI A. Learning U-Net without forgetting for near real-time wildfire monitoring by the fusion of SAR and optical time series[J]. Remote Sensing of Environment, 2021, 261:112467.
[123] JIANG Xiao, LI Gang, LIU Yu, et al. Change detection in heterogeneous optical and SAR remote sensing images via deep homogeneous feature fusion[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13:1551-1566.
[124] CERVONE G, SAVA E, HUANG Qunying, et al. Using twitter for tasking remote-sensing data collection and damage assessment:2013 Boulder flood case study[J]. International Journal of Remote Sensing, 2016, 37(1):100-124.
[125] ROSSER J F, LEIBOVICI D G, JACKSON M J. Rapid flood inundation mapping using social media, remote sensing and topographic data[J]. Natural Hazards, 2017, 87(1):103-120.
[126] HUANG Qunying, CERVONE G, ZHANG Guiming. A cloud-enabled automatic disaster analysis system of multi-sourced data streams:an example synthesizing social media, remote sensing and Wikipedia data[J]. Computers, Environment and Urban Systems, 2017, 66:23-37.
[127] CHI Mingmin, SUN Zhongyi, QIN Yiqing, et al. A novel methodology to label urban remote sensing images based on location-based social media photos[J]. Proceedings of the IEEE, 2017, 105(10):1926-1936.
[128] LIU Xiaoping, HE Jialv, YAO Yao, et al. Classifying urban land use by integrating remote sensing and social media data[J]. International Journal of Geographical Information Science, 2017, 31(8):1675-1696.
[129] QIN Yiqing, CHI Mingmin, LIU Xuan, et al. Classification of high resolution urban remote sensing images using deep networks by integration of social media photos[C]//Proceedings of 2018 IEEE International Geoscience and Remote Sensing Symposium. Valencia, Spain:IEEE, 2018:7243-7246.
[130] SHI Yan, QI Zhixin, LIU Xiaoping, et al. Urban land use and land cover classification using multisource remote sensing images and social media data[J]. Remote Sensing, 2019, 11(22):2719.
[131] CAO Rui, TU Wei, YANG Cuixin, et al. Deep learning-based remote and social sensing data fusion for urban region function recognition[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163:82-97.
[132] TU Wei, ZHANG Yatao, LI Qingquan, et al. Scale effect on fusing remote sensing and human sensing to portray urban functions[J]. IEEE Geoscience and Remote Sensing Letters, 2021, 18(1):38-42.
[133] MA Ting. Multi-level relationships between satellite-derived nighttime lighting signals and social media-derived human population dynamics[J]. Remote Sensing, 2018, 10(7):1128.
[134] ZHAO Naizhuo, CAO Guofeng, ZHANG Wei, et al. Remote sensing and social sensing for socioeconomic systems:a comparison study between nighttime lights and location-based social media at the 500m spatial resolution[J]. International Journal of Applied Earth Observation and Geoinformation, 2020, 87:102058.
[135] 李小文.定量遥感的发展与创新[J].河南大学学报(自然科学版), 2005, 35(4):49-56. LI Xiaowen. Retrospect, prospect and innovation in quantitative remote sensing[J]. Journal of Henan University (Natural Science), 2005, 35(4):49-56.
[136] 岳林蔚,沈焕锋,袁强强,等.基于深度置信网络的多源DEM点面融合模型[J].武汉大学学报(信息科学版), 2021, 46(7):1090-1097. YUE Linwei, SHEN Huanfeng, YUAN Qiangqiang, et al. A multi-source DEM point-surface fusion model based on deep belief network[J]. Geomatics and Information Science of Wuhan University, 2021, 46(7):1090-1097.
[137] LI Tongwen, SHEN Huanfeng, ZENG Chao, et al. Point-surface fusion of station measurements and satellite observations for mapping PM_2.5 distribution in China:methods and assessment[J]. Atmospheric Environment, 2017, 152:477-489.
[138] 李同文.顾及时空特征的大气PM_2.5神经网络遥感反演[D].武汉:武汉大学, 2020. DOI:10.27379/d.cnki.gwhdu.2020.000817. LI Tongwen. Research on atmospheric PM_2.5 neural network remote sensing retrieval considering spatiotemporal characteristics[D]. Wuhan:Wuhan University, 2020. DOI:10.27379/d.cnki.gwhdu.2020.000817.
[139] YANG Qianqian, YUAN Qiangqiang, YUE Linwei, et al. Mapping PM_2.5 concentration at a sub-km level resolution:a dual-scale retrieval approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 165:140-151.
[140] YUAN Qiangqiang, SHEN Huanfeng, LI Tongwen, et al. Deep learning in environmental remote sensing:achievements and challenges[J]. Remote Sensing of Environment, 2020, 241:111716.
[141] 杨倩倩,靳才溢,李同文,等.数据驱动的定量遥感研究进展与挑战[J].遥感学报, 2022, 26(2):268-285 YANG Qianqian, JIN Caiyi, LI Tongwen, et al. Research progress and challenges of data-driven quantitative remote sensing[J]. National Remote Sensing Bulletin, 2022, 26(2):268-285.
[142] MCCOLL K A, ALEMOHAMMAD S H, AKBAR R, et al. The global distribution and dynamics of surface soil moisture[J]. Nature Geoscience, 2017, 10(2):100-104.
[143] VITERBO P, BETTS A K. Impact of the ECMWF reanalysis soil water on forecasts of the July 1993 Mississippi flood[J]. Journal of Geophysical Research:Atmospheres, 1999, 104(D16):19361-19366.
[144] TRENBERTH K E, GUILLEMOT C J. Evaluation of the global atmospheric moisture budget as seen from analyses[J]. Journal of Climate, 1995, 8(9):2255-2272.
[145] BAGHDADI N, GAULTIER S, KING C. Retrieving surface roughness and soil moisture from synthetic aperture radar (SAR) data using neural networks[J]. Canadian Journal of Remote Sensing, 2002, 28(5):701-711.
[146] SANTI E, PETTINATO S, PALOSCIA S, et al. An algorithm for generating soil moisture and snow depth maps from microwave spaceborne radiometers:HydroAlgo[J]. Hydrology and Earth System Sciences, 2012, 16(10):3659-3676.
[147] AIRES F, PRIGENT C, ROSSOW W B. Sensitivity of satellite microwave and infrared observations to soil moisture at a global scale:2. Global statistical relationships[J]. Journal of Geophysical Research:Atmospheres, 2005, 110(D11):D11103.
[148] EROGLU O, KURUM M, BOYD D, et al. High spatio-temporal resolution CYGNSS soil moisture estimates using artificial neural networks[J]. Remote Sensing, 2019, 11(19):2272.
[149] SONG Xiaodong, ZHANG Ganlin, LIU Feng, et al. Modeling spatiotemporal distribution of soil moisture by deep learning-based cellular automata model[J]. Journal of Arid Land, 2016, 8(5):734-748.
[150] FANG Kuai, SHEN Chaopeng, KIFER D, et al. Prolongation of SMAP to spatiotemporally seamless coverage of continental U.S. using a deep learning neural network[J]. Geophysical Research Letters, 2017, 44(21):11030-11039.
[151] HEGAZI E H, YANG Lingbo, HUANG Jingfeng. A convolutional neural network algorithm for soil moisture prediction from Sentinel-1 SAR images[J]. Remote Sensing, 2021, 13(24):4964.
[152] YUAN Qiangqiang, XU Hongzhang, LI Tongwen, et al. Estimating surface soil moisture from satellite observations using a generalized regression neural network trained on sparse ground-based measurements in the continental US[J]. Journal of Hydrology, 2020, 580:124351.