Hyperspectral remote sensing image intrinsic information decomposition: advances and challenges

doi:10.11947/j.AGCS.2023.20220563

Abstract

Abstract: Hyperspectral imaging is a powerful image acquisition method which can record the rich spectral and spatial information of the scene in a high dimensional data cube. Due to this advantage, hyperspectral imaging has been very useful in many practical applications of earth observation and aerospace. However, as a branch of optical remote sensing, the performance of hyperspectral imaging may be affected by many factors such as atmosphere and illumination. The objective of hyperspectral intrinsic image decomposition is to decrease the influence of complex environmental factors, extract and represent the intrinsic spectral and spatial information of hyperspectral images accurately, so as to improve the performance of hyperspectral image recognition and interpretation. This paper reviews some representative work in hyperspectral intrinsic image decomposition. The principle, advantages, and disadvantages of some typical intrinsic image decomposition methods have been analyzed. Moreover, the challenging problems of intrinsic image decomposition faced in real remote sensing applications have been illustrated. At last, based on the requirements of practical remote sensing applications, we discuss the development trends of hyperspectral intrinsic image decomposition. This review could be a good guide for those researchers who are interested in the advances and applications of hyperspectral remote sensing. More importantly, it gives some important future research directions that could be investigated in the future.

Key words: hyperspectral remote sensing, artificial intelligence, intrinsic image decomposition, image recognition and interpretation

CLC Number:

P237

LI Shutao, WU Qiong, KANG Xudong. Hyperspectral remote sensing image intrinsic information decomposition: advances and challenges[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(7): 1059-1073.

References

[1] 赵英时. 遥感应用分析原理与方法[M]. 北京:科学出版社, 2003:34-37. ZHAO Yingshi. Principles and methods of remote sensing application analysis[M]. Beijing:Science Press, 2003:34-37.
[2] HU X, XIE C, FAN Z,et al.Hyperspectral anomaly detection using deep learning:a review[J]. Remote Sensing, 2022, 14(9):1973.
[3] JIA J, WANG Y, CHEN J, et al.Status and application of advanced airborne hyperspectral imaging technology:a review[J]. Infrared Physics and Technology, 2020, 104:103115.
[4] QIAN S.Hyperspectral satellites, evolution, and development history[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:7032-7056.
[5] LU B, DAO P D, LIU J, et al.Recent advances of hyperspectral imaging technology and applications in agriculture[J]. Remote Sensing, 2020, 12(16):2659.
[6] LU L, GONG Z, LIANG Y, et al.Retrieval of chlorophyll-a concentrations of class II water bodies of inland lakes and reservoirs based on ZY1-02D satellite hyperspectral data[J]. Remote Sensing, 2022, 14(8):1842.
[7] ZHANG B, GUO B, ZOU B, et al.Retrieving soil heavy metals concentrations based on GaoFen-5 hyperspectral satellite image at an opencast coal mine, Inner Mongolia, China[J]. Environmental Pollution, 2022, 300:118981.
[8] SHIMONI M, HAELTERMAN R, PERNEEL C.Hyperspectral imaging for military and security applications:combining myriad processing and sensing techniques[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(2):101-117.
[9] 刘冰, 左溪冰, 谭熊, 等. 高光谱影像分类的深度少样例学习方法[J]. 测绘学报, 2020, 49(10):1331-1342.DOI:10.11947/j.AGCS.2020.20190486. LIU Bing, ZUO Xibing, TAN Xiong, et al. A deep few-shot learning algorithm for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10):1331-1342.DOI:10.11947/j.AGCS.2020.20190486.
[10] 孙一帆, 余旭初, 谭熊, 等. 面向小样本高光谱影像分类的轻量化关系网络[J]. 武汉大学学报信息科学版, 2022, 47(8):1336-1348. SUN Yifan, YU Xuchu, TAN Xiong, et al. Lightweight relational network for small sample hyperspectral image classification[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8):1336-1348.
[11] XUE Z, NIE X. Low-rank and sparse representation with adaptive neighborhood regularization for hyperspectral image classification[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(1):73.DOI:10.11947/j.JGGS.2022.0108.
[12] WANG K, ZHENG S, RUI L, et al. A deep double-channel dense network for hyperspectral image classification[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(4):46-62.DOI:10.11947/j.JGGS.2021.0404.
[13] 任智伟, 吴玲达. 基于本征图像分解的高光谱图像空谱联合分类[J]. 航天返回与遥感, 2019, 40(3):111-120. REN Zhiwei, WU Lingda. Spectral-spatial classification for hyperspectral imagery based on intrinsic image decomposition[J]. Spacecraft Recovery and Remote Sensing, 2019, 40(3):111-120.
[14] GU Y, XIE W, LI X, et al.Hyperspectral intrinsic image decomposition with enhanced spatial information[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:5523014.
[15] BARROW H, TENENBAUM J, HANSON A, et al. Recovering intrinsic scene characteristics[J]. Computer Vision Systems, 1978, 2:3-26.
[16] KANG X, LI S, FANG L, et al.Intrinsic image decomposition for feature extraction of hyperspectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(4):2241-2253.
[17] JIN X, GU Y, XIE W. Intrinsic hyperspectral image decomposition with DSM cues[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-13.
[18] GAO G, LIU B, ZHANG X, et al.Multitemporal intrinsic image decomposition with temporal-spatial energy constraints for remote sensing image analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-16.
[19] HUANG Q, ZHU W, ZHAO Y, et al. Multispectral image intrinsic decomposition via subspace constraint[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:6430-6439.
[20] JIN X, GU Y.Superpixel-based intrinsic image decomposition of hyperspectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8):4285-4295.
[21] JIN X, GU Y, LIU T. Intrinsic image recovery from remote sensing hyperspectral images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 57(1):224-238.
[22] BARRON J T, MALIK J. Shape, illumination, and reflectance from shading[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence,2015,37(8):1670-1687.
[23] BARRON J T, MALIK J. High-frequency shape and albedo from shading using natural image statistics[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition. Colorado:IEEE, 2011:2521-2527.
[24] BARRON J T, MALIK J. Shape, albedo, and illumination from a single image of an unknown object[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Rhode Island:IEEE, 2012:334-341.
[25] LAND E H, MCCANN J J. Lightness and retinex theory[J]. Josa, 1971, 61(1):1-11.
[26] GROSSE R, JOHNSON M K, ADELSON E H, et al.Ground truth dataset and baseline evaluations for intrinsic image algorithms[C]//Proceedings of 2009 IEEE International Conference on Computer Vision.Kyoto:IEEE, 2009:2335-2342.
[27] BELL M, FREEMAN E T. Learning local evidence for shading and reflectance[C]//Proceedings of 2001 IEEE International Conference on Computer Vision. Columbia:IEEE, 2001:670-677.
[28] SHEN L, TAN P, LIN S. Intrinsic image decomposition with non-local texture cues[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Alaska:IEEE, 2008:1-7.
[29] GARCES E, MUNOZ A, LOPEZ-MORENO J, et al. Intrinsic images by clustering[C]//Proceedings of 2012 Computer Graphics Forum.Oxford:Blackwell Publishing Ltd, 2012, 31(4):1415-1424.
[30] CHEN Q, KOLTUN V. A simple model for intrinsic image decomposition with depth cues[C]//Proceedings of 2013 IEEE International Conference on Computer Vision.Sydney:IEEE, 2013:241-248.
[31] SHEN L, YEO C, HUA B S. Intrinsic image decomposition using a sparse representation of reflectance[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(12):2904-2915.
[32] LI K, WANG Y, YE X, et al.Sparse intrinsic decomposition and applications[J]. Signal Processing:Image Communication, 2021, 95:116281.
[33] NARIHIRA T, MAIRE M, YU S X. Learning lightness from human judgement on relative reflectance[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:2965-2973.
[34] ZHOU T, KRAHENBUHL P, EFROS A A. Learning data-driven reflectance priors for intrinsic image decomposition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Chile:IEEE, 2015:3469-3477.
[35] SHEN J, YANG X, Li X, et al. Intrinsic image decomposition using optimization and user scribbles[J]. IEEE Transactions on Cybernetics, 2013, 43(2):425-436.
[36] CHENG B, YANG J,YAN S, et al. Learning with L1-graph for image analysis[J]. IEEE Transactions on Image Processing, 2010, 19(4):858-866.
[37] KAROUI M S, DEVILLE Y, BENHALOUCHE F Z, et al. Hypersharpening by joint-criterion nonnegative matrix factorization[J].IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(3):1660-1670.
[38] ZHANG K, ZUO W, ZHANG L. Learning a single convolutional super-resolution network for multiple degradations[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:3262-3271.
[39] PIELLA G. Image fusion for enhanced visualization:avariational approach[J]. International Journal of Computer Vision, 2009, 83(1):1-11.
[40] AIAZZI B, ALPARONE L, BARONTI S, et al. MTF-tailored multiscale fusion of high-resolution ms and pan imagery[J]. Photogrammetric Engineering and Remote Sensing, 2006, 72(5):591-596.
[41] RUBIN J M, RICHARDS W A. Color vision and image intensities:when are changes material?[J]. Biological Cybernetics, 1982, 45(3):215-226.
[42] MOREL J M, PETRO A B, SBERT C. A PDE formalization of retinex theory[J]. IEEE Transactions on Image Processing, 2010, 19(11):2825-2837.
[43] LI Z, SNAVELY N. Learning intrinsic image decomposition from watching the world[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:9039-9048.
[44] KONG N, BLACK M J. Intrinsic depth:improving depth transfer with intrinsic images[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Chile:IEEE, 2015:3514-3522.
[45] HACHAMA M, GHANEM B, WONKA P. Intrinsic scene decomposition from RGB-D images[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Chile:IEEE, 2015:810-818.
[46] LAFFONT P Y, BAZIN J C. Intrinsic decomposition of image sequences from local temporal variations[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Chile:IEEE, 2015:433-441.
[47] JEON J, CHO S, TONG X, et al. Intrinsic image decomposition using structure-texture separation and surface normals[C]//Proceedings of 2014 European Conference on Computer Vision. Switzerland:Springer, 2014:218-233.
[48] YU L F, YEUNG S K, TAI Y W, et al.Shading-based shape refinement of RGB-D images[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland:IEEE, 2013:1415-1422.
[49] BARRON J T, MALIK J. High-frequency shape and albedo from shading using natural image statistics[C]//Proceedings of 2011 IEEE Conference on Computer Vision and Pattern Recognition.Colorado Springs:IEEE, 2011:2521-2528.
[50] BASRI R, JACOBS D W, EMELMACHER I K. Photometric stereo with general,unknown lighting[J]. International Journal of Computer Vision, 2007, 72(3):239-257.
[51] RAMAMOORTHI R, HANRAHAN P. An efficient representation for irradiance environment maps[C]//Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM Press, 2001:497-500.
[52] CHOMBOON K, CHUJAI P, TEERARASSAMEE P, et al. An empirical study of distance metrics for k-nearest neighbor algorithm[C]//Proceedings of the 3rd International Conference on Industrial Application Engineering. Kitakyushu:IIAE, 2015:280-285.
[53] JIN X, GU Y. Intrinsic scene properties from hyperspectral images and LiDAR[C]//Proceedings of 2019 IEEE Conference on Computer Vision and Pattern Recognition. Long Beach:IEEE, 2019:1423-1431.
[54] JIN X, GU Y, LIU T, et al.Supervoxel-based intrinsic scene properties from hyperspectral images and LiDAR[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-13.
[55] DEBES C, MERENTITIS A, HEREMANS R, et al.Hyperspectral and LiDAR data fusion:outcome of the 2013 GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014, 7(6):2405-2418.
[56] XU Y, DU B, ZHANG L, et al.Advanced multi-sensor optical remote sensing for urban and use and land cover classification:outcome of the 2018 IEEE GRSS data fusion contest[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(6):1709-1724.
[57] GAO G, LIU B, ZHANG X, et al.Multitemporal intrinsic image decomposition with temporal-spatial energy constraints for remote sensing image analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-16.
[58] DUAN P, LAI J, GHAMISI P, et al. Component decomposition-based hyperspectral resolution enhancement for mineral mapping[J]. Remote Sensing, 2020, 12(18):2903.