Artificial intelligence for reliable object recognition from remotely sensed data: overall framework design, review and prospect

doi:10.11947/j.AGCS.2021.20210095

Abstract

Abstract: Reliability is one of the important features in remotely sensed data-based land use monitoring. Artificial intelligence (AI) technology promotes the rapid development of object recognition from remotely sensed data. However, the un-explainability in such image processing causes reliability problems. Based on the reliability theory and the basic theory of AI, this paper first presents the idea and the overall framework of intelligent and reliable object recognition. Second, the core research directions, including analysis of influencing factors, improvement methods, evaluation methods, and process control for reliability are sequentially introduced. Finally, the future development trend of AI for reliable object recognition from remotely sensed data is outlined.

Key words: artificial intelligence, reliability, object recognition, remote sensing

CLC Number:

P237

SHI Wenzhong, ZHANG Min. Artificial intelligence for reliable object recognition from remotely sensed data: overall framework design, review and prospect[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1049-1058.

References

[1] 龚健雅. 人工智能时代测绘遥感技术的发展机遇与挑战[J]. 武汉大学学报(信息科学版), 2018, 43(12):1788-1796. GONG Jianya. Chances and challenges for development of surveying, mapping and remote sensing in the era of artificial intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1788-1796.
[2] HOESER T, BACHOFER F, KUENZER C. Object detection and image segmentation with deep learning on earth observation data:a review-part II:applications[J]. Remote Sensing, 2020, 12(18):47.
[3] MA L, LIU Y, ZHANG X L, et al. Deep learning in remote sensing applications:a meta-analysis and review[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 152:166-177.
[4] SHI W Z, ZHANG M, ZHANG R, et al. Change detection based on artificial intelligence:state-of-the-art and challenges[J]. Remote Sensing, 2020, 12(10):35.
[5] SHI Wenzhong. Principles of modeling uncertainties in spatial data and spatial analyses[M]. Boca Raton:CRC press, 2009.
[6] 史文中, 秦昆, 陈江平, 等. 可靠性地理国情动态监测的理论与关键技术探讨[J]. 科学通报, 2012, 57(24):2239-2248. SHI Wenzhong, QIN Kun, CHEN Jiangping, et al. Key theories and technologies on reliable dynamic monitoring for national geographical state[J]. Chinese Science Bulletin, 2012, 57(24):2239-2248.
[7] 张钹. 人工智能进入后深度学习时代[J]. 智能科学与技术学报, 2019, 1(1):4-6. ZHANG Bo. Artificial intelligence is entering the post deep learning era[J]. Chinese Journal of Intelligent Science and Technology, 2019, 1(01):4-6.
[8] 国务院. 国务院关于印发新一代人工智能发展规划的通知[J].中华人民共和国国务院公报, 2017(22):7-21. State Council. Notice of the State Council on issuing the development plan for the new generation of artificial intelligence[J]. Bulletin of the State Council of the People's Republic of China, 2017(22):7-21.
[9] 史文中, 陈鹏飞, 张效康. 地理国情监测可靠性分析[J]. 测绘学报, 2017, 46(10):1620-1626. SHI Wenzhong, CHEN Pengfei, ZHANG Xiaokang. Reliability analysis geographical conditions monitoring[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46 (10):1620-1626.
[10] 史文中. 空间数据与空间分析不确定性原理[M]. 北京:科学出版社, 2015. SHI Wenzhong. Principles of modeling uncertainties in spatial data and spatial analyses[M]. Beijing:Science Press, 2015.
[11] 舒红, 史文中. 浅谈测量平差到空间数据分析的可靠性理论延伸[J]. 武汉大学学报(信息科学版), 2018, 43(12):1979-1985+1993. SHU Hong, SHI Wenzhong. Extension of reliability theory of surveying adjustment into spatial data analytics[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1979-1985+1993.
[12] 张过, 管志超. 卫星成像质量可靠性研究初探[J]. 武汉大学学报(信息科学版), 2018, 43(12):1954-1961. ZHANG Guo, GUAN Zhichao. Primary research on reliability of satellite imaging quality[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1954-1961.
[13] 张华. 遥感数据可靠性分类方法研究[D]. 徐州:中国矿业大学, 2012. ZHANG Hua. Study on reliable classification methods based on remotely sensed image[D]. Xuzhou:China University of Mining and Technology, 2012.
[14] 张璨, 张明英. 人工智能深度学习算法可靠性评估方法研究[J]. 信息技术与标准化, 2018, (8):38-42. ZHANG Can, ZHANG Mingying. Research on the reliability assessment method of artificial intelligence deep learning algorithms[J]. Standardization Research,2018, (8):38-42.
[15] 孙金彦, 徐南, 董丹丹, 等. 不同空间分辨率遥感数据识别湖泊的误差分析[J]. 人民长江, 2019, 50(4):25-31. SUN Jinyan, XU Nan, Dong Dandan, et al. Error analysis on lakes mapping accuracy by remote sensing data of different spatial resolution[J]. Yangtze River, 2019, 50(4):25-31.
[16] SHI W Z, HAO M. Analysis of spatial distribution pattern of change-detection error caused by misregistration[J]. International journal of remote sensing, 2013, 34(19):6883-6897.
[17] KAPLAN A, HAENLEIN M. Siri, Siri, in my hand:who's the fairest in the land? on the interpretations, illustrations, and implications of artificial intelligence[J]. Business Horizons, 2019, 62(1):15-25.
[18] 高志宏, 周旭, 程滔. 地理国情普查中容易混分地表覆盖类型定量统计与分析[J]. 测绘通报, 2015, (6):32-34. GAO Zhihong, ZHOU Xu, CHENG Tao. Statistical analysis of the confusing land cover types in China geography census[J], Bulletin of Surveying and Mapping, 2015, (6):32-34.
[19] 成科扬, 王宁, 师文喜, 等. 深度学习可解释性研究进展[J]. 计算机研究与发展, 2020, 57(06):1208-1217. CHENG Keyang, WANG Ning,SHI Wenxi, et al. Research advances in the interpretability of deep learning[J]. Journal of Computer Research and Development, 2020, 57(06):1208-1217.
[20] QIU S L, LIU Q H, ZHOU S J, et al. Review of artificial intelligence adversarial attack and defense technologies[J]. Applied Sciences, 2019, 9(5):909.
[21] 史文中, 张鹏林, 陈江平等. 可靠性时空数据分析[M]. 北京:科学出版社, 2021. SHI Wenzhong, ZHANG Penglin, CHEN Jiangping, et al. Reliability in spatiotemporal data analysis[M]. Beijing:Science Press, 2021.
[22] 晁剑, 张慧芳, 许长军, 等. 双时相影像联合不确定性对变化检测精度的影响机理探索[J]. 应用科学学报, 2020, 38(6):916-923. CHAO Jian, ZHANG Huifang, XU Changjun, et al. Research on influence mechanism of joint uncertainty of bio-images on change detection accuracy[J]. Journal of Applied Sciences-Electronics and Information Engineering, 2020, 38(6):916-923.
[23] HE P F, SHI W Z, ZHANG H, et al. A novel dynamic threshold method for unsupervised change detection from remotely sensed images[J]. Remote sensing letters, 2014, 5(4):396-403.
[24] HE P F, SHI W Z, MIAO Z L, et al. Advanced Markov random field model based on local uncertainty for unsupervised change detection[J]. Remote sensing letters, 2015, 6(9):667-676.
[25] ZHANG X K, SHI W Z, LU Z Y, et al. Land cover change detection from high-resolution remote sensing imagery using multitemporal deep feature collaborative learning and a semi-supervised Chan-Vese model[J]. Remote Sensing, 2019, 11(23):20.
[26] SHI W Z, SHAO P, HAO M, et al. Fuzzy topology-based method for unsupervised change detection[J]. Remote sensing letters, 2016, 7(1):81-90.
[27] ZHANG P L, LU Z Y, SHI W Z. Local spectrum-trend similarity approach for detecting land-cover change by using SPOT-5 satellite images[J]. IEEE Geoscience and Remote Sensing Letters, 2013, 11(4):738-742.
[28] SUN Y, ZHANG X C, ZHAO X Y, et al. Extracting building boundaries from high resolution optical images and LiDAR data by integrating the convolutional neural network and the active contour model[J]. Remote Sensing, 2018, 10(9):1459.
[29] CAI L P, SHI W Z, HAO M, et al. A multi-feature fusion-based change detection method for remote sensing images[J]. Journal of the Indian Society of Remote Sensing, 2018, 46(12):2015-2022.
[30] ZHAN T, GONG M G, JIANG X M, et al. Unsupervised scale-driven change detection with deep spatial-spectral features for VHR images[J]. Ieee Transactions on Geoscience and Remote Sensing, 2020, 58(8):5653-5665.
[31] ZHANG P L, SHI W Z, WONG M S, et al. A reliability-based multi-algorithm fusion technique in detecting changes in land cover[J]. Remote Sensing, 2013, 5(3):1134-1151.
[32] CAI L P, SHI W Z, ZHANG H, et al. Object-oriented change detection method based on adaptive multi-method combination for remote-sensing images[J]. International Journal of Remote Sensing, 2016, 37(22):5457-5471.
[33] TAN K, ZHANG Y S, WANG X, et al. Object-based change detection using multiple classifiers and multi-scale uncertainty analysis[J]. Remote Sensing, 2019, 11(3):17.
[34] SONG A, KIM Y, HAN Y. Uncertainty analysis for object-based change detection in very high-resolution satellite images using deep learning network[J]. Remote Sensing, 2020, 12(15):26.
[35] WANG Q M, ATKINSON P M, SHI W Z. Fast subpixel mapping algorithms for subpixel resolution change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 53(4):1692-1706.
[36] WANG S, QUAN D, LIANG X F, et al. A deep learning framework for remote sensing image registration[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 145:148-164.
[37] SHI W Z, ZHANG M, KE H F, et al. Landslide recognition by deep convolutional neural network and change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020:1-19.
[38] HARTLING S, SAGAN V, SIDIKE P, et al. Urban tree species classification using a WorldView-2/3 and LiDAR data fusion approach and deep learning[J]. Sensors, 2019, 19(6):1284.
[39] YANG G, ZHANG Q, ZHANG G X. EANet:edge-aware network for the extraction of buildings from aerial images[J]. Remote Sensing, 2020, 12(13):2161.
[40] LU X Y, ZHONG Y F, ZHENG Z, et al. Multi-scale and multi-task deep learning framework for automatic road extraction[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):9362-9377.
[41] CHENG G, ZHOU P C, HAN J W. Learning rotation-invariant convolutional neural networks for object detection in VHR optical remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12):7405-7415.
[42] HOESER T, KUENZER C. Object detection and image segmentation with deep learning on earth observation data:a review-part I:evolution and recent trends[J]. Remote Sensing, 2020, 12(10):1667.
[43] YOU Y N, CAO J Y, ZHOU W L. A survey of change detection methods based on remote sensing images for multi-source and multi-objective scenarios[J]. Remote Sensing, 2020, 12(15):40.
[44] CHERIYADAT A M. Unsupervised feature learning for aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014, 52(1):439-451.
[45] LI Y S, CHEN W, ZHANG Y J, et al. Accurate cloud detection in high-resolution remote sensing imagery by weakly supervised deep learning[J]. Remote Sensing of Environment, 2020, 250:112045.
[46] FU Q, YU X C, WEI X P, et al. Semi-supervised classification of hyperspectral imagery based on stacked autoencoders[C]//Proceedings of the 8th International Conference on Digital Image Processing. Bellingham:[s.n.], 2016.
[47] RABBI J, RAY N, SCHUBERT M, et al. Small-object detection in remote sensing images with end-to-end edge-enhanced GAN and object detector network[J]. Remote Sensing, 2020, 12(9):1432.
[48] YAN Y M, TAN Z C, SU N. A data augmentation strategy based on simulated samples for ship detection in rgb remote sensing images[J]. ISPRS International Journal of Geo-Information, 2019, 8(6):276.
[49] SUN X, WANG B, WANG Z R, et al. Research progress on few-shot learning for remote sensing image interpretation[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14:2387-2402.
[50] HU F, XIA G S, HU J, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11):14680-14707.
[51] LIU J F, CHEN K M, XU G L, et al. Convolutional neural network-based transfer learning for optical aerial images change detection[J]. Ieee Geoscience and Remote Sensing Letters, 2020, 17(1):127-131.
[52] GILPIN L H, BAU D, YUAN B Z, et al. Explaining explanations:an overview of interpretability of machine learning[C]//Proceedings of 2018 IEEE International Conference on Data Science and Advanced Analytics. New York:IEEE, 2018:80-89.
[53] ARRIETA A B, DIAZ-RODRIGUEZ N, DEL SER J, et al. Explainable artificial intelligence:concepts, taxonomies, opportunities and challenges toward responsible AI[J]. Information Fusion, 2020, 58:82-115.
[54] 化盈盈, 张岱墀, 葛仕明. 深度学习模型可解释性的研究进展[J]. 信息安全学报, 2020, 5(03):1-12. HUA Yingying, ZHANG Daichi, GE Shiming. Research progress in the interpretability of deep learning models[J]. Journal of Cyber Security, 2020, 5(03):1-12.
[55] HUNG S C, WU H C, TSENG M H. Remote sensing scene classification and explanation using RSSCNet and LIME[J]. Applied Sciences-Basel, 2020, 10(18):24.
[56] XIE X, ZHOU X R, LI J Z, et al. An ontology-based framework for complex urban object recognition through integrating visual features and interpretable semantics[J]. Complexity, 2020, 2020:15.
[57] CAMPOS-TABERNER M, GARCIA-HARO F J, MARTINEZ B, et al. Understanding deep learning in land use classification based on Sentinel-2 time series[J]. Scientific Reports, 2020, 10(1):12.
[58] WOLANIN A, MATEO-GARCIA G, CAMPS-VALLS G, et al. Estimating and understanding crop yields with explainable deep learning in the Indian Wheat Belt[J]. Environmental Research Letters, 2020, 15(2):12.
[59] YAN X, ZANG Z, JIANG Y Z, et al. A spatial-temporal interpretable deep learning model for improving interpretability and predictive accuracy of satellite-based PM2. 5[J]. Environmental Pollution, 2021, 273, 116459.
[60] GOODFELLOW I J, SHLENS J, SZEGEDY C. Explaining and harnessing adversarial examples[C]//Proceedings of 2015 International Conference on learning Representations. San Diego:[s.n.], 2015.
[61] 中国电子工业标准化技术协会. 人工智能深度学习算法评估规范:T/CESA1026-2018[S]. 北京:[s.n.], 2018. China Electronics Standardization Association. Artificial intelligence-assessment specification for deep learning algorithms:T/CESA 1026-2018[S]. Beijing:[s.n.], 2018.
[62] 全国信息安全标准化技术委员会. 人工智能安全标准化白皮书[EB/OL].[2020-12-10]. https://www.tc260.org.cn/front/postDetail.html?id=2019031151659. National Information Security Standardization Technical Committee. White paper on standardization of artificial intelligence security[EB/OL].[2020-12-10]. https://www.tc260.org.cn/front/postDetail.html?id=2019031151659.
[63] International Electro-Technical Commission. Information technology-artificial intelligence-overview of trustworthiness in artificial intelligence:ISO/IEC TR 24028:2020[S]. Geneva:International Organization for Standardization. 2020.
[64] 张效康. 地理国情监测数据可靠性分析与控制方法研究[D]. 武汉:武汉大学, 2017. ZHANG Xiaokang. Reliability analysis and controlling methods for national geographic statemonitoring data[D]. Wuhan:Wuhan University, 2017.
[65] 武旭芳. 地理国情遥感监测数据变化检测可靠性控制算法[J]. 北京测绘, 2020, 34(11):1559-1563. WU Xufang. Reliability control method of remote sensing monitoring data change detectionin geographical situation[J]. Beijing Surveying and Mapping, 2020, 34(11):1559-1563.