The design of deep learning framework and model for intelligent remote sensing

doi:10.11947/j.AGCS.2022.20220027

Abstract

Abstract: The rapid development of remote sensing technology has achieved massive remote sensing images, and the deep-learning-based remote sensing image interpretation has shown certain advantages in image feature extraction and representation. However, the intelligent processing framework and information service capabilities are relatively lagging. Open-source deep learning frameworks and models cannot yet meet the requirements of intelligent remote sensing processing. Based on the analysis of existing intelligent frameworks and models, we design a dedicated deep learning framework and model with remote sensing characteristics for the problems of large remote sensing image size, large-scale changes, and multiple data channels. The focus is on the construction of a dedicated framework that takes into account remote sensing data characteristics and the preliminary experimental results on remote sensing image classification. The design of this remote sensing image interpretation framework will provide strong support for the construction of a dedicated deep learning framework and models that integrate the temporal, spatial, and spectral features of remote sensing data.

Key words: remote sensing intelligent interpretation, deep learning, dedicated framework and model, remote sensing feature

CLC Number:

P227

GONG Jianya, ZHANG Mi, HU Xiangyun, ZHANG Zhan, LI Yansheng, Jiang Liangcun. The design of deep learning framework and model for intelligent remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4): 475-487.

References

[1] 李德仁, 童庆禧, 李荣兴, 等. 高分辨率对地观测的若干前沿科学问题[J]. 中国科学: 地球科学, 2012, 42(6): 805-813. LI Deren, TONG Qingxi, LI Rongxing, et al. Some frontier scientific problems of high resolution Earth observation[J]. Scientia Sinica (Terrae), 2012, 42(6): 805-813.
[2] 李德仁, 王密, 沈欣, 等. 从对地观测卫星到对地观测脑[J]. 武汉大学学报(信息科学版), 2017, 42(2): 143-149. LI Deren, WANG Mi, SHEN Xin, et al. From earth observation satellite to earth observation brain[J]. Geomatics and Information Science of Wuhan University, 2017, 42(2): 143-149.
[3] 龚健雅. 人工智能时代测绘遥感技术的发展机遇与挑战[J]. 武汉大学学报(信息科学版), 2018, 43(12): 1788-1796. GONG Jianya. Chances and challenges for development of surveying and remote sensing in the age of artificial intelligence[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12): 1788-1796.
[4] 龚健雅, 许越, 胡翔云, 等. 遥感影像智能解译样本库现状与研究[J]. 测绘学报, 2021, 50(8): 1013-1022.DOI: 10.11947/j.AGCS.2021.20210085. GONG Jianya, XU Yue, HU Xiangyun, et al. Status analysis and research of sample database for intelligent interpretation of remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1013-1022.DOI: 10.11947/j.AGCS.2021.20210085.
[5] 张永军, 万一, 史文中, 等. 多源卫星影像的摄影测量遥感智能处理技术框架与初步实践[J]. 测绘学报, 2021, 50(8): 1068-1083.DOI: 10.11947/j.AGCS.2021.20210079. ZHANG Yongjun, WAN Yi, SHI Wenzhong, et al. Technical framework and preliminary practices of photogrammetric remote sensing intelligent processing of multi-source satellite images[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1068-1083.DOI: 10.11947/j.AGCS.2021.20210079.
[6] CHEN Wuyang, JIANG Ziyu, WANG Zhangyang, et al. Collaborative global-local networks for memory-efficient segmentation of ultra-high resolution images[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA: IEEE, 2019: 8916-8925.
[7] XING Jin, SIEBER R, CAELLI T. A scale-invariant change detection method for land use/cover change research[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018,141: 252-264.
[8] WANG Xinyu, ZHONG Yanfei, ZHANG Liangpei, et al. Spatial group sparsity regularized nonnegative matrix factorization for hyperspectral unmixing[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(11): 6287-6304.
[9] ZHONG Yanfei, LI Wenqing, WANG Xinyu, et al. Satellite-ground integrated destriping network: a new perspective for EO-1 Hyperion and Chinese hyperspectral satellite datasets[J]. Remote Sensing of Environment, 2020,237: 111416.
[10] FRENCH R M. Catastrophic forgetting in connectionist networks[J]. Trends in Cognitive Sciences, 1999, 3(4): 128-135.
[11] LEE Sangwoo,KIM Jinhwa,JUN Jaehyun,et al.Overcoming catastrophic forgetting by incremental moment matching[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. New York, USA: Red Hook,2017: 4655-4665.
[12] PAN Ting.Dragon: a computation graph virtual machine based deep learning framework[EB/OL].[2022-02-28]. https://arxiv.org/abs/1707.08265.
[13] TEAM TTD, AL-RFOU R, ALAIN G, et al.Theano: a Python framework for fast computation of mathematical expressions[EB/OL].[2022-02-28].https://arxiv.org/abs/1605.02688.
[14] JIA Yangqing,SHELHAMER E,DONAHUE J,et al.Caffe:convolutional architecture for fast feature embedding[C]//Proceedings of the 22nd ACM international conference on Multimedia.New York, USA:Association for Computing Machinery,2014:675-678.
[15] ABADI M, AGARWAL A, BARHAM P,et al. Tensorflow: large-scale machine learning on heterogeneous distributed systems[EB/OL].[2022-02-28].https://arxiv.org/pdf/1603.04467.pdf.
[16] PASZKE A, GROSS S, MASSA F,et al.PyTorch: an imperative style, high-performance deep learning library[EB/OL].[2022-02-28].https://arxiv.org/abs/1912.01703.
[17] CHEN Tianqi, LI Mu, LI Yutian, et al. MXNet: a flexible and efficient machine learning library for heterogeneous distributed systems[EB/OL].[2022-02-28].https://arxiv.org/pdf/1512.01274.pdf.
[18] KRIZHEVSKY Alex,SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[19] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL, USA:IEEE,2009: 248-255.
[20] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL].[2022-02-28].https://arxiv.org/abs/1409.1556.
[21] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA,USA: IEEE, 2015: 1-9.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, USA: IEEE, 2016: 770-778.
[23] HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017.
[24] SUN Ke, XIAO Bin, LIU Dong, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA: IEEE, 2019: 5686-5696.
[25] LEVINE S, FINN C, DARRELL T, et al. End-to-end training of deep visuomotor policies[EB/OL].[2022-02-28].https://doi.org/10.48550/arXiv.1504.00702.
[26] REAL E, MOORE S, SELLE A, et al. Large-scale evolution of image classifiers[EB/OL].[2022-02-28].https://arxiv.org/abs/1703.01041.
[27] 黄樟灿.演化计算的搜索策略研究[D]. 武汉: 武汉大学, 2004. HUANG Zhangcan. Research on search strategy of evolutionary computation[D].Wuhan: Wuhan University, 2004.
[28] TAN Mingxing, CHEN Bo, PANG Ruoming, et al. MnasNet: platform-aware neural architecture search for mobile[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA: IEEE, 2019: 2815-2823.
[29] LIPTON Z C, BERKOWITZ J, ELKAN C. A critical review of recurrent neural networks for sequence learning[EB/OL].[2022-02-28].https://arxiv.org/abs/1506.00019.
[30] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL].[2022-02-28].https://arxiv.org/abs/1706.05587.
[31] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
[32] SCHULMAN J, WOLSKI F, DHARIWAL P, et al. Proximal policy optimization algorithms[EB/OL].[2022-02-28].https://arxiv.org/abs/1707.06347.
[33] BOTTOU L. Stochastic gradient descent tricks[EB/OL].[2022-02-28].http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.419.1465.
[34] NOGUEIRA K, PENATTI O A B, DOS SANTOS J A. Towards better exploiting convolutional neural networks for remote sensing scene classification[J]. Pattern Recognition, 2017,61: 539-556.
[35] XIAO Zhifeng, LONG Yang, LI Deren, et al. High-resolution remote sensing image retrieval based on CNNs from a dimensional perspective[J]. Remote Sensing, 2017, 9(7): 725.
[36] LONG Yang, GONG Yiping, XIAO Zhifeng, et al. Accurate object localization in remote sensing images based on convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(5): 2486-2498.
[37] DING Jian, XUE Nan, XIA Guisong, et al. Object detection in aerial images: a large-scale benchmark and challenges[EB/OL].[2022-02-28].https://arxiv.org/abs/2102.12219v2.
[38] HUANG Bo, ZHAO Bei, SONG Yimeng. Urban land-use mapping using a deep convolutional neural network with high spatial resolution multispectral remote sensing imagery[J]. Remote Sensing of Environment,2018,214:73-86.
[39] HU HANGTAO, CAI SHUO, WANG WEI, et al. A semantic segmentation approach based on DeepLab network in high-resolution remote sensing images [C]//Proceedings of the 10th International Conference on Image and Graphics. Beijing,China:Springer, 2019.
[40] KHELIFI L, MIGNOTTE M. Deep learning for change detection in remote sensing images: comprehensive review and meta-analysis[J]. IEEE Access, 2020(8): 126385-126400.
[41] ZHANG Lin, HU Xiangyun, ZHANG Mi, et al. Object-level change detection with a dual correlation attention-guided detector[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021,177: 147-160.
[42] YU Dawen, JI Shunping, LIU Jin, et al. Automatic 3D building reconstruction from multi-view aerial images with deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021,171: 155-170.
[43] ALIDOOST F, AREFI H, TOMBARI F. 2D image-to-3D model: knowledge-based 3D building reconstruction (3DBR) using single aerial images and convolutional neural networks (CNNs)[J]. Remote Sensing, 2019, 11(19): 2219.