Dual-channel parallel hybrid convolutional neural networks based classification method for high-resolution remote sensing image

doi:10.11947/j.AGCS.2023.20220163

Abstract

Abstract: High spatial resolution remote sensing images have rich spatial detail information and multi-spectral information. Previous studies have shown that two-dimensional convolutional neural networks (CNN) are suitable for extracting spatial information, while three-dimensional CNN are more suitable for extracting spectral information. In order to make better use of spatio-spectral information, this paper innovatively proposes a dual-channel parallel hybrid convolutional neural networks (DPHCNN), which fully combines the advantages of two-dimensional and three-dimensional CNN in spatio-spectral information extraction. Simultaneously, the hybrid attention mechanism and multi-scale convolution are introduced to enhance the extraction ability of spatial detail features to achieve accurate classification of high-resolution images. In the experiment, the GF-2 image dataset was used for verification. Compared with state-of-the-art deep learning classification methods, the DPHCNN method proposed in this paper not only has the highest classification accuracy and better classification efficiency but maintains the highest robustness in multi-temporal images classification, which has more advantages in comprehensive evaluation.

Key words: hybrid convolutional neural network, high-resolution remote sensing images, multi-scale convolution, hybrid attention mechanism, image classification

CLC Number:

P227

GU Xiaohu, LI Zhengjun, MIAO Jianhao, LI Xinghua, SHEN Huanfeng. Dual-channel parallel hybrid convolutional neural networks based classification method for high-resolution remote sensing image[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 798-807.

References

[1] ZHANG Xiao, LIU Liangyun, CHEN Xidong, et al. Fine land-cover mapping in China using landsat datacube and an operational SPECLib-based approach[J]. Remote Sensing, 2019, 11(9):1056.
[2] 李道纪, 郭海涛, 卢俊, 等. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8):1051-1064.DOI:10.11947/j.AGCS.2020.20190407. LI Daoji, GUO Haitao, LU Jun, et al. A remote sensing image classification procedure based on multilevel attention fusion U-Net[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8):1051-1064.DOI:10.11947/j.AGCS.2020.20190407.
[3] GISLASON P O, BENEDIKTSSON J A, SVEINSSON J R. Random forests for land cover classification[J]. Pattern Recognition Letters, 2006, 27(4):294-300.
[4] HUANG Xin, ZHANG Liangpei. An SVM ensemble approach combining spectral, structural, and semantic features for the classification of high-resolution remotely sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(1):257-272.
[5] YUAN Qiangqiang, SHEN Huanfeng, LI Tongwen, et al. Deep learning in environmental remote sensing:achievements and challenges[J]. Remote Sensing of Environment, 2020, 241(11):111716.
[6] ZHANG Liangpei, ZHANG Lefei, DU Bo. Deep learning for remote sensing data:a technical tutorial on the state of the art[J]. IEEE Geoscience and Remote Sensing Magazine, 2016, 4(2):22-40.
[7] LÜ Qi, DOU Yong, NIU Xin, et al. Urban land use and land cover classification using remotely sensed SAR data through deep belief networks[J]. Journal of Sensors, 2015, 20(15):1-10.
[8] LIU Ying, WU Linzhi. Geological disaster recognition on optical remote sensing images using deep learning[J]. Procedia Computer Science, 2016, 91:566-575.
[9] MOU Lichao, PEDRAM G, ZHU Xiaoxiang. Deep recurrent neural networks for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7):3639-3655.
[10] AUDEBERT N, SAUX B L, SEBASTIEN L. Deep learning for classification of hyperspectral data:a comparative review[J]. IEEE Geoscience and Remote Sensing Magazine, 2019, 7(2):159-173.
[11] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16x16 words:transformers for image recognition at scale[C]//Proceedings of 2021 International Conference on Learning Representation (ICLR).Virtual Event:ICLR, 2021.
[12] RONNEBERGER O, FISCHER P, BROX T. U-Net:Convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer- Assisted Intervention (MICCAI). Munich:Springer, 2015:234-241.
[13] 杨瑞, 祁元, 苏阳. 深度学习U-Net方法及其在高分辨卫星影像分类中的应用[J]. 遥感技术与应用, 2020, 35(4):767-774. YANG Rui, QI Yuan, SU Yang. U-net neural networks and its application in high resolution satellite image classification[J]. Remote Sensing Technology and Application, 2020, 35(4):767-774.
[14] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab:semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4):834-848.
[15] 叶沅鑫, 谭鑫, 孙苗苗, 等. 基于增强DeepLabV3网络的高分辨率遥感影像分类[J]. 测绘通报, 2021(4):40-44. YE Yuanxin, TAN Xin, SUN Miaomiao, et al. High-resolution remote sensing image classification based on improved DeepLabV3 network[J]. Bulletin of Surveying and Mapping, 2021(4):40-44.
[16] STRUDEL R, GARCIA R, LAPTEV I, et al. Segmenter:transformer for semantic segmentation[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision (ICCV). Montreal:IEEE, 2022:7242-7252.
[17] LI Ying, ZHANG Haokui, XUE Xizhe, et al. Deep learning for remote sensing image classification:a survey[J]. Wiley Interdisciplinary Reviews-Data Mining and Knowledge Discovery, 2018, 8(6):e1264.
[18] HU Wei, HUANG Yangyu, WEI Li, et al. Deep convolutional neural networks for hyperspectral image classification[J]. Journal of Sensors, 2015, 2015(2):1-12.
[19] MAKANTASIS K, KARANTZALOS K, DOULAMIS A, et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]//Proceedings of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Milan:IEEE, 2015:4959-4962.
[20] LEE H, KWON H. Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2017, 26(10):4843-4855.
[21] LI Ying, ZHANG Haokui, SHEN Qiang. Spectral-spatial classification of hyperspectral imagery with 3D convolutional neural network[J]. Remote Sensing, 2017, 9(1):67-88.
[22] ROY S K, KRISHNA G, DUBEY S R, et al. HybridSN:exploring 3D-2D CNN feature hierarchy for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(2):277-281.
[23] 刘冰, 余旭初, 张鹏强, 等. 联合空-谱信息的高光谱影像深度三维卷积网络分类[J]. 测绘学报, 2019, 48(1):53-63.DOI:10.11947/j.AGCS.2019.20170578. LIU Bing, YU Xuchu, ZHANG Pengqiang, et al. Deep 3D convolutional network combined with spatial-spectral features for hyperspectral image classification[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):53-63.DOI:10.11947/j.AGCS.2019.20170578.
[24] SRIVASTAVA N, HINTON G, KRIZHEVSKY A, et al. Dropout:prevent NN from overfitting[J]. Journal of Machine Learning Research, 2014, 15:1929-1958.
[25] 李星华, 白学辰, 李正军, 等. 面向高分影像建筑物提取的多层次特征融合网络[J]. 武汉大学学报(信息科学版), 2022, 47(8):1236-1244. LI Xinghua, BAI Xuechen, LI Zhengjun, et al. High-resolution image building extraction based on multi-level feature fusion network[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8):1236-1244.
[26] HE Mingyi, LI Bo, CHEN Huahui. Multi-scale 3D deep convolutional neural network for hyperspectral image classification[C]//Proceedings of 2017 IEEE International Conference on Image Processing (ICIP). Beijing:IEEE, 2018:3904-3908.
[27] WOO S, PARK J, LEE J Y, et al. CBAM:convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision (ECCV). Munich:Springer, 2018:3-19.
[28] CAO Yue, XU Jiarui, LIN S, et al. GCNet:Non-local networks meet squeeze-excitation networks and beyond[C]//Proceedings of 2019 International Conference on Computer Vision (ICCV). Seoul:IEEE, 2019:1971-1980.
[29] BOTTOU Leon. Large-scale machine learning with stochastic gradient descent[C]//Proceedings of 2010 COMPSTAT. Paris:Springer, 2010:177-186.
[30] KANG Xudong, ZHUO Binbin, DUAN Puhong. Dual-path network-based hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(3):447-451.
[31] BEN HAMIDA A, BENOIT A, LAMBERT P, et al. 3D deep learning approach for remote sensing image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8):4420-4434.