GLFFNet model for remote sensing image scene classification

doi:10.11947/j.AGCS.2023.20220286

Abstract

Abstract: Traditional scene classification models cannot perform multi-scale key feature extraction in remote sensing images in a lightweight and efficient manner. Deep learning methods generally have shortcomings such as large amount of calculation and slow convergence speed. In view of the above problems, this paper makes full use of the ability of CNN structure and Transformer structure to extract features at different scales, and proposes a feature extract module, named global and local features fused (GLFF) block. Based on this module, a lightweight remote sensing image scene classification model, GLFFNet, is designed, which has better local information and global information extraction ability. In order to verify the effectiveness of GLFFNet, this paper uses the open-source remote sensing image datasets RSSCN7 and SIRI-WHU to verify the complexity and recognition ability of GLFFNet and other deep learning networks. Finally, GLFFNet achieves recognition accuracy of up to 94.82% and 95.83% on RSSCN7 and SIRI-WHU datasets, respectively, which is better than other state-of-the-art models.

Key words: remote sensing image, scene classification, convolutional neural network, Transformer, GLFFNet model

CLC Number:

P237

WANG Wei, DENG Jiwei, WANG Xin, LI Zhiyong, YUAN Ping. GLFFNet model for remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(10): 1693-1702.

References

[1] 龚健雅,张觅,胡翔云,等.智能遥感深度学习框架与模型设计[J].测绘学报, 2022, 51(4):475-487. DOI:10.11947/j.AGCS.2022. 20220027. GONG Jianya, ZHANG Mi, HU Xiangyun, et al. The design of deep learning framework and model for intelligent remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(4):475-487. DOI:10.11947/j.AGCS.2022. 20220027.
[2] 乔星星,施文灶,刘芫汐,等.基于ResNet双注意力机制的遥感图像场景分类[J].计算机系统应用, 2021, 30(8):243-248. QIAO Xingxing, SHI Wenzao, LIU Yuanxi, et al. Remote sensing image scene classification based on ResNet and dual attention mechanism[J]. Computer Systems&Applications, 2021, 30(8):243-248.
[3] WANG Wang, LI Shaochun, WANG Wang, et al. A simple deep learning network for classification of 3 D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3):49-59.
[4] 边小勇,费雄君,穆楠.基于尺度注意力网络的遥感图像场景分类[J].计算机应用, 2020, 40(3):872-877. BIAN Xiaoyong, FEI Xiongjun, MU Nan. Remote sensing image scene classification based on scale-attention network[J]. Journal of Computer Applications, 2020, 40(3):872-877.
[5] 许夙晖,慕晓冬,赵鹏,等.利用多尺度特征与深度网络对遥感影像进行场景分类[J].测绘学报, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016. 20150623. XU Suhui, MU Xiaodong, ZHAO Peng, et al. Scene classification of remote sensing image based on multi-scale feature and deep neural network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(7):834-840. DOI:10.11947/j.AGCS.2016. 20150623.
[6] 蔡之灵,翁谦,叶少珍,等.基于Inception-V3模型的高分遥感影像场景分类[J].国土资源遥感, 2020, 32(3):80-89. CAI Zhiling, WENG Qian, YE Shaozhen, et al. Remote sensing image scene classification based on Inception-V3[J]. Remote Sensing for Land&Resources, 2020, 32(3):80-89.
[7] PAN S J, YANG Qiang. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2010, 22(10):1345-1359.
[8] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:2818-2826.
[9] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31 st International Conference on Neural Information Processing Systems. Long Beach:ACM Press, 2017:6000-6010.
[10] SWALPA K, ANKUR D, DANFENG H, et al. Multimodal fusion transformer for remote sensing image classification[EB/OL].[2022-04-29]. https://arxiv.org/abs/2203.16952.
[11] HONG D, HAN Z, YAO J, et al. Spectral Former:rethinking hyperspectral image classification with transformers[EB/OL].[2022-04-29]. https://arxiv.org/abs/2107.02988.
[12] BAZI Y, BASHMAL L, AL RAHHAL M M, et al. Vision transformers for remote sensing image classification[J]. Remote Sensing, 2021, 13(3):516.
[13] 宋中山,梁家锐,郑禄,等.基于双向门控尺度特征融合的遥感场景分类[J].计算机应用, 2021, 41(9):2726-2735. SONG Zhongshan, LIANG Jiarui, ZHENG Lu, et al. Remote sensing scene classification based on bidirectional gated scale feature fusion[J]. Journal of Computer Applications, 2021, 41(9):2726-2735.
[14] 龚希,吴亮,谢忠,等.融合全局和局部深度特征的高分辨率遥感影像场景分类方法[J].光学学报, 2019, 39(3):0301002. GONG Xi, WU Liang, XIE Zhong, et al. Classification method of high-resolution remote sensing scenes based on fusion of global and local deep features[J]. Acta Optica Sinica, 2019, 39(3):0301002.
[15] 郑卓,方芳,刘袁缘,等.高分辨率遥感影像场景的多尺度神经网络分类法[J].测绘学报, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018. 20170191. ZHENG Zhuo, FANG Fang, LIU Yuanyuan, et al. Joint multi-scale convolution neural network for scene classification of high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(5):620-630. DOI:10.11947/j.AGCS.2018. 20170191.
[16] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:770-778.
[17] LIU Ze, LIN Yutong, CAO Yue, et al. Swin transformer:hierarchical vision transformer using shifted windows[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2022:9992-10002.
[18] XIE Saining, GIRSHICK R, DOLLÁR P, et al. Aggregated residual transformations for deep neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017:5987-5995.
[19] GLOROT X, BOR DE S A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14 th International Conference on Artificial Intelligence and Statistics. Sardinia:IEEE, 2011:315-323.
[20] WU Hang, LIU Baozhen, SU Weihua, et al. Deep filter banks for land-use scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(12):1895-1899.
[21] ZHU Qiqi, ZHONG Yanfei, ZHAO Bei, et al. Bag-of-visual-words scene classifier with local and global features for high spatial resolution remote sensing imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(6):747-751.
[22] IOFFE S, SZEGEDY C. Batch normalization:accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32 nd International Conference on International Conference on Machine Learning. New York:ACM Press, 2015:448-456.
[23] KINGMA D, BA J. Adam:a method for stochastic optimization[C]//Proceedings of the 3 rd International Conference for Learning Representations. Xi'an:IEEE, 2015, 1-13.
[24] ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. ShuffleNet:an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:6848-6856.
[25] HU Jie, SHEN Li, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8):2011-2023.
[26] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//Proceedings of 2015 International Conference for Learning Representations. San Diego:ICLR, 2015:463-476.
[27] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:1-9.
[28] TAN M, LE Q V. EfficientNetV2:smaller models and faster training[C]//Proceedings of 2021 International Conference on Machine Learning. Xi'an:IEEE, 2021:10096-10106.
[29] LIU Z, MAO H, WU C Y, et al. A ConvNet for the 2020 s[EB/OL].[2022-4-29]. https://arxiv.org/abs/2201.03545.
[30] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16 x16 words:transformers for image recognition at scale[C]//Proceedings of the 9 th International Conference on Learning Representations. Virtual Event:IEEE, 2020:1-12.
[31] CHEN Y, DAI X, CHEN D, et al. Mobile-Former:bridging mobile net and transformer[EB/OL].[2022-04-29]. https://arxiv.org/abs/2108.05895.
[32] PENG Zhiliang, HUANG Wei, GU Shanzhi, et al. Conformer:local features coupling global representations for visual recognition[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. Montreal:IEEE, 2022:357-366.
[33] GUO J, HAN K, WU H, et al. CMT:convolutional neural networks meet vision transformers[EB/OL].[2022-04-29]. https://arxiv.org/abs/2107.06263.
[34] GUO M, LU C, LIU Z, et al. Visual attention network[EB/OL].[2022-04-29]. https://arxiv.org/abs/2202.09741.
[35] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM:visual explanations from deep networks via gradient-based localization[J]. International Journal of Computer Vision, 2020, 128(2):336-359.
[36] SPRINGENBERG J T, DOSOVITSKIY A, BROX T, et al. Striving for simplicity:the all convolutional net[EB/OL].[2022-04-29]. https://arxiv.org/abs/1412.6806 v1