Remote sensing image scene classification method integrating spatial and semantic information of transferred features

doi:10.11947/j.AGCS.2025.20250199

Abstract

Abstract:

To address scene confusion and low classification accuracy caused by complex spatial distributions of ground objects in remote sensing (RS) scenes, a novel classification method integrating spatial and semantic information from transferred features of RS scenes is proposed. Leveraging the representation capabilities of different-level transferred features from a deep convolutional neural network for local detail and global semantic information, a deep spatial co-occurrence matrix is constructed to quantify the spatial co-occurrence patterns of local features, which are then fused with high-level semantic features. The resulting spatial-semantic joint feature synergistically represents scene spatial and semantic information, thereby enhancing recognition capability for complex RS scenes. Experiments on several RS scene classification datasets demonstrate the proposed method effectively discriminates complex and confusing scenes, showing advantages in spatial information representation and classification performance improvement.

Key words: remote sensing image, scene classification, transferred features, deep spatial co-occurrence matrix, spatial-semantic information fusion

CLC Number:

P237

Xi GONG, Zhanlong CHEN, Hengqiang ZHENG, Sheng HU, Hongyan ZHANG. Remote sensing image scene classification method integrating spatial and semantic information of transferred features[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(11): 2009-2025.

Figures/Tables 27

Fig. 1

Tab. 1

Fig. 2

Fig. 3

Fig. 4

Tab. 2

Tab. 3

Fig. 5

Fig. 6

Tab. 4

Fig. 7

Tab. 5

Tab. 6

Tab. 7

Fig. 8

Fig. 9

Fig. 10

Tab. 8

Tab. 9

Fig. 11

Tab. 10

Fig. 12

Tab. 11

Fig. 13

Fig. 14

Tab. 12

Tab. 13

References 34

[1]	ROWLEY J. The wisdom hierarchy: representations of the DIKW hierarchy[J]. Journal of Information Science, 2007, 33(2): 163-180.
[2]	郑卓, 方芳, 刘袁缘, 等. 高分辨率遥感影像场景的多尺度神经网络分类法[J]. 测绘学报, 2018, 47(5): 620-630. DOI: 10.11947/j.AGCS.2018.20170191.
	ZHENG Zhuo, FANG Fang, LIU Yuanyuan, et al. Joint multi-scale convolution neural network for scene classification of high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(5): 620-630. DOI: 10.11947/j.AGCS.2018.20170191.
[3]	李彦胜, 吴敏郎, 张永军. 知识图谱约束深度网络的高分辨率遥感影像场景分类[J]. 测绘学报, 2024, 53(4): 677-688. DOI: 10.11947/j.AGCS.2024.20230125.
	LI Yansheng, WU Minlang, ZHANG Yongjun. Knowledge graph-guided deep network for high-resolution remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(4): 677-688. DOI: 10.11947/j.AGCS.2024.20230125.
[4]	LI Yansheng, WU Yuning, CHENG Gong, et al. MEET: a million-scale dataset for fine-grained geospatial scene classification with zoom-free remote sensing imagery[J]. CAA Journal of Automatica Sinica, 2025, 12(5): 1004-1023.
[5]	HU Fan, XIA Guisong, HU Jingwen, et al. Transferring deep convolutional neural networks for the scene classification of high-resolution remote sensing imagery[J]. Remote Sensing, 2015, 7(11): 14680-14707.
[6]	NOGUEIRA K, PENATTI O A B, DOS SANTOS J A. Towards better exploiting convolutional neural networks for remote sensing scene classification[J]. Pattern Recognition, 2017, 61: 539-556.
[7]	CASTELLUCCIO M, POGGI G, SANSONE C, et al. Land use classification in remote sensing images by convolutional neural networks[J]. Acta Ecologica Sinica, 2015, 28(2): 627-635.
[8]	ZHOU Weixun, SHAO Zhenfeng, CHENG Qimin. Deep feature representations for high-resolution remote sensing scene classification[C]//Proceedings of 2016 International Workshop on Earth Observation and Remote Sensing Applications. Guangzhou: IEEE, 2016: 338-342.
[9]	CHAIB S, LIU Huan, GU Yanfeng, et al. Deep feature fusion for VHR remote sensing scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(8): 4775-4784.
[10]	ÖZYURT F. Efficient deep feature selection for remote sensing image recognition with fused deep learning architectures[J]. The Journal of Supercomputing, 2020, 76: 8413-8431.
[11]	PENATTI O A B, NOGUEIRA K, DOS SANTOS J A. Do deep features generalize from everyday objects to remote sensing and aerial scenes domains?[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Boston: IEEE, 2015: 44-51.
[12]	YANDEX A B, LEMPITSKY V. Aggregating local deep features for image retrieval[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 1269-1277.
[13]	CSURKA G, DANCEC R, FAN L X, et al. Visual categorization with bags of keypoints[C]//Proceedings of 2014 Workshop on Statistical Learning in Computer Vision. Zurich: Springer, 2014: 1-22.
[14]	JEGOU H, DOUZE M, SCHMID C, et al. Aggregating local descriptors into a compact image representation[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 3304-3311.
[15]	WANG Jinjun, YANG Jianchao, YU Kai, et al. Locality-constrained linear coding for image classification[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco: IEEE, 2010: 3360-3367.
[16]	龚希, 吴亮, 谢忠, 等. 融合全局和局部深度特征的高分辨率遥感影像场景分类方法[J]. 光学学报, 2019, 39(3): 19-29.
	GONG Xi, WU Liang, XIE Zhong, et al. Classification method of high-resolution remote sensing scenes based on fusion of global and local deep features[J]. Acta Optica Sinica, 2019, 39(3): 19-29.
[17]	BRATASANU D, NEDELCU I, DATCU M. Bridging the semantic gap for satellite image annotation and automatic mapping applications[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2011, 4(1): 193-204.
[18]	LAZEBNIK S, SCHMID C, PONCE J. Beyond bags of features: spatial pyramid matching for recognizing natural scene categories[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2006: 2169-2178.
[19]	YANG Yi, NEWSAM S. Spatial pyramid co-occurrence for image classification[C]//Proceedings of 2011 International Conference on Computer Vision. Barcelona: IEEE, 2011: 1465-1472.
[20]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large scale image recognition[EB/OL]. [2025-05-08]. https://arxiv.org/abs/1409.1556.
[21]	龚希, 陈占龙, 吴亮, 等. 用于高分辨遥感影像场景分类的迁移学习混合专家分类模型[J]. 光学学报, 2021, 41(23): 19-31.
	GONG Xi, CHEN Zhanlong, WU Liang, et al. Transfer learning based mixture of experts classification model for high-resolution remote sensing scene classification[J]. Acta Optica Sinica, 2021, 41(23): 19-31.
[22]	BREIMAN L. Random forests[J]. Machine Learning, 2001, 45(1): 5-32.
[23]	王威, 李希杰, 王新. ADC-CPANet：一种局部—全局特征融合的遥感图像分类方法[J]. 遥感学报, 2024, 28(10): 2661-2672.
	WANG Wei, LI Xijie, WANG Xin. ADC-CPANet: a remote sensing image classification method based on local-global feature fusion[J]. National Remote Sensing Bulletin, 2024, 28(10): 2661-2672.
[24]	王威, 郑薇, 王新. 面向遥感图像场景分类的LAG-MANet模型[J]. 测绘学报, 2024, 53(7): 1371-1383. DOI: 10.11947/j.AGCS.2024.20230074.
	WANG Wei, ZHENG Wei, WANG Xin. LAG-MANet model for remote sensing image scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(7): 1371-1383. DOI: 10.11947/j.AGCS.2024.20230074.
[25]	XU Yin, GUO Weiwei, ZHANG Zenghui, et al. Multiple embeddings contrastive pretraining for remote sensing image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19(6): 451060501-451060505.
[26]	WU Hang, LIU Baozhen, SU Weihua, et al. Deep filter banks for land-use scene classification[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(12): 1895-1899.
[27]	GAO Yue, SHI Jun, LI Jun, et al. Remote sensing scene classification with dual attention-aware network[C]//Proceedings of 2020 IEEE International Conference on Image, Vision and Computing. Beijing: IEEE, 2020: 171-175.
[28]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of 2012 International Conference on Neural Information Processing Systems. Lake Tahoe: NIPS, 2012: 1097-1105.
[29]	SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 1-9.
[30]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vega: IEEE, 2016: 770-778.
[31]	HUANG Gao, LIU Zhuang, VAN DER MAATEN L, et al. Densely connected convolutional networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 2261-2269.
[32]	ZHANG Xiangyu, ZHOU Xinyu, LIN Mengxiao, et al. ShuffleNet: an extremely efficient convolutional neural network for mobile devices[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 6848-6856.
[33]	BAI Lin, LIU Qingxin, LI Cuiling, et al. Remote sensing image scene classification using multiscale feature fusion covariance network with octave convolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5620214.
[34]	TANG Xu, LI Mingteng, MA Jingjing, et al. EMTCAL: efficient multiscale transformer and cross-level attention learning for remote sensing scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 5626915.

组别	卷积层数目	特征尺寸
conv1	2	64×224×224
conv2	2	128×112×112
conv3	4	256×56×56
conv4	4	512×28×28
conv5	4	512×14×14

数据集	场景尺寸	场景数目	训练样本占比/（%）
UCM	256像素×256像素	2100	80
SIRI	200像素×200像素	2400	80
RSSCN7	400像素×400像素	2800	50

卷积层	r值	K值
conv4：conv4_1、conv4_2、conv4_3、conv4_4	2、4、6、8、10、12、14	UCM：50、200、400、600、800、1000、1200、1400、1600、1800、2000
conv5：conv5_1、conv5_2、conv5_3、conv5_4	1、2、3、4、5、6、7	SIRI：50、100、200、300、400、500、600、700、800、900、1000

分类特征	准确率
分类特征	UCM	SIRI
方向梯度直方图特征	52.14	44.79
尺度不变特征变换特征	58.33	53.96
局部二值模式特征	31.43	46.25
SIFT+BoVW	76.81	75.63
VGG-19卷积层级联深度特征^[21]	93.33	93.48
空间特征的Min-acc	84.01	84.79
空间特征的Mid-acc	95.47	94.38
空间特征的Max-acc	98.33	96.25

数据集	Low_K	Mid_K	High_K
UCM	{400，600，800}	{1000，1200，1400}	{1600，1800，2000}
SIRI	{200，300，400}	{500，600，700}	{800，900，1000}