Segmentation method of high-score remote sensing target based on road neighborhood relationship

doi:10.11947/j.AGCS.2025.20240276

Abstract

Abstract:

In recent years, the advancement of deep learning technology has been a continuous process. The application of remote sensing image instance segmentation to a variety of datasets has yielded effective and efficient segmentation results. However, existing methods for the instances segmentation of remote sensing image usually only fuse spatial context information at the pixel level, while neglecting the mining of spatial relationships between feature targets. In this paper, we propose a research project on the high-resolution remote sensing target segmentation method fusing road neighborhood relations based on YOLOv8. This method introduces a coordinate attention module and a redesigned distance loss function, which focus on the spatial relations among feature targets and combine them with visual information to enhance the semantic understanding and pixel-level segmentation accuracy. This approach significantly improves the accuracy and efficiency of target segmentation.

Key words: spatial relation, instance segmentation, YOLO, attention module, road neighborhood relations

CLC Number:

P231

Chaoyang WANG, Yishao SU, Jiancheng LUO, Xiaodong HU, Liegang XIA. Segmentation method of high-score remote sensing target based on road neighborhood relationship[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(7): 1294-1304.

Figures/Tables 11

Fig. 1

Fig. 2

Tab. 1

Tab. 2

Fig. 3

Tab. 3

Tab. 4

Tab. 5

Fig. 4

Fig. 5

Fig. 6

References 27

[1]	LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[2]	HINTON G E, SALAKHUTDINOV R R. Reducing the dimensionality of data with neural networks[J]. Science, 2006, 313(5786): 504-507.
[3]	LIU Li, OUYANG Wanli, WANG Xiaogang, et al. Deep learning for generic object detection: a survey[J]. International Journal of Computer Vision, 2020, 128(2): 261-318.
[4]	张兵, 高连如, 李嘉鑫, 等. 高/多光谱遥感图像超分辨率融合研究进展与展望[J]. 测绘学报, 2023, 52(7): 1074-1089. DOI: . doi: 10.11947/j.AGCS.2023.20220499
	ZHANG Bing, GAO Lianru, LI Jiaxin, et al. Advances and prospects in hyperspectral and multispectral remote sensing image super-resolution fusion[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(7): 1074-1089. DOI: . doi: 10.11947/j.AGCS.2023.20220499
[5]	李德仁, 张良培, 夏桂松. 遥感大数据自动分析与数据挖掘[J]. 测绘学报, 2014, 43(12): 1211-1216.
	LI Deren, ZHANG Liangpei, XIA Guisong. Automatic analysis and mining of remote sensing big data[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(12): 1211-1216.
[6]	BLASCHKE T. Object based image analysis for remote sensing[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(1): 2-16.
[7]	陈军, 赵仁亮. GIS空间关系的基本问题与研究进展[J]. 测绘学报, 1999, 28(2): 95-102.
	CHEN Jun, ZHAO Renliang. Spatial relations in GIS: a survey on its key issues and research progress[J]. Acta Geodaetica et Cartographic Sinica, 1999, 28(2): 95-102.
[8]	尤淑撑, 刘顺喜, 李小文, 等. 基于空间约束关系的土地利用/覆被遥感分类方法研究[J]. 农业工程学报, 2005, 21(9): 51-55.
	YOU Shucheng, LIU Shunxi, LI Xiaowen, et al. Land-use/land-cover classification method under spatial constraint[J]. Transactions of the Chinese Society of Agricultural Engineering, 2005, 21(9): 51-55.
[9]	赵红蕊, 阎广建, 邓小炼, 等. 一种简单加入空间关系的实用图像分类方法[J]. 遥感学报, 2003, 7(5): 358-363.
	ZHAO Hongrui, YAN Guangjian, DENG Xiaolian, et al. A classification method based on spatial information[J]. Journal of Remote Sensing, 2003, 7(5): 358-363.
[10]	舒宁. 关于遥感影像处理分析的理论与方法之若干问题[J]. 武汉大学学报(信息科学版), 2007, 32(11): 1007-1010, 1015.
	SHU Ning. Aspects of the theories and methodologies for remote sensed image processing and analysis[J]. Geomatics and Information Science of Wuhan University, 2007, 32(11): 1007-1010, 1015.
[11]	胡圣武, 王宏涛. 空间关系的研究进展[J]. 测绘科学, 2007, 32(1): 153-156, 159, 166.
	HU Shengwu, WANG Hongtao. Recent research of spatial relation[J]. Science of Surveying and Mapping, 2007, 32(1): 153-156, 159, 166.
[12]	吕可枫, 张永生, 于英, 等. 语义信息与地理配准相结合的实例目标定位[J]. 测绘学报, 2023, 52(8): 1375-1386. DOI: . doi: 10.11947/j.AGCS.2023.20220008
	LÜ Kefeng, ZHANG Yongsheng, YU Ying, et al. Instance object localization based on semantic information and geo-registration[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(8): 1375-1386. DOI: . doi: 10.11947/j.AGCS.2023.20220008
[13]	CUI Wei, WANG Fei, HE Xin, et al. Multi-scale semantic segmentation and spatial relationship recognition of remote sensing images based on an attention model[J]. Remote Sensing, 2019, 11(9): 1044.
[14]	王骁, 钱海忠, 何海威, 等. 顾及邻域居民地群组相似性的道路网匹配方法[J]. 测绘学报, 2016, 45(1): 103-111. DOI: . doi: 10.11947/j.AGCS.2016.20140588
	WANG Xiao, QIAN Haizhong, HE Haiwei, et al. Matching method for road networks considering the similarity of the neighborhood habitation cluster[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(1): 103-111. DOI: . doi: 10.11947/j.AGCS.2016.20140588
[15]	MUFFORD J T, HILL D J, FLOOD N J, et al. Use of unmanned aerial vehicles (UAVs) and photogrammetric image analysis to quantify spatial proximity in beef cattle[J]. Journal of Unmanned Vehicle Systems, 2019, 7(3): 194-206.
[16]	ZHENG Z, ZHONG Y, WANG J, et al. Foreground-aware relation network for geospatial object segmentation in high spatial resolution remote sensing imagery[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Berlin: IEEE, 2020.
[17]	刘万增, 陈杭, 任加新, 等. 基于混合智能的街景影像知识提取方法[J]. 测绘学报, 2024, 53(9): 1817-1828. DOI: . doi: 10.11947/j.AGCS.2024.20220720
	LIU Wanzeng, CHEN Hang, REN Jiaxin, et al. Research on knowledge extraction from street scene images based on hybrid intelligence[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(9): 1817-1828. DOI: . doi: 10.11947/j.AGCS.2024.20220720
[18]	XIA Liegang, SU Yishao, LIU Ruiyan, et al. HRRSODN: high-resolution remote sensing object detection network fusion of road-adjacent relationships[J]. Geocarto International, 2023, 38(1): 213-221.
[19]	LI Ke, WAN Gang, CHENG Gong, et al. Object detection in optical remote sensing images: a survey and a new benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 159: 296-307.
[20]	HE K, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Berlin: Springer, 2017.
[21]	BOLYA D, ZHOU Chong, XIAO Fanyi, et al. YOLACT: real-time instance segmentation[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 9156-9165.
[22]	WANG X, ZHANG R, KONG T, et al. Solov2: dynamic and fast instance segmentation[J]. Advances in Neural Information Processing Systems, 2020, 33: 17721-17732.
[23]	TIAN Zhi, SHEN Chunhua, WANG Xinlong, et al. BoxInst: high-performance instance segmentation with box annotations[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 5439-5448.
[24]	KIRILLOV A, WU Y, HE K, GIRSHICK R. Pointrend: image segmentation as rendering[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New York: IEEE, 2020.
[25]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. Advances in Neural Information Processing Systems, 2017, 30: 657-661.
[26]	LIU Z, LIN Y, CAO Y, et al. SwinTransformer: hierarchical vision transformer using shifted windows[C]//Proceedings of 2021 IEEE/CVF International Conference on Computer Vision. New York: IEEE, 2021.
[27]	CHEN Keyan, LIU Chenyang, CHEN Hao, et al. RSPrompter: learning to prompt for remote sensing instance segmentation based on visual foundation model[J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-17.

数据集	方法	Tollstation	Servicearea	加油站	Box AP（IoU=0.5）	F₁值
SR	YOLOv8	91.0	92.9	88.1	90.7	87.6
	YOLOv8+HROR-CA	92.4	93.7	87.8	91.3	88.0
	YOLOv8+HROR-Rloss	92.6	92.9	89.7	91.7	89.0
	YOLOv8+HROR-CA+HROR-Rloss	95.2	93.9	88.6	92.6	89.3

数据集	方法	Tollstation	Servicearea	加油站	Mask AP（IoU=0.5）	F₁值
SR	YOLOv8	89.9	94.5	88.2	90.9 8	8.5
	YOLOv8+HROR-CA	91.1	95.4	88.2	91.6 8	9.0
	YOLOv8+HROR-Rloss	92.3	93.7	90.0	92.0 8	9.0
	YOLOv8+HROR-CA+HROR-Rloss	94.9	95.1	89.2	93.0 9	0.0

研究方法	Box AP	Mask AP	Box F₁值	Mask F₁值
Mask R-CNN	82.4	82.1	81.5	81.1
SAM-det	84.0	81.0	82.6	80.8
RSPrompter	82.6	81.2	81.1	80.3
Yolact	85.9	83.3	83.4	82.3
Pointrend	84.2	84.6	82.0	82.8
Solov2	84.1	84.1	83.0	83.0
BoxInst	87.6	87.6	85.2	85.0
SwinTransformer	91.1	91.3	88.0	89.0
HRRSOSN	92.6	93.0	89.3	90.0

研究方法	Box AP	Mask AP	Box F₁值	Mask F₁值
Mask R-CNN	84.0	83.3	83.0	82.8
SAM-det	85.7	85.3	84.9	83.8
RSPrompter	84.8	84.6	83.9	83.3
Yolact	87.5	86.9	86.2	86.3
Pointrend	85.7	85.8	83.5	83.8
Solov2	—	85.2	—	84.9
BoxInst	88.4	88.7	87.3	87.1
SwinTransformer	92.4	92.5	89.6	90.0
HRRSOSN	93.9	94.1	90.4	91.2

研究方法	GFLOPs	Params/MB
Mask R-CNN	155	31.33
Yolact	82.7	50.14
BoxInst	300.9	34.73
SAM-det	127	83.75
RSPrompter	122	86.01
Pointrend	135	60.22
Solov2	34.7	18.27
SwinTransformer	252	69.11
HRRSOSN	12.1	3.28