Road extraction networks fusing multiscale and edge features

doi:10.11947/j.AGCS.2024.20230291

Abstract

Abstract:

Extracting roads using remote sensing images is of great significance to urban development. However, due to factors such as variable scale of roads and easy to be obscured, it leads to problems such as road miss detection and incomplete edges. To address the above problems, this paper proposes a network (MeD-Net) for road extraction from remote sensing images integrating multi-scale features and focusing on edge detail features. MeD-Net consists of two parts: road segmentation and edge extraction. The road segmentation network uses multi-scale deep feature processing (MDFP) module to extract multi-scale features taking into account global and local information, and is trained using group normalization optimization model after convolution. The edge extraction network uses detail-guided fusion algorithms to enhance the detail information of deep edge features and uses attention mechanisms for feature fusion. To verify the algorithm performance, this paper conducts experiments using the Massachusetts road dataset and the GF-2 road dataset in Qingdao area. The experiments show that MeD-Net achieves the highest accuracy in both datasets in terms of intersection-over-union ratio and F1 value, and is able to extract roads at different scales and maintain road edges more completely.

Key words: road extraction, semantic segmentation, multi-scale features, edge extraction

CLC Number:

P231

Genyun SUN, Chao SUN, Aizhu ZHANG. Road extraction networks fusing multiscale and edge features[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(12): 2233-2243.

Figures/Tables 14

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Tab. 1

Fig. 8

Tab. 2

Fig. 9

Tab. 3

Tab. 4

Fig. 10

References 31

[1]	ZHU Qiqi, ZHANG Yanan, WANG Lizeng, et al. A global context-aware and batch-independent network for road extraction from VHR satellite imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175(12): 353-365.
[2]	连仁包, 张振敏, 廖一鹏, 等. 结合测地距离场与曲线平滑的遥感图像道路中心线快速提取[J]. 测绘学报, 2023, 52(8): 1317-1329. DOI:. doi: 10.11947/j.AGCS.2023.20220002
	LIAN Renbao, ZHANG Zhenmin, LIAO Yipeng, et al. A quick road centreline extraction method from remote sensing images combining with geodesic distance field and curve smoothing[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(8): 1317-1329. DOI:. doi: 10.11947/j.AGCS.2023.20220002
[3]	周绍光, 陈超, 岳建平. 形状先验和图割的高分辨率遥感影像道路段提取[J]. 测绘学报, 2014, 43(1): 60-65.
	ZHOU Shaoguang, CHEN Chao, YUE Jianping. Extracting roads from high-resolution RS images based on shape priors and graph cuts[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(1): 60-65.
[4]	ZHANG Zhengxin, LIU Qingjie, WANG Yunhong. Road extraction by deep residual U-Net[J]. IEEE Geoscience and Remote Sensing Letters, 2018, 15(5): 749-753.
[5]	贺浩, 王仕成, 杨东方, 等. 基于Encoder-Decoder网络的遥感影像道路提取方法[J]. 测绘学报, 2019, 48(3): 330-338. DOI:. doi: 10.11947/j.AGCS.2019.20180005
	HE Hao, WANG Shicheng, YANG Dongfang, et al. An road extraction method for remote sensing image based on Encoder-Decoder network[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(3): 330-338. DOI:. doi: 10.11947/j.AGCS.2019.20180005
[6]	YANG Xiaofei, LI Xutao, YE Yunming, et al. Road detection via deep residual dense U-Net[C]//Proceedings of 2019 International Joint Conference on Neural Networks. Budapest: IEEE, 2019: 1-7.
[7]	韩洁, 郭擎, 李安. 结合非监督分类和几何-纹理-光谱特征的高分影像道路提取[J]. 中国图象图形学报, 2017, 22(12): 1788-1797.
	HAN Jie, GUO Qing, LI An. High-resolution image road extraction combining unsupervised classification and geometric-texture-spectral features[J]. Journal of Image and Graphics in China, 2017, 22(12): 1788-1797.
[8]	房玉品, 王小鹏, 李新娜. 基于自适应形态学的遥感图像道路提取[J]. 激光与光电子学进展, 2022, 59(16): 135-142.
	FANG Yuping, WANG Xiaopeng, LI Xinna. Remote sensing image road extraction based on adaptive morphology[J]. Advances in Laser and Optoelectronics, 2022, 59(16): 135-142.
[9]	王艳梅, 李金雨, 冯海霞. 改进支持向量机的遥感影像道路提取技术研究[J]. 浙江水利水电学院学报, 2021, 33(3): 74-76.
	WANG Yanmei, LI Jinyu, FENG Haixia. Research on improved support vector machine for remote sensing image road extraction technology[J]. Journal of Zhejiang Water Conservancy and Hydropower College, 2021, 33(3): 74-766.
[10]	潘博众. 基于改进的半随机森林算法的道路识别技术研究[D]. 南京: 南京大学, 2017.
	PAN Bozhong. Research on road recognition technology based on improved semi-random forest algorithm[D]. Nanjing: Nanjing University, 2017.
[11]	梁双凤. 基于马尔科夫随机场的遥感影像道路提取[J]. 测绘与空间地理信息, 2022, 45(9): 186-189.
	LIANG Shuangfeng. Road extraction of remote sensing images based on Markov random field[J]. Geomatics and Spatial Information Technology, 2022, 45(9): 186-189.
[12]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe: ACM Press, 2012: 1097-1105.
[13]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[14]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large scale image recognition[C]//Proceedings of 2015 International Conference on Learning Representations. San Diego: ICLR, 2015: 463-476.
[15]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[16]	许苗, 李元祥, 钟娟娟, 等. L-UNet：轻量化云遮挡道路提取网络[J]. 中国图象图形学报, 2021, 26(11): 2670-2679.
	XU Miao, LI Yuanxiang, ZHONG Juanjuan, et al. L-UNet: lightweight network for road extraction in cloud occlusion scene[J]. Journal of Image and Graphics, 2021, 26(11): 2670-2679.
[17]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich: Springer, 2015: 234-241.
[18]	SUN Yueming, SHAO Zhenfeng, CHENG Gui, et al. Road and car extraction using UAV images via efficient dual contextual parsing Network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-13.
[19]	WANG Y, SEO J, JEON T. NL-LinkNet: toward lighter but more accurate road extraction with non-local operations[J]. IEEE Geoscience and Remote Sensing Letters, 2022, 19: 1-5.
[20]	宋廷强, 刘童心, 宗达, 等. 改进U-Net网络的遥感影像道路提取方法研究[J]. 计算机工程与应用, 2021, 57(14): 209-216.
	SONG Tingqiang, LIU Tongxin, ZONG Da, et al. Research on improving the road extraction method of remote sensing images based on U-Net network[J]. Computer Engineering and Applications, 2021, 57(14): 209-216.
[21]	赫晓慧, 宋定君, 李盼乐, 等. 融合多尺度特征的遥感影像道路提取方法[J]. 计算机工程, 2022, 48(8): 196-205.
	HE Xiaohui, SONG Dingjun, LI Panle, et al. Remote sensing image road extraction method with fusion of multi-scale features[J]. Computer Engineering, 2022, 48(8): 196-205.
[22]	LI Xingang, WANG Yuebin, ZHANG Liqiang, et al. Topology-enhanced urban road extraction via a geographic feature-enhanced network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(12): 8819-8830.
[23]	LIAO Cheng, HU Han, LI Haifeng, et al. Joint learning of contour and structure for boundary-preserved building extraction[J]. Remote Sensing, 2021, 13(6): 1049.
[24]	QUAN Yujun, YU Anzhu, CAO Xuefeng, et al. Building extraction from remote sensing images with DoG as prior constraint[J]. Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 6559-6570.
[25]	WU Yuxin, HE Kaiming. Group normalization[C]//Proceedings of 2018 European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
[26]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision. Munich: Springer, 2018: 3-19.
[27]	MNIH V. Machine learning for aerial image labeling[D]. Toronto: University of Toronto, 2013.
[28]	MEI Jie, LI Roujing, GAO Wang, et al. CoANet: connectivity attention network for road extraction from satellite imagery[J]. IEEE Transactions on Image Processing, 2021, 30: 8540-8552.
[29]	ZHOU Lichen, ZHANG Chuang, MING Wu. D-LinkNet: LinkNet with pretrained encoder and dilated convolution for high resolution satellite imagery road extraction[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Salt Lake City: IEEE, 2018: 192-1924.
[30]	ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6230-6239.
[31]	BOLEI Zhou, ADITYA Khosla, AGATA Lapedriza, et al. Learning deep features for discriminative localization[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 2921-2929.

模型	precision	recall	IoU	F₁值
CoANet	66.77	83.47	58.97	74.19
D-LinkNet	81.94	71.16	61.51	76.17
PSPNet	71.19	59.47	47.94	64.81
SobelNet	79.96	69.84	59.44	74.56
MeD-Net	84.84	71.93	63.74	77.85

模型	precision	recall	IoU	F₁值
CoANet	57.08	93.58	54.93	70.91
D-LinkNet	77.11	70.79	58.50	73.81
PSPNet	82.56	71.33	61.99	76.53
SobelNet	82.66	72.04	62.58	76.97
MeD-Net	82.40	74.37	64.17	78.18

模型	precision	recall	IoU	F₁值
UNet	88.20	65.78	60.46	75.36
UNet+EdgeNet	85.02	71.45	63.45	77.65
UNet+MDFP	77.26	76.10	62.18	76.68
MeD-Net	84.84	71.93	63.74	77.85

模型	precision	recall	IoU	F₁值
UNet	83.31	69.66	61.13	75.88
UNet+EdgeNet	75.21	77.74	61.88	76.45
UNet+MDFP	75.12	78.26	62.15	76.66
MeD-Net	82.40	74.37	64.17	78.18