顾及拓扑结构的道路标线提取

doi:10.11947/j.AGCS.2024.20230289

摘要/Abstract

摘要：

道路标线是重要的交通标志信息，车载激光雷达点云为道路标线的提取提供了高精度的三维坐标和反射强度信息。由于扫描距离、目标材质等因素影响，不同目标会表现为相近的强度值，给道路标线的提取带来干扰；道路使用过程中的磨损老化会破坏标线原有的结构，造成标线提取后出现间断不连续问题；此外，道路标线种类多样且实际中出现的概率不同，导致分割网络提取结果中样本较少的类别分类精度较低。为此，本文提出了一种可准确提取各类标线并具有拓扑稳健性的分割+分类两阶段提取方法。首先，使用多层感知机自适应学习强度与其影响因素之间的关系，对路面点云进行强度改正；然后，提出链式空间拓扑网络LST-Net对道路上的所有标线进行语义分割，引入行列卷积、注意力机制捕捉标线结构信息，加入拓扑惩罚对其训练，确定标线的位置；最后，使用YOLOv5对标线进行检测，单独训练分类网络，解决分割中样本不均衡的问题，完成对标线的分类。在不同场景的3份车载点云上进行试验，结果表明本文方法标线提取精度达到94.1%，召回率达到95.6%，具有较强的实用性和有效性。

关键词: 车载激光点云, 道路标线提取, 拓扑结构, 语义分割, 目标检测

Abstract:

Road markings are important traffic sign information, and onboard LiDAR point clouds provide high precision 3D coordinates and reflectance intensity information for their extraction. Due to factors such as scanning distance and target material, the different object may exhibit similar intensity values, causing interference in the extraction of road markings. Wear and aging during road use can also damage the original structure of the markings, resulting in discontinuities after extraction. In addition, the diversity of road markings and their varying occurrence frequencies in practice can lead to low classification accuracy for categories with fewer samples in the segmentation network extraction results. To address these issues, this paper proposes a two-stage segmentation and classification extraction method that accurately extracts various types of markings and has topological robustness. Firstly, a multi-layer perceptron is used to adaptively learn the relationship between intensity and its influencing factors, and to perform intensity correction on the road point clouds. Secondly, the semantic segmentation network link spatial topology net (LST-Net) is proposed to segment all road markings, which captures line structure information using row-column convolution and attention mechanisms, and is trained with topological punishment to determine the positions of markings. Finally, YOLOv5 is used to detect the markings, and a separate classification network is trained to address the issue of sample imbalance in segmentation. Experiments are conducted on three sets of point clouds from different driving scenarios, and the results show that our approach achieves a marking extraction accuracy of 94.1% and a recall rate of 95.6%, demonstrating strong practicality and effectiveness.

Key words: vehicle-mounted LiDAR point clouds, road markings extraction, topological structure, semantic segmentation, object detection

中图分类号:

P237

刘家兴, 黄玉春, 石文轩, 叶曦, 杨鹤. 顾及拓扑结构的道路标线提取[J]. 测绘学报, 2024, 53(11): 2213-2227.

Jiaxing LIU, Yuchun HUANG, Wenxuan SHI, Xi YE, He YANG. Road markings extraction considering topological structure[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(11): 2213-2227.

图/表 25

图1

图2

图3

图4

图5

图6

图7

图8

图9

图10

图11

表1

图12

图13

图14

图15

表2

表3

图16

表4

图17

表5

图18

表6

表7

参考文献 35

[1]	MATHIBELA B, NEWMAN P, POSNER I. Reading the road: road marking classification and interpretation[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(4): 2072-2081.
[2]	李德仁, 洪勇, 王密, 等. 测绘遥感能为智能驾驶做什么?[J]. 测绘学报, 2021, 50(11): 1421-1431.DOI:. doi: 10.11947/j.AGCS.2021.20210280
	LI Deren, HONG Yong, WANG Mi, et al. What can surveying and remote sensing do for intelligent driving?[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1421-1431.DOI:. doi: 10.11947/j.AGCS.2021.20210280
[3]	WU Tao, RANGANATHAN A. A practical system for road marking detection and recognition[C]//Proceedings of 2012 IEEE Intelligent Vehicles Symposium. Madrid: IEEE, 2012: 25-30.
[4]	刘经南, 吴杭彬, 郭迟, 等. 高精度道路导航地图的进展与思考[J]. 中国工程科学, 2018, 20(2): 99-105.
	LIU Jingnan, WU Hangbin, GUO Chi, et al. Progress and consideration of high precision road navigation map[J]. Strategic Study of CAE, 2018, 20(2): 99-105.
[5]	KIM J G, YOO J H, KOO J C. Road and lane detection using stereo camera[C]//Proceedings of 2018 IEEE International Conference on Big Data and Smart Computing. Shanghai: IEEE, 2018: 649-652.
[6]	AZIMI S M, FISCHER P, KORNER M, et al. Aerial LaneNet: lane-marking semantic segmentation in aerial imagery using wavelet-enhanced cost-sensitive symmetric fully convolutional neural networks[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(5): 2920-2938.
[7]	YIN Weiling, QIAN Mingyang, WANG Lijun, et al. Road extraction from satellite images with iterative cross-task feature enhancement[J]. Neurocomputing, 2022, 506: 300-310.
[8]	YANG Bisheng, WEI Zheng, LI Qingquan, et al. Automated extraction of street-scene objects from mobile LiDAR point clouds[J]. International Journal of Remote Sensing, 2012, 33(18): 5839-5861.
[9]	SMADJA L, NINOT J, GAVRILOVIC T. Road extraction and environment interpretation from LiDAR sensors[J]. IAPRS, 2010, 38: 281-286.
[10]	TOTH C, PASKAA E, BRZEZINSKA D. Using road pavement markings as ground control for LiDAR data[J]. International Archives of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2008, 37(Part B1): 189-195.
[11]	HATA A, WOLF D. Road marking detection using LiDAR reflective intensity data and its application to vehicle localization[C]//Proceedings of the 17th International IEEE Conference on Intelligent Transportation Systems. Qingdao: IEEE, 2014: 584-589.
[12]	谭凯, 程效军, 张吉星. TLS强度数据的入射角及距离效应改正方法[J]. 武汉大学学报(信息科学版), 2017, 42(2): 223-228.
	TAN Kai, CHENG Xiaojun, ZHANG Jixing. Correction for incidence angle and distance effects on TLS intensity data[J]. Geomatics and Information Science of Wuhan University, 2017, 42(2): 223-228.
[13]	CHEN Xin, KOHLMEYER B, STROILA M, et al. Next generation map making: geo-referenced ground-level LiDAR point clouds for automatic retro-reflective road feature extraction[C]//Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. Seattle: ACM Press, 2009: 488-491.
[14]	WAN Rui, HUANG Yuchun, XIE Rongchang, et al. Combined lane mapping using a mobile mapping system[J]. Remote Sensing, 2019, 11(3): 305.
[15]	HUANG Pengdi, CHENG Ming, CHEN Yiping, et al. Traffic sign occlusion detection using mobile laser scanning point clouds[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(9): 2364-2376.
[16]	JUNG J, CHE Erzhuo, OLSEN M J, et al. Efficient and robust lane marking extraction from mobile LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 147: 1-18.
[17]	PAN Yue, YANG Bisheng, LI Shengfu, et al. Automatic road markings extraction, classification and vectorization from mobile laser scanning data[EB/OL]. [2023-07-01].https://isprs-archives.copernicus.org/articles/XLII-2-W13/1089/2019/isprs-archives-XLII-2-W13-1089-2019.pdf.
[18]	YE Chengming, ZHAO He, MA Lingfei, et al. Robust lane extraction from MLS point clouds towards HD maps especially in curve road[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(2): 1505-1518.
[19]	WEN Chenglu, SUN Xiaotian, LI J, et al. A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 147: 178-192.
[20]	CHENG Yiting, PATEL A, WEN Chenglu, et al. Intensity thresholding and deep learning based lane marking extraction and lane width estimation from mobile light detection and ranging (LiDAR) point clouds[J]. Remote Sensing, 2020, 12(9): 1379.
[21]	MA Lingfei, LI Ying, LI J, et al. Capsule-based networks for road marking extraction and classification from mobile LiDAR point clouds[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(4): 1981-1995.
[22]	CHEN Siyun, ZHANG Zhenxin, ZHONG Ruofei, et al. A dense feature pyramid network-based deep learning model for road marking instance segmentation using MLS point clouds[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(1): 784-800.
[23]	CHAURASIA A, CULURCIELLO E. LinkNet: exploiting encoder representations for efficient semantic segmentation[C]//Proceedings of 2017 IEEE Visual Communications and Image Processing. St. Petersburg: IEEE, 2017: 1-4.
[24]	PAN Xingang, SHI Jianping, LUO Ping, et al. Spatial as deep: spatial CNN for traffic scene understanding[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence. [S.l.]: AAAI, 2018.
[25]	ZHANG Han, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[C]//Proceedings of the 36th International Conference on Machine Learning. Long Beach: [s.n.], 2019: 7354-7363.
[26]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[27]	SASAKI K, IIZUKA S, SIMO-SERRA E, et al. Joint gap detection and inpainting of line drawings[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 5725-5733.
[28]	CHENG Yiting, LIN Yichun, HABIB A. Generalized LiDAR intensity normalization and its positive impact on geometric and learning-based lane marking detection[J]. Remote Sensing, 2022, 14(17): 4393.
[29]	BAI Jie, NIU Zheng, GAO Shuai, et al. An exploration, analysis, and correction of the distance effect on terrestrial hyperspectral LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 198: 60-83.
[30]	ZHENG Tu, FANG Hao, ZHANG Yi, et al. RESA: recurrent feature-shift aggregator for lane detection[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2021, 35(4): 3547-3554.
[31]	FENG Zhengyang, GUO Shaohua, TAN Xin, et al. Rethinking efficient lane detectionvia curve modeling[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 17062-17070.
[32]	WANG Jinsheng, MA Yinchao, HUANG Shaofei, et al. A keypoint-based global association network for lane detection[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 1392-1401.
[33]	MOSINSKA A, MARQUEZ-NEILA P, KOZINSKI M, et al. Beyond the pixel-wise loss for topology-aware delineation[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018.
[34]	SHIT S, PAETZOLD J C, SEKUBOYINA A, et al. clDice—a novel topology-preserving loss function for tubular structure segmentation[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021.
[35]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 MICCAI. Cham: Springer International Publishing, 2015: 234-241.

路段名称	数据总量	训练集	测试集
路段1	1856	1299	557
路段2	2560	1792	768
路段3	4800	3360	1440

试验数据	网络结构	精度	召回率	F₁值	IOU
路段1	RESA^[30]	0.883 1	0.925 2	0.90 36	0.824 7
路段1	LST-Net	0.938 9	0.967 9	0.953 2	0.910 5
路段2	RESA^[30]	0.876 0	0.881 1	0.878 5	0.783 3
路段2	LST-Net	0.941 5	0.948 8	0.945 2	0.896 2
路段3	RESA^[30]	0.885 6	0.899 6	0.892 6	0.806 4
路段3	LST-Net	0.942 7	0.953 3	0.947 9	0.901 1

网络结构名称	编解码部分	注意力模块	行列卷积模块
A	√
B	√	√
C	√		√
D（LST-Net）	√	√	√

网络结构名称	精度	召回率	F₁值	IOU
A	0.927 5	0.606 2	0.733 2	0.575 7
B	0.906 4	0.848 1	0.876 3	0.779 3
C	0.882 4	0.825 6	0.853 1	0.739 7
D（LSA-Net）	0.890 1	0.902 2	0.896 1	0.811 9

所用损失函数	精度	召回率	F₁值	IOU
Dice损失	0.890 1	0.902 2	0.896 1	0.811 9
文献[33]的损失函数	0.879 8	0.861 5	0.870 5	0.770 4
文献[34]的损失函数	0.901 4	0.894 9	0.898 1	0.815 6
本文提出的损失函数	0.941 5	0.948 8	0.945 2	0.896 2