Automatic classification and vectorization of road markings from mobile laser point clouds

doi:10.11947/j.AGCS.2021.20200351

Abstract

Abstract: Road markings are important traffic safety facilities. Its location, attribute, and topological relationship finely describe road traffic structure, and it is the basic data for applications such as intelligent traffic, high-precision maps, location, and navigation. This paper proposes a graph attention network with spatial context information (GAT_SCNet) to classify the road markings from mobile LiDAR point clouds. GAT_SCNet explores the graph structure to establish the appearance and dependence information among road markings. Meanwhile, GAT_SCNet incorporates the multi-head attention mechanism into the node propagation step, which computes the hidden states of each node based on the geometric, topological, and spatial structure relationships of the neighboring nodes. Finally, road markings classification is realized by the classification of nodes. Then, some schemes are designed for road markings vectorization. Four test datasets consisting of urban and highway scenes by different mobile laser scanning systems are used to evaluate the validities of the proposed method. Four accuracy evaluation metrics precision and recall of 9 types of road markings on the selected test datasets achieve (100.00%, 93.77%, 100.00%, 100.00%, 100.00%, 96.73%, 97.96%, 100.00%, 98.39%) and (100.00%, 96.36%, 100.00%, 10.000%, 100.00%, 97.26%, 85.72%, 100.00%, 94.16%), respectively. Accuracy evaluations and comparative studies prove that the proposed method has the capability of classifying multi-type road markings simultaneously and distinguishing similar road markings like dashed markings, zebra crossings, and stop lines in complex urban scenes.

Key words: MLS points clouds, road markings classification, graph structure, attention mechanism, road markings vectorization

CLC Number:

P307

FANG Lina, WANG Shuang, ZHAO Zhiyuan, FU Huasheng, CHEN Chongcheng. Automatic classification and vectorization of road markings from mobile laser point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(9): 1251-1265.

References

[1] WU Tao, RANGANATHAN A. A practical system for road marking detection and recognition[C]//Proceedings of 2012 IEEE Intelligent Vehicles Symposium. Madrid: IEEE, 2012.
[2] MATHIBELA B, NEWMAN P, POSNER I. Reading the road: road marking classification and interpretation[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(4): 2072-2081.
[3] DANESCU R, NEDEVSCHI S. Detection and classification of painted road objects for intersection assistance applications[C]//Proceedings the 13th International IEEE Conference on Intelligent Transportation Systems. Funchal, Portugal:IEEE, 2010.
[4] 刘经南, 詹骄, 郭迟, 等. 智能高精地图数据逻辑结构与关键技术[J]. 测绘学报, 2019, 48(8): 939-953. LIU Jingnan, ZHAN Jiao, GUO Chi, et al. Data logic structure and key technologies on intelligent high-precision map[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(8): 939-953.
[5] CHEN Xin, KOHLMEYER B, STROILA M, et al. Next generation map making: geo-referenced ground-level LiDAR point clouds for automatic retro-reflective road feature extraction[C]//Proceedings of the 17th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems. Seattle, Washington:ACM, 2009.
[6] RIVEIRO B, GONZÁLEZ-JORGE H, MARTÍNEZ-SÁNCHEZ J, et al. Automatic detection of zebra crossings from mobile LiDAR data[J]. Optics & Laser Technology, 2015, 70: 63-70.
[7] MANANDHAR D, SHIBASAKI R. Vehicle-borne laser mapping system (VLMS) for 3D GIS[C]//Proceedings of 2001 IEEE International Geoscience and Remote Sensing Symposium: Scanning the Present and Resolving the Future. Sydney, Australia: IEEE, 2001.
[8] JAAKKOLA A, HYYPPÄ J, HYYPPÄ H, et al. Retrieval algorithms for road surface modelling using laser-based mobile mapping[J]. Sensors, 2008, 8(9): 5238-5249.
[9] 姚连璧, 秦长才, 张邵华, 等. 车载激光点云的道路标线提取及语义关联[J]. 测绘学报, 2020, 49(4): 480-488. DOI: 10.11947/j.AGCS.2020.20190241. YAO Lianbi, QIN Changcai, ZHANG Shaohua, et al. Road marking extraction and semantic correlation based on vehicle-borne laser point cloud[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4): 480-488. DOI: 10.11947/j.AGCS.2020.20190241.
[10] CHENG Ming, ZHANG Haocheng, WANG Cheng, et al. Extraction and classification of road markings using mobile laser scanning point clouds[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3): 1182-1196.
[11] 方莉娜, 黄志文, 罗海峰, 等. 结合SVM与图匹配的车载激光点云道路标线识别[J]. 地球信息科学学报, 2019, 21(7): 994-1008. FANG Lina, HUANG Zhiwen, LUO Haifeng, et al. Integrating SVM and graph matching for identifying road markings from mobile LiDAR point clouds[J]. Journal of Geo-information Science, 2019, 21(7): 994-1008.
[12] MANCINI A, FRONTONI E, ZINGARETTI P. Automatic road object extraction from mobile mapping systems[C]//Proceedings of 2012 IEEE/ASME 8th IEEE/ASME International Conference on Mechatronic and Embedded Systems and Applications. Suzhou, China:IEEE, 2012.
[13] YANG Mengmeng, WAN Youchuan, LIU Xianlin, et al. Laser data based automatic recognition and maintenance of road markings from MLS system[J]. Optics & Laser Technology, 2018, 107: 192-203.
[14] SOILÁN M, RIVEIRO B, MARTÍNEZ-SÁNCHEZ J, et al. Segmentation and classification of road markings using MLS data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 123(1): 94-103.
[15] YU Yongtao, LI J, GUAN Haiyan, et al. Learning hierarchical features for automated extraction of road markings from 3-D mobile LiDAR point clouds[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(2): 709-726.
[16] WEN Chenglu, SUN Xiaotian, LI J, et al. A deep learning framework for road marking extraction, classification and completion from mobile laser scanning point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 147(1): 178-192.
[17] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-assisted Intervention.Munich, Germany:Springer, 2015.
[18] FOGGIA P, PERCANNELLA G, VENTO M. Graph matching and learning in pattern recognition in the last 10 years[J]. International Journal of Pattern Recognition and Artificial Intelligence, 2014, 28(1): 1450001.
[19] 项英倬, 谭菊仙, 韩杰思, 等. 图匹配技术研究[J]. 计算机科学, 2018, 45(6): 27-31, 45. XIANG Yingzhuo, TAN Juxian, HAN Jiesi, et al. Survey of graph matching algorithms[J]. Computer Science, 2018, 45(6): 27-31, 45.
[20] WU Zonghan, PAN Shirui, CHEN Fengwen, et al. A comprehensive survey on graph neural networks[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 32(1): 4-24.
[21] ZHANG Ziwei, CUI Peng, ZHU Wenwu. Deep learning on graphs: a survey[J]. IEEE Transactions on Knowledge and Data Engineering, 2020,14(8):1-23. DOI: 10.1109/TKDE.2020.2981333.
[22] WANG Yue, SUN Yongbin, LIU Ziwei, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 146.
[23] LAN Shiyi, YU Ruichi, YU Gang, et al. Modeling local geometric structure of 3D point clouds using Geo-CNN[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA: IEEE, 2019.
[24] WANG Lei, HUANG Yuchun, HOU Yaolin, et al. Graph attention convolution for point cloud semantic segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach, CA: IEEE, 2019.
[25] 方莉娜, 黄志文, 罗海峰, 等. 车载激光扫描数据中实线型交通标线提取[J]. 测绘学报, 2019, 48(8): 960-974. DOI: 10.11947/j.AGCS.2019.20180579. FANG Lina, HUANG Zhiwen, LUO Haifeng, et al. Solid lanes extraction from mobile laser scanning point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(8): 960-974. DOI: 10.11947/j.AGCS.2019.20180579.
[26] KAMPFFMEYER M, SALBERG A B, JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops. Las Vegas, NV:IEEE, 2016.
[27] KAMPFFMEYER M, SALBERG A B, JENSSEN R. Urban land cover classification with missing data modalities using deep convolutional neural networks[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2018, 11(6): 1758-1768.
[28] LAPTEV I, MAYER H, LINDEBERG T, et al. Automatic extraction of roads from aerial images based on scale space and snakes[J]. Machine Vision and Applications, 2000, 12(1): 23-31.