融合图卷积与多尺度特征的接触网点云语义分割

doi:10.11947/j.AGCS.2024.20230198

测绘学报 ›› 2024, Vol. 53 ›› Issue (8): 1624-1633.doi: 10.11947/j.AGCS.2024.20230198

融合图卷积与多尺度特征的接触网点云语义分割

徐涛¹(), 杨元维¹(), 高贤君¹^,², 王志威³, 潘越³, 李少华¹, 许磊⁴, 王艳军⁵^,⁶, 刘波², 余静⁷, 吴凤敏⁷, 孙浩宇¹

^1.长江大学地球科学学院，湖北　武汉　430100
^2.东华理工大学自然资源部环鄱阳湖区域矿山环境监测与治理重点实验室，江西　南昌　330013
^3.内蒙古自治区测绘地理信息中心，内蒙古　呼和浩特　010050
^4.中国铁路设计集团有限公司，天津　300308
^5.湖南科技大学地理空间信息技术国家地方联合工程实验室，湖南　湘潭　411201
^6.湖南科技大学测绘遥感信息工程湖南省重点实验室，湖南　湘潭　411201
^7.重庆市地理信息和遥感应用中心，重庆　401147

收稿日期:2023-06-08 发布日期:2024-09-25
通讯作者: 杨元维 E-mail:2021720578@yangtze.edu.cn;2021720578@yangtze.edu.cn;yyw_08@yangtzeu.edu.cn
作者简介:徐涛（1998—），男，硕士生，研究方向为三维点云数据语义分割。E-mail：2021720578@yangtze.edu.cn
基金资助:
城市轨道交通数字化建设与测评技术国家工程实验室开放课题基金(2023ZH01);天津市科技计划(23YFYSHZ00190);重庆市自然科学基金(CSTB2022NSCQ-MSX1484);自然资源部环鄱阳湖区域矿山环境监测与治理重点实验室开放基金(MEMI-2021-2022-08);湖南科技大学测绘遥感信息工程湖南省重点实验室开放基金(E22205);湖南省自然科学基金项目部门联合基金(2024JJ8327);江西省自然科学基金(20232ACB204032);长江大学大学生创新项目(Yz2023013)

Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation

Tao XU¹(), Yuanwei YANG¹(), Xianjun GAO¹^,², Zhiwei WANG³, Yue PAN³, Shaohua LI¹, Lei XU⁴, Yanjun WANG⁵^,⁶, Bo LIU², Jing YU⁷, Fengmin WU⁷, Haoyu SUN¹

^1.School of Geosciences, Yangtze University, Wuhan 430100, China
^2.Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake of Ministry of Natural Resources, East China University of Technology, Nanchang 330013, China
^3.Region Surveying and Mapping Geographic Information Center, Hohhot 010050, China
^4.China Railway Design Corporation, Tianjin 300308, China
^5.National-local Joint Engineering Laboratory of Geo-spatial Information Technology, Hunan University of Science and Technology, Xiangtan 411201, China
^6.Hunan Provincial Key Laboratory of Geo-Information Engineering in Surveying, and Remote Sensing, Hunan University of Science and Technology, Xiangtan 411201, China
^7.Chongqing Geomatics and Remote Sensing Center, Chongqing 401147, China

Received:2023-06-08 Published:2024-09-25
Contact: Yuanwei YANG E-mail:2021720578@yangtze.edu.cn;2021720578@yangtze.edu.cn;yyw_08@yangtzeu.edu.cn
About author:XU Tao (1998—), male, postgraduate, majors in the semantic segmentation of 3D point cloud data. E-mail: 2021720578@yangtze.edu.cn
Supported by:
The Open Fund of National Engineering Laboratory for Digital Construction and Evaluation Technology of Urban Rail Transit(2023ZH01);Tianjin Science and Technology Plan Project(23YFYSHZ00190);The Natural Science Foundation of Chongqing Province(CSTB2022NSCQ-MSX1484);Open Fund of Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake, Ministry of Natural Resources(MEMI-2021-2022-08);The Open Fund of Hunan Provincial Key Laboratory of Geo-Information Engineering in Surveying, Mapping and Remote Sensing, Hunan University of Science and Technology(E22205);Hunan Provincial Natural Science Foundation Project Department Union Fund(2024JJ8327);Jiangxi Provincial Natural Science Foundation(20232ACB204032);Yangtze University College Student Innovation Project(Yz2023013)

摘要/Abstract

摘要：

准确的接触网语义分割结果对于接触网组件提取和几何参数检测具有重要的意义。实际上，接触网场景复杂，部件之间的尺寸差异较大，并且存在着较多语义信息接近且相连的部件，导致现有的深度学习方法难以高精度地完成接触网点云语义分割任务。基于上述问题，本文提出一种基于图卷积和多尺度特征的神经网络GDM-Net。该网络包含基于图的局部特征提取器，增强了对接触网点云局部特征提取；双重通道注意力模块，同时兼顾了接触网点云的全局和显著特征的提取；多尺度特征融合的细化模块，通过提取并融合接触网的多尺度信息，提升了分割精度。受益于上述几个模块，该网络提升了对于接触网部件交界处的点云分割能力。基于接触网数据集对该方法进行定性和定量的验证分析，GDM-Net相较于5种其他的点云深度学习方法，精度最高，OA、mIoU和F₁值这3个精度指标分别可以达到96.73%,91.06%和95.28%。定性比较表示，本文提出的网络可以有效减少部件连接部分的错分问题，提高接触网部件分割的完整性。

关键词: 激光雷达, 接触网系统, 图卷积, 注意力机制, 多尺度特征融合, 点云语义分割

Abstract:

Accurately segmenting the catenary is essential for extracting its components and detecting geometric parameters. In fact, the catenary scene is complex, with significant differences in size between internal components. There are many components with similar and connected semantic information, which makes it difficult for existing deep learning methods to accurately complete catenary point cloud semantic segmentation tasks. To address this issue, this paper proposes a neural network named GDM-Net that leverages graph convolution and multi-level features. GDM-Net includes a graph-based local feature extractor that enhances local feature extraction of the catenary point cloud, a double efficient channel attention module that considers the extraction of global and salient features of the catenary point cloud, and a refinement module of multi-scale feature fusion that improves segmentation accuracy by extracting and fusing multi-scale information of the catenary. The proposed network significantly improves the point cloud segmentation ability of catenary components, particularly at the intersection. Based on qualitative and quantitative analysis of the overhead catenary system dataset, the method is verified to achieve the highest accuracy among five other point cloud deep learning methods. The OA, mIoU, and F₁ accuracy indices reach 96.73%, 91.06%, and 95.28%, respectively. Qualitative comparisons demonstrate that the proposed network effectively reduces the misclassification problem of component links and improves the integrity of catenary component segmentation.

Key words: LiDAR, overhead catenary system, graph convolution, attention mechanism, multi-scale feature fusion, point cloud semantic segmentation

中图分类号:

P237

徐涛, 杨元维, 高贤君, 王志威, 潘越, 李少华, 许磊, 王艳军, 刘波, 余静, 吴凤敏, 孙浩宇. 融合图卷积与多尺度特征的接触网点云语义分割[J]. 测绘学报, 2024, 53(8): 1624-1633.

Tao XU, Yuanwei YANG, Xianjun GAO, Zhiwei WANG, Yue PAN, Shaohua LI, Lei XU, Yanjun WANG, Bo LIU, Jing YU, Fengmin WU, Haoyu SUN. Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(8): 1624-1633.

图/表 15

图1

图2

图3

图4

图5

图6

表1

图7

表2

表3

图8

表4

表5

图9

图10

参考文献 27

[1]	LIN Shuai, XU Cheng, CHEN Lipei, et al. LiDAR point cloud recognition of overhead catenary system with deep learning[J]. Sensors, 2020, 20(8): 2212-2228.
[2]	TU Xiaohan, XU Cheng, LIU Siping, et al. LiDAR point cloud recognition and visualization with deep learning for overhead contact inspection[J]. Sensors, 2020, 20(21): 6387-6404.
[3]	杨元维, 张跃, 高贤君, 等. 顾及空间关系的3D LiDAR铁路支持装置自动提取[J]. 浙江大学学报(工学版), 2022, 56(10): 2066-2076.
	YANG Yuanwei, ZHANG Yue, GAO Xianjun, et al. Automatic extraction of spatially relevant 3D LiDAR railway support facility[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(10): 2066-2076.
[4]	杨必胜, 梁福逊, 黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报, 2017, 46(10): 1509-1516. DOI: 10.11947/j.AGCS.2017.20170351.
	YANG Bisheng, LIANG Fuxun, HUANG Ronggang. Progress, challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1509-1516. DOI: 10.11947/j.AGCS.2017.20170351.
[5]	周靖松, 韩志伟, 杨长江. 基于三维点云的接触网几何参数检测方法[J]. 仪器仪表学报, 2018, 39(4): 239-246.
	ZHOU Jingsong, HAN Zhiwei, YANG Changjiang. Catenary geometric parameters detection method based on 3D point cloud[J]. Chinese Journal of Scientific Instrument, 2018, 39(4): 239-246.
[6]	张东, 余朝刚. 基于激光扫描的接触网几何参数检测方法研究[J]. 计算机测量与控制, 2016, 24(1): 57-60.
	ZHANG Dong, YU Chaogang. Catenary geometry parameter detection method based on laser scanning[J]. Computer Measurement & Control, 2016, 24(1): 57-60.
[7]	罗隆福, 叶威, 王健. 基于深度学习的高铁接触网顶紧螺栓的缺陷检测[J]. 铁道科学与工程学报, 2021, 18(3): 605-614.
	LUO Longfu, YE Wei, WANG Jian. Defect detection of the puller bolt in high-speed railway catenary based on deep learning[J]. Journal of Railway Science and Engineering, 2021, 18(3): 605-614.
[8]	XIE Yuxing, TIAN Jiaojiao, ZHU Xiaoxiang. Linking points with labels in 3D: a review of point cloud semantic segmentation[J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 8(4): 38-59.
[9]	DERPANIS K G. Overview of the RANSAC algorithm[J]. Image Rochester NY, 2010, 4(1): 2-3.
[10]	SCHUBERT E, SANDER J, ESTER M, et al. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN[J]. ACM Transactions on Database Systems, 2017, 42(3): 1-21.
[11]	AHMAD A, DEY L. A k-mean clustering algorithm for mixed numeric and categorical data[J]. Data & Knowledge Engineering, 2007, 63(2): 503-527.
[12]	ZHANG Jixian, LIN Xiangguo, NING Xiaogang. SVM-based classification of segmented airborne LiDAR point clouds in urban areas[J]. Remote Sensing, 2013, 5(8): 3749-3775.
[13]	孙杰, 赖祖龙. 利用随机森林的城区机载LiDAR数据特征选择与分类[J]. 武汉大学学报(信息科学版), 2014, 39(11): 1310-1313.
	SUN Jie, LAI Zulong. Airborne LiDAR feature selection for urban classification using random forests[J]. Geomatics and Information Science of Wuhan University, 2014, 39(11): 1310-1313.
[14]	LALONDE J F, UNNIKRISHNAN R, VANDAPEL N, et al. Scale selection for classification of point-sampled 3D surfaces[C]//Proceedings of 2005 International Conference on 3D Digital Imaging and Modeling. Ottawa: IEEE, 2005: 285-292.
[15]	张广斌, 高贤君, 冉树浩, 等. 高分遥感影像云雪共存区轻量云高精度检测方法[J]. 测绘学报, 2023, 52(1): 93-107. DOI: 10.11947/j.AGCS.2023.20210686.
	ZHANG Guangbin, GAO Xianjun, RAN Shuhao, et al. Accurate and lightweight cloud detection method based on cloud and snow coexistence region of high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(1): 93-107. DOI: 10.11947/j.AGCS.2023.20210686.
[16]	张佳颖, 赵晓丽, 陈正. 基于深度学习的点云语义分割综述[J]. 激光与光电子学进展, 2020, 57(4): 28-46.
	ZHANG Jiaying, ZHAO Xiaoli, CHEN Zheng. Review of semantic segmentation of point cloud based on deep learning[J]. Laser & Optoelectronics Progress, 2020, 57(4): 28-46.
[17]	YANG Zhixin, TANG Lulu, ZHANG Kun, et al. Multi-view CNN feature aggregation with ELM auto-encoder for 3D shape recognition[J]. Cognitive Computation, 2018, 10(6): 908-921.
[18]	RIEGLER G, ULUSOY A O, GEIGER A. OctNet: learning deep 3D representations at high resolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6620-6629.
[19]	CHARLES R Q, SU H, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 77-85.
[20]	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[J]. Advances in Neural Information Processing Systems, 2017, 30: 5099-5108.
[21]	WANG Yue, SUN Yongbin, LIU Ziwei, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[22]	GRAHAM B, ENGELCKE M, VAN DER MAATEN L. 3D semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 9224-9232.
[23]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
[24]	JIANG Tengping, YANG Bisheng, WANG Yongjun, et al. RailSeg: learning local-global feature aggregation with contextual information for railway point cloud semantic segmentation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-29.
[25]	RUBINSTEIN R Y, KROESE D P. A tutorial introduction to the cross-entropy method[M]//Information science and statistics. New York: Springer, 2004: 29-58.
[26]	CHEN Zhanlong, LI Shuangjiang, XU Yongyang, et al. Correg-Yolov3: a method for dense buildings detection in high-resolution remote sensing images[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(2): 51-61.
[27]	QIAN Guocheng, LI Yuchen, PENG Houwen, et al. Pointnext: revisiting Pointnet++ with improved training and scaling strategies[J]. Advances in Neural Information Processing Systems, 2022, 35: 23192-23204.

数据集	单臂	双臂	训练集		验证集		测试集
数据集	单臂	双臂	单臂	双臂	单臂	双臂	单臂	双臂
场景数	68	20	55	15	11	4	2	1
点的数量	3.54×10⁷	1.4×10⁷	2.9×10⁷	1.1×10⁷	5.7×10⁶	2.8×10⁶	1.04×10⁶	7×10⁵

方法	评价精度	承力索	定位器	斜腕臂	直腕臂	弹性吊索	定位管	背景	吊弦	接触线
PointNet	P	83.20	70.23	77.09	74.70	79.42	62.24	50.04	88.17	98.29
	R	90.54	81.88	92.94	98.16	63.18	87.72	48.32	57.58	97.03
	mIoU	76.55	60.79	72.82	73.67	64.29	61.25	50.09	53.46	95.42
	F₁值	86.72	75.61	84.28	84.84	70.38	72.82	49.17	69.67	97.66
PointNet＋＋	P	86.86	96.36	94.63	94.56	84.65	91.63	87.76	83.66	96.1
	R	90.65	93.65	96.28	92.34	86.33	93.36	87.36	83.09	98.19
	mIoU	76.12	86.95	91.36	91.92	72.58	88.50	79.36	80.91	94.93
	F₁值	88.71	94.99	95.46	93.45	85.48	92.49	87.56	83.37	97.13
DGCNN	P	92.81	98.55	96.62	96.56	86.80	93.07	87.98	85.61	99.33
	R	92.69	94.74	94.89	96.56	87.80	95.51	87.29	85.69	99.19
	mIoU	86.48	93.44	91.85	93.34	77.46	89.17	77.99	74.90	98.53
	F₁值	92.75	96.61	95.75	96.56	87.30	94.27	87.63	85.65	99.26
PointNeXt	P	90.34	98.56	99.27	94.70	95.18	96.55	93.31	94.25	98.87
	R	96.65	96.07	94.98	98.87	85.69	98.69	92.86	88.56	99.71
	mIoU	87.60	94.75	94.32	93.69	82.13	95.33	87.06	84.03	98.58
	F₁值	93.39	97.30	97.08	96.74	90.19	97.61	93.08	91.32	99.29
Pix4point	P	96.49	96.59	99.36	94.76	91.56	87.91	91.12	89.71	99.56
	R	97.26	93.67	95.05	98.48	93.86	97.85	92.13	88.46	99.44
	mIoU	91.13	90.67	94.47	93.39	86.39	86.24	89.90	77.89	99.00
	F₁值	96.87	95.11	97.16	96.59	92.70	92.61	91.62	89.08	99.50
GDM－Net	P	96.25	98.69	98.15	96.24	94.53	91.64	93.61	91.84	99.53
	R	96.85	94.67	95.16	97.81	90.18	97.91	92.94	87.38	99.61
	mIoU	91.45	93.49	93.48	94.21	87.71	89.88	89.06	81.09	99.15
	F₁值	95.53	96.64	96.63	97.02	92.31	94.67	93.27	89.56	99.57

方法	P	R	OA	mIoU	F₁值
PointNet	75.93	79.70	88.26	64.09	77.77
PointNet＋＋	90.69	91.25	92.34	84.37	90.96
DGCNN	93.04	92.71	94.93	87.02	92.87
PointNeXt	95.67	94.68	96.05	90.83	95.17
Pix4point	94.12	95.13	96.54	89.90	94.62
GDM－Net	95.61	94.94	96.73	91.06	95.28

方法	参数	FLOPs	每点测试时间/μs
PointNet	3.55×10⁶	58.53×10⁹	16.21
PointNet＋＋	1.24×10⁶	10.72×10⁹	20.51
DGCNN	2.64×10⁶	64.24×10⁹	17.35
PointNeXt	41.60×10⁶	84.80×10⁹	18.81
Pix4Point	23.70×10⁶	190.00×10⁹	21.36
GDM－Net	3.42×10⁶	133.87×10⁹	24.41

方法	mIoU	类别名称
方法	mIoU	轨道	道床	悬链线	围栏	植被	立柱	钢结构	地面	建筑物	支持装置	背景点
PointNet	11.72	5.51	6.83	13.66	7.01	38.20	2.36	7.66	25.49	6.45	7.56	8.21
PointNet＋＋	34.28	25.86	36.63	56.36	15.63	60.82	9.62	23.36	57.36	32.36	28.69	30.35
DGCNN	24.05	21.08	16.89	26.31	17.21	61.15	9.63	8.65	38.15	19.63	9.56	36.30
PointNeXt	75.19	80.72	73.23	62.92	81.61	88.28	65.39	74.64	83.36	90.82	60.61	65.59
Pix4Point	71.54	71.84	44.66	74.19	45.19	89.99	64.34	67.81	79.72	91.58	62.64	94.94
GDM－Net	81.52	90.63	78.69	76.33	88.37	93.61	68.37	72.63	85.39	85.68	68.36	88.65

融合图卷积与多尺度特征的接触网点云语义分割

Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 27

相关文章 15

编辑推荐

Metrics

本文评价

[1]	鄢薪, 慎利, 潘俊杰, 戴延帅, 王继成, 郑晓莉, 李志林. 多尺度特征融合与空间优化的弱监督高分遥感建筑变化检测[J]. 测绘学报, 2024, 53(8): 1586-1597.
[2]	蒋亚楠, 郑林枫, 许强, 汤明高, 朱星. 机理引导下的阶跃型滑坡位移预测深度学习模型[J]. 测绘学报, 2024, 53(6): 1128-1139.
[3]	彭代锋, 翟晨晨, 周顶蔚, 张永军, 管海燕, 臧玉府. 基于金字塔语义token全局信息增强的高分光学遥感影像变化检测[J]. 测绘学报, 2024, 53(6): 1195-1211.
[4]	丁少鹏, 卢秀山, 刘如飞, 杨懿, 顾海燕, 李海涛. 联合目标特征引导与多重注意力的建筑物变化检测[J]. 测绘学报, 2024, 53(6): 1224-1235.
[5]	纪长琦, 郭肇捷, 孙海丽, 钟若飞. 基于移动激光扫描的地铁隧道渗漏水定位及快速检测方法[J]. 测绘学报, 2024, 53(6): 1236-1250.
[6]	肖天元, 艾廷华, 余华飞, 杨敏, 刘鹏程. 地图综合图卷积神经网络点群简化方法[J]. 测绘学报, 2024, 53(1): 158-172.
[7]	钱闯, 张红娟, 李文卓, 刘晖, 李必军. LiDAR标签和栅格占有图结合的LiDAR/IMU空间标定方法[J]. 测绘学报, 2023, 52(9): 1469-1479.
[8]	江宝得, 黄威, 许少芬, 巫勇. 融合分散自适应注意力机制的多尺度遥感影像建筑物实例细化提取[J]. 测绘学报, 2023, 52(9): 1504-1514.
[9]	安晓亚, 朱余德, 晏雄锋. 卷积神经网络支持下的建筑物选取方法[J]. 测绘学报, 2023, 52(9): 1574-1583.
[10]	张艺超, 郑向涛, 卢孝强. 基于层级Transformer的高光谱图像分类方法[J]. 测绘学报, 2023, 52(7): 1139-1147.
[11]	胡功明, 杨春成, 徐立, 尚海滨, 王泽凡, 秦志龙. 改进U-Net的遥感图像语义分割方法[J]. 测绘学报, 2023, 52(6): 980-989.
[12]	顾小虎, 李正军, 缪健豪, 李星华, 沈焕锋. 高分遥感影像双通道并行混合卷积分类方法[J]. 测绘学报, 2023, 52(5): 798-807.
[13]	胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817.
[14]	王培晓, 张彤, 聂士超, 杨瑾萱, 王天骄. 顾及缺失值的因果图时空预测网络[J]. 测绘学报, 2023, 52(5): 818-830.
[15]	赵冰冰, 谭骁勇, 杨学习, 石岩, 邓敏. 融合地理条件驱动效应和图卷积的土地利用演化模拟CA模型[J]. 测绘学报, 2023, 52(5): 831-842.