Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation

doi:10.11947/j.AGCS.2024.20230198

Acta Geodaetica et Cartographica Sinica ›› 2024, Vol. 53 ›› Issue (8): 1624-1633.doi: 10.11947/j.AGCS.2024.20230198

• Photogrammetry and Remote Sensing • Previous Articles Next Articles

Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation

Tao XU¹(), Yuanwei YANG¹(), Xianjun GAO¹^,², Zhiwei WANG³, Yue PAN³, Shaohua LI¹, Lei XU⁴, Yanjun WANG⁵^,⁶, Bo LIU², Jing YU⁷, Fengmin WU⁷, Haoyu SUN¹

^1.School of Geosciences, Yangtze University, Wuhan 430100, China
^2.Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake of Ministry of Natural Resources, East China University of Technology, Nanchang 330013, China
^3.Region Surveying and Mapping Geographic Information Center, Hohhot 010050, China
^4.China Railway Design Corporation, Tianjin 300308, China
^5.National-local Joint Engineering Laboratory of Geo-spatial Information Technology, Hunan University of Science and Technology, Xiangtan 411201, China
^6.Hunan Provincial Key Laboratory of Geo-Information Engineering in Surveying, and Remote Sensing, Hunan University of Science and Technology, Xiangtan 411201, China
^7.Chongqing Geomatics and Remote Sensing Center, Chongqing 401147, China

Received:2023-06-08 Published:2024-09-25
Contact: Yuanwei YANG E-mail:2021720578@yangtze.edu.cn;2021720578@yangtze.edu.cn;yyw_08@yangtzeu.edu.cn
About author:XU Tao (1998—), male, postgraduate, majors in the semantic segmentation of 3D point cloud data. E-mail: 2021720578@yangtze.edu.cn
Supported by:
The Open Fund of National Engineering Laboratory for Digital Construction and Evaluation Technology of Urban Rail Transit(2023ZH01);Tianjin Science and Technology Plan Project(23YFYSHZ00190);The Natural Science Foundation of Chongqing Province(CSTB2022NSCQ-MSX1484);Open Fund of Key Laboratory of Mine Environmental Monitoring and Improving around Poyang Lake, Ministry of Natural Resources(MEMI-2021-2022-08);The Open Fund of Hunan Provincial Key Laboratory of Geo-Information Engineering in Surveying, Mapping and Remote Sensing, Hunan University of Science and Technology(E22205);Hunan Provincial Natural Science Foundation Project Department Union Fund(2024JJ8327);Jiangxi Provincial Natural Science Foundation(20232ACB204032);Yangtze University College Student Innovation Project(Yz2023013)

Abstract

Abstract:

Accurately segmenting the catenary is essential for extracting its components and detecting geometric parameters. In fact, the catenary scene is complex, with significant differences in size between internal components. There are many components with similar and connected semantic information, which makes it difficult for existing deep learning methods to accurately complete catenary point cloud semantic segmentation tasks. To address this issue, this paper proposes a neural network named GDM-Net that leverages graph convolution and multi-level features. GDM-Net includes a graph-based local feature extractor that enhances local feature extraction of the catenary point cloud, a double efficient channel attention module that considers the extraction of global and salient features of the catenary point cloud, and a refinement module of multi-scale feature fusion that improves segmentation accuracy by extracting and fusing multi-scale information of the catenary. The proposed network significantly improves the point cloud segmentation ability of catenary components, particularly at the intersection. Based on qualitative and quantitative analysis of the overhead catenary system dataset, the method is verified to achieve the highest accuracy among five other point cloud deep learning methods. The OA, mIoU, and F₁ accuracy indices reach 96.73%, 91.06%, and 95.28%, respectively. Qualitative comparisons demonstrate that the proposed network effectively reduces the misclassification problem of component links and improves the integrity of catenary component segmentation.

Key words: LiDAR, overhead catenary system, graph convolution, attention mechanism, multi-scale feature fusion, point cloud semantic segmentation

CLC Number:

P237

Tao XU, Yuanwei YANG, Xianjun GAO, Zhiwei WANG, Yue PAN, Shaohua LI, Lei XU, Yanjun WANG, Bo LIU, Jing YU, Fengmin WU, Haoyu SUN. Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(8): 1624-1633.

Figures/Tables 15

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Tab.1

Fig.7

Tab.2

Tab.3

Fig.8

Tab.4

Tab.5

Fig.9

Fig.10

References 27

[1]	LIN Shuai, XU Cheng, CHEN Lipei, et al. LiDAR point cloud recognition of overhead catenary system with deep learning[J]. Sensors, 2020, 20(8): 2212-2228.
[2]	TU Xiaohan, XU Cheng, LIU Siping, et al. LiDAR point cloud recognition and visualization with deep learning for overhead contact inspection[J]. Sensors, 2020, 20(21): 6387-6404.
[3]	杨元维, 张跃, 高贤君, 等. 顾及空间关系的3D LiDAR铁路支持装置自动提取[J]. 浙江大学学报(工学版), 2022, 56(10): 2066-2076.
	YANG Yuanwei, ZHANG Yue, GAO Xianjun, et al. Automatic extraction of spatially relevant 3D LiDAR railway support facility[J]. Journal of Zhejiang University (Engineering Science), 2022, 56(10): 2066-2076.
[4]	杨必胜, 梁福逊, 黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报, 2017, 46(10): 1509-1516. DOI: 10.11947/j.AGCS.2017.20170351.
	YANG Bisheng, LIANG Fuxun, HUANG Ronggang. Progress, challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10): 1509-1516. DOI: 10.11947/j.AGCS.2017.20170351.
[5]	周靖松, 韩志伟, 杨长江. 基于三维点云的接触网几何参数检测方法[J]. 仪器仪表学报, 2018, 39(4): 239-246.
	ZHOU Jingsong, HAN Zhiwei, YANG Changjiang. Catenary geometric parameters detection method based on 3D point cloud[J]. Chinese Journal of Scientific Instrument, 2018, 39(4): 239-246.
[6]	张东, 余朝刚. 基于激光扫描的接触网几何参数检测方法研究[J]. 计算机测量与控制, 2016, 24(1): 57-60.
	ZHANG Dong, YU Chaogang. Catenary geometry parameter detection method based on laser scanning[J]. Computer Measurement & Control, 2016, 24(1): 57-60.
[7]	罗隆福, 叶威, 王健. 基于深度学习的高铁接触网顶紧螺栓的缺陷检测[J]. 铁道科学与工程学报, 2021, 18(3): 605-614.
	LUO Longfu, YE Wei, WANG Jian. Defect detection of the puller bolt in high-speed railway catenary based on deep learning[J]. Journal of Railway Science and Engineering, 2021, 18(3): 605-614.
[8]	XIE Yuxing, TIAN Jiaojiao, ZHU Xiaoxiang. Linking points with labels in 3D: a review of point cloud semantic segmentation[J]. IEEE Geoscience and Remote Sensing Magazine, 2020, 8(4): 38-59.
[9]	DERPANIS K G. Overview of the RANSAC algorithm[J]. Image Rochester NY, 2010, 4(1): 2-3.
[10]	SCHUBERT E, SANDER J, ESTER M, et al. DBSCAN revisited, revisited: why and how you should (still) use DBSCAN[J]. ACM Transactions on Database Systems, 2017, 42(3): 1-21.
[11]	AHMAD A, DEY L. A k-mean clustering algorithm for mixed numeric and categorical data[J]. Data & Knowledge Engineering, 2007, 63(2): 503-527.
[12]	ZHANG Jixian, LIN Xiangguo, NING Xiaogang. SVM-based classification of segmented airborne LiDAR point clouds in urban areas[J]. Remote Sensing, 2013, 5(8): 3749-3775.
[13]	孙杰, 赖祖龙. 利用随机森林的城区机载LiDAR数据特征选择与分类[J]. 武汉大学学报(信息科学版), 2014, 39(11): 1310-1313.
	SUN Jie, LAI Zulong. Airborne LiDAR feature selection for urban classification using random forests[J]. Geomatics and Information Science of Wuhan University, 2014, 39(11): 1310-1313.
[14]	LALONDE J F, UNNIKRISHNAN R, VANDAPEL N, et al. Scale selection for classification of point-sampled 3D surfaces[C]//Proceedings of 2005 International Conference on 3D Digital Imaging and Modeling. Ottawa: IEEE, 2005: 285-292.
[15]	张广斌, 高贤君, 冉树浩, 等. 高分遥感影像云雪共存区轻量云高精度检测方法[J]. 测绘学报, 2023, 52(1): 93-107. DOI: 10.11947/j.AGCS.2023.20210686.
	ZHANG Guangbin, GAO Xianjun, RAN Shuhao, et al. Accurate and lightweight cloud detection method based on cloud and snow coexistence region of high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(1): 93-107. DOI: 10.11947/j.AGCS.2023.20210686.
[16]	张佳颖, 赵晓丽, 陈正. 基于深度学习的点云语义分割综述[J]. 激光与光电子学进展, 2020, 57(4): 28-46.
	ZHANG Jiaying, ZHAO Xiaoli, CHEN Zheng. Review of semantic segmentation of point cloud based on deep learning[J]. Laser & Optoelectronics Progress, 2020, 57(4): 28-46.
[17]	YANG Zhixin, TANG Lulu, ZHANG Kun, et al. Multi-view CNN feature aggregation with ELM auto-encoder for 3D shape recognition[J]. Cognitive Computation, 2018, 10(6): 908-921.
[18]	RIEGLER G, ULUSOY A O, GEIGER A. OctNet: learning deep 3D representations at high resolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 6620-6629.
[19]	CHARLES R Q, SU H, MO K C, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 77-85.
[20]	QI C R, YI L, SU H, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[J]. Advances in Neural Information Processing Systems, 2017, 30: 5099-5108.
[21]	WANG Yue, SUN Yongbin, LIU Ziwei, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[22]	GRAHAM B, ENGELCKE M, VAN DER MAATEN L. 3D semantic segmentation with submanifold sparse convolutional networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 9224-9232.
[23]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.
[24]	JIANG Tengping, YANG Bisheng, WANG Yongjun, et al. RailSeg: learning local-global feature aggregation with contextual information for railway point cloud semantic segmentation [J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-29.
[25]	RUBINSTEIN R Y, KROESE D P. A tutorial introduction to the cross-entropy method[M]//Information science and statistics. New York: Springer, 2004: 29-58.
[26]	CHEN Zhanlong, LI Shuangjiang, XU Yongyang, et al. Correg-Yolov3: a method for dense buildings detection in high-resolution remote sensing images[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(2): 51-61.
[27]	QIAN Guocheng, LI Yuchen, PENG Houwen, et al. Pointnext: revisiting Pointnet++ with improved training and scaling strategies[J]. Advances in Neural Information Processing Systems, 2022, 35: 23192-23204.

数据集	单臂	双臂	训练集		验证集		测试集
数据集	单臂	双臂	单臂	双臂	单臂	双臂	单臂	双臂
场景数	68	20	55	15	11	4	2	1
点的数量	3.54×10⁷	1.4×10⁷	2.9×10⁷	1.1×10⁷	5.7×10⁶	2.8×10⁶	1.04×10⁶	7×10⁵

方法	评价精度	承力索	定位器	斜腕臂	直腕臂	弹性吊索	定位管	背景	吊弦	接触线
PointNet	P	83.20	70.23	77.09	74.70	79.42	62.24	50.04	88.17	98.29
	R	90.54	81.88	92.94	98.16	63.18	87.72	48.32	57.58	97.03
	mIoU	76.55	60.79	72.82	73.67	64.29	61.25	50.09	53.46	95.42
	F₁值	86.72	75.61	84.28	84.84	70.38	72.82	49.17	69.67	97.66
PointNet＋＋	P	86.86	96.36	94.63	94.56	84.65	91.63	87.76	83.66	96.1
	R	90.65	93.65	96.28	92.34	86.33	93.36	87.36	83.09	98.19
	mIoU	76.12	86.95	91.36	91.92	72.58	88.50	79.36	80.91	94.93
	F₁值	88.71	94.99	95.46	93.45	85.48	92.49	87.56	83.37	97.13
DGCNN	P	92.81	98.55	96.62	96.56	86.80	93.07	87.98	85.61	99.33
	R	92.69	94.74	94.89	96.56	87.80	95.51	87.29	85.69	99.19
	mIoU	86.48	93.44	91.85	93.34	77.46	89.17	77.99	74.90	98.53
	F₁值	92.75	96.61	95.75	96.56	87.30	94.27	87.63	85.65	99.26
PointNeXt	P	90.34	98.56	99.27	94.70	95.18	96.55	93.31	94.25	98.87
	R	96.65	96.07	94.98	98.87	85.69	98.69	92.86	88.56	99.71
	mIoU	87.60	94.75	94.32	93.69	82.13	95.33	87.06	84.03	98.58
	F₁值	93.39	97.30	97.08	96.74	90.19	97.61	93.08	91.32	99.29
Pix4point	P	96.49	96.59	99.36	94.76	91.56	87.91	91.12	89.71	99.56
	R	97.26	93.67	95.05	98.48	93.86	97.85	92.13	88.46	99.44
	mIoU	91.13	90.67	94.47	93.39	86.39	86.24	89.90	77.89	99.00
	F₁值	96.87	95.11	97.16	96.59	92.70	92.61	91.62	89.08	99.50
GDM－Net	P	96.25	98.69	98.15	96.24	94.53	91.64	93.61	91.84	99.53
	R	96.85	94.67	95.16	97.81	90.18	97.91	92.94	87.38	99.61
	mIoU	91.45	93.49	93.48	94.21	87.71	89.88	89.06	81.09	99.15
	F₁值	95.53	96.64	96.63	97.02	92.31	94.67	93.27	89.56	99.57

方法	P	R	OA	mIoU	F₁值
PointNet	75.93	79.70	88.26	64.09	77.77
PointNet＋＋	90.69	91.25	92.34	84.37	90.96
DGCNN	93.04	92.71	94.93	87.02	92.87
PointNeXt	95.67	94.68	96.05	90.83	95.17
Pix4point	94.12	95.13	96.54	89.90	94.62
GDM－Net	95.61	94.94	96.73	91.06	95.28

方法	参数	FLOPs	每点测试时间/μs
PointNet	3.55×10⁶	58.53×10⁹	16.21
PointNet＋＋	1.24×10⁶	10.72×10⁹	20.51
DGCNN	2.64×10⁶	64.24×10⁹	17.35
PointNeXt	41.60×10⁶	84.80×10⁹	18.81
Pix4Point	23.70×10⁶	190.00×10⁹	21.36
GDM－Net	3.42×10⁶	133.87×10⁹	24.41

方法	mIoU	类别名称
方法	mIoU	轨道	道床	悬链线	围栏	植被	立柱	钢结构	地面	建筑物	支持装置	背景点
PointNet	11.72	5.51	6.83	13.66	7.01	38.20	2.36	7.66	25.49	6.45	7.56	8.21
PointNet＋＋	34.28	25.86	36.63	56.36	15.63	60.82	9.62	23.36	57.36	32.36	28.69	30.35
DGCNN	24.05	21.08	16.89	26.31	17.21	61.15	9.63	8.65	38.15	19.63	9.56	36.30
PointNeXt	75.19	80.72	73.23	62.92	81.61	88.28	65.39	74.64	83.36	90.82	60.61	65.59
Pix4Point	71.54	71.84	44.66	74.19	45.19	89.99	64.34	67.81	79.72	91.58	62.64	94.94
GDM－Net	81.52	90.63	78.69	76.33	88.37	93.61	68.37	72.63	85.39	85.68	68.36	88.65

Integrated graph convolution and multi-scale features for the overhead catenary system point cloud semantic segmentation

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 27

Related Articles 15

Recommended Articles

Metrics

Comments

[1]	Xin YAN, Li SHEN, Junjie PAN, Yanshuai DAI, Jicheng WANG, Xiaoli ZHENG, Zhi-lin LI. Weakly supervised building change detection integrating multi-scale feature fusion and spatial refinement for high resolution remote sensing images [J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(8): 1586-1597.
[2]	Yanan JIANG, Linfeng ZHENG, Qiang XU, Minggao TANG, Xing ZHU. Step-like displacement prediction of landslides guided by deformation mechanism [J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1128-1139.
[3]	Daifeng PENG, Chenchen ZHAI, Dingwei ZHOU, Yongjun ZHANG, Haiyan GUAN, Yufu ZANG. High-resolution optical images change detection based on global information enhancement by pyramid semantic token [J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1195-1211.
[4]	Shaopeng DING, Xiushan LU, Rufei LIU, Yi YANG, Haiyan GU, Haitao LI. Building change detection method combining object feature guidance and multiple attention mechanism [J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1224-1235.
[5]	Changqi JI, Zhaojie GUO, Haili SUN, Ruofei ZHONG. Location and rapid detection method of water leakage in subway tunnels based on mobile laser scanning [J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 1236-1250.
[6]	QIAN Chuang, ZHANG Hongjuan, LI Wenzhuo, LIU Hui, LI Bijun. A LiDAR/IMU spatial calibration method based on LiDAR labels and occupancy grid map [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(9): 1469-1479.
[7]	JIANG Baode, HANG Wei, XU Shaofen, WU Yong. Multi-scale building instance refinement extraction from remote sensing images by fusing with decentralized adaptive attention mechanism [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(9): 1504-1514.
[8]	AN Xiaoya, ZHU Yude, YAN Xiongfeng. A building selection method supported by graph convolutional network [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(9): 1574-1583.
[9]	LI Pengfei, LI Dou, HU Jinfei, YAO Wanqiang, ZANG Yuzhe. Assessing the ability of airborne LiDAR to monitor soil erosion on the Chinese Loess Plateau [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(8): 1342-1354.
[10]	ZHANG Yichao, ZHENG Xiangtao, LU Xiaoqiang. Hyperspectral image classification method based on hierarchical transformer network [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(7): 1139-1147.
[11]	WANG Dandi, XU Qing, XING Shuai, LIN Yuzhun, ZHANG Guoping. Semi-empirical waveform decomposition method for correction of near water surface penetration error in airborne LiDAR bathymetry [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(6): 944-955.
[12]	HU Gongming, YANG Chuncheng, XU Li, SHANG Haibin, WANG Zefan, QIN Zhilong. Improved U-Net remote sensing image semantic segmentation method [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(6): 980-989.
[13]	GU Xiaohu, LI Zhengjun, MIAO Jianhao, LI Xinghua, SHEN Huanfeng. Dual-channel parallel hybrid convolutional neural networks based classification method for high-resolution remote sensing image [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 798-807.
[14]	HU Minghong, LI Jiatian, YAO Yanji, A Xiaohui, LU Mei, LI Wen. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 808-817.
[15]	WANG Peixiao, ZHANG Tong, NIE Shichao, YANG Jinxuan, WANG Tianjiao. A causal graph convolutional network considering missing values for spatio-temporal prediction [J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 818-830.