A dynamic weighted fusion SLAM method using multi-source sensor data in complex underground spaces

doi:10.11947/j.AGCS.2025.20230586

Abstract

Abstract:

Simultaneous localization and mapping (SLAM) is pivotal for autonomous detection, automatic inspection, and emergency rescue in underground spaces. However, the challenges of narrow and long tunnels, complex terrain, and uneven illumination in underground spaces make LiDAR point cloud and visual image matching highly susceptible to degradation. This, in turn, results in insufficient accuracy or even failure of SLAM when fusing multi-sensor data. To address this challenge, we propose a dynamic weighted fusion SLAM method for multi-source sensor data with enhanced robustness. First, during the visual image preprocessing stage, an image enhancement technique based on the hue, saturation, and value (HSV) color space is employed. This method combines single-parameter homomorphic filtering with contrast limited adaptive histogram equalization (CLAHE) to effectively enhance the brightness and contrast of the image. This improvement strengthens the robustness of visual odometry. Next, the data quality of each sensor is evaluated using a Mahalanobis distance consistency test, which analyzes potential data degradation and adaptively selects the most suitable sensor data for fusion based on the current scene. Finally, considering the key parameters of each sensor, we construct the multi-source sensor factor graph model. The dynamic combination of multi-source sensor data weights is then formed according to the data quality model, allowing for the dynamic adjustment of the weight of each sensor data fusion factor. To verify the effectiveness of the proposed method, experiments were conducted in typical underground spaces such as underground corridors, excavated subway tunnels, and coal mine tunnels using self-designed and integrated mobile robots. Qualitative and quantitative comparative analyses were performed against various state-of-the-art methods. The results demonstrate that the maximum root mean square error (RMSE) of the proposed method is only 0.19 m. The average cloud to cloud (C2C) distance is less than 0.13 m, referencing the point cloud acquired by high-precision terrestrial 3D laser scanning. Additionally, the constructed point cloud maps exhibit superior global consistency and geometric structure authenticity. These findings confirm that the proposed method offers higher accuracy and robustness in complex underground spaces.

Key words: complex underground spaces, image enhancement, multi-source sensor data fusion, dynamic weight, intelligent perception

CLC Number:

P232

Xiaohu LIN, Xin YANG, Wanqiang YAO, Hongwei MA, Bolin MA, Xiongwei MA. A dynamic weighted fusion SLAM method using multi-source sensor data in complex underground spaces[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(3): 523-535.

Figures/Tables 12

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Tab. 1

Fig. 8

Tab. 2

Fig. 9

Tab. 3

References 32

[1]	艾琳, 于轩. 提升城市承载力推动新型城镇化可持续发展[J]. 宏观经济管理, 2021(3): 54-60.
	AI Lin, YU Xuan. Enhance urban carrying capacity and promote sustainable development of new urbanization[J]. Macroeconomic Management, 2021(3): 54-60.
[2]	周哲, 胡钊政, 李娜, 等. 面向智能车的地下停车场环视特征地图构建与定位[J]. 测绘学报, 2021, 50(11): 1574-1584. DOI:. doi: 10.11947/j.AGCS.2021.20210205
	ZHOU Zhe, HU Zhaozheng, LI Na, et al. Visual map from around view system for intelligent vehicle localization in underground parking lots[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1574-1584. DOI:. doi: 10.11947/j.AGCS.2021.20210205
[3]	白师宇, 赖际舟, 吕品, 等. 面向地下空间探测的移动机器人定位与感知方法[J]. 机器人, 2022, 44(4): 463-470.
	BAI Shiyu, LAI Jizhou, LÜ Pin, et al. Mobile robot localization and perception method for subterranean space exploration[J]. Robot, 2022, 44(4): 463-470.
[4]	EBADI K, BERNREITER L, BIGGIE H, et al. Present and future of SLAM in extreme environments: the DARPA SubT challenge[J]. IEEE Transactions on Robotics, 2023, 40: 936-959.
[5]	CHANG Yun, EBADI K, DENNISTON C E, et al. LAMP 2.0: a robust multi-robot SLAM system for operation in challenging large-scale underground environments[J]. IEEE Robotics and Automation Letters, 7(4): 9175-9182.
[6]	王保兵, 王凯, 王丹丹, 等. 地下复杂空间无人机研究进展及其面临的挑战[J]. 工矿自动化, 2023, 49(7): 6-13.
	WANG Baobing, WANG Kai, WANG Dandan, et al. Research progress and challenges faced by unmanned aerial vehicles in complex underground spaces[J]. Industry and Mine Automation, 2023, 49(7): 6-13.
[7]	LI Xingxing, ZHANG Xiaohong, NIU Xiaoji, et al. Progress and achievements of multi-sensor fusion navigation in China during 2019—2023[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(3): 102-114.
[8]	SIZINTSEV M, RAJVANSHI A, CHIU H P, et al. Multi-sensor fusion for motion estimation in visually-degraded environments[C]//Proceedings of 2019 IEEE International Symposium on Safety, Security, and Rescue Robotics. Würzburg: IEEE, 2019: 7-14.
[9]	李帅鑫, 李九人, 田滨, 等. 面向点云退化的隧道环境的无人车激光SLAM方法[J]. 测绘学报, 2021, 50(11): 1487-1499. DOI:. doi: 10.11947/j.AGCS.2021.20210248
	LI Shuaixin, LI Jiuren, TIAN Bin, et al. A laser SLAM method for unmanned vehicles in point cloud degenerated tunnel environments[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1487-1499. DOI:. doi: 10.11947/j.AGCS.2021.20210248
[10]	PFEIFER T, PROTZEL P. Robust sensor fusion with self-tuning mixture models[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2018: 3678-3685.
[11]	ZUO Xingxing, GENEVA P, LEE W, et al. LIC-fusion: LiDAR-inertial-camera odometry[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. Macau: IEEE, 2019: 5848-5854.
[12]	赵玏洋, 闫利. 移动机器人SLAM位姿估计的改进四元数无迹卡尔曼滤波[J]. 测绘学报, 2022, 51(2): 212-223. DOI:. doi: 10.11947/j.AGCS.2022.20210082
	ZHAO Leyang, YAN Li. Advanced quaternion unscented Kalman filter based on SLAM of mobile robot pose estimation[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(2): 212-223. DOI:. doi: 10.11947/j.AGCS.2022.20210082
[13]	LI Shengyu, LI Xingxing, WANG Huidan, et al. Multi-GNSS PPP/INS/Vision/LiDAR tightly integrated system for precise navigation in urban environments[J]. Information Fusion, 2023, 90: 218-232.
[14]	INDELMAN V, WILLIAMS S, KAESS M, et al. Information fusion in navigation systems via factor graph based incremental smoothing[J]. Robotics and Autonomous Systems, 2013, 61(8): 721-738.
[15]	WENW S, KAN Y C, HSU L T. Performance comparison of GNSS/INS integrations based on EKF and factor graph optimization[C]//Proceedings of the 32nd International Technical Meeting of the Satellite Division of the Institute of Navigation. Miami: Institute of Navigation, 2019: 3019-3032.
[16]	ZHANG Ji, SINGH S. LOAM: LiDAR odometry and mapping in real-time[J]. Robotics: Science and systems. 2014, 2(9): 1-9.
[17]	SHAO Weizhao, VIJAYARANGAN S, LI Cong, et al. Stereo visual inertial LiDAR simultaneous localization and mapping[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. Macau: IEEE, 2019: 370-377.
[18]	WANG Zengyuan, ZHANG Jianhua, CHEN Shengyong, et al. Robust high accuracy visual-inertial-laser SLAM system[C]//Proceedings of 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems. Macau: IEEE, 2019: 6636-6641.
[19]	王铉彬, 李星星, 廖健驰, 等. 基于图优化的紧耦合双目视觉/惯性/激光雷达SLAM方法[J]. 测绘学报, 2022, 51(8): 1744-1756. DOI:. doi: 10.11947/j.AGCS.2022.20210503
	WANG Xuanbin, LI Xingxing, LIAO Jianchi, et al. Tightly-coupled stereo visual-inertial-LiDAR SLAM based on graph optimization[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(8): 1744-1756. DOI:. doi: 10.11947/j.AGCS.2022.20210503
[20]	蔺小虎. 城市复杂环境下Visual/INS/LiDAR融合高精度位姿估计方法研究[D]. 武汉: 武汉大学, 2021.
	LIN Xiaohu. Visual/INS/LiDAR integration for high-precision pose estimation in complex urban environment[D]. Wuhan: Wuhan University, 2021.
[21]	QIN Tong, LI Peiliang, SHEN Shaojie. VINS-mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4): 1004-1020.
[22]	SHAN Tixiao, ENGLOT B, MEYERS D, et al. LIO-SAM: tightly-coupled LiDAR inertial odometry via smoothing and mapping[C]//Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020: 5135-5142.
[23]	YANG Pangbin, GONG Yun. Uneven illumination image matching algorithm combined with single-parameter homomorphic filtering[C]//Proceedings of the 13th International Conference on Graphics and Image Processing. Kunming: SPIE, 2022: 355-363.
[24]	HANA F M, MAULIDA I D. Analysis of contrast limited adaptive histogram equalization (CLAHE) parameters on finger knuckle print identification[J]. Journal of Physics: Conference Series, 2021, 1764(1): 012049.
[25]	江松, 崔智翔, 代碧波, 等. 暗环境适应性的基于SLAM的煤矿井下机器人定位方法[J/OL]. 煤炭科学技术, 1-14.[2024-02-03]. https://link.cnki.net/urlid/11.2402.td.20240325.1341.004.
	JIANG Song, CUI Zhixiang, DAI Bibo, et al. SLAM-based localization method of coal mine underground robot with adaptability to dark illumination environment[J]. Coal Science and Technology, 1-14.[2024-02-03]. https://link.cnki.net/urlid/11.2402.td.20240325.1341.004.
[26]	GRAETER J, WILCZYNSKI A, LAUER M, et al. LIMO: LiDAR-monocular visual odometry[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2018: 7872-7879.
[27]	HACKETT J K, SHAH M. Multi-sensor fusion: a perspective[C]//Proceedings of 1990 IEEE International Conference on Robotics and Automation. Cincinnati: IEEE, 1990: 1324-1330.
[28]	WANG Fengjuan, ZHANG Baoju, ZHANG Cuiping, et al. Low-light image joint enhancement optimization algorithm based on frame accumulation and multi-scale Retinex[J]. Ad Hoc Networks, 2021, 113: 102398.
[29]	DONG Shuai, MA Jia, SU Zhilong, et al. Robust circular marker localization under non-uniform illuminations based on homomorphic filtering[J]. Measurement, 2021, 170: 108700.
[30]	CAMPOS C, ELVIRA R, RODRIGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 37(6): 1874-1890.
[31]	SHAN Tixiao, ENGLOT B, RATTI C, et al. LVI-SAM: tightly-coupled LiDAR-visual-inertial odometry via smoothing and mapping[C]//Proceedings of 2021 IEEE International Conference on Robotics and Automation). Xi'an: IEEE, 2021: 5692-5698.
[32]	LIN Jiarong, ZHENG Chunran, XU Wei, et al. R2LIVE: a robust, real-time, LiDAR-inertial-visual tightly-coupled state estimator and mapping[J]. IEEE Robotics and Automation Letters, 6(4): 7469-7476.

算法	匹配数量			匹配成功率/（%）
算法	A7	B1	C6	A7	B1	C6
原始图像	38	36	53	20.6	19.5	27.2
Retinex	102	125	134	54.3	66.1	67.0
同态滤波	114	147	121	59.9	74.2	61.1
CLAHE	137	159	149	71.8	79.5	74.5
本文算法	148	162	173	76.0	81.0	86.5

方法	MIN			MAX			MEAN			RMSE
方法	图8（a）	图8（b）	图8（c）	图8（a）	图8（b）	图8（c）	图8（a）	图8（b）	图8（c）	图8（a）	图8（b）	图8（c）
本文方法	0.03	0.04	0.04	0.14	0.19	0.45	0.08	0.11	0.17	0.09	0.12	0.19
本文方法无动态加权	0.03	0.04	0.04	0.32	0.28	3.11	0.22	0.16	0.71	0.28	0.20	1.32
IE-R2LIVE	0.03	0.04	0.03	0.25	0.30	2.67	0.18	0.17	0.59	0.23	0.21	1.05
IE-LVI-SAM	0.02	0.04	0.04	0.18	0.59	3.54	0.13	0.24	0.83	0.14	0.28	1.51
LIO-SAM	0.03	0.03	0.06	0.49	0.36	16.81	0.26	0.20	6.90	0.18	0.25	10.56
IE-ORB-SLAM3	0.09	0.08	N/A	1.53	5.02	N/A	0.91	2.41	N/A	1.05	3.08	N/A
ORB-SLAM3	0.17	N/A	N/A	21.67	N/A	N/A	5.13	N/A	N/A	7.72	N/A	N/A
LOAM	2.05	N/A	N/A	9.18	N/A	N/A	5.80	N/A	N/A	6.45	N/A	N/A

算法	预处理	前端里程计	后端优化
本文方法	35.1	30.7	42.8
IE-R2LIVE	38.3	26.0	44.6
IE-LVI-SAM	34.6	28.1	45.2
LIO-SAM	10.8	22.5	29.3
IE-ORB-SLAM3	26.7	13.4	37.5
ORB-SLAM3	5.4	14.9	37.0
LOAM	8.3	16.2	15.7