Intelligent perception measurement technology of autonomous UAV for unknown environment

doi:10.11947/j.AGCS.2024.20230389

Abstract

Abstract:

The development of intelligent surveying and mapping puts higher requirements for efficient, complete, and intelligent data collection, especially in GNSS-denied environments such as under-canopy, where traditional methods frequently struggle to achieve efficient and high-coverage measurements. To address the need for intelligent perception measurement of unknown environments, this paper introduces a novel intelligent perception measurement unmanned aerial vehicle (UAV) technology and framework, using a UAV as a mobile platform. This paper integrates visual online autonomous localization with global exploration path planning. Initially, the framework incorporates a novel visual-inertial odometry (VIO) online localization algorithm based on point and line features, which solves the initial pose estimation through feature extraction and matching of point and line features, and then high-precision pose information of the UAV is generated in real-time using factor graph optimization. Furthermore, to ensure efficient and high-coverage autonomous UAV measurements in unknown environments, this paper employs a global optimal exploration path planning method that considers multi-level information to determine the local exploration targets, and then generates high-quality exploration trajectories in real time through trajectory search and optimization algorithm. Moreover, the framework was validated by a customized UAV platform, the step-by-step comparison and overall real-world test demonstrates that the localization and space exploration methods designed and adopted by the framework have significant advantages compared with the current representative methods. In addition, it achieves efficient and high-coverage full autonomous measurements in GNSS-denied local under-canopy environments, which establishes a solid theoretical and framework foundation for the further development of online intelligent perception in unknown scenarios.

Key words: autonomous UAV, intelligent perception measurements technology and framework, online visual localization, autonomous planning and space exploration

CLC Number:

P232

Li YAN, Yinghao ZHAO, Jicheng DAI, Bo XU, Hong XIE, Yuquan ZHOU. Intelligent perception measurement technology of autonomous UAV for unknown environment[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 999-1012.

Figures/Tables 15

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Fig.6

Fig.7

Fig.8

Tab.1

Tab.2

Fig.9

Fig.10

Tab.3

Fig.11

Fig.12

References 51

[1]	陈军, 刘万增, 武昊, 等. 智能化测绘的基本问题与发展方向[J]. 测绘学报, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021.20210235. DOI:10.11947/j.AGCS.2021.20210235.
	CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping: fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021.20210235. DOI:10.11947/j.AGCS.2021.20210235.
[2]	陈军, 武昊, 刘万增, 等. 自然资源时空信息的技术内涵与研究方向[J]. 测绘学报, 2022, 51(7):1130-1140. DOI:10.11947/j.AGCS.2022.20210643.
	CHEN Jun, WU Hao, LIU Wanzeng, et al. Technical connotation and research agenda of natural resources spatio-temporal information[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7):1130-1140. DOI:10.11947/j.AGCS.2022.20210643.
[3]	CHEN Chen, LI Zhilin, LI Songnian, et al. From digitalized to intelligentized surveying and mapping: fundamental issues and research agenda[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(2):148-160.
[4]	郭庆华, 胡天宇, 刘瑾, 等. 轻小型无人机遥感及其行业应用进展[J]. 地理科学进展, 2021, 40(9):1550-1569.
	GUO Qinghua, HU Tianyu, LIU Jin, et al. Advances in light weight unmanned aerial vehicle remote sensing and major industrial applications[J]. Progress in Geography, 2021, 40(9):1550-1569.
[5]	闫利, 陈宇, 谢洪, 等. 测量机器人的关键技术[J]. 测绘学报, 2021, 50(9):1159-1169. DOI:10.11947/j.AGCS.2021.20210090.
	YAN Li, CHEN Yu, XIE Hong, et al. Surveying robot and its key technology[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(9):1159-1169. DOI:10.11947/j.AGCS.2021.20210090.
[6]	HE Yijia, XU Bo, OUYANG Zhanpeng, et al. A rotation-translation-decoupled solution for robust and efficient visual-inertial initialization[C]//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 739-748.
[7]	LIN H Y, ZHAN Jiarong. GNSS-denied UAV indoor navigation with UWB incorporated visual inertial odometry[J]. Measurement, 2023, 206:112256.
[8]	DUAN Ran, PAUDEL D P, FU Changhong, et al. Stereo orientation prior for UAV robust and accurate visual odometry[J]. ASME Transactions on Mechatronics, 2022, 27(5):3440-3450.
[9]	DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):38-49.
[10]	BARTOLOMEI L, TEIXEIRA L, CHLI M. Perception-aware path planning for UAVs using semantic segmentation[C]//Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020: 5808-5815.
[11]	LEE E M, CHOI J, LIM H, et al. REAL: rapid exploration with active loop-closing toward large-scale 3D mapping using UAVs[C]//Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. [S.l.]: ACM Press, 2021: 4194-4198.
[12]	ZHOU Quan, REDDING C J, QI Hairong, et al. Agile path planning for radiation source searching with aerial drones[C]//Proceedings of 2021 IEEE Nuclear Science Symposium and Medical Imaging Conference. Piscataway: IEEE, 2021: 1-2.
[13]	QIN Tong, LI Peiliang, SHEN Shaojie. VINS-mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020.
[14]	STEENBEEK A, NEX F. CNN-based dense monocular visual SLAM for real-time UAV exploration in emergency conditions[J]. Drones, 2022, 6(3):79.
[15]	RHODES C, LIU Cunjia, CHEN Wenhua. Autonomous source term estimation in unknown environments: from a dual control concept to UAV deployment[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2274-2281.
[16]	MUR-ARTAL R, TARDÓS J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[17]	CAMPOS C, ELVIRA R, RODRIGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6):1874-1890.
[18]	SUN Ke, MOHTA K, PFROMMER B, et al. Robust stereo visual inertial odometry for fast autonomous flight[J]. IEEE Robotics and Automation Letters, 2018, 3(2):965-972.
[19]	SUMIKURA S, SHIBUYA M, SAKURADA K. OpenVSLAM: a versatile visual SLAM framework[C]//Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM Press, 2019: 2292-2295.
[20]	BLOESCH M, OMARI S, HUTTER M, et al. Robust visual inertial odometry using a direct EKF-based approach[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: ACM Press, 2015: 298-304.
[21]	ROSINOL A, ABATE M, CHANG Yun, et al. Kimera: an open-source library for real-time metric-semantic localization and mapping[C]//Proceedings of 2020 IEEE International Conference on Robotics and Automation. Paris: IEEE, 2020: 1689-1696.
[22]	王晨捷, 罗斌, 李成源, 等. 无人机视觉SLAM协同建图与导航[J]. 测绘学报, 2020, 49(6):767-776. DOI:10.11947/j.AGCS.2020.20190145.
	WANG Chenjie, LUO Bin, LI Chengyuan, et al. The collaborative mapping and navigation based on visual SLAM in UAV platform[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6):767-776. DOI:10.11947/j.AGCS.2020.20190145.
[23]	GOMEZ-OJEDA R, MORENO F A, ZUNIGA-NOEL D, et al. PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Transactions on Robotics, 2019, 35(3):734-746.
[24]	GOMEZ-OJEDA R, BRIALES J, GONZALEZ-JIMENEZ J. PL-SVO: semi-direct monocular visual odometry by combining points and line segments[C]//Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: ACM Press, 2016: 4211-4216.
[25]	ZHOU Lipu, WANG Shengze, KAESS M. DPLVO: direct point-line monocular visual odometry[J]. IEEE Robotics and Automation Letters, 2021, 6(4):7113-7120.
[26]	ZHAO Yinghao, YAN Li, DAI Jicheng, et al. Robust planning system for fast autonomous flight in complex unknown environment using sparse directed frontier points[J]. Drones, 2023, 7(3):219.
[27]	BIRCHER A, KAMEL M, ALEXIS K, et al. Receding horizon “next-best-view” planner for 3D exploration[C]//Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA). Sweden: IEEE, 2016: 1462-1468.
[28]	WITTING C, FEHR M, BEHNEMANN R, et al. History-aware autonomous exploration in confined environments using MAVs[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid: IEEE, 2018: 1-9.
[29]	SELIN M, TIGER M, DUBERG D, et al. Efficient autonomous exploration planning of large-scale 3D environments[J]. IEEE Robotics and Automation Letters, 2019, 4(2):1699-1706.
[30]	WANG Chaoqun, ZHU Delong, LI Teng, et al. Efficient autonomous robotic exploration with semantic road map in indoor environments[J]. IEEE Robotics and Automation Letters, 2019, 4(3):2989-2996.
[31]	BATINOVIC A, IVANOVIC A, PETROVIC T, et al. A shadow casting-based next-best-view planner for autonomous 3D exploration[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2969-2976.
[32]	DUBERG D, JENSFELT P. UFOExplorer: fast and scalable sampling-based exploration with a graph-based planning structure[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2487-2494.
[33]	NAAZARE M, ROSAS F G, SCHULZ D. Online next-best-view planner for 3D-exploration and inspection with a mobile manipulator robot[J]. IEEE Robotics and Automation Letters, 2022, 7(2):3779-3786.
[34]	RESPALL V M, DEVITT D, FEDORENKO R, et al. Fast sampling-based next-best-view exploration algorithm for a MAV[C]//Proceedings of 2021 IEEE International Conference on Robotics and Automation. Xi'an: IEEE, 2021: 89-95.
[35]	YAMAUCHI B. A frontier-based approach for autonomous exploration[C]//Proceedings of 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Monterey: IEEE, 1997.
[36]	ZHOU Boyu, ZHANG Yichen, CHEN Xinyi, et al. FUEL: fast UAV exploration using incremental frontier structure and hierarchical planning[J]. IEEE Robotics and Automation Letters, 2021, 6(2):779-786.
[37]	CIESLEWSKI T, KAUFMANN E, SCARAMUZZA D. Rapid exploration with multi-rotors: a frontier selection method for high speed flight[C]//Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. [S.l.]: ACM Press, 2017: 2135-2142.
[38]	SHI Jianbo, TOMAS I. Good features to track[C]//Proceedings of 1994 IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 1994: 593-600.
[39]	BAKER S, MATTHEWS I. Lucas-kanade 20 years on: a unifying framework[J]. International Journal of Computer Vision, 2004, 56(3):221-255.
[40]	ZHANG Lilian, KOCH R. An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J]. Journal of Visual Communication and Image Representation, 2013, 24(7):794-805.
[41]	ELFES A. Using occupancy grids for mobile robot perception and navigation[J]. Computer, 1989, 22(6):46-57.
[42]	ZHOU Boyu, ZHANG Yichen, CHEN Xinyi, et al. FUEL: fast UAV exploration using incremental frontier structure and hierarchical planning[J]. IEEE Robotics and Automation Letters, 2021, 6(2):779-786.
[43]	ZHOU Xin, WANG Zhepei, YE Hongkai, et al. EGO-planner: an ESDF-free gradient-based local planner for quadrotors[J]. IEEE Robotics and Automation Letters, 2021, 6(2):478-485.
[44]	WANG Zhepei, ZHOU Xin, XU Chao, et al. Geometrically constrained trajectory optimization for multicopters[J]. IEEE Transactions on Robotics, 2022, 38(5):3259-3278.
[45]	REN Yunfan, ZHU Fangcheng, LIU Wenyi, et al. Bubble planner: planning high-speed smooth quadrotor trajectories using receding corridors[C]//Proceedings of 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems. Kyoto: IEEE, 2022: 6332-6339.
[46]	ZHAO Yinghao, YAN Li, CHEN Yu, et al. Robust and efficient trajectory replanning based on guiding path for quadrotor fast autonomous flight[J]. Remote Sensing, 2021, 13(5):972.
[47]	ZHOU Boyu, GAO Fei, WANG Luqi, et al. Robust and efficient quadrotor trajectory generation for fast autonomous flight[J]. IEEE Robotics and Automation Letters, 2019, 4(4):3529-3536.
[48]	BURRI M, NIKOLIC J, GOHL P, et al. The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35(10):1157-1163.
[49]	LEUTENEGGER S, FURGALE P, RABAUD V, et al. Key frame-based visual-inertial SLAM using nonlinear optimization[C]//Proceedings of 2013 Science and Systems IX. Robotics: Science and Systems Foundation, 2013.
[50]	HE Y, ZHAO J, GUO Y, et al. PL-VIO: tightly-coupled monocular visual-inertial odometry using point and line features[J]. Sensors (Basel), 2018, 18(4):E1159.
[51]	SCHÖNBERGER J L, ZHENG Enliang, FRAHM J M, et al. Pixelwise view selection for unstructured multi-view stereo[C]//Proceedings of 2016 Computer Vision. Cham: Springer, 2016: 501-518.

数据序列	OKVIS-Mono	VINS-Mono	PL-VIO	PL-SLAM	本文算法
MH_01_easy	29.5	14.8	20.1	15.7	11.1
MH_02_easy	30.7	17.1	13.1	14.2	9.3
MH_03_medium	33.4	19.4	26.1	14.7	15.7
MH_04_difficult	38.9	34.6	35.8	12.4	17.1
MH_05_difficult	46.7	29.2	24.4	55.5	14.4
V1_02_medium	22.2	7.9	17.0	16.9	8.9
V1_03_difficult	28.1	20.7	27.0	42.0	14.3
V2_01_easy	14.0	8.2	9.3	19.4	7.4
V2_02_medium	21.1	15.7	12.3	25.2	12.2

数据序列	OKVIS-Mono	VINS-Mono	PL-VIO	PL-SLAM	本文算法
MH_01_easy	3.2	2.0	1.6	6.0	1.6
MH_02_easy	3.9	2.3	1.7	2.5	0.9
MH_03_medium	3.3	1.6	1.7	3.4	0.8
MH_04_difficult	2.3	1.5	1.6	6.8	1.4
MH_05_difficult	2.4	0.7	1.1	9.9	0.7
V1_02_medium	6.0	2.6	3.2	5.6	1.5
V1_03_difficult	8.1	6.2	3.4	9.1	4.2
V2_01_easy	2.2	2.0	2.2	2.3	2.3
V2_02_medium	4.9	4.3	2.9	4.6	1.7

场景	方法	探索耗时/s			飞行距离/m			覆盖范围/m³
场景	方法	Avg	Min	Std	Avg	Min	Std	Avg	Min	Std
office	FUEL	167.5	156.7	8.0	239.0	225.5	11.9	1 082.2	1 079.6	2.0
office	本文算法	133.2	127.4	4.3	193.8	177.9	8.2	1 077.7	1 075.5	2.0
pillar	FUEL	163.6	147.0	9.1	215.8	189.7	14.8	756.8	753.0	1.8
pillar	本文算法	142.3	130.2	7.7	156.7	136.0	11.3	752.3	748.9	2.0
bridge	FUEL	146.2	126.1	13.8	208.5	186.8	14.5	871.6	868.7	1.6
bridge	本文算法	122.4	109.8	7.3	167.7	146.7	10.6	867.4	865.4	1.6