面向未知环境的自主无人机智能感知测量技术

doi:10.11947/j.AGCS.2024.20230389

测绘学报 ›› 2024, Vol. 53 ›› Issue (6): 999-1012.doi: 10.11947/j.AGCS.2024.20230389

面向未知环境的自主无人机智能感知测量技术

闫利¹^,²(), 赵英豪³, 戴集成¹, 徐博¹, 谢洪¹^,²(), 周玉泉¹^,²

^1.武汉大学测绘学院，湖北　武汉　430079
^2.湖北珞珈实验室，湖北　武汉　430079
^3.信息工程大学地理空间信息学院，河南　郑州　450001

收稿日期:2023-09-06 发布日期:2024-07-22
通讯作者: 谢洪 E-mail:lyan@sgg.whu.edu.cn;hxie@sgg.whu.edu.cn
作者简介:闫利（1966—），男，教授，博士生导师，研究方向为摄影测量、遥感和三维激光扫描技术。　E-mail：lyan@sgg.whu.edu.cn
基金资助:
国家自然科学基金(42394061);湖北省重大科技项目(2021AAA010);湖北珞珈实验室开放基金(220100053)

Intelligent perception measurement technology of autonomous UAV for unknown environment

Li YAN¹^,²(), Yinghao ZHAO³, Jicheng DAI¹, Bo XU¹, Hong XIE¹^,²(), Yuquan ZHOU¹^,²

^1.School of Geodesy and Geomatics, Wuhan University, Wuhan 430079, China
^2.Hubei Luojia Laboratory, Wuhan 430079, China
^3.Institute of Surveying and Mapping, Information Engineering University, Zhengzhou 450001, China

Received:2023-09-06 Published:2024-07-22
Contact: Hong XIE E-mail:lyan@sgg.whu.edu.cn;hxie@sgg.whu.edu.cn
About author:YAN Li (1966—), male, professor, PhD supervisor, majors in photogrammetry, remote sensing and LiDAR.　E-mail: lyan@sgg.whu.edu.cn
Supported by:
The National Natural Science Foundation of China(42394061);The Science and Technology Major Project of Hubei Province(2021AAA010);Open Fund of Hubei Luojia Laboratory(220100053)

摘要/Abstract

摘要：

智能化测绘的发展对数据采集高效性、完备性和智能性提出了更高的要求。尤其是在林下等GNSS拒止环境下，现有传统手段往往难以完成高效率、高覆盖率测量。为了满足未知环境的智能化感知测量需求，以无人机为移动平台，本文设计并提出了一种融合视觉在线自主定位及全局探测路径规划的自主无人机智能感知测量技术与总体框架。本文首先设计并采用了一种基于点线特征的VIO（visual-inertial odometry）在线定位算法，通过点线特征的提取和匹配进行初始位姿的解算，之后利用因子图优化实时地输出无人机高精度的位姿信息。进一步地，为了实现无人机对于未知环境高效且高覆盖率的自主测量，采用了一种顾及多层次信息的全局最优探测路径规划方法确定局部最佳探测目标，然后通过轨迹搜索和优化算法实时地生成高质量的探测运动轨迹。通过自主搭建无人机平台对该技术框架进行了验证，分步对比和总体真实试验表明框架设计并采用的定位及空间探测方法相较于目前具有代表性的方法具有明显优势，并且在GNSS拒止局部林下环境中实现了高效高覆盖率的全自主测量，为未知场景进一步的在线智能化感知奠定了良好的理论方法与框架基础。

关键词: 自主无人机, 智能感知测量技术与框架, 在线视觉定位, 自主规划与空间探测

Abstract:

The development of intelligent surveying and mapping puts higher requirements for efficient, complete, and intelligent data collection, especially in GNSS-denied environments such as under-canopy, where traditional methods frequently struggle to achieve efficient and high-coverage measurements. To address the need for intelligent perception measurement of unknown environments, this paper introduces a novel intelligent perception measurement unmanned aerial vehicle (UAV) technology and framework, using a UAV as a mobile platform. This paper integrates visual online autonomous localization with global exploration path planning. Initially, the framework incorporates a novel visual-inertial odometry (VIO) online localization algorithm based on point and line features, which solves the initial pose estimation through feature extraction and matching of point and line features, and then high-precision pose information of the UAV is generated in real-time using factor graph optimization. Furthermore, to ensure efficient and high-coverage autonomous UAV measurements in unknown environments, this paper employs a global optimal exploration path planning method that considers multi-level information to determine the local exploration targets, and then generates high-quality exploration trajectories in real time through trajectory search and optimization algorithm. Moreover, the framework was validated by a customized UAV platform, the step-by-step comparison and overall real-world test demonstrates that the localization and space exploration methods designed and adopted by the framework have significant advantages compared with the current representative methods. In addition, it achieves efficient and high-coverage full autonomous measurements in GNSS-denied local under-canopy environments, which establishes a solid theoretical and framework foundation for the further development of online intelligent perception in unknown scenarios.

Key words: autonomous UAV, intelligent perception measurements technology and framework, online visual localization, autonomous planning and space exploration

中图分类号:

P232

闫利, 赵英豪, 戴集成, 徐博, 谢洪, 周玉泉. 面向未知环境的自主无人机智能感知测量技术[J]. 测绘学报, 2024, 53(6): 999-1012.

Li YAN, Yinghao ZHAO, Jicheng DAI, Bo XU, Hong XIE, Yuquan ZHOU. Intelligent perception measurement technology of autonomous UAV for unknown environment[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(6): 999-1012.

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

图9

图10

表3

图11

图12

参考文献 51

[1]	陈军, 刘万增, 武昊, 等. 智能化测绘的基本问题与发展方向[J]. 测绘学报, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021.20210235. DOI:10.11947/j.AGCS.2021.20210235.
	CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping: fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021.20210235. DOI:10.11947/j.AGCS.2021.20210235.
[2]	陈军, 武昊, 刘万增, 等. 自然资源时空信息的技术内涵与研究方向[J]. 测绘学报, 2022, 51(7):1130-1140. DOI:10.11947/j.AGCS.2022.20210643.
	CHEN Jun, WU Hao, LIU Wanzeng, et al. Technical connotation and research agenda of natural resources spatio-temporal information[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7):1130-1140. DOI:10.11947/j.AGCS.2022.20210643.
[3]	CHEN Chen, LI Zhilin, LI Songnian, et al. From digitalized to intelligentized surveying and mapping: fundamental issues and research agenda[J]. Journal of Geodesy and Geoinformation Science, 2022, 5(2):148-160.
[4]	郭庆华, 胡天宇, 刘瑾, 等. 轻小型无人机遥感及其行业应用进展[J]. 地理科学进展, 2021, 40(9):1550-1569.
	GUO Qinghua, HU Tianyu, LIU Jin, et al. Advances in light weight unmanned aerial vehicle remote sensing and major industrial applications[J]. Progress in Geography, 2021, 40(9):1550-1569.
[5]	闫利, 陈宇, 谢洪, 等. 测量机器人的关键技术[J]. 测绘学报, 2021, 50(9):1159-1169. DOI:10.11947/j.AGCS.2021.20210090.
	YAN Li, CHEN Yu, XIE Hong, et al. Surveying robot and its key technology[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(9):1159-1169. DOI:10.11947/j.AGCS.2021.20210090.
[6]	HE Yijia, XU Bo, OUYANG Zhanpeng, et al. A rotation-translation-decoupled solution for robust and efficient visual-inertial initialization[C]//Proceedings of 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Vancouver: IEEE, 2023: 739-748.
[7]	LIN H Y, ZHAN Jiarong. GNSS-denied UAV indoor navigation with UWB incorporated visual inertial odometry[J]. Measurement, 2023, 206:112256.
[8]	DUAN Ran, PAUDEL D P, FU Changhong, et al. Stereo orientation prior for UAV robust and accurate visual odometry[J]. ASME Transactions on Mechatronics, 2022, 27(5):3440-3450.
[9]	DI Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):38-49.
[10]	BARTOLOMEI L, TEIXEIRA L, CHLI M. Perception-aware path planning for UAVs using semantic segmentation[C]//Proceedings of 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2020: 5808-5815.
[11]	LEE E M, CHOI J, LIM H, et al. REAL: rapid exploration with active loop-closing toward large-scale 3D mapping using UAVs[C]//Proceedings of 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems. [S.l.]: ACM Press, 2021: 4194-4198.
[12]	ZHOU Quan, REDDING C J, QI Hairong, et al. Agile path planning for radiation source searching with aerial drones[C]//Proceedings of 2021 IEEE Nuclear Science Symposium and Medical Imaging Conference. Piscataway: IEEE, 2021: 1-2.
[13]	QIN Tong, LI Peiliang, SHEN Shaojie. VINS-mono: a robust and versatile monocular visual-inertial state estimator[J]. IEEE Transactions on Robotics, 2018, 34(4):1004-1020.
[14]	STEENBEEK A, NEX F. CNN-based dense monocular visual SLAM for real-time UAV exploration in emergency conditions[J]. Drones, 2022, 6(3):79.
[15]	RHODES C, LIU Cunjia, CHEN Wenhua. Autonomous source term estimation in unknown environments: from a dual control concept to UAV deployment[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2274-2281.
[16]	MUR-ARTAL R, TARDÓS J D. ORB-SLAM2: an open-source SLAM system for monocular, stereo, and RGB-D cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[17]	CAMPOS C, ELVIRA R, RODRIGUEZ J J G, et al. ORB-SLAM3: an accurate open-source library for visual, visual-inertial, and multimap SLAM[J]. IEEE Transactions on Robotics, 2021, 37(6):1874-1890.
[18]	SUN Ke, MOHTA K, PFROMMER B, et al. Robust stereo visual inertial odometry for fast autonomous flight[J]. IEEE Robotics and Automation Letters, 2018, 3(2):965-972.
[19]	SUMIKURA S, SHIBUYA M, SAKURADA K. OpenVSLAM: a versatile visual SLAM framework[C]//Proceedings of the 27th ACM International Conference on Multimedia. Nice: ACM Press, 2019: 2292-2295.
[20]	BLOESCH M, OMARI S, HUTTER M, et al. Robust visual inertial odometry using a direct EKF-based approach[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: ACM Press, 2015: 298-304.
[21]	ROSINOL A, ABATE M, CHANG Yun, et al. Kimera: an open-source library for real-time metric-semantic localization and mapping[C]//Proceedings of 2020 IEEE International Conference on Robotics and Automation. Paris: IEEE, 2020: 1689-1696.
[22]	王晨捷, 罗斌, 李成源, 等. 无人机视觉SLAM协同建图与导航[J]. 测绘学报, 2020, 49(6):767-776. DOI:10.11947/j.AGCS.2020.20190145.
	WANG Chenjie, LUO Bin, LI Chengyuan, et al. The collaborative mapping and navigation based on visual SLAM in UAV platform[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6):767-776. DOI:10.11947/j.AGCS.2020.20190145.
[23]	GOMEZ-OJEDA R, MORENO F A, ZUNIGA-NOEL D, et al. PL-SLAM: a stereo SLAM system through the combination of points and line segments[J]. IEEE Transactions on Robotics, 2019, 35(3):734-746.
[24]	GOMEZ-OJEDA R, BRIALES J, GONZALEZ-JIMENEZ J. PL-SVO: semi-direct monocular visual odometry by combining points and line segments[C]//Proceedings of 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). [S.l.]: ACM Press, 2016: 4211-4216.
[25]	ZHOU Lipu, WANG Shengze, KAESS M. DPLVO: direct point-line monocular visual odometry[J]. IEEE Robotics and Automation Letters, 2021, 6(4):7113-7120.
[26]	ZHAO Yinghao, YAN Li, DAI Jicheng, et al. Robust planning system for fast autonomous flight in complex unknown environment using sparse directed frontier points[J]. Drones, 2023, 7(3):219.
[27]	BIRCHER A, KAMEL M, ALEXIS K, et al. Receding horizon “next-best-view” planner for 3D exploration[C]//Proceedings of 2016 IEEE International Conference on Robotics and Automation (ICRA). Sweden: IEEE, 2016: 1462-1468.
[28]	WITTING C, FEHR M, BEHNEMANN R, et al. History-aware autonomous exploration in confined environments using MAVs[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Madrid: IEEE, 2018: 1-9.
[29]	SELIN M, TIGER M, DUBERG D, et al. Efficient autonomous exploration planning of large-scale 3D environments[J]. IEEE Robotics and Automation Letters, 2019, 4(2):1699-1706.
[30]	WANG Chaoqun, ZHU Delong, LI Teng, et al. Efficient autonomous robotic exploration with semantic road map in indoor environments[J]. IEEE Robotics and Automation Letters, 2019, 4(3):2989-2996.
[31]	BATINOVIC A, IVANOVIC A, PETROVIC T, et al. A shadow casting-based next-best-view planner for autonomous 3D exploration[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2969-2976.
[32]	DUBERG D, JENSFELT P. UFOExplorer: fast and scalable sampling-based exploration with a graph-based planning structure[J]. IEEE Robotics and Automation Letters, 2022, 7(2):2487-2494.
[33]	NAAZARE M, ROSAS F G, SCHULZ D. Online next-best-view planner for 3D-exploration and inspection with a mobile manipulator robot[J]. IEEE Robotics and Automation Letters, 2022, 7(2):3779-3786.
[34]	RESPALL V M, DEVITT D, FEDORENKO R, et al. Fast sampling-based next-best-view exploration algorithm for a MAV[C]//Proceedings of 2021 IEEE International Conference on Robotics and Automation. Xi'an: IEEE, 2021: 89-95.
[35]	YAMAUCHI B. A frontier-based approach for autonomous exploration[C]//Proceedings of 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation. Monterey: IEEE, 1997.
[36]	ZHOU Boyu, ZHANG Yichen, CHEN Xinyi, et al. FUEL: fast UAV exploration using incremental frontier structure and hierarchical planning[J]. IEEE Robotics and Automation Letters, 2021, 6(2):779-786.
[37]	CIESLEWSKI T, KAUFMANN E, SCARAMUZZA D. Rapid exploration with multi-rotors: a frontier selection method for high speed flight[C]//Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. [S.l.]: ACM Press, 2017: 2135-2142.
[38]	SHI Jianbo, TOMAS I. Good features to track[C]//Proceedings of 1994 IEEE Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 1994: 593-600.
[39]	BAKER S, MATTHEWS I. Lucas-kanade 20 years on: a unifying framework[J]. International Journal of Computer Vision, 2004, 56(3):221-255.
[40]	ZHANG Lilian, KOCH R. An efficient and robust line segment matching approach based on LBD descriptor and pairwise geometric consistency[J]. Journal of Visual Communication and Image Representation, 2013, 24(7):794-805.
[41]	ELFES A. Using occupancy grids for mobile robot perception and navigation[J]. Computer, 1989, 22(6):46-57.
[42]	ZHOU Boyu, ZHANG Yichen, CHEN Xinyi, et al. FUEL: fast UAV exploration using incremental frontier structure and hierarchical planning[J]. IEEE Robotics and Automation Letters, 2021, 6(2):779-786.
[43]	ZHOU Xin, WANG Zhepei, YE Hongkai, et al. EGO-planner: an ESDF-free gradient-based local planner for quadrotors[J]. IEEE Robotics and Automation Letters, 2021, 6(2):478-485.
[44]	WANG Zhepei, ZHOU Xin, XU Chao, et al. Geometrically constrained trajectory optimization for multicopters[J]. IEEE Transactions on Robotics, 2022, 38(5):3259-3278.
[45]	REN Yunfan, ZHU Fangcheng, LIU Wenyi, et al. Bubble planner: planning high-speed smooth quadrotor trajectories using receding corridors[C]//Proceedings of 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems. Kyoto: IEEE, 2022: 6332-6339.
[46]	ZHAO Yinghao, YAN Li, CHEN Yu, et al. Robust and efficient trajectory replanning based on guiding path for quadrotor fast autonomous flight[J]. Remote Sensing, 2021, 13(5):972.
[47]	ZHOU Boyu, GAO Fei, WANG Luqi, et al. Robust and efficient quadrotor trajectory generation for fast autonomous flight[J]. IEEE Robotics and Automation Letters, 2019, 4(4):3529-3536.
[48]	BURRI M, NIKOLIC J, GOHL P, et al. The EuRoC micro aerial vehicle datasets[J]. International Journal of Robotics Research, 2016, 35(10):1157-1163.
[49]	LEUTENEGGER S, FURGALE P, RABAUD V, et al. Key frame-based visual-inertial SLAM using nonlinear optimization[C]//Proceedings of 2013 Science and Systems IX. Robotics: Science and Systems Foundation, 2013.
[50]	HE Y, ZHAO J, GUO Y, et al. PL-VIO: tightly-coupled monocular visual-inertial odometry using point and line features[J]. Sensors (Basel), 2018, 18(4):E1159.
[51]	SCHÖNBERGER J L, ZHENG Enliang, FRAHM J M, et al. Pixelwise view selection for unstructured multi-view stereo[C]//Proceedings of 2016 Computer Vision. Cham: Springer, 2016: 501-518.

数据序列	OKVIS-Mono	VINS-Mono	PL-VIO	PL-SLAM	本文算法
MH_01_easy	29.5	14.8	20.1	15.7	11.1
MH_02_easy	30.7	17.1	13.1	14.2	9.3
MH_03_medium	33.4	19.4	26.1	14.7	15.7
MH_04_difficult	38.9	34.6	35.8	12.4	17.1
MH_05_difficult	46.7	29.2	24.4	55.5	14.4
V1_02_medium	22.2	7.9	17.0	16.9	8.9
V1_03_difficult	28.1	20.7	27.0	42.0	14.3
V2_01_easy	14.0	8.2	9.3	19.4	7.4
V2_02_medium	21.1	15.7	12.3	25.2	12.2

数据序列	OKVIS-Mono	VINS-Mono	PL-VIO	PL-SLAM	本文算法
MH_01_easy	3.2	2.0	1.6	6.0	1.6
MH_02_easy	3.9	2.3	1.7	2.5	0.9
MH_03_medium	3.3	1.6	1.7	3.4	0.8
MH_04_difficult	2.3	1.5	1.6	6.8	1.4
MH_05_difficult	2.4	0.7	1.1	9.9	0.7
V1_02_medium	6.0	2.6	3.2	5.6	1.5
V1_03_difficult	8.1	6.2	3.4	9.1	4.2
V2_01_easy	2.2	2.0	2.2	2.3	2.3
V2_02_medium	4.9	4.3	2.9	4.6	1.7

场景	方法	探索耗时/s			飞行距离/m			覆盖范围/m³
场景	方法	Avg	Min	Std	Avg	Min	Std	Avg	Min	Std
office	FUEL	167.5	156.7	8.0	239.0	225.5	11.9	1 082.2	1 079.6	2.0
office	本文算法	133.2	127.4	4.3	193.8	177.9	8.2	1 077.7	1 075.5	2.0
pillar	FUEL	163.6	147.0	9.1	215.8	189.7	14.8	756.8	753.0	1.8
pillar	本文算法	142.3	130.2	7.7	156.7	136.0	11.3	752.3	748.9	2.0
bridge	FUEL	146.2	126.1	13.8	208.5	186.8	14.5	871.6	868.7	1.6
bridge	本文算法	122.4	109.8	7.3	167.7	146.7	10.6	867.4	865.4	1.6

面向未知环境的自主无人机智能感知测量技术

Intelligent perception measurement technology of autonomous UAV for unknown environment

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 51

相关文章 15

编辑推荐

Metrics

本文评价

[1]	康俊华. 基于全卷积神经网络的端到端立体密集匹配研究[J]. 测绘学报, 2022, 51(5): 785-785.
[2]	李佳田, 王聪聪, 阿晓荟, 晏玲, 朱志浩, 高鹏. 多向前方交会与单隐层神经网络结合的近景前方交会法[J]. 测绘学报, 2020, 49(6): 736-745.
[3]	闫利, 戴集成, 谭骏祥, 刘华, 陈长军. SLAM激光点云整体精配准位姿图技术[J]. 测绘学报, 2019, 48(3): 313-321.
[4]	张春森, 张萌萌, 郭丙轩. 影像信息驱动的三角网格模型优化方法[J]. 测绘学报, 2018, 47(7): 959-967.
[5]	闫利, 谭骏祥, 刘华, 陈长军. 融合遗传算法和ICP的地面与车载激光点云配准[J]. 测绘学报, 2018, 47(4): 528-536.
[6]	张帆, 黄印, 黄先锋, 徐思奇. 非直视区域的普通平面镜辅助地面三维激光扫描方法[J]. 测绘学报, 2017, 46(12): 1950-1958.
[7]	邓非, 吴幼丝, 胡玉雷, 崔红霞. 球形全景影像位姿估计的改进EPnP算法[J]. 测绘学报, 2016, 45(6): 677-684.
[8]	王祥, 张永军, 黄山, 谢勋伟. 旋转多基线摄影光束法平差法方程矩阵带宽优化[J]. 测绘学报, 2016, 45(2): 170-177.
[9]	刘祥磊, 童小华, 马静. 视频测量影像序列椭圆形人工目标点快速识别和跟踪方法[J]. 测绘学报, 2015, 44(6): 663-669.
[10]	姚吉利, 马宁, 贾象阳, 徐广鹏, 谢建春. 球形标靶的固定式扫描大点云自动定向方法[J]. 测绘学报, 2015, 44(4): 431-437.
[11]	浦石李京伟郭四清. 融合语义特征与GPS位置的地面激光点云拼接方法[J]. 测绘学报, 2014, 43(5): 545-550.
[12]	李畅. 利用灭点进行相机检校与定姿的充要条件及其不确定性分析[J]. 测绘学报, 2012, 41(6): 851-856.
[13]	刘华. 车载激光点云地物提取与分类研究[J]. 测绘学报, 2020, 49(11): 1506-1506.
[14]	李帅鑫, 李九人, 田滨, 陈龙, 王力, 李广云. 面向点云退化的隧道环境的无人车激光SLAM方法[J]. 测绘学报, 2021, 50(11): 1487-1499.
[15]	孙喜亮, 关宏灿, 苏艳军, 徐光彩, 郭庆华. 面向高精度城市测绘的激光紧耦合SLAM方法[J]. 测绘学报, 2021, 50(11): 1585-1593.