点-线特征联合的全景图像位姿解算方法

doi:10.11947/j.AGCS.2023.20210565

测绘学报 ›› 2023, Vol. 52 ›› Issue (2): 218-229.doi: 10.11947/j.AGCS.2023.20210565

点-线特征联合的全景图像位姿解算方法

朱宁宁, 杨必胜, 陈驰, 董震

武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430079

收稿日期:2021-10-10 修回日期:2021-12-27 发布日期:2023-03-07
通讯作者: 杨必胜 E-mail:bshyang@whu.edu.cn
作者简介:朱宁宁(1988-),男,博士后,研究方向为摄影测量与遥感。E-mail:ningningzhu@whu.edu.cn
基金资助:
国家杰出青年科学基金（41725005）；国家自然科学基金（42101446；42071451）；测绘遥感信息工程国家重点实验室专项科研经费资助；中国博士后科学基金（2022T150488）

Position-attitude calculation of panoramic image based on point-line feature combination

ZHU Ningning, YANG Bisheng, CHEN Chi, DONG Zhen

State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Received:2021-10-10 Revised:2021-12-27 Published:2023-03-07
Supported by:
The National Science Fund for Distinguished Young Scholars of China (No. 41725005);The National Natural Sciences Foundation of China (Nos. 42101446;42071451);LIESMARS Special Research Funding;China Postdoctoral Science Foundation (No. 2022T150488)

摘要/Abstract

摘要： 目前，全景图像位置和姿态参数的解算多基于点特征，而场景中普遍存在的线特征尚未得到充分利用。本文提出一种点-线特征联合的全景图像位姿解算方法，不仅可用于点特征缺失场景中全景图像位姿参数的解算，而且在点特征充足的场景中可提高位姿解算的精度和稳健性。该方法中的线特征使用线上的任意两点表示，不要求全景图像和三维场景同名线上的选点具有对应关系，因而易于选取，具有极大的实用性。首先，使用直接线性变换构建点-线特征联合的全景图像位姿解算模型，并针对水平线和垂直线获取简化后的模型；然后，利用仿真道路场景，从特征点和线的不同组合方式及大姿态角两方面分析该模型的适用性，并通过人工引入不同类型及量级的点-线误差分析该模型的容差性；最后，将本文方法应用于全景图像与激光点云的融合，从理论和实践两方面证明点-线特征联合的位姿解算方法在精度、稳健性和容差性方面优于单纯的点特征解算方法。

关键词: 全景图像, 点-线特征联合, 位姿解算, 仿真分析, 全景图像与激光点云融合

Abstract: At present, the position and attitude parameters of panoramic images are mostly solved by point features, while the line features commonly existing in the scene have not been fully utilized. In this paper, a solution method based on point-line feature combination is proposed, which can not only be used to solve the position and attitude of panoramic image in the scene with missing point features, but also improve the accuracy and robustness in the scene with sufficient point features. The line feature in this method is represented by any two points on the line, which does not require the corresponding relationship between the panoramic image and the 3D scene, so it is easy to select and has great practicability. Firstly, the direct linear transform (DLT) is used to construct the point-line feature combination model of panoramic image, and the simplified model is obtained for horizontal and vertical lines; Then, using the simulated road scene, the applicability of the model is analyzed from the two aspects:different combinations of feature points and lines, large attitude angle, and the tolerance is analyzed by manually introducing different types and magnitude of point-line errors; Finally, this method is applied to the fusion of panoramic image and LiDAR points. It is proved that this method of point-line feature combination is better than the simple point method in accuracy, robustness and tolerance.

Key words: panoramic image, point-line feature combination, position and attitude calculation, simulation analysis, panoramic image and LiDAR points fusion

中图分类号:

P227

朱宁宁, 杨必胜, 陈驰, 董震. 点-线特征联合的全景图像位姿解算方法[J]. 测绘学报, 2023, 52(2): 218-229.

ZHU Ningning, YANG Bisheng, CHEN Chi, DONG Zhen. Position-attitude calculation of panoramic image based on point-line feature combination[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 218-229.

参考文献

[1] LAN Jinhui, LI Jian, HU Guangda, et al. Distance estimation using a panoramic sensor based on a novel spherical optical flow algorithm[J]. Optics & Laser Technology, 2013, 45:168-176.
[2] TEHRANI M H, GARRATT M A, ANAVATTI S. Horizon-based attitude estimation from a panoramic vision sensor[J]. IFAC Proceedings Volumes, 2012, 45(1):185-188.
[3] 李德仁. 论地球空间信息的三维可视化:基于图形还是基于影像[J]. 测绘学报, 2010, 39(2):111-114. LI Deren. 3D visualization of geospatial information:graphics based or imagery based[J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(2):111-114.
[4] ALMARAZ-CABRAL C C, GONZALEZ-BARBOSA J J, VILLA J, et al. Fringe projection profilometry for panoramic 3D reconstruction[J]. Optics and Lasers in Engineering, 2016, 78:106-112.
[5] BRENNAN P F, PONTO K, CASPER G, et al. Virtualizing living and working spaces:proof of concept for a biomedical space-replication methodology[J]. Journal of Biomedical Informatics, 2015, 57:53-61.
[6] 杨化超, 张磊, 姚国标, 等. 局部单应约束的高精度图像自动配准方法[J]. 测绘学报, 2012, 41(3):401-408. YANG Huachao, ZHANG Lei, YAO Guobiao, et al. An automated image registration method with high accuracy based on local homography constraint[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(3):401-408.
[7] 吴凡路, 刘建军, 任鑫, 等. 嫦娥三号全景相机图像全景镶嵌方法的研究[J]. 光学学报, 2014, 34(9):182-190. WU Fanlu, LIU Jianjun, REN Xin, et al. Research on panoramic mosaics method of Chang'E-3 Pancam images[J]. Acta Optica Sinica, 2014, 34(9):182-190.
[8] 霍春宝, 童帅, 赵立辉, 等. SIFT特征匹配的显微全景图拼接[J]. 辽宁工程技术大学学报(自然科学版), 2015, 34(1):93-96. HUO Chunbao, TONG Shuai, ZHAO Lihui, et al. Microscopic panorama splicing based on sift features matching[J]. Journal of Liaoning Technical University (Natural Science), 2015, 34(1):93-96.
[9] 黄立勤, 陈财淦. 全景图拼接中图像融合算法的研究[J]. 电子与信息学报, 2014, 36(6):1292-1298. HUANG Liqin, CHEN Caigan. Study on image fusion algorithm of panoramic image stitching[J]. Journal of Electronics & Information Technology, 2014, 36(6):1292-1298.
[10] KIM D H, YOON Y I, CHOI J S. An efficient method to build panoramic image mosaics[J]. Pattern Recognition Letters, 2003, 24(14):2421-2429.
[11] CHEN Haipeng, SHEN Xuanjing, LI Xiaofei, et al. Bionic mosaic method of panoramic image based on compound eye of fly[J]. Journal of Bionic Engineering, 2011, 8(4):440-448.
[12] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018(1):1-15.
[13] XU Yusheng, HUANG Huang, TONG Tong, et al. Exploiting robust estimators in phase correlation of 3D point clouds for 6 DoF pose estimation[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3):72-90.
[14] 季顺平, 史云. 车载全景相机的影像匹配和光束法平差[J]. 测绘学报, 2013, 42(1):94-100, 107. JI Shunping, SHI Yun. Image matching and bundle adjustment using vehicle-based panoramic camera[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(1):94-100, 107.
[15] 谢东海, 钟若飞, 吴俣, 等. 球面全景影像相对定向与精度验证[J]. 测绘学报, 2017, 46(11):1822-1829.DOI:10.11947/j.AGCS.2017.20160645. XIE Donghai, ZHONG Ruofei, WU Yu, et al. Relative pose estimation and accuracy verification of spherical panoramic image[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(11):1822-1829.DOI:10.11947/j.AGCS.2017.20160645.
[16] NISTER D. An efficient solution to the five-point relative pose problem[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(6):756-770.
[17] 季顺平, 史云. 多镜头组合型全景相机两种成像模型的定位精度比较[J]. 测绘学报, 2014, 43(12):1252-1258. JI Shunping, SHI Yun. Comparison of two sensor models for multi-camera rig system in measurements[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(12):1252-1258.
[18] 朱宁宁. 柱面全景图像拼接方法的仿真分析[J]. 测绘学报, 2017, 46(4):487-497.DOI:10.11947/j.AGCS.2017.20160456. ZHU Ningning. Simulation analysis of cylindrical panoramic image mosaic[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(4):487-497.DOI:10.11947/j.AGCS.2017.20160456.
[19] 马洪超, 姚春静, 邬建伟. 利用线特征进行高分辨率影像与LiDAR点云的配准[J]. 武汉大学学报(信息科学版), 2012, 37(2):136-140, 159. MA Hongchao, YAO Chunjing, WU Jianwei. Registration of LiDAR point clouds and high resolution images based on linear features[J]. Geomatics and Information Science of Wuhan University, 2012, 37(2):136-140, 159.
[20] 何培培, 万幼川, 杨威, 等. 基于线特征的城区激光点云与影像自动配准[J].光学学报, 2015, 35(5):360-368. HE Peipei, WAN Youchuan, YANG Wei, et al. Automatic registration of urban laser point cloud with aerial image data based on straight-lines[J]. Acta Optica Sinica, 2015, 35(5):360-368.
[21] 陈驰, 杨必胜, 彭向阳. 低空UAV激光点云和序列影像的自动配准方法[J]. 测绘学报, 2015, 44(5):518-525.DOI:10.11947/j.AGCS.2015.20130558. CHEN Chi, YANG Bisheng, PENG Xiangyang. Automatic registration of low altitude UAV sequent images and laser point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(5):518-525.DOI:10.11947/j.AGCS.2015.20130558.
[22] ZHU Ningning, JIA Yonghong, JI Shunping. Registration of panoramic/fish-eye image sequence and LiDAR points using skyline features[J]. Sensors (Basel, Switzerland), 2018, 18(5):1651.
[23] 闫利, 索一凡, 曹亮. 车载激光雷达点云与全景影像的自动配准[J]. 测绘科学, 2016, 41(4):113-117, 123. YAN Li, SUO Yifan, CAO Liang. Registration of vehicle-borne LiDAR point cloud and panoramic image[J]. Science of Surveying and Mapping, 2016, 41(4):113-117, 123.
[24] YANG Bisheng, CHEN Chi. Automatic registration of UAV-borne sequent images and LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101(3):262-274.
[25] ZHU Ningning. Simulation analysis of spherical panoramic mosaic[J]. Signal Processing, 2019, 158(5):190-200.
[26] 杨必胜, 董震. 点云智能处理[M]. 北京:科学出版社, 2020. YANG Bisheng, DONG Zhen. Point cloud intelligent processing[M]. Beijing:Science Press, 2020.
[27] 杨必胜, 梁福逊, 黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报, 2017, 46(10):1509-1516.DOI:10.11947/j.AGCS.2017.20170351. YANG Bisheng, LIANG Fuxun, HUANG Ronggang. Progress, challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1509-1516.DOI:10.11947/j.AGCS.2017.20170351.
[28] 杨必胜, 董震. 点云智能研究进展与趋势[J]. 测绘学报, 2019, 48(12):1575-1585.DOI:10.11947/j.AGCS.2019.20190465. YANG Bisheng, DONG Zhen. Progress and perspective of point cloud intelligence[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1575-1585.DOI:10.11947/j.AGCS.2019.20190465.
[29] WANG Wang, LI Shaochun, WANG Wang, et al.. A simple deep learning network for classification of 3D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3):49-59.
[30] ZHU Ningning, JIA Yonghong, HUANG Xia. Semiautomatically register MMS LiDAR points and panoramic image sequence using road lamp and lane[J]. Photogrammetric Engineering & Remote Sensing, 2019, 85(11):829-840.
[31] ZHU Ningning, YANG Bisheng, DONG Zhen, et al. Automatic registration of mobile mapping system lidar points and panoramic-image sequences by relative orientation model[J].Photogrammetric Engineering & Remote Sensing, 2021,87(12):913-922.

点-线特征联合的全景图像位姿解算方法

Position-attitude calculation of panoramic image based on point-line feature combination

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 1

编辑推荐

Metrics

本文评价