车载MMS激光点云与序列全景影像自动配准方法

doi:10.11947/j.AGCS.2018.20170520

测绘学报 ›› 2018, Vol. 47 ›› Issue (2): 215-224.doi: 10.11947/j.AGCS.2018.20170520

车载MMS激光点云与序列全景影像自动配准方法

陈驰^1,2, 杨必胜^1,2, 田茂^1,2, 李健平^1,2, 邹响红^1,2, 吴唯同^1,2, 宋易恒^1,2

1. 武汉大学测绘遥感信息工程国家重点实验室, 湖北武汉 430079;
2. 武汉大学时空数据智能获取技术与应用教育部工程研究中心, 湖北武汉 430079

收稿日期:2017-09-14 修回日期:2017-12-04 出版日期:2018-02-20 发布日期:2018-03-02
通讯作者: 杨必胜 E-mail:bshyang@whu.edu.cn
作者简介:陈驰(1989-),男,博士,助理研究员,研究方向为无人机、车载、机器人移动测量测绘数据融合。E-mail:chenchi_liesmars@foxmail.com
基金资助:
国家自然科学基金（41701530；41725005；41531177）；国家重点研发计划（2016YFF0103501）；中国博士后科学基金（2016M600614）

Automatic Registration of Vehicle-borne Mobile Mapping Laser Point Cloud and Sequent Panoramas

CHEN Chi^1,2, YANG Bisheng^1,2, TIAN Mao^1,2, LI Jianping^1,2, ZOU Xianghong^1,2, WU Weitong^1,2, SONG Yiheng^1,2

1. State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China;
2. Engineering Research Center for Spatio-temporal Data Smart Acquisition and Application, Ministry of Education of China, Wuhan University, Wuhan 430079, China

Received:2017-09-14 Revised:2017-12-04 Online:2018-02-20 Published:2018-03-02
Supported by:
The National Natural Science Foundation of China (Nos. 41701530;41725005;41531177);The National Key Research and Development Program of China (No. 2016YFF0103501);The China Postdoctoral Science Foundation (No. 2016M600614)

摘要/Abstract

摘要： 提出一种车载移动测量系统（MMS）激光点云与序列全景影像自动配准方法。首先采用层次化城市场景目标提取方法自激光点云提取天际线矢量，在全景影像中经虚拟成像与分割角点提取算法生成天际线矢量。然后，将提取结果作为几何配准基元，构建配准基元图，通过最小化配准基元图编辑距离进行匹配，组成共轭配准基元对，解算2D-3D粗配准模型，获得全景影像与LiDAR点云参考坐标系之间的初始转换关系。最后，为消除几何配准基元提取与匹配误差对配准结果的影响，自序列全景影像虚拟成像影像生成多视立体密集匹配点云，继而使用变种ICP算法优化其与激光点云数据间3D-3D配准参数，间接优化全景影像与激光点云间的配准参数，精化配准结果。试验结果表明，本文提出的自动配准方法可以实现车载MMS激光点云与序列全景影像的1.5像素级自动配准，配准成果可应用于真彩色点云生成等点云/影像数据融合应用。

关键词: 激光点云, 全景影像, 配准, 特征提取, 车载移动测量系统

Abstract: An automatic registration method of mobile mapping system laser point cloud and sequence panoramic image is proposed in this paper.Firstly,hierarchical object extraction method is applied on LiDAR data to extract the building façade and outline polygons are generated to construct the skyline vectors.A virtual imaging method is proposed to solve the distortion on panoramas and corners on skylines are further detected on the virtual images combining segmentation and corner detection results.Secondly,the detected skyline vectors are taken as the registration primitives.Registration graphs are built according to the extracted skyline vector and further matched under graph edit distance minimization criteria.The matched conjugate primitives are utilized to solve the 2D-3D rough registration model to obtain the initial transformation between the sequence panoramic image coordinate system and the LiDAR point cloud coordinate system.Finally,to reduce the impact of registration primitives extraction and matching error on the registration results,the optimal transformation between the multi view stereo matching dens point cloud generated from the virtual imaging of the sequent panoramas and the LiDAR point cloud are solved by a 3D-3D ICP registration algorithm variant,thus,refine the exterior orientation parameters of panoramas indirectly.Experiments are undertaken to validate the proposed method and the results show that 1.5 pixel level registration results are achieved on the experiment dataset.The registration results can be applied to point cloud and panoramas fusion applications such as true color point cloud generation.

Key words: LiDAR point cloud, panoramas, registration, feature extraction, vehicle-borne mobile mapping system

中图分类号:

P237

陈驰, 杨必胜, 田茂, 李健平, 邹响红, 吴唯同, 宋易恒. 车载MMS激光点云与序列全景影像自动配准方法[J]. 测绘学报, 2018, 47(2): 215-224.

CHEN Chi, YANG Bisheng, TIAN Mao, LI Jianping, ZOU Xianghong, WU Weitong, SONG Yiheng. Automatic Registration of Vehicle-borne Mobile Mapping Laser Point Cloud and Sequent Panoramas[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2): 215-224.

参考文献

[1] 侯艳芳, 叶泽田, 杨勇. 基于POS数据的车载面阵CCD影像与激光点云融合处理研究[J]. 遥感信息, 2011(4):76-79. HOU Yanfang, YE Zetian, YANG Yong, Research on Integrated Processing of Vehicle-borne Array CCD Images and Laser Point Cloud Based on POS Data[J]. Remote Sensing Information, 2011(4):76-79.
[2] 吴胜浩, 钟若飞. 基于移动平台的激光点云与数字影像融合方法[J]. 首都师范大学学报(自然科学版), 2011, 32(4):57-61. WU Shenghao, ZHONG Ruofei. The 3D Visualization's Approach Based on Vehicle-borne Laser Scanning[J]. Journal of Capital Normal University (Natural Science Edition), 2011, 32(4):57-61.
[3] PYLVÄNÄINEN T, ROIMELA K, VEDANTHAM R, et al. Automatic Alignment and Multi-View Segmentation of Street View Data Using 3D Shape Priors[C]//Proceedings of the 5th International Symposium on 3D Data Processing, Visualization and Transmission (3DPVT). Paris, France:Espace Saint Martin, 2010:738-739.
[4] WANG Ruisheng, FERRIE F P, MACFARLANE J. Automatic Registration of Mobile LiDAR and Spherical Panoramas[C]//Proceedings of 2012 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). Providence, RI:IEEE, 2012:33-40.
[5] SWART A, BROERE J, VELTKAMP R, et al. Refined Non-rigid Registration of A Panoramic Image Sequence to a LiDAR Point Cloud[M]//STILLA U, ROTTENSTEINER F, MAYER H, et al. Photogrammetric Image Analysis. Berlin:Springer, 2011:73-84.
[6] CHEN C, YANG B, DENG C. Registration of the Panoramic Image Sequence onto Mobile Laser Scanning Point Cloud Using Linear Features[C]//The International Symposium on Mobile Mapping Technology (MMT2013). Taiwan, China:2013:1-9.
[7] TANEJA A, BALLAN L, POLLEFEYS M. Geometric Change Detection in Urban Environments Using Images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(11):2193-2206.
[8] HOFMANN S, EGGERT D, BRENNER C. Skyline Matching Based Camera Orientation from Images and Mobile Mapping Point Clouds[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2014, Ⅱ-5:181-188.
[9] VIOLA P, WELLS Ⅲ W M. Alignment by Maximization of Mutual Information[J]. International Journal of Computer Vision, 1997, 24(2):137-154.
[10] NELDER J A, MEAD R. A Simplex Method for Function Minimization[J]. The Computer Journal, 1965, 7(4):308-313.
[11] RUSINKIEWICZ S, LEVOY M. Efficient Variants of the ICP Algorithm[C]//Proceedings of the 3rd International Conference on 3-D Digital Imaging and Modeling. Quebec City, Canada:IEEE, 2001:145-152.
[12] YANG Bisheng, DONG Zhen, ZHAO Gang, et al. Hierarchical Extraction of Urban Objects from Mobile Laser Scanning Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 99:45-57.
[13] ROTHER C, KOLMOGOROV V, BLAKE A. "GrabCut":Interactive Foreground Extraction Using Iterated Graph Cuts[C]//Proceedings of ACM SIGGRAPH 2004. New York, NY:ACM, 2004:309-314.
[14] AWRANGJEB M, LU Guojun, FRASER C S. Performance Comparisons of Contour-based Corner Detectors[J]. IEEE Transactions on Image Processing, 2012, 21(9):4167-4179.
[15] CHRISTY S, HORAUD R. Iterative Pose Computation from Line Correspondences[J]. Computer Vision and Image Understanding, 1999, 73(1):137-144.
[16] 邓非, 张祖勋, 张剑清. 利用激光扫描和数码相机进行古建筑三维重建研究[J]. 测绘科学, 2007, 32(2):29-30. DENG Fei, ZHANG Zuxun, ZHANG Jianqing. 3D Reconstruction of Old Architecture by Laser Scanner and Digital Camera[J]. Science of Surveying and Mapping, 2007, 32(2):29-30.
[17] 王晏民, 胡春梅. 一种地面激光雷达点云与纹理影像稳健配准方法[J]. 测绘学报, 2012, 41(2):266-272. WANG Yanmin, HU Chunmei. A Robust Registration Method for Terrestrial LiDAR Point Clouds and Texture Image[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(2):266-272.
[18] LIU L, STAMOS I. Automatic 3D to 2D Registration for the Photorealistic Rendering of Urban Scenes[C]//Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego:IEEE, 2005:137-143.
[19] RAMALINGAM S, BOUAZIZ S, STURM P, et al. SKYLINE2GPS:Localization in Urban Canyons Using Omni-skylines[C]//Proceedings of 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Taipei, Taiwan, China:IEEE, 2010:3816-3823.
[20] SATTLER T, LEIBE B, KOBBELT L. Fast Image-based Localization Using Direct 2D-to-3D Matching[C]//Proceedings of 2011 IEEE International Conference on Computer Vision (ICCV). Barcelona, Spain:IEEE, 2011:667-674.
[21] KNOPP J, SIVIC J, PAJDLA T. Avoiding Confusing Features in Place Recognition[M]//DANⅡLIDIS K, MARAGOS P, PARAGIOS N. Computer Vision-ECCV 2010. Berlin:Springer, 2010:748-761.
[22] LI Zhuqiang, ZHANG Liqiang, ZHONG Ruofei, et al. Classification of Urban Point Clouds:A Robust Supervised Approach with Automatically Generating Training Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(3):1207-1220.
[23] KWAK E. Automatic 3D Building Model Generation by Integrating LiDAR and Aerial Images Using a Hybrid Approach[D]. Calgary:University of Calgary, 2013.
[24] AWRANGJEB M, LU Guojun, FRASER C S, et al. A Fast Corner Detector Based on the Chord-to-point Distance Accumulation Technique[C]//Proceedings of 2009 Digital Image Computing:Techniques and Applications. Melbourne, VIC, Australia:IEEE, 2009:519-525.
[25] 袁修孝, 张雪萍, 付建红. 高斯-克吕格投影坐标系下POS角元素的转换方法[J]. 测绘学报, 2011, 40(3):338-344. YUAN Xiuxiao, ZHANG Xueping, FU Jianhong. Transformation of Angular Elements Obtained Via A Position and Orientation System in Gauss-Kruger Projection Coordinate System[J]. Acta Geodaetica et Cartographica Sinica, 2011, 40(3):338-344.
[26] ZHOU Kun, HOU Qinming, WANG Rui, et al. Real-Time KD-Tree Construction on Graphics Hardware[J]. ACM Transactions on Graphics (TOG), 2008, 27(5):126.
[27] SNAVELY N, SEITZ S M, SZELISKI R. Modeling the World from Internet Photo Collections[J]. International Journal of Computer Vision, 2008, 80(2):189-210.
[28] TOLA E, LEPETIT V, FUA P. DAISY:An Efficient Dense Descriptor Applied to Wide-baseline Stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):815-830.
[29] YANG Bisheng, CHEN Chi. Automatic Registration of UAV-borne Sequent Images and LiDAR Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101:262-274.
[30] CHEN Y, MEDIONI G. Object Modeling by Registration of Multiple Range Images[C]//Proceedings of 1991 IEEE International Conference on Robotics and Automation. Sacramento, CA:IEEE, 1991:2724-2729.
[31] POMERLEAU F, COLAS F, FERLAND F, et al. Relative Motion Threshold for Rejection in ICP Registration[M]//HOWARD A, IAGNEMMA K, KELLY A. Field and Service Robotics. Berlin:Springer, 2010:229-238.

车载MMS激光点云与序列全景影像自动配准方法

Automatic Registration of Vehicle-borne Mobile Mapping Laser Point Cloud and Sequent Panoramas

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	金飞, 官恺, 刘智, 韩佳容, 芮杰, 李庆高. 无监督密集匹配特征提取网络性能分析[J]. 测绘学报, 2022, 51(3): 426-436.
[2]	方莉娜, 王爽, 赵志远, 付化胜, 陈崇成. 车载激光点云中交通标线自动分类与矢量化[J]. 测绘学报, 2021, 50(9): 1251-1265.
[3]	杨必胜, 韩旭, 董震. 适用于城市场景大规模点云语义标识的深度学习网络[J]. 测绘学报, 2021, 50(8): 1059-1067.
[4]	李培, 姜刚, 马千里, 薛万峰, 杨伟华. 结合张量与互信息的混合模型多模态图像配准方法[J]. 测绘学报, 2021, 50(7): 916-929.
[5]	马张烽, 蒋弥, 丁琪瑄. 时序Sentinel-1TOPS模式SAR数据精配准[J]. 测绘学报, 2021, 50(5): 634-640.
[6]	蒋腾平, 王永君, 张林淇, 梁冲, 孙剑. 融合CNN和MRF的激光点云层次化语义分割方法[J]. 测绘学报, 2021, 50(2): 215-225.
[7]	李帅鑫, 李九人, 田滨, 陈龙, 王力, 李广云. 面向点云退化的隧道环境的无人车激光SLAM方法[J]. 测绘学报, 2021, 50(11): 1487-1499.
[8]	孙喜亮, 关宏灿, 苏艳军, 徐光彩, 郭庆华. 面向高精度城市测绘的激光紧耦合SLAM方法[J]. 测绘学报, 2021, 50(11): 1585-1593.
[9]	李彩林, 王志勇, 俞路路, 郭宝云. 最邻近曲面约束的近景光学影像与地面激光点云几何配准[J]. 测绘学报, 2020, 49(8): 1014-1022.
[10]	李永强, 李鹏鹏, 董亚涵, 范辉龙. 车载LiDAR点云数据中杆状地物自动提取与分类[J]. 测绘学报, 2020, 49(6): 724-735.
[11]	王密, 郭贝贝, 龙小祥, 薛麟, 程宇峰, 金淑英, 周晓. 高分六号宽幅相机在轨几何定标及精度验证[J]. 测绘学报, 2020, 49(2): 171-180.
[12]	方莉娜, 卢丽靖, 赵志远, 王昀昀, 陈崇成. 车载激光点云道路边界提取的Snake方法[J]. 测绘学报, 2020, 49(11): 1438-1450.
[13]	张瑞辰, 边少锋, 刘雁春, 李厚朴. 大面积水深异常检测的条件变化自编码算法[J]. 测绘学报, 2019, 48(9): 1182-1189.
[14]	方莉娜, 黄志文, 罗海峰, 陈崇成. 车载激光扫描数据中实线型交通标线提取[J]. 测绘学报, 2019, 48(8): 960-974.
[15]	余岸竹, 刘冰, 邢志鹏, 杨帆, 杨其淼. 面向高光谱影像分类的显著性特征提取方法[J]. 测绘学报, 2019, 48(8): 985-995.