分区优化混合SfM方法

doi:10.11947/j.AGCS.2021.20210105

测绘学报 ›› 2022, Vol. 51 ›› Issue (1): 115-126.doi: 10.11947/j.AGCS.2021.20210105

分区优化混合SfM方法

许彪¹, 董友强^2,3, 张力¹, 孙钰珊¹, 刘玉轩¹, 查冰⁴, 韩晓霞¹

1. 中国测绘科学研究院, 北京 100830;
2. 北京建筑大学测绘与城市空间信息学院, 北京 100044;
3. 北京市建筑遗产精细重构与健康监测重点实验室, 北京 100044;
4. 俄亥俄州立大学工程学院土木环境和大地测量系, 美国哥伦布 43210

收稿日期:2021-02-28 修回日期:2021-05-24 发布日期:2022-02-15
通讯作者: 董友强 E-mail:dongyouqiang@bucea.edu.cn
作者简介:许彪(1986-),男,博士,助理研究员,研究方向为数字摄影测量与遥感。E-mail:biaoxv@casm.ac.cn
基金资助:
国家重点研发计划（2017YFB0503000）；北京市教委科技计划面上项目（KM202110016005）

A hybrid SfM method based on partition optimization

XU Biao¹, DONG Youqiang^2,3, ZHANG Li¹, SUN Yushan¹, LIU Yuxuan¹, ZHA Bing⁴, HAN Xiaoxia¹

1. Chinese Academy of Surveying & Mapping, Beijing 100830, China;
2. Beijing University of Civil Engineering and Architecture, Beijing 100044, China;
3. Beijing Key Laboratory for Architectural Heritage Fine Reconstruction & Health Monitoring, Beijing 100044, China;
4. Environmental and Geodetic Engineering, Department of Civil, Ohio State University, Columbus, OH 43210, USA

Received:2021-02-28 Revised:2021-05-24 Published:2022-02-15
Supported by:
The National Key Research and Development Program of China(No. 2017YFB0503000);The Scientific Research Project of Beijing Educational Committee (No. KM202110016005)

摘要/Abstract

摘要： 针对大规模无序影像稀疏三维重建问题，本文提出一种稳健、高效且易于并行的分区优化的混合式SfM方法。首先，利用SIFT算法进行影像匹配，无须GPS/INS等其他辅助信息，仅利用影像间的匹配结果计算得到的影像关联度完成影像分区。然后，提出一种改进的增量式SfM方法实现每个分区内快速重建，以及提出多项标准自动剔除不可靠分区并将这些分区内影像重新划分至其他分区，实现分区的动态调整。最后，提出一种稳健高精度的分区融合算法，实现相机参数、影像姿态和场景三维信息的准确融合。多组不同规模、不同影像类型以及不同场景的典型数据试验结果表明本文方法对不同数据集具有很好的稳健性，在保持高精度的同时能大大提高重建效率，尤其适用于大规模影像数据集。

关键词: 混合式SfM, 大规模影像, 影像分区, 分区融合

Abstract: Aiming at solving the problem of sparse 3D reconstruction of large-scale unordered images, this paper proposes a robust, efficient, and easy-to-parallel hybrid SfM method based on partition optimization. Firstly, the SIFT algorithm is used for image matching, and image partitioning is completed using image correlation scores calculated from the matching results without other auxiliary information such as GPS/INS. Secondly, an improved incremental SfM method is applied to achieve rapid reconstruction in each partition, and a number of standards are introduced to automatically eliminate the unreliable partitions and re-divide the images in these partitions into other partitions to achieve dynamic adjustment. Finally, a robust and high-precision partition fusion algorithm is proposed to realize accurate fusion of camera parameters, image posture, and 3D information of the scene. The experimental results of multiple challenging data sets of different scales, different image types, and different scenes show that our proposed hybrid SfM method has good robustness to different data sets and dramatically improves efficiency while maintaining high precision,which especially suitable for large-scale image sets.

Key words: hybrid SfM, large-scale image sets, image partition, partition fusion

中图分类号:

P231

许彪, 董友强, 张力, 孙钰珊, 刘玉轩, 查冰, 韩晓霞. 分区优化混合SfM方法[J]. 测绘学报, 2022, 51(1): 115-126.

XU Biao, DONG Youqiang, ZHANG Li, SUN Yushan, LIU Yuxuan, ZHA Bing, HAN Xiaoxia. A hybrid SfM method based on partition optimization[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(1): 115-126.

参考文献

[1] AGARWAL S, SNAVELY N, SIMON I,et al. Building Rome in a day[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:72-79. DOI:10.1109/ICCV.2009.5459148.
[2] IRSCHARA A, HOPPE C, BISCHOF H, et al. Efficient structure from motion with weak position and orientation priors[C]//Proceedings of IEEE CVPR 2011 WORKSHOPS. ColoradoSprings, CO:IEEE, 2011:21-28.
[3] JIANG Nianjuan, CUI Zhaopeng, TAN Ping. A global linear method for camera pose registration[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney:IEEE, 2013:481-488.
[4] MOULON P, MONASSE P, MARLET R. Adaptive structure from motion with a Contrario model estimation[C]//Proceedings of the 11th Asian Conference on Computer Vision. Daejeon, Korea:Springer, 2012:257-270.
[5] 郭复胜, 高伟. 基于辅助信息的无人机图像批处理三维重建方法[J]. 自动化学报, 2013, 39(6):834-845. GUO Fusheng, GAO Wei. Batch reconstruction from UAV images with prior information[J]. Acta Automatica Sinica, 2013, 39(6):834-845.
[6] WILSON K, SNAVELY N. Robust global translations with 1DSfM[M]. London:Springer International Publishing, 2014:61-75.
[7] SWEENEY C. Theia multiview geometry library:totorial & reference[EB/OL]. (2016-03-26)[2020-11-18]. http://www.theia-sfm.org.
[8] 何海清, 陈敏, 陈婷, 等. 低空影像SfM三维重建的耦合单-多旋转平均迭代优化法[J]. 测绘学报, 2019, 48(6):688-697. DOI:10.11947/j.AGCS.2019.20180063. HE Haiqing, CHEN Min, CHEN Ting, et al. Single and multiple rotation averaging iterative optimization coupled 3D reconstruction for low-altitude images using SfM algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):688-697. DOI:10.11947/j.AGCS.2019.20180063.
[9] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:4104-4113.
[10] 薛武, 张永生, 赵玲, 等. 增量式SfM与POS辅助光束法平差精度比较[J]. 测绘学报, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274. XUE Wu, ZHANG Yongsheng, ZHAO Ling, et al. Compareison of the accuracy of incremental SfM with POS-aided bundle adjustment[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274.
[11] 李劲澎, 姜挺, 肖达, 等. 基于关系图的无人机影像三维重建[J]. 光学精密工程, 2016, 24(6):1501-1509. LI Jinpeng, JIANG Ting, XIAO Da, et al. On diagram-based three-dimensional reconstruction of UAV image[J]. Optics and Precision Engineering, 2016, 24(6):1501-1509.
[12] 薛武. 无人机影像定位优化技术研究[D]. 郑州:信息工程大学, 2017. XUE Wu. Research on optimization technology of UAV images positioning[D]. Zhengzhou:Information Engineering University, 2017.
[13] CHEN Yisong, CHAN A B, LIN Zhouchen, et al. Efficient tree-structured SfM by RANSAC generalized procrustes analysis[J]. Computer Vision and Image Understanding, 2017, 157:179-189.
[14] SZELISKI R. Structure from motion[J]. Near Eastern Archaeology, 2017, 77(3):187-191.
[15] KLOPSCHITZ M, IRSCHARA A, REITMAYR G, et al. Robust incremental structure frommotion[C]//Proceedings of the Fifth International Symposium on 3D Data Processing, Visualization and Transmission. Toronto:Gerhard Reitmayr, 2010:1-8.
[16] FRAHM J M, FITE-GEORGEL P, GALLUP D, et al. Building Rome on a cloudless day[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece:Springer, 2010:368-381.
[17] 吴子敬, 邓非. SfM的倾斜航空影像空中三角测量[J]. 测绘科学, 2017, 42(11):97-101. WU Zijing, DENG Fei. Aerial triangulation of oblique aerial images based on SfM[J]. Science of Surveying and Mapping, 2017, 42(11):97-101.
[18] 卢俊. 基于无序多视影像的三维重建关键技术研究[D]. 郑州:信息工程大学, 2015. LU Jun. Study on 3D reconstruction key technology of unordered multi-view images[D]. Zhengzhou:Information Engineering University, 2015.
[19] 于英, 张永生, 薛武, 等. 一种稳健性增强和精度提升的增量式运动恢复结构方法[J]. 测绘学报, 2019, 48(2):207-215. DOI:10.11947/j.AGCS.2019.20170665. YU Ying, ZHANG Yongsheng, XUE Wu, etal. A incremental structure from motion methodof robustness enhancement and accuracy improvement[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(2):207-215. DOI:10.11947/j.AGCS.2019.20170665.
[20] 许志华, 吴立新, 刘军, 等. 顾及影像拓扑的SfM算法改进及其在灾场三维重建中的应用[J]. 武汉大学学报(信息科学版), 2015, 40(5):599-606. XU Zhihua, WU Lixin, LIU Jun, etal. Modification of SfM algorithm referring to image topology and its application in 3-dimension reconstruction of disaster area[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5):599-606.
[21] 姜三, 许志海, 张峰, 等. 面向无人机倾斜影像的高效SfM重建方案[J]. 武汉大学学报(信息科学版), 2019, 44(8):1153-1161. JIANG San, XU Zhihai, ZHANG Feng, et al. Solution for efficient SfM reconstruction of oblique UAV images[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8):1153-1161.
[22] WU Changchang. Visual SfM:a visual structure from motion system[EB/OL].(2011-06-30)[2021-02-07].http://ccwu.me/vsfm/.
[23] FARENZENA M, FUSIELLO A, GHERARDI R. Structure-and-motion pipeline on ahierarchical cluster tree[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops). Kyoto, Japan:IEEE,2009:1489-1496.
[24] GHERARDI R, FARENZENA M, FUSIELLO A. Improving the efficiency of hierarchical structure-and-motion[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco, CA:IEEE, 2010:1594-1600.
[25] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[26] 王豪, 张力, 艾海滨, 等. 三维重建中的大规模航空影像检索方法[J]. 测绘科学, 2019, 44(2):136-144. WANG Hao, ZHANG Li, AI Haibin, et al. Large scale aerial image retrieval method in 3D reconstruction[J]. Science of Surveying and Mapping, 2019, 44(2):136-144.
[27] SCHÖNBERGER J L, PRICE T, SATTLER T, et al. A vote-and-verify strategy for fast spatial verification in image retrieval[C]//Asian Conference on Computer Vision.Taipei, China:Springer, 2016:321-337.

分区优化混合SfM方法

A hybrid SfM method based on partition optimization

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	何海清, 陈敏, 陈婷, 李大军, 陈晓勇. 低空影像SfM三维重建的耦合单-多旋转平均迭代优化法[J]. 测绘学报, 2019, 48(6): 688-697.
[2]	张一, 姜挺, 江刚武, 余岸竹, 于英. 特征法视觉SLAM逆深度滤波的三维重建[J]. 测绘学报, 2019, 48(6): 708-717.
[3]	南轲, 齐华, 叶沅鑫. 深度卷积特征表达的多模态遥感影像模板匹配方法[J]. 测绘学报, 2019, 48(6): 727-736.
[4]	周凡. 视频影像增强虚拟三维场景的注册与渲染方法研究[J]. 测绘学报, 2019, 48(6): 801-801.
[5]	于英, 张永生, 薛武, 王涛. 一种稳健性增强和精度提升的增量式运动恢复结构方法[J]. 测绘学报, 2019, 48(2): 207-215.
[6]	皮英冬, 谢宝蓉, 杨博, 张昳玲, 李欣, 王密. 利用稀少控制点的线阵推扫式光学卫星在轨几何定标方法[J]. 测绘学报, 2019, 48(2): 216-225.
[7]	李斌, 魏俊博, 马博超, 王璐, 徐明霞. 不规则体体积计算三维激光点云切片法[J]. 测绘学报, 2019, 48(1): 42-52.
[8]	王丹菂, 徐青, 邢帅, 林雨准, 李鹏程. 一种由粗到精的机载激光测深信号检测方法[J]. 测绘学报, 2018, 47(8): 1148-1159.
[9]	靳国旺, 熊新, 张红敏, 徐青, 刘辉, 王新田. 相同侧视方向异轨SAR图像差异化约束连接点提取[J]. 测绘学报, 2018, 47(1): 91-101.
[10]	杨博, 王密, 皮英冬. 仅用虚拟控制点的超大区域无控制区域网平差[J]. 测绘学报, 2017, 46(7): 874-881.
[11]	李忠美, 边少锋, 瞿勇. 多像空间前方交会的抗差总体最小二乘估计[J]. 测绘学报, 2017, 46(5): 593-604.
[12]	张春森, 朱师欢, 臧玉府, 肖雄武, 薛万唱. 顾及曝光延迟的无人机GPS辅助光束法平差方法[J]. 测绘学报, 2017, 46(5): 565-572.
[13]	朱庆, 陈崇泰, 胡翰, 丁雨淋. 顾及纹理特征的航空影像自适应密集匹配方法[J]. 测绘学报, 2017, 46(1): 62-72.
[14]	于英, 张永生, 薛武, 李磊. 影像连接点均衡化高精度自动提取[J]. 测绘学报, 2017, 46(1): 90-97.
[15]	马振玲. 卫星测绘空三定位网理论研究[J]. 测绘学报, 2016, 45(9): 1134-1134.