A hybrid SfM method based on partition optimization

doi:10.11947/j.AGCS.2021.20210105

Abstract

Abstract: Aiming at solving the problem of sparse 3D reconstruction of large-scale unordered images, this paper proposes a robust, efficient, and easy-to-parallel hybrid SfM method based on partition optimization. Firstly, the SIFT algorithm is used for image matching, and image partitioning is completed using image correlation scores calculated from the matching results without other auxiliary information such as GPS/INS. Secondly, an improved incremental SfM method is applied to achieve rapid reconstruction in each partition, and a number of standards are introduced to automatically eliminate the unreliable partitions and re-divide the images in these partitions into other partitions to achieve dynamic adjustment. Finally, a robust and high-precision partition fusion algorithm is proposed to realize accurate fusion of camera parameters, image posture, and 3D information of the scene. The experimental results of multiple challenging data sets of different scales, different image types, and different scenes show that our proposed hybrid SfM method has good robustness to different data sets and dramatically improves efficiency while maintaining high precision,which especially suitable for large-scale image sets.

Key words: hybrid SfM, large-scale image sets, image partition, partition fusion

CLC Number:

P231

XU Biao, DONG Youqiang, ZHANG Li, SUN Yushan, LIU Yuxuan, ZHA Bing, HAN Xiaoxia. A hybrid SfM method based on partition optimization[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(1): 115-126.

References

[1] AGARWAL S, SNAVELY N, SIMON I,et al. Building Rome in a day[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision. Kyoto, Japan:IEEE, 2009:72-79. DOI:10.1109/ICCV.2009.5459148.
[2] IRSCHARA A, HOPPE C, BISCHOF H, et al. Efficient structure from motion with weak position and orientation priors[C]//Proceedings of IEEE CVPR 2011 WORKSHOPS. ColoradoSprings, CO:IEEE, 2011:21-28.
[3] JIANG Nianjuan, CUI Zhaopeng, TAN Ping. A global linear method for camera pose registration[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision. Sydney:IEEE, 2013:481-488.
[4] MOULON P, MONASSE P, MARLET R. Adaptive structure from motion with a Contrario model estimation[C]//Proceedings of the 11th Asian Conference on Computer Vision. Daejeon, Korea:Springer, 2012:257-270.
[5] 郭复胜, 高伟. 基于辅助信息的无人机图像批处理三维重建方法[J]. 自动化学报, 2013, 39(6):834-845. GUO Fusheng, GAO Wei. Batch reconstruction from UAV images with prior information[J]. Acta Automatica Sinica, 2013, 39(6):834-845.
[6] WILSON K, SNAVELY N. Robust global translations with 1DSfM[M]. London:Springer International Publishing, 2014:61-75.
[7] SWEENEY C. Theia multiview geometry library:totorial & reference[EB/OL]. (2016-03-26)[2020-11-18]. http://www.theia-sfm.org.
[8] 何海清, 陈敏, 陈婷, 等. 低空影像SfM三维重建的耦合单-多旋转平均迭代优化法[J]. 测绘学报, 2019, 48(6):688-697. DOI:10.11947/j.AGCS.2019.20180063. HE Haiqing, CHEN Min, CHEN Ting, et al. Single and multiple rotation averaging iterative optimization coupled 3D reconstruction for low-altitude images using SfM algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):688-697. DOI:10.11947/j.AGCS.2019.20180063.
[9] SCHÖNBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:4104-4113.
[10] 薛武, 张永生, 赵玲, 等. 增量式SfM与POS辅助光束法平差精度比较[J]. 测绘学报, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274. XUE Wu, ZHANG Yongsheng, ZHAO Ling, et al. Compareison of the accuracy of incremental SfM with POS-aided bundle adjustment[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274.
[11] 李劲澎, 姜挺, 肖达, 等. 基于关系图的无人机影像三维重建[J]. 光学精密工程, 2016, 24(6):1501-1509. LI Jinpeng, JIANG Ting, XIAO Da, et al. On diagram-based three-dimensional reconstruction of UAV image[J]. Optics and Precision Engineering, 2016, 24(6):1501-1509.
[12] 薛武. 无人机影像定位优化技术研究[D]. 郑州:信息工程大学, 2017. XUE Wu. Research on optimization technology of UAV images positioning[D]. Zhengzhou:Information Engineering University, 2017.
[13] CHEN Yisong, CHAN A B, LIN Zhouchen, et al. Efficient tree-structured SfM by RANSAC generalized procrustes analysis[J]. Computer Vision and Image Understanding, 2017, 157:179-189.
[14] SZELISKI R. Structure from motion[J]. Near Eastern Archaeology, 2017, 77(3):187-191.
[15] KLOPSCHITZ M, IRSCHARA A, REITMAYR G, et al. Robust incremental structure frommotion[C]//Proceedings of the Fifth International Symposium on 3D Data Processing, Visualization and Transmission. Toronto:Gerhard Reitmayr, 2010:1-8.
[16] FRAHM J M, FITE-GEORGEL P, GALLUP D, et al. Building Rome on a cloudless day[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece:Springer, 2010:368-381.
[17] 吴子敬, 邓非. SfM的倾斜航空影像空中三角测量[J]. 测绘科学, 2017, 42(11):97-101. WU Zijing, DENG Fei. Aerial triangulation of oblique aerial images based on SfM[J]. Science of Surveying and Mapping, 2017, 42(11):97-101.
[18] 卢俊. 基于无序多视影像的三维重建关键技术研究[D]. 郑州:信息工程大学, 2015. LU Jun. Study on 3D reconstruction key technology of unordered multi-view images[D]. Zhengzhou:Information Engineering University, 2015.
[19] 于英, 张永生, 薛武, 等. 一种稳健性增强和精度提升的增量式运动恢复结构方法[J]. 测绘学报, 2019, 48(2):207-215. DOI:10.11947/j.AGCS.2019.20170665. YU Ying, ZHANG Yongsheng, XUE Wu, etal. A incremental structure from motion methodof robustness enhancement and accuracy improvement[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(2):207-215. DOI:10.11947/j.AGCS.2019.20170665.
[20] 许志华, 吴立新, 刘军, 等. 顾及影像拓扑的SfM算法改进及其在灾场三维重建中的应用[J]. 武汉大学学报(信息科学版), 2015, 40(5):599-606. XU Zhihua, WU Lixin, LIU Jun, etal. Modification of SfM algorithm referring to image topology and its application in 3-dimension reconstruction of disaster area[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5):599-606.
[21] 姜三, 许志海, 张峰, 等. 面向无人机倾斜影像的高效SfM重建方案[J]. 武汉大学学报(信息科学版), 2019, 44(8):1153-1161. JIANG San, XU Zhihai, ZHANG Feng, et al. Solution for efficient SfM reconstruction of oblique UAV images[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8):1153-1161.
[22] WU Changchang. Visual SfM:a visual structure from motion system[EB/OL].(2011-06-30)[2021-02-07].http://ccwu.me/vsfm/.
[23] FARENZENA M, FUSIELLO A, GHERARDI R. Structure-and-motion pipeline on ahierarchical cluster tree[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops (ICCV Workshops). Kyoto, Japan:IEEE,2009:1489-1496.
[24] GHERARDI R, FARENZENA M, FUSIELLO A. Improving the efficiency of hierarchical structure-and-motion[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco, CA:IEEE, 2010:1594-1600.
[25] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[26] 王豪, 张力, 艾海滨, 等. 三维重建中的大规模航空影像检索方法[J]. 测绘科学, 2019, 44(2):136-144. WANG Hao, ZHANG Li, AI Haibin, et al. Large scale aerial image retrieval method in 3D reconstruction[J]. Science of Surveying and Mapping, 2019, 44(2):136-144.
[27] SCHÖNBERGER J L, PRICE T, SATTLER T, et al. A vote-and-verify strategy for fast spatial verification in image retrieval[C]//Asian Conference on Computer Vision.Taipei, China:Springer, 2016:321-337.