A incremental structure from motion method of robustness enhancement and accuracy improvement

doi:10.11947/j.AGCS.2019.20170665

Abstract

Abstract:

The Incremental structure from motion(ISFM) method realizes 3D reconstruction based on unordered images and plays an important role in fields such as fine modeling, 3D recording of realistic scenes and 3D reconstruction of Internet images. When facing with the complexity of the scene structure, the ISFM method has some problems, such as poor robustness and low accuracy, which often leads to unsatisfactory 3D reconstruction results and even failure of 3D reconstruction, which severely limits its development and application. An incremental structure from motion method of robustness enhancement and accuracy improvement is constucted. The main improvements are:① Facing the problem that there are many error points in stereo image feature matching, a parameter adaptive RANSAC method which takes into account the feature response value is proposed, which can effectively remove the mismatch and keep the correct matching point to the maximum extent. ② A strategy of adding the next image to avoid robustness and reconstruction accuracy is proposed, which makes the process of reconstruction more reasonable. ③ The elimination of outer points into the adjustment process is introduced, which significantly improves the robustness and accuracy of the adjustment. Finally, by using UAV low-altitude image data, close-range shooting data and Internet-downloaded image data respectively, the experimental results show that this method can effectively eliminate the mismatch points, optimize the image reconstruction sequence and weaken the effect of error points on the adjustment results.

Key words: incremental structure from motion, RANSAC, scoring strategy, error elimination

CLC Number:

P231

YU Ying, ZHANG Yongsheng, XUE Wu, WANG Tao. A incremental structure from motion method of robustness enhancement and accuracy improvement[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(2): 207-215.

References

[1] SZELISKI R. Structure from motion[J]. Near Eastern Archaeology, 2017, 77(3):187-191.
[2] DIETRICH J T. Riverscape mapping with helicopter-based structure-from-motion photogrammetry[J]. Geomorphology, 2016, 252(1):144-157.
[3] 李劲澎, 姜挺, 肖达, 等. 基于关系图的无人机影像三维重建[J]. 光学精密工程, 2016, 24(6):1501-1509. LI Jinpeng, JIANG Ting, XIAO Da, et al. On diagram-based three-dimensional reconstruction of UAV image[J]. Optics and Precision Engineering, 2016, 24(6):1501-1509.
[4] SNAVELY N, SEITZ S M, SZELISKI R. Skeletal graphs for efficient structure from motion[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK:IEEE, 2008:1-8.
[5] FARENZENA M, FUSIELLO A, GHERARDI R. Structure-and-motion pipeline on a hierarchical cluster tree[C]//Proceedings of the 12th IEEE International Conference on Computer Vision Workshops, ICCV Workshops. Kyoto, Japan:IEEE, 2009:1489-1496.
[6] KLOPSCHITZ M, IRSCHARA A, REITMAYR G, et al. Robust incremental structure from motion[C]//Proceedings of the 3rd International Symposium on 3D Data Processing, Visualization and Transmission. Toronto:[s.n.], 2010:1-8.
[7] MOULON P, MONASSE P, MARLET R. Adaptive structure from motion with a contrario model estimation[C]//Proceedings of the 11th Asian Conference on Computer Vision. Berlin:Springer, 2012:257-270.
[8] SHAH R, DESHPANDE A, NARAYANAN P J. Multistage SFM:revisiting incremental structure from motion[C]//Proceedings of the 2nd International Conference on 3D Vision. Tokyo, Japan:IEEE, 2014:417-424.
[9] 许志华, 吴立新, 刘军, 等. 顾及影像拓扑的SfM算法改进及其在灾场三维重建中的应用[J]. 武汉大学学报(信息科学版), 2015, 40(5):599-606. XU Zhihua, WU Lixin, LIU Jun, et al. Modification of SfM algorithm referring to image topology and its application in 3-dimension reconstruction of disaster area[J]. Geomatics and Information Science of Wuhan University, 2015, 40(5):599-606.
[10] XU Zhihua, WU Lixin, GERKE M, et al. Skeletal camera network embedded structure-from-motion for 3D scene reconstruction from UAV images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 121(1):113-127.
[11] WU Changchang. Visual SfM:a visual structure from motion system[CP/OL].[2017-05-05]. http://ccwu.me/vsfm/.
[12] FUHRMANN S, LANGGUTH F, GOESELE M. MVE:a multi-view reconstruction environment[C]//Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage. Darmstadt, Germany:ACM, 2014:11-18.
[13] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:4104-4113.
[14] FRAHM J M, FITE-GEORGEL P, GALLUP D, et al. Building rome on a cloudless day[C]//Proceedings of the 11th European Conference on Computer Vision. Heraklion, Crete, Greece:Springer-Verlag, 2010:368-381.
[15] 闫利, 费亮, 叶志云, 等. 大范围倾斜多视影像连接点自动提取的区域网平差法[J]. 测绘学报, 2016, 45(3):310-317, 338. YAN Li, FEI Liang, YE Zhiyun, et al. Automatic tie-points extraction for triangulation of large-scale oblique multi-view images[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3):310-317, 338.
[16] 于英, 张永生, 薛武, 等. 影像连接点均衡化高精度自动提取[J]. 测绘学报, 2017, 46(1):90-97. DOI:10.11947/j.AGCS.2017.20160320. YU Ying, ZHANG Yongsheng, XUE Wu, et al. Automatic tie points extraction with uniform distribution and high precision[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(1):90-97. DOI:10.11947/j.AGCS.2017.20160320.
[17] CHEN Yisong, CHAN A B, LIN Zhouchen, et al. Efficient tree-structured SfM by RANSAC generalized procrustes analysis[J]. Computer Vision and Image Understanding, 2017, 157(2):179-189.
[18] WU Changchang. SiftGPU:A GPU implementation of scale invariant feature transform (SIFT)[EB/OL]. (2012-02-07). https://blog.csdn.net/archielau/article/details/7907387.
[19] LEPETIT V, MORENO-NOGUER F, FUA P. EPnP:An accurate O(n) solution to the pnp problem[J]. International Journal of Computer Vision, 2009, 81(2):155-166.
[20] 于英. 无人机动态摄影测量若干关键技术研究[D]. 郑州:信息工程大学, 2014. YU Ying. Research on key technologies of UAV dynamic photogrammetry[D]. Zhengzhou:Information Engineering University, 2014.
[21] 薛武. 无人机影像定位优化技术研究[D]. 郑州:信息工程大学, 2017. XUE Wu. Research on optimization technology of UAV images positioning[D]. Zhengzhou:Information Engineering University, 2017.
[22] 薛武, 张永生, 赵玲, 等. 增量式SFM与POS辅助光束法平差精度比较[J]. 测绘学报, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274. XUE Wu, ZHANG Yongsheng, ZHAO Ling, et al. Compareison of the accuracy of incremental SFM with POS-aided bundle adjustment[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(2):198-207. DOI:10.11947/j.AGCS.2017.20160274.
[23] 李德仁, 袁修孝. 误差处理与可靠性理论[M]. 2版. 武汉:武汉大学出版社, 2013. LI Deren, YUAN Xiuxiao. Error processing and reliability theory[M]. 2nd ed. Wuhan:Wuhan University Press, 2013.