摄影测量局部场景稳健合并的并行式运动恢复结构方法

doi:10.11947/j.AGCS.2024.20220321

测绘学报 ›› 2024, Vol. 53 ›› Issue (2): 332-343.doi: 10.11947/j.AGCS.2024.20220321

摄影测量局部场景稳健合并的并行式运动恢复结构方法

肖腾¹, 王鑫², 梅熙³, 叶志伟¹, 颜青松², 邓非^2,4

1. 湖北工业大学计算机学院, 湖北武汉 430068;
2. 武汉大学测绘学院, 湖北武汉 430079;
3. 中铁二院工程集团有限责任公司, 四川成都 610083;
4. 武汉天际航信息科技股份有限公司, 湖北武汉 430223

收稿日期:2022-05-10 修回日期:2023-04-11 发布日期:2024-03-08
作者简介:肖腾(1990-),男,博士,讲师,研究方向为摄影测量与实景三维重建。E-mail:xiao@hbut.edu.cn
基金资助:
国家自然科学基金（42301491；42301507）；湖北省重点研发计划项目（2022BAA035）；湖北工业大学科研启动基金（XJ2022002001）

Robust merging of subblock reconstructions for parallel structure from motion in photogrammetry

XIAO Teng¹, WANG Xin², MEI Xi³, YE Zhiwei¹, YAN Qingsong², DENG Fei^2,4

1. School of Computer Science, Hubei University of Technology, Wuhan 430068, China;
2. School of Geodesy and Geomatics, Wuhan University, Wuhan 430079, China;
3. China Railway Eryuan Engineering Group Co., Ltd., Chengdu 610083, China;
4. Wuhan Tianjihang Information Technology Co., Ltd., Wuhan 430223, China

Received:2022-05-10 Revised:2023-04-11 Published:2024-03-08
Supported by:
The National Natural Science Foundation of (Nos. 42301491; 42301507); The Hubei Key Research and Development Project (No. 2022BAA035); Research Startup Fund of Hubei University of Technology (No. XJ2022002001)

摘要/Abstract

摘要： 针对并行式运动恢复结构（SfM）在局部场景合并时稳健性差的问题，提出一种摄影测量局部场景稳健合并的并行式SfM方法。对整个场景的影像关联图进行分块及扩展处理，得到相互重叠的子区块，并利用一种改进的增量式SfM方法生成局部场景重建结果。在局部场景合并时，首先利用局部场景的重叠关系构建子区块关联图，并以子区块三元组为单元，进行粗差剔除；然后，利用子区块三元组的代数性质，优化得到更符合几何一致性的子区块间的相对变换；最后，从上述结果中计算得到更准确的局部场景到统一坐标系下的尺度、旋转、平移变换。试验采用无人机影像，结果表明本文方法在局部场景合并时有更好的稳健性，而且SfM结果的精确度也要优于其他并行式方法和COLMAP，在摄影测量和实景三维重建中有较大的应用潜力。

关键词: 摄影测量, 实景三维, 无人机影像, 并行式运动恢复结构

Abstract: In this paper, we proposed an improved parallel structure from motion pipeline in photogrammetry by robustifying the merging of subblock reconstructions in a better fashion. Specifically, the whole block represented by a view graph is divided into a number of overlapped subblocks via graph partition and expansion, and an improved incremental SfM is employed to generate the SfM reconstruction of each subblock. To merge these subblock SfM reconstructions in a more robust manner, a subblock graph indicating the overlapping relationship of subblock reconstructions is first built. By considering the geometry consistencies of subblock triplets, gross errors are detected. Then, we leverage the algebraic properties of subblock triplets, which aims to make them more geometrically consistent, to refine the relative transformations between subblock reconstructions. Finally, more accurate relative transformations between subblock reconstructions can be obtained to boost the subsequential merging. Experimental results using UAV images show that the proposed method can guarantee robustness in the subblock reconstruction merging stage. The precision of our SfM results is better than several state-of-the-art parallel SfM methods and the popular COLMAP. Furthermore, it has significant potential for use in photogrammetry and 3D Real Scene reconstruction.

Key words: photogrammetry, 3D real scene, UAV images, parallel structure from motion

中图分类号:

P227

肖腾, 王鑫, 梅熙, 叶志伟, 颜青松, 邓非. 摄影测量局部场景稳健合并的并行式运动恢复结构方法[J]. 测绘学报, 2024, 53(2): 332-343.

XIAO Teng, WANG Xin, MEI Xi, YE Zhiwei, YAN Qingsong, DENG Fei. Robust merging of subblock reconstructions for parallel structure from motion in photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(2): 332-343.

参考文献

[1] SNAVELY N, SEITZ S M, SZELISKI R. Photo tourism:exploring photo collections in 3D[J]. ACM Transactions on Graphics, 25(3):835-846.
[2] AGARWAL S, FURUKAWA Y, SNAVELY N, et al. Building Rome in a day[J]. Communications of the ACM, 2011, 54(10):105-112.
[3] WU Changchang. Towards linear-time incremental structure from motion[C]//Proceedings of 2013 International Conference on 3D Vision. Seattle:IEEE, 2013:127-134.
[4] SCHONBERGER J L, FRAHM J M. Structure-from-motion revisited[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:4104-4113.
[5] WANG X, XIAO T, KASTEN Y.A hybrid global structure from motion method for synchronously estimating global rotations and global translations[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 174:35-55.
[6] XIAO Teng, YAN Qingsong, MA Weile, et al. Progressive structure from motion by iteratively prioritizing and refining match pairs[J]. Remote Sensing, 2021, 13(12):2340.
[7] JIANG S, JIANG C, JIANG W. Efficient structure from motion for large-scale UAV images:a review and a comparison of SfM tools[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 167:230-251.
[8] 许彪, 董友强, 张力, 等. 分区优化混合SfM方法[J]. 测绘学报, 2022, 51(1):115-126.DOI:10.11947/j.AGCS.2021.20210105. XU Biao, DONG Youqiang, ZHANG Li, et al. A hybrid SfM method based on partition optimization[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(1):115-126.DOI:10.11947/j.AGCS.2021.20210105.
[9] 李清泉, 邵成立, 万剑华, 等. 优视摄影测量与泛在实景三维数据采集:以实景三维青岛为例[J]. 武汉大学学报(信息科学版), 2022, 47(10):1587-1597. LI Qingquan, SHAO Chengli, WAN Jianhua, et al. Optimized views photogrammetry and ubiquitous real 3D data acquisition with the application case in Qingdao[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10):1587-1597.
[10] LYU Jingguo, YANG Xingbin, ZHANG Danlu, et al. High-resolution remote sensing image semi-global matching method considering geometric constraints of connection points and image texture information [J]. Journal of Geodesy and Geoinformation Science, 2021,4(4):97-112.
[11] BHOWMICK B, PATRA S, CHATTERJEE A, et al.Divide and conquer:a hierarchical approach to large-scale structure-from-motion[J]. Computer Vision and Image Understanding, 2017, 157:190-205.
[12] CHEN Shu, LIANG Luming, OUYANG Jianquan. Accurate structure from motion using consistent cluster merging[J].Multimedia Tools and Applications, 2022, 81(17):24913-24935.
[13] CHEN Yu, SHEN Shuhan, CHEN Yisong, et al. Graph-based parallel large scale structure from motion[J]. Pattern Recognition, 2020, 107:107537.
[14] CUI Hainan, SHEN Shuhan, GAO Xiang, et al. CSFM:Community-based structure from motion[C]//Proceedings of 2017 IEEE International Conference on Image Processing.Beijing:IEEE, 2017:4517-4521.
[15] FANG Meiling, POLLOK T, QU Chengchao. Merge-SfM:merging partial reconstructions[C]//Proceedings of the 30th British Machine Vision Conference. Cardiff:British Machine Vision Association,2019:29.
[16] XIE Xiuchuan, YANG Tao, LI Dongdong, et al. Hierarchical clustering-aligning framework based fast large-scale 3D reconstruction using aerial imagery[J]. Remote Sensing, 2019, 11(3):315.
[17] ZHU S. Accurate, scalable and parallel structure from motion[D].Hong Kong:Hong Kong University of Science and Technology, 2017.
[18] ZHU Siyu, ZHANG Runze, ZHOU Lei, et al. Very large-scale global SfM by distributed motion averaging[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE, 2018:4568-4577.
[19] FARENZENA M, FUSIELLO A, GHERARDI R. Structure-and-motion pipeline on a hierarchical cluster tree[C]//Proceedings of 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops. Kyoto:IEEE, 2009:1489-1496.
[20] GHERARDI R, FARENZENA M, FUSIELLO A. Improving the efficiency of hierarchical structure-and-motion[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE, 2010:1594-1600.
[21] TOLDO R, GHERARDI R, FARENZENA M, et al. Hierarchical structure-and-motion recovery from uncalibrated images[J]. Computer Vision and Image Understanding, 2015, 140:127-143.
[22] CUI Zhaopeng, JIANG Nianjuan, TANG Chengzhou, et al. Linear global translation estimation with feature tracks[C]//Proceedings of 2015 British Machine Vision Conference. Swansea:British Machine Vision Association, 2015.
[23] SHI Jianbo, MALIK J. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8):888-905.
[24] KASTEN Y, GEIFMAN A, GALUN M, et al. Algebraic characterization of essential matrices and their averaging in multiview settings[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul:IEEE, 2019:5895-5903.
[25] MOULON P, MONASSE P, PERROT R, et al. OpenMVG:open multiple view geometry[C].Proceedings of 2016 International Workshop on Reproducible Research in Pattern Recognition. Cham:Springer, 2016:60-74.
[26] BOYD S, PARIKH N, CHU E, et al.Distributed optimization and statistical learning via the alternating direction method of multipliers[J]. Foundations and Trends ® in Machine Learning, 2010, 3(1):1-122.
[27] TRIGGS B, MCLAUCHLAN P F, HARTLEY R I, et al.Bundle adjustment-a modern synthesis[C]//Proceedings of 1999 International workshop on vision algorithms. Berlin:Springer,1999:298-372.
[28] YAO Yao, LUO Zixin, LI Shiwei, et al. BlendedMVS:a large-scale dataset for generalized multi-view stereo networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle:IEEE, 2020:1790-1799.

摄影测量局部场景稳健合并的并行式运动恢复结构方法

Robust merging of subblock reconstructions for parallel structure from motion in photogrammetry

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	姜三, 刘凯, 李清泉, 江万寿. 融合深度特征的无人机影像SfM重建[J]. 测绘学报, 2024, 53(2): 321-331.
[2]	王建荣, 杨元喜, 卢学良, 缪毓喆. 光轴位置测量数据辅助立体影像无控定位技术[J]. 测绘学报, 2024, 53(1): 1-7.
[3]	雷臻, 张帆, 向瀚宇, 杨冲, 黄先锋. 多视角翻转拍摄物体的完整三维重建[J]. 测绘学报, 2023, 52(8): 1305-1316.
[4]	张荣庭, 张广运, 尹继豪. 复杂城市动态图卷积网络三维场景语义分割法[J]. 测绘学报, 2023, 52(10): 1703-1713.
[5]	王建荣, 杨元喜, 胡燕, 吕源, 杨秀策, 卢学良, 曹彬才. 高分十四号立体测绘卫星无控定位精度初步评估[J]. 测绘学报, 2023, 52(1): 8-14.
[6]	张祖勋, 姜慧伟, 庞世燕, 胡翔云. 多时相遥感影像的变化检测研究现状与展望[J]. 测绘学报, 2022, 51(7): 1091-1107.
[7]	许强, 朱星, 李为乐, 董秀军, 戴可人, 蒋亚楠, 陆会燕, 郭晨. “天-空-地”协同滑坡监测技术进展[J]. 测绘学报, 2022, 51(7): 1416-1436.
[8]	张力, 刘玉轩, 孙洋杰, 蓝朝桢, 艾海滨, 樊仲藜. 数字航空摄影三维重建理论与技术发展综述[J]. 测绘学报, 2022, 51(7): 1437-1457.
[9]	王任享, 王建荣. 我国卫星摄影测量发展及其进步[J]. 测绘学报, 2022, 51(6): 804-810.
[10]	张祖勋, 郑顺义, 王晓南. 工业摄影测量技术发展与应用[J]. 测绘学报, 2022, 51(6): 843-853.
[11]	李清泉, 黄惠, 姜三, 胡庆武, 于文率. 优视摄影测量方法及精度分析[J]. 测绘学报, 2022, 51(6): 996-1007.
[12]	朱庆, 张利国, 丁雨淋, 胡翰, 葛旭明, 刘铭崴, 王玮. 从实景三维建模到数字孪生建模[J]. 测绘学报, 2022, 51(6): 1040-1049.
[13]	唐晓芳, 詹总谦, 丁久婕, 刘佳辉, 熊子柔. 顾及超像素光谱特征的无人机影像自动模糊聚类分割法[J]. 测绘学报, 2022, 51(5): 677-690.
[14]	张春森, 葛英伟, 郭丙轩, 张月莹. 无人机影像DSM自动生成随机传播COLVLL算法[J]. 测绘学报, 2022, 51(11): 2346-2354.
[15]	许文涛, 李晓昭, 章杨松, 朱鸿鹄, 张巍, 宣程强. 基于摄影测量系统的岩体结构面精细识别表征及应用[J]. 测绘学报, 2022, 51(10): 2093-2106.