A GPU-PatchMatch multi-view dense matching algorithm based on parallel propagation

doi:10.11947/j.AGCS.2020.20180459

Abstract

Abstract: Aiming at the problem of low efficiency of multi-view dense matching, a GPU-PatchMatch multi-view dense matching algorithm is proposed. The algorithm uses GPU to improve the computational efficiency of PatchMatch. At the same time, it also makes full use of sparse scene information to initialize the depth information. In addition, in order to improve the propagation efficiency, it uses the pyramid red-blackboard to propagate the depth information in parallel. Finally, the experiments are carried out on the DTU, Strecha and Vaihigen datasets, and compared with the commonly used multi-view dense matching algorithms. The results show that our algorithm has a significant improvement in reconstruction efficiency, which is 7 times higher than the CPU algorithm (such as PMVS, MVE and OpenMVS), and 2.5 times higher than the GPU algorithm (such as Gipuma), which proves the effectiveness of the proposed method.

Key words: 3D reconstruction, multi-view dense matching, PatchMatch, parallel computation

CLC Number:

P237

DENG Fei, YAN Qingsong, XIAO Teng. A GPU-PatchMatch multi-view dense matching algorithm based on parallel propagation[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2): 181-190.

References

[1] SEITZ S M, CURLESS B, DIEBEL J, et al. A comparison and evaluation of multi-view stereo reconstruction algorithms[C]//Proceedings of 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. New York:IEEE, 2006:519-528.
[2] REMONDINO F, SPERA M G, NOCERINO E, et al. State of the art in high density image matching[J]. Photogrammetric Record, 2014, 29(146):144-166.
[3] FURUKAWA Y, PONCE J. Accurate, dense, and robust multiview stereopsis[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(8):1362-1376.
[4] 赵红蕊, 陆胜寒. 基于特征尺度分布与对极几何约束的高清影像快速密集匹配方法[J]. 测绘学报, 2018, 47(6):790-798. DOI:10.11947/j.AGCS.2018.20170630. ZHAO Hongrui, LU Shenghan. Dense high-definition image matching strategy based on scale distribution of feature and geometric constrain[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):790-798. DOI:10.11947/j.AGCS.2018.20170630.
[5] 王竞雪, 朱庆, 王伟玺. 多匹配基元集成的多视影像密集匹配方法[J]. 测绘学报, 2013, 42(5):691-698. WANG Jingxue, ZHU Qing, WANG Weixi. A dense matching algorithm of multi-view image based on the integrated multiple matching primitives[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(5):691-698.
[6] GOESELE M, SNAVELY N, CURLESS B, et al. Multi-view stereo for community photo collections[C]//Proceedings of 2007 IEEE 11th International Conference on Computer Vision. Rio de Janeiro:IEEE, 2007:1-8.
[7] ROTHERMEL M, WENZEL K, FRITSCH D, et al. SURE:photogrammetric surface reconstruction from imagery[C]//Proceedings of LC3D Workshop. Berlin:ISPRS, 2012:1-9.
[8] SHEN Shuhan. Accurate multiple view 3D reconstruction using patch-based stereo for large-scale scenes[J]. IEEE Transactions on Image Processing, 2013, 22(5):1901-1914.
[9] GALLIANI S, LASINGER K, SCHINDLER K. Massively parallel multiview stereopsis by surface normal diffusion[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago:IEEE, 2015:873-881.
[10] 闫利, 费亮, 陈长海, 等. 利用网络图进行高分辨率航空多视影像密集匹配[J]. 测绘学报, 2016, 45(10):1171-1181. DOI:10.11947/j.AGCS.2016.20160068. YAN Li, FEI Liang, CHEN Changhai, et al. A multi-view dense matching algorithm of high resolution aerial images based on graph network[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(10):1171-1181. DOI:10.11947/j.AGCS.2016.20160068.
[11] 吴军, 姚泽鑫, 程门门. 融合SIFT与SGM的倾斜航空影像密集匹配[J]. 遥感学报, 2015, 19(3):431-442. WU Jun, YAO Zexin, CHENG Menmen. Airborne oblique stereo image dense matching by integrating SIFT and SGM algorithm[J]. Journal of Remote Sensing, 2015, 19(3):431-442.
[12] TOLDO R, FANTINI F, GIONA L, et al. Accurate multiview stereo reconstruction with fast visibility integration and tight disparity bounding[C]//Proceedings of 3D Virtual Reconstruction and Visualization of Complex Architectures. Trento:ISPRS, 2013:243-249.
[13] KAR A, HÄNE C, MALIK J. Learning a multi-view stereo machine[C]//Proceedings of the 30st Conference on Neural Information Processing Systems. Long Beach:Neural Information Processing Systems Foundation, 2017:365-376.
[14] YAO Yao, LUO Zixin, LI Shiwei, et al. MVSNet:depth inference for unstructured multi-view stereo[C]//Proceedings of the 15th European Conference on Computer Vision. Munich:Springer, 2018:785-801.
[15] JI Mengqi, GALL J, ZHENG Haitian, et al. SurfaceNet:an end-to-end 3D neural network for multiview stereopsis[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice:IEEE, 2017:2326-2334.
[16] BARNES C, SHECHTMAN E, FINKELSTEIN A, et al. PatchMatch:a randomized correspondence algorithm for structural image editing[J]. ACM Transactions on Graphics (TOG), 2009, 28(3):24.
[17] BLEYER M, RHEMANN C, ROTHER C. PatchMatch stereo-stereo matching with slanted support windows[C]//Proceedings of 2011 22nd British Machine Vision Conference. Dundee:BMVA, 2011:1-11.
[18] HIRSCHMULLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):328-341.
[19] YOON K J, KWEON I S. Adaptive support-weight approach for correspondence search[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(4):650-656.
[20] TOLA E, LEPETIT V, FUA P DAISY:an efficient dense descriptor applied to wide-baseline stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(5):815-830.
[21] BOISSONNAT J D, DEVILLERS O, PION S, et al. Triangulations in CGAL[C]//Proceedings of the 16th Annual Symposium on Computational Geometry. Hong Kong, China:ACM, 2000:11-18.
[22] AANAES H, JENSEN R R, VOGIATZIS G, et al. Large-scale data for multiple-view stereopsis[J]. International Journal of Computer Vision, 2016, 120(2):153-168.
[23] STRECHA C, VON HANSEN W, VAN GOOL L, et al. On benchmarking camera calibration and multi-view stereo for high resolution imagery[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage:IEEE, 2008:1-8.
[24] CRAMER M. The DGPF-test on digital airborne camera evaluation-overview and test design[J]. Photogrammetrie-Fernerkundung-Geoinformation, 2010, 2010(2):73-82.
[25] ALTMAN S, XIAO Wen, GRAYSON B. Evaluation of LOW-COST terrestrial Photogrammetry for 3D reconstruction of complex buildings[C]//Proceedings of ISPRS Geospatial Week 2017. Wuhan:Copernicus GmbH, 2017:199-206.