Variational refinement of mesh with line constraint for photogrammetry

doi:10.11947/j.AGCS.2020.20190255

Abstract

Abstract: In order to solve the problem of that urban 3D reconstruction is too smooth in the edge area with line features, it is proposed a variational refinement of mesh with line constraint. The algorithm takes the initial reconstruction mesh as input and introduces three energy terms. Transforming the refinement problem into energy reduction problem. Firstly, photo-consistency constraints are constructed by combining all image information, and then regularization constraints are added to all the vertices of the mesh. Finally, 3D line constraint is added to energy term. The gradient difference value is obtained by discretizing sum of three weighted energy term to each vertex. Using gradient descent method to make each vertex move along the gradient vector. When the energy no longer decreases or reaches enough iterations, the refined mesh model is obtained. Calculating the coordinate corrections of each vertex by variational refinement iteration method. Thus, vertices naturally migrate to the object edges if any. The results show that the proposed algorithm can preserve the edge features better. Compared with the existing Poisson surface reconstruction algorithm, the quality of mesh is higher and the visual effect is better.

Key words: 3D reconstruction, photo consistency, regularization, line constraint, variational refinement

CLC Number:

P237

DENG Fei, CHEN Xin, YAN Qingsong, QU Yingjie. Variational refinement of mesh with line constraint for photogrammetry[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4): 469-479.

References

[1] MOULON P, MONASSE P, MARLET R. Adaptive structure from motion with a contrario model estimation[C]//Proceedings of the 11th Asian Conference of Computer Vision.Daejeon:Springer, 2012:257-270.
[2] WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! large-scale texturing of 3D reconstructions[C]//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014:836-850.
[3] 欧阳欢, 范大昭, 纪松, 等. 结合离散化描述与同名点约束的线特征匹配[J]. 测绘学报, 2018, 47(10):1363-1371. DOI:10.11947/j.AGCS.2018.20170231. OUYANG Huan, FAN Dazhao, JI Song, et al. Line matching based on discrete description and conjugate point constraint[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(10):1363-1371. DOI:10.11947/j.AGCS.2018.20170231.
[4] 李明, 张卫龙, 范丁元. 城市三维重建中的自动纹理优化方法[J]. 测绘学报, 2017, 46(3):338-345.DOI:10.11947/j.AGCS.2017.20160467. LI Ming, ZHANG Weilong, FAN Dingyuan. Automatic texture optimization for 3D urban reconstruction[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(3):338-345. DOI:10.11947/j.AGCS.2017.20160467.
[5] 姜翰青, 王博胜, 章国锋, 等. 面向复杂三维场景的高质量纹理映射[J]. 计算机学报, 2015, 38(12):2349-2360. JIANG Hanqing, WANG Bosheng, ZHANG Guofeng et al. High-quality texture mapping for complex 3D scenes[J]. Chinese Journal of Computers, 2015, 38(12):2349-2360.
[6] KAZHDAN M, BOLITHO M, HOPPE H. Poisson surface reconstruction[C]//Proceedings of the 4th Eurographics Symposium on Geometry Processing.Aire-la-Ville:ACM, 2006:61-70.
[7] KAZHDAN M, HOPPE H. Screened Poisson surface reconstruction[J]. ACM Transactions on Graphics, 2013, 32(3):61-70.
[8] GUARDA A F R, BIOUCAS-DIAS J M, RODRIGUES N M M, et al. Improving point cloud to surface reconstruction with generalized Tikhonov regularization[C]//Proceedings of the 19th International Workshop on Multimedia Signal Processing. Luton:IEEE, 2017:1-6.
[9] ESTELLERS V, SCOTT M, TEW K, et al. Robust Poisson surface reconstruction[C]//Proceedings of the 5th International Conference on Scale Space and Variational Methods in Computer Vision.Lège-Cap Ferret, France:Springer, 2015:525-537.
[10] DELAUNOY A, PRADOS E. Gradient flows for optimizing triangular mesh-based surfaces:applications to 3D reconstruction problems dealing with visibility[J]. International Journal of Computer Vision, 2011, 95(2):100-123.
[11] DELAUNOY A, POLLEFEYS M. Photometric bundle adjustment for dense multi-view 3D modeling[C]//Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014:1486-1493.
[12] VU H H, LABATUT P, PONS J P, et al. High accuracy and visibility-consistent dense multiview stereo[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(5):889-901.
[13] HEISE P, JENSEN B, KLOSE S, et al. Variational Patch-Match multiview reconstruction and refinement[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago:IEEE, 2015:882-890.
[14] LI S, SIU S Y, FANG T, et al. Efficient multi-view surface refinement with adaptive resolution control[C]//Proceedings of the 14th European Conference on Computer Vision. Amsterdam:Springer, 2016:349-364.
[15] 张春森, 张萌萌, 郭丙轩. 影像信息驱动的三角网格模型优化方法[J]. 测绘学报, 2018, 47(7):959-967. DOI:10.11947/j.AGCS.2018.20170733. ZHANG Chunsen, ZHANG Mengmeng, GUO Bingxuan. Refinement of the 3D mesh model driven by the image information[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(7):959-967. DOI:10.11947/j.AGCS.2018.20170733.
[16] MORREALE L, ROMANONI A, MATTEUCCI M. Predicting the next best view for 3D mesh refinement[C]//Proceedings of the 15th International Conference Intelligent Autonomous Systems. Berlin:Springer-Verlag, 2019:760-772.
[17] WANG C C L. Incremental reconstruction of sharp edges on mesh surfaces[J]. Computer Aided Design, 2006, 38(6):689-702.
[18] BLÁHA M, ROTHERMEL M, OSWALD M R, et al. Semantically informed multiview surface refinement[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:3839-3847.
[19] BÓDIS-SZOMORU' A, RIEMENSCHNEIDER H, VAN GOOL L. Superpixel meshes for fast edge-preserving surface reconstruction[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition. Boston:IEEE, 2015:2011-2020.
[20] LI Shiwei, YAO Yao, FANG Tian, et al. Reconstructing thin structures of manifold surfaces by integrating spatial curves[C]//Proceedings of IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:2887-2896.
[21] PONS J P, KERIVEN R, FAUGERAS O. Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score[J]. International Journal of Computer Vision, 2007, 72(2):179-193.
[22] KOBBELT L, CAMPAGNA S, VORSATZ J, et al. Interactive multi-resolution modeling on arbitrary meshes[C]//Proceedings of the 25th Annual Conference on Computer Graphics and Interactive Techniques. New York:ACM, 1998:105-114.
[23] HOFER M, MAURER M, BISCHOF H. Line3D:efficient 3D scene abstraction for the built environment[C]//Proceedings of German Conference on Pattern Recognition. Aachen:Springer-Verlag, 2015:237-248.
[24] HOFER M, MAURER M, BISCHOF H. Efficient 3D scene abstraction using line segments[J]. Computer Vision and Image Understanding, 2017, 157(4):167-178.
[25] STRECHA C, VON HANSEN W, VAN GOOL L, et al. On benchmarking camera calibration and multi-view stereo for high resolution imagery[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage:IEEE, 2008:1-8.