Multiple view complete 3D model reconstruction of manually turned object

doi:10.11947/j.AGCS.2023.20220439

Abstract

Abstract: In close-range photogrammetry, manually turning objects and taking highly overlapping images from multiple views is significant to complete 3D model reconstruction. However, the background of the images is inconsistent with the object in the process of turning, leading to the failure of bundle adjustment, and the deformation and incompleteness of object models. This paper proposes a complete 3D reconstruction method based on multi-view geometric consistency. We analyze the distribution pattern of rotation unit quaternion to judge whether the object is turned in the image pair and group the images. Based on the grouping results and the geometric consistency of feature point matching, we reconstruct object regions without the interference in the background of the images, improving the precision of aerial triangulation and obtaining complete mesh models. Experiments show that our method could improve the accuracy and completeness of 3D models effectively when the object has been overturned.

Key words: close-range photogrammetry, manually turned object, complete 3D model, image grouping, rotation unit quaternion

CLC Number:

P227

LEI Zhen, ZHANG Fan, XIANG Hanyu, YANG Chong, HUANG Xianfeng. Multiple view complete 3D model reconstruction of manually turned object[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(8): 1305-1316.

References

[1] 郑太雄, 黄帅, 李永福, 等. 基于视觉的三维重建关键技术研究综述[J]. 自动化学报, 2020, 46(4):631-652. ZHENG Taixiong, HUANG Shuai, LI Yongfu, et al. Key techniques for vision based 3D reconstruction: a review[J]. Acta Automatica Sinica, 2020, 46(4):631-652.
[2] 杨荣, 冯有前, 袁修久. 利用现有模型修复不完整三维模型[J]. 计算机辅助设计与图形学学报, 2015, 27(1): 98-105. YANG Rong, FENG Youqian, YUAN Xiujiu. Restoration of fragmentary 3D models using existing models[J]. Journal of Computer-Aided Design & Computer Graphics, 2015, 27(1): 98-105.
[3] SHASHUA A, LEVIN A. Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects[C]//Proceedings of the 8th IEEE International Conference on Computer Vision.Vancouver:IEEE, 2002: 592-599.
[4] 秦爽. 转台拍摄的多视图像前后景自动分割方法研究[D]. 武汉: 武汉大学, 2019. QIN Shuang. Research on automatic segmentation method of foreground and background of multi-view images taken by turntable[D]. Wuhan: Wuhan University, 2019.
[5] VIDAL R, MA Yi, SOATTO S, et al. Two-view multibody structure from motion[J].International Journal of Computer Vision, 2006, 68(1): 7-25.
[6] 周武根. 动态环境下稠密视觉同时定位与地图构建方法研究[D]. 北京: 中国科学院大学, 2020. ZHOU Wugen. Research on simultaneous location and map construction method of dense vision in dynamic environment[D]. Beijing: University of Chinese Academy of Sciences, 2020.
[7] WANGSIRIPITAK S, MURRAY D W. Avoiding moving outliers in visual SLAM by tracking moving objects[C]//Proceedings of 2009 IEEE International Conference on Robotics and Automation. Kobe: IEEE, 2009: 375-380.
[8] CHHAYA F, REDDY D, UPADHYAY S, et al. Monocular reconstruction of vehicles: combining SLAM with shape priors[C]//Proceedings of 2016 IEEE International Conference on Robotics and Automation. Stockholm: IEEE, 2016: 5758-5765.
[9] YANG Chong, ZHANG Fan, GAO Yunlong, et al. Moving car recognition and removal for 3D urban modelling using oblique images[J]. Remote Sensing, 2021, 13(17): 3458.
[10] HE Kaiming, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2980-2988.
[11] YU Chao, LIU Zuxin, LIU Xinjun, et al. DS-SLAM: a semantic visual SLAM towards dynamic environments[C]//Proceedings of 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems. Madrid: IEEE, 2019: 1168-1174.
[12] BADRINARAYANAN V, KENDALL A, CIPOLLA R.SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[13] WANG Chenjie, LUO Bin, ZHANG Yun, et al. DymSLAM: 4D dynamic scene reconstruction based on geometrical motion segmentation[J]. IEEE Robotics and Automation Letters, 2021, 6(2): 550-557.
[14] KUNDU A, KRISHNA K M, SIVASWAMY J. Moving object detection by multi-view geometric techniques from a single camera mounted robot[C]//Proceedings of 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems. St. Louis:IEEE, 2009: 4306-4312.
[15] MIGLIORE D, RIGAMONTI R, MARZORATI D, et al. Use asingle camera for simultaneous localization and mapping with mobile object tracking in dynamic environments[C]//Proceedings of 2009 IEEE International Conference on Robotics and Automation.[S.l.]: IEEE, 2009.
[16] ZOU Danping, TAN Ping. CoSLAM: collaborative visual SLAM in dynamic environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(2): 354-366.
[17] BESCOS B, FÁCIL J M, CIVERA J, et al.DynaSLAM: tracking, mapping, and inpainting in dynamic scenes[J]. IEEE Robotics and Automation Letters, 2018, 3(4): 4076-4083.
[18] SAPUTRA M R U, MARKHAM A, TRIGONI N. Visual SLAM and structure from motion in dynamic environments: a survey[J]. ACM Computing Surveys, 2018, 51(2): 1-36.
[19] HRUBY P, PAJDLA T. Reconstructing small 3D objects in front of a textured background[EB/OL]. [202-07-08]. https://arxiv.org/abs/2105.11352.
[20] 盛庆红, 柳建锋, 虞梦昕, 等. 对偶四元数近景影像空中三角测量法[J]. 测绘学报, 2015, 44(5): 503-509. DOI: 10.11947/j.AGCS.2015.20130533. SHENG Qinghong, LIU Jianfeng, YU Mengxin, et al. Aerial triangulation close-range images with dual quaternion[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(5): 503-509. DOI: 10.11947/j.AGCS.2015.20130533.
[21] MOISAN L, MOULON P, MONASSE P. Automatic homographic registration of a pair of images, with A contrario elimination of outliers[J]. Image Processing on Line, 2012, 2: 56-73.
[22] RODRIGUEZ A, LAIO A. Clustering by fast search and find of density peaks[J]. Science, 2014, 344(6191): 1492-1496.
[23] PIOREK M, JABLONSKI B. A quaternion clustering framework[J]. International Journal of Applied Mathematics and Computer Science, 2020, 30(1): 133-147.
[24] ZELNIK-MANOR L, PERONA P. Self-tuning spectral clustering[C]//Proceedings of the 17th International Conference on Neural Information Processing Systems. Vancouver: ACM Press, 2004: 1601-1608.
[25] GOESELE M, SNAVELY N, CURLESS B, et al.Multi-view stereo for community photo collections[C]//Proceedings of IEEE 11th International Conference on Computer Vision. Rio de Janeiro:IEEE, 2007: 1-8.
[26] RAND W M. Objective criteria for the evaluation of clustering methods[J]. Journal of the American Statistical Association, 1971, 66(336): 846-850.
[27] ANAS H, JENSEN R R, VOGIATZIS G, et al. Large-scale data for multiple-view stereopsis[J].International Journal of Computer Vision, 2016, 120(2): 153-168.