3D face reconstruction based on global ICP and improved Poisson

doi:10.11947/j.AGCS.2023.20220128

Abstract

Abstract: In order to realize fast and accurate 3D face digitization at low cost, a high precision and robust automatic 3D face reconstruction method is proposed in this paper. Taking the left and right face point clouds and RGB image data collected by structured light camera as input, we first propose an adaptive sub-sampling global optimization ICP(iterative closest point) registration method to integrate the left and right point clouds, and then use a improved Poisson reconstruction method through normal vector optimization to reconstruct the surface of the point cloud after registration . The Poisson reconstruction method has good reconstruction effect and reconstruction accuracy for the face point cloud, which is a non-closed point cloud. Based on generated grid model, the RBG image was mapped to the mesh model, and finally a 3D face model with detailed texture is reconstructed. Extensive face reconstruction experiments demonstrate that the proposed method has the characteristics of high robustness and high precision, and can efficiently and accurately reconstruct the 3D face.

Key words: point cloud depth information, face model reconstruction, point cloud registration, Poisson reconstruction, point cloud surface reconstruction

CLC Number:

P237
TP391

LI Haoran, MEI Tiancan, GAO Zhi. 3D face reconstruction based on global ICP and improved Poisson[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(3): 454-463.

References

[1] 叶日藏. 基于Kinect深度传感器的三维重建技术应用研究[D]. 广州:华南理工大学, 2013. YE Ricang. Three-dimensional reconstruction technique application research based on Kinect depth sensor[D]. Guangzhou:South China University of Technology, 2013.
[2] 白云峰. Photoscan多视角三维重建技术在土遗址保护中的应用研究[D]. 西安:西北大学, 2017. BAI Yunfeng. Application of Photoscan multi-view 3D reconstruction technique in soil site protection[D]. Xi'an:Northwest University, 2017.
[3] 张文园, 刘雪莹, 王福. 生物医学图像三维重建技术应用研究进展[J]. 中华实用诊断与治疗杂志, 2016, 30(7):629-631. ZHANG Wenyuan, LIU Xueying, WANG Fu. Application of three-dimensional reconstruction based on biomedical images[J]. Journal of Chinese Practical Diagnosis and Therapy, 2016, 30(7):629-631.
[4] YANG Shanmin, LIN Yi, ZHANG Jianwei. Triangular patch based texture fusion for multi-view 3D face model[C]//Proceedings of 2018 International Conference on Graphics and Image Processing. Chengdu, China:SPIE, 2019,11069:288-298.
[5] 马洁莹, 田暄, 翟庆, 等. 基于点到面度量的多视角点云配准方法[J]. 西安交通大学学报, 2022, 56(6):120-132. MA Jieying, TIAN Xuan, ZHAI Qing, et al. Registration of multi-view point sets based on point-to-plane measurement[J]. Journal of Xi'an Jiaotong University, 2022, 56(6):120-132.
[6] 石峰源,郑循江,姜丽辉,等.基于霍夫变换的空间非合作目标点云配准算法[J/OL].北京航空航天大学学报:1-10[2023-02-21].DOI:10.13700/j.bh.1001-5965.2021.0575. SHI Fengyuan, ZHENG Xunjiang, JIANG Lihui,et al. Point cloud registration algorithm for non-cooperative targets based on Hough transform[J]. Journal of Beijing University of Aeronautics and Astronautics:1-10[2023-02-21].DOI:10.13700/j.bh.1001-5965.2021.0575.
[7] FAHANDEZH-SAADI S, WANG Di, TOMIZUKA M. Robust feature-based point registration using directional mixture model[J]. IFAC-PapersOnLine, 2020, 53(2):15454-15460.
[8] 刘若晗, 郭丙轩. 顾及点云曲率的快速点云表面模型重建算法[J]. 测绘科学技术, 2020, 8(2):88-95. LIU Ruohan, GUO Bingxuan. A fast surface reconstruction algorithm considering the curvature of point cloud[J]. Geomatics Science and Technology, 2020, 8(2):88-95.
[9] JONASSON E T, BOEUF J, MURCUTT P, et al. Improved reconstruction of JET using LiDAR-vision fusion[J]. Fusion Engineering and Design, 2020, 161:112061.
[10] 朱若岭, 张昊. 基于Delaunay三角网的高效车载雷达点云表面重建算法[J]. 国外电子测量技术, 2021, 40(12):44-49. ZHU Ruoling, ZHANG Hao. Efficient point cloud surface reconstruction algorithm for vehicle radar based on Delaunay triangulation[J]. Foreign Electronic Measurement Technology, 2021, 40(12):44-49.
[11] SHANG Shang, SUN Sun, ZHANG Zhang, et al. Research on 3D target pose tracking and modeling[J]. Journal of Geodesy and Geoinformation Science, 2019(2):60-69.
[12] XU Yusheng, HUANG Huang, TONG Tong, et al. Exploiting robust estimators in phase correlation of 3D point clouds for 6 DoF pose estimation[J]. Journal of Geodesy and Geoinformation Science, 2021(3):72-90.
[13] LI Bin, ZHAO Xiaofa, CHEN Yong, et al. Three dimensional laser point cloud slicing method for calculating irregular volume[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(4):31-43.
[14] 刘博文. 基于二维特征点的三维人脸点云配准技术研究[D]. 北京:北方工业大学, 2018. LIU Bowen. Research on 3D face registration based on two dimensional feature points[D]. Beijing:North China University of Technology, 2018.
[15] 张倩, 吕丽平. 基于ORB与局部仿射一致性约束的快速人脸特征配准[J]. 电子测量与仪器学报, 2019, 33(6):38-44. ZHANG Qian, LÜ Liping. Fast face feature registration based on ORB and local affine consistency constraints[J]. Journal of Electronic Measurement and Instrumentation, 2019, 33(6):38-44.
[16] 孙硕, 嵇晓强, 刘丹. 用于面部虚拟整形的三维人脸重建系统设计[J]. 科学技术与工程, 2021, 21(25):10806-10813. SUN Shuo, JI Xiaoqiang, LIU Dan. 3D face reconstruction system for face virtual shaping[J]. Science Technology and Engineering, 2021, 21(25):10806-10813.
[17] 张维康. 基于双目立体视觉的三维人脸重建方法研究[D]. 沈阳:沈阳工业大学, 2020. ZHANG Weikang. Research on 3D face reconstruction method based on binocular stereo vision[D]. Shenyang:Shenyang University of Technology, 2020.
[18] 夏颖. 基于双目立体视觉的三维人脸重建算法的研究[D]. 南京:东南大学, 2020. XIA Ying. The study of 3D face reconstruction algorithm based on biocular stereo vision[D]. Nanjing:Southeast University, 2020.
[19] 王刚. 基于双目立体视觉的人脸三维模型重建研究[D]. 长春:吉林大学, 2014. WANG Gang. Research of human faces reconstruction on binocular stereo vision[D]. Changchun:Jilin University, 2014.
[20] ZHU Rui, MA Shan, XU Degang. Guided filter simplification method for noisy point cloud data[C]//Proceedings of 2020 Chinese Automation Congress. Shanghai, China:IEEE, 2020:6951-6955.
[21] XIE Bingqian, GU Yanfeng, CAO Zhimin. Building LiDAR point cloud denoising processing through sparse representation[C]//Proceedings of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS).Milan, Italy:IEEE, 2015:585-588.
[22] LIU Haoyue, SONG Rui, ZHANG Xuebo, et al. Point cloud segmentation based on Euclidean clustering and multi-plane extraction in rugged field[J]. Measurement Science and Technology, 2021, 32(9):095106.
[23] YANG Jiaolong, LI Hongdong, JIA Yunde. Go-ICP:solving 3D registration efficiently and globally optimally[C]//Proceedings of 2013 IEEE International Conference on Computer Vision.Sydney, Australia:IEEE, 2014:1457-1464.
[24] HAZAMA K, EBARA H. Branch and bound algorithm for parallel many-core architecture[C]//Proceedings of the 6th International Symposium on Computing and Networking Workshops (CANDARW). Takayama, Japan:IEEE, 2018:272-277.
[25] YANG Jiaolong, LI Hongdong, CAMPBELL D, et al. Go-ICP:a globally optimal solution to 3D ICP point-set registration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(11):2241-2254.
[26] 康豆. 点云的三维重建与纹理映射[D]. 太原:中北大学, 2021. KANG Dou. 3D reconstruction and texture mapping of point cloud[D]. Taiyuan:North University of China, 2021.
[27] 张凯. 基于泊松方程的三维表面重建算法的研究[D]. 天津:河北工业大学, 2014. ZHANG Kai. Research on 3d surface reconstruction algorithm based on Poisson equation[D]. Tianjin:Hebei University of Technology, 2014.
[28] KENMOCHI Y, KOTANI K, IMIYA A. Marching cubes method with connectivity[C]//Proceedings of 1999 International Conference on Image Processing (Cat. 99CH36348). Kobe, Japan:IEEE, 1999,4:361-365.
[29] WAECHTER M, MOEHRLE N, GOESELE M. Let there be color! Large-scale texturing of 3D reconstructions[C]//Computer Vision-ECCV 2014:13th European Conference. Zurich, Switzerland:Springer International Publishing, 2014:836-850.
[30] CHENG S W, DEY T K, SHEWCHUK J. Three-dimensional delaunay triangulations[M]//Delaunay Mesh Generation. New York,USA:Chapman and Hall/CRC, 2016:101-120.
[31] CHAKRABORTY T. Interior ball-pivoting on point clouds for offsetting triangular meshes[C]//Proceedings of CAD'21. Barcelona, Spain:CAD Solutions LLC,2021:83-87.
[32] LI Hui, MA Xiuli, LI Jinbo, et al. Research on model correction based on scattered point cloud data surface reconstruction[C]//Proceedings of 2011 IET International Communication Conference on Wireless Mobile and Computing (CCWMC 2011). Shanghai. London:IET, 2012:97-101.
[33] HOPPE H. Poisson surface reconstruction and its applications[C]//Proceedings of 2008 ACM symposium on solid and physical modeling. New York,USA:ACM Press, 2008:10.
[34] YANG-TURNER F, VOLK D, FOWLER P, et al. Scalable pathogen pipeline platform (SP3):enabling unified genomic data analysis with elastic cloud computing[C]//Proceedings of 2019 IEEE International Conference on Cloud Computing (CLOUD). Milan, Italy:IEEE, 2019:478-480.