3D Reconstruction of High-reflective and Textureless Targets Based on Multispectral Polarization and Machine Vision

doi:10.11947/j.AGCS.2018.20170624

Abstract

Abstract: With the rapid development of photogrammetry and machine vision technology,a higher universality of three-dimensional reconstruction is required.For high-reflective and non-metal targets with smooth surface or extreme simple texture,a large area of data void may appear on the reconstruction surface since the traditional 3D reconstruction methods depend on texture and reflective characteristics.To solve this problem,a 3D reconstruction method based on multispectral polarization imaging is proposed in this paper,which integrates photogrammetry and machine vision and achieves accurate reconstruction by obtaining accurate multispectral polarization characteristics of targets.The proposed method does not rely on the texture information on the surface,and it can solve the problem of joint estimating refractive index and zenith angle simultaneously,which cannot be achieved only by Fresnel theory.Due to straylights and diffuse reflection light have different polarization and spectral characteristics at different wavelengths,we can remove the highlight to improve the reconstruction accuracy.The 3D reconstruction method based on multiband polarization imaging is the guiding progress of 3D reconstruction after the fusion of photogrammetry and machine vision,which has a wider application range.

Key words: 3D reconstruction, multispectral polarization, textureless targets

CLC Number:

TP391

HAO Jinglei, ZHAO Yongqiang, ZHAO Haimeng, Peter BREZANY, SUN Jiayu. 3D Reconstruction of High-reflective and Textureless Targets Based on Multispectral Polarization and Machine Vision[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 816-824.

References

[1] SONG Weitao, WENG Dongdong, LIU Yue, et al. Three-dimensional Reconstruction for Photon Counting Imaging Using a Planar Catadioptric Method[C]//Proceedings of 2017 Conference on Lasers and Electro-optics Pacific Rim. Singapore:IEEE, 2017:1-2.
[2] WANG Binquan, KONG Lingcheng, ZHAO Jianghai, et al. Design of Three-dimensional Reconstruction and Robot Path Planning Based on Kinect System[C]//Proceedings of the 29th Chinese Control and Decision Conference. Chongqing, China:IEEE, 2017:3829-3834.
[3] ZHANG Junning, PAN Wei, SHI Haobin, et al. Three-Dimensional Reconstruction Method Based on Target Recognition for Binocular Humanoid Robot[C]//Proceedings of 2016 International Conference on Progress in Informatics and Computing. Shanghai, China:IEEE, 2017:355-359.
[4] HONDA J, YOKOYAMA H, TAJIMA H, et al. Influences of 3D Aircraft Model to ILS Localizer[C]//Proceedings of the 201610th International Conference on Complex, Intelligent, and Software Intensive Systems. Fukuoka, Japan:IEEE, 2016:180-185.
[5] SUN Huabo, LUO Boren, YU Liling, et al. Mosaic Research with 3D LiDar Point Cloud of Civil Aircraft[C]//Proceedings of the 20132nd International Symposium on Instrumentation and Measurement, Sensor Network and Automation. Toronto, ON, Canada:IEEE, 2013:683-686.
[6] GODARD C, HEDMAN P, LI Wenbin, et al. Multi-View Reconstruction of Highly Specular Surfaces in Uncontrolled Environments[C]//Proceedings of 2015 International Conference on 3D Vision. Lyon, France:IEEE, 2015:19-27.
[7] ZHAO Yongqiang, YI Chen, KONG S G, et al. Multi-band Polarization Imaging and Applications[M]. Berlin, Heidelberg:Springer, 2016.
[8] RAGHEB H, HANCOCK E R. A Probabilistic Framework for Specular Shape-from-shading[J]. Pattern Recognition, 2003, 36(2):407-427.
[9] KOSHIKAWA K. A Polarimetric Approach to Shape Understanding of Glossy Objects[C]//Proceedings of the 6th International Joint Conference on Artificial Intelligence. Tokyo, Japan:Morgan Kaufmann Publishers Inc., 1979.
[10] GHOSH A, CHEN Tongbo, PEERS P, et al. Circularly Polarized Spherical Illumination Reflectometry[J]. ACM Transactions on Graphics, 2010, 29(6):162.
[11] MIYAZAKI D, KAGESAWA M, IKEUCHI K. Transparent Surface Modeling from a Pair of Polarization Images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(1):73-82.
[12] ATKINSON G A, HANCOCK E R. Multi-view Surface Reconstruction Using Polarization[C]//Proceedings of the 10th IEEE International Conference on Computer Vision. Beijing, China:IEEE, 2005:309-316.
[13] MIYAZAKI D, SAITO M, SATO Y, et al. Determining Surface Orientations of Transparent Objects Based on Polarization Degrees in Visible and Infrared Wavelengths[J]. Journal of the Optical Society of America A, 2002, 19(4):687-694.
[14] MOREL O, GORRIA P. Polarization Imaging for 3D Inspection of Highly Reflective Metallic Objects[J]. Optics and Spectroscopy, 2006, 101(1):11-17.
[15] STOLZ C, FERRATON M, MÉRIAUDEAU F. Shape from Polarization:A Method for Solving Zenithal Angle Ambiguity[J]. Optics Letters, 2012, 37(20):4218-4220.
[16] ZHAO Yongqiang, WANG Miaomiao, YANG Guang, et al. FOV Expansion of Bioinspired Multiband Polarimetric Imagers with Convolutional Neural Networks[J]. IEEE Photonics Journal, 2018, 10(1):1-14.
[17] MIYAZAKI D, SHIGETOMI T, BABA M, et al. Polarization-Based Surface Normal Estimation of Black Specular Objects from Multiple Viewpoints[C]//Proceedings of the 2nd International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission. Zurich, Switzerland:IEEE, 2012:104-111.
[18] DROUET F, STOLZ C, LALIGANT O, et al. 3D Reconstruction of External and Internal Surfaces of Transparent Objects from Polarization State of Highlights[J]. Optics Letters, 2014, 39(10):2955-2958.
[19] ATKINSON G A, HANCOCK E R. Surface Reconstruction Using Polarization and Photometric Stereo[C]//Proceedings of the 12th International Conference on Computer Analysis of Images and Patterns. Vienna, Austria:Springer, 2007:466-473.
[20] SAMAN G, HANCOCK E. Refractive Index Estimation Using Photometric Stereo[C]//Proceedings of the 18th IEEE International Conference on Image Processing. Brussels, Belgium:IEEE, 2011:1925-1928.
[21] HUYNH C P, ROBLES-KELLY A, HANCOCK E R. Shape and Refractive Index from Single-View Spectro-Polarimetric Images[M]. Hingham:Kluwer Academic Publishers, 2013.
[22] KADAMBI A, TAAMAZYAN V, SHI Boxin, et al. Polarized 3D:High-quality Depth Sensing with Polarization Cues[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015:3370-3378.
[23] NGO T T, NAGAHARA H, TANIGUCHI R I. Shape and Light Directions from Shading and Polarization[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:2310-2318.
[24] 平茜茜, 刘勇, 董欣明, 等. 基于偏振双目视觉的无纹理高反光目标三维重构[J]. 红外与毫米波学报, 2017, 36(4):432-438. Ping Xixi, LIU Yong, DONG Xinming, et al. 3-D Reconstruction of Textureless and high-Reflective target by Polarization and Binocular Stereo Vision[J]. Journal of Infrared and Millimeter Waves, 2017, 36(4):432-438.
[25] AHMAD J E, TAKAKURA Y. Stereo-polarimetric Measurement of Pair of Mueller Images for Three Dimensional Partial Reconstruction[C]//Proceedings of the 200614th European Signal Processing Conference. Florence, Italy:IEEE, 2006:1-4.
[26] BORN M, WOLF E. Principles of Optics:Electromagnetic Theory of Propagation, Interference and Diffraction of Light[M]. 7th ed. Oxford:Cambridge University Press, 1999.
[27] ZHAO Yongqiang, PENG Qunnie, YI Chen, et al. Multiband Polarization Imaging[J/OL]. Journal of Sensors,(2016)[2016-04-15].http://dx.doi.org/10.1155/2016/5985673.
[28] ZHAO Yongqiang, PENG Qunnie, XUE Jize, et al. Specular Reflection Removal Using Local Structural Similarity and Chromaticity Consistency[C]//Proceedings of 2015 IEEE International Conference on Image Processing. Quebec City, QC, Canada:IEEE, 2015:3397-3401.