Solving camera pose with multiplexing mirror constraint

doi:10.11947/j.AGCS.2024.20220362

Abstract

Abstract: Aiming at the problem that the reference object is not in the field of view, a method with constraints of mirror camera to solve camera pose is proposed.The steps are:① Mirror camera pose is solved by using the method of camera pose estimation. ② The linear constraint relationship between the pose of camera and single image mirror camera is constructed with mirror reflection symmetry. ③ To establish constraints between multiple images, the mirror camera poses were multiplexed, 18 linear constraints can be obtained using 3 images. This makes the accuracy of camera pose solution improved. Experiments show that:the errors of rotation matrix and translation vector are 0.009° and 1.866mm respectively, and the corresponding errors are reduced by 0.009°and 2.292mm on average, and has a strong adaptability. In real experiments, the average reprojection error is 0.492 pixels. This suggests that the method in this paper is better than the comparison methods, and the pose of the camera where the reference object is not in the field of view can be accurately solved.

Key words: camera pose, mirror reflection symmetry, mirror camera, least squares

CLC Number:

P234

A Xiaohui, LI Jiatian, LIU Jiayin, HU Hao, HE Rixing, LU Mei, DUAN Ye. Solving camera pose with multiplexing mirror constraint[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(3): 548-557.

References

[1] WANG Ping, CHOU Yongxin, AN Aimin, et al. Solving the PnL problem using the hidden variable method:an accurate and efficient solution[J]. The Visual Computer, 2022, 38(1):95-106.
[2] 李佳田, 贾成林, 牛一如, 等. 一种求解单像空间后方交会的监督学习方法[J]. 武汉大学学报(信息科学版), 2019, 44(8):1144-1152. LI Jiatian, JIA Chenglin, NIU Yiru, et al. A supervised learning method for solving space resection of single image[J]. Geomatics and Information Science of Wuhan University, 2019, 44(8):1144-1152.
[3] 尚洋, 孙晓亮, 张跃强, 等. 三维目标位姿跟踪与模型修正[J]. 测绘学报, 2018, 47(6):799-808.DOI:10.11947/j.AGCS.2018.20170626. SHANG Yang, SUN Xiaoliang, ZHANG Yueqiang, et al. Research on 3D target pose tracking and modeling[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):799-808.DOI:10.11947/j.AGCS.2018.20170626.
[4] 张帆, 黄印, 黄先锋, 等. 非直视区域的普通平面镜辅助地面三维激光扫描方法[J]. 测绘学报, 2017, 46(12):1950-1958.DOI:10.11947/j.AGCS.2016.20150411. ZHANG Fan, HUANG Yin, HUANG Xianfeng, et al. 3D laser scanning assisted by ordinary plane mirror for non-direct viewing area[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(12):1950-1958.DOI:10.11947/j.AGCS.2016.20150411.
[5] AGRAWAL A. Extrinsic camera calibration without a direct view using spherical mirror[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney:IEEE, 2013:2368-2375.
[6] FRANCKEN Y, HERMANS C, BEKAERT P. Screen-camera calibration using a spherical mirror[C]//Proceedings of the 4th Canadian Conference on Computer and Robot Vision.Piscataway:ACM Press, 2007:11-20.
[7] FRANCKEN Y, HERMANS C, BEKAERT P. Screen-camera calibration using gray codes[C]//Proceedings of 2009 Canadian Confe-rence on Computer and Robot Vision. Kelowna:IEEE, 2009:155-161.
[8] 付生鹏, 赵吉宾, 夏仁波, 等. 基于环形镜面的相机外部参数自动标定方法[J]. 机器人, 2015, 37(3):277-285. FU Shengpeng, ZHAO Jibin, XIA Renbo, et al. Annular mirror based extrinsic camera automatic calibration[J]. Robot, 2015, 37(3):277-285.
[9] NITSCHKE C, NAKAZAWA A, TAKEMURA H. Display-camera calibration from eye reflections[C]//Proceedings of the 12th IEEE International Conference on Computer Vision. Kyoto:IEEE, 2009:1226-1233.
[10] FUNK N, YANG Y H. Using a raster display for photometric stereo[C]//Proceedings of the 4th Canadian Conference on Computer and Robot Vision. Montreal:IEEE, 2007:201-207.
[11] STURM P, BONFORT T. How to compute the pose of an object without a direct view?[C]//Proceedings of the 7th Asian Conference on Computer Vision-Volume Part II.Hyderabad:ACM Press, 2006:21-31.
[12] HESCH J A, MOURIKIS A I, ROUMELIOTIS S I. Mirror-based extrinsiccamera calibration[J]. Springer Tracts in Advanced Robotic,2009, 57:285-299.
[13] RODRIGUES R, BARRETO J P, NUNES U. Camera pose estimation using images of planar mirror reflections[C]//Proceedings of the 11th European conference on Computer vision:Part IV.Heraklion:ACM Press, 2010:382-395.
[14] TAKAHASHI K, NOBUHARA S, MATSUYAMA T. A new mirror-based extrinsic camera calibration using an orthogonality constraint[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition.Providence:IEEE, 2012:1051-1058.
[15] TAKAHASHI K, NOBUHARA S, MATSUYAMA T. Mirror-based camera pose estimation using an orthogonality constraint[J]. IPSJ Transactions on Computer Vision and Applications, 2016, 8:11-19.
[16] LONG Gucan, KNEIP L, LI Xin, et al. Simplified mirror-based camera pose computation via rotation averaging[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE, 2015:1247-1255.
[17] KUMAR R K, ILIE A, FRAHM J M, et al. Simple calibration of non-overlapping cameras with a mirror[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage:IEEE, 2008:1-7.
[18] 付生鹏, 赵吉宾, 夏仁波, 等. 基于平面镜的相机-显示器快速标定方法[J]. 仪器仪表学报, 2013, 34(12):2770-2776. FU Shengpeng, ZHAO Jibin, XIA Renbo, et al. A mirror-based fast camera-screen calibration method[J]. Chinese Journal of Scientific Instrument, 2013, 34(12):2770-2776.
[19] ZHANG Zhengyou. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[20] 邓非, 吴幼丝, 胡玉雷, 等. 球形全景影像位姿估计的改进EPnP算法[J]. 测绘学报, 2016, 45(6):677-684.DOI:10.11947/j.AGCS.2016.20150411. DENG Fei, WU Yousi, HU Yulei,et al. Position and pose estimation of spherical panoramic image with improved EPnP algorithm[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(6):677-684.DOI:10.11947/j.AGCS.2016.20150411.
[21] LI Shiqi, XU Chi, XIE Ming. A robust O(n) solution to the perspective-n-point problem[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 34(7):1444-1450.
[22] GOLUB G H, VAN LOAN C F. Matrix computations[M]. 3rd ed. Baltimore:Johns Hopkins University Press, 1996.
[23] 李佳田, 王聪聪, 阿晓荟, 等. 多向前方交会与单隐层神经网络结合的近景前方交会法[J]. 测绘学报, 2020, 49(6):736-745.DOI:10.11947/j.AGCS.2020.20180600. LI Jiatian, WANG Congcong, A Xiaohui, et al. Method of close-range space intersection combining multi-image forward intersection with single hidden layer neural network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(6):736-745.DOI:10.11947/j.AGCS.2020.20180600.
[24] LI Jiatian, WANG Congcong, JIA Cheng, et al. A hybrid conjugate gradient algorithm for solving relative orientation of big rotation angle stereo pair[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):62-70.
[25] SATTLER T, MADDERN W, TOFT C, et al. Benchmarking 6DOF outdoor visual localization in changing conditions[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City:IEEE, 2018:8601-8610.