Calibration of multi-head panoramic camera using 2D checkerboard under rotation photography

doi:10.11947/j.AGCS.2022.20210337

Abstract

Abstract: Multi-head panoramic camera (MPC) with ultra-wide field of view and high resolution is widely used, but its calibration depends on high-precision 3D control information. In this paper, a MPC calibration method is proposed using one common 2D checkerboard and rotatable platform. Proposed method is distinctly divided into two stages: firstly, side camera calibration using rotation sequence images. With elaborate consideration of setup of coordinate reference frame of checkerboard and rotatable platform, one strict rotating photography equation is first established for the side-camera of MPC. Based on it, initial external parameter values of the image sequence outputed from any side camera of MPC in rotatable platform are given, and further optimized by taking the corner of the chessboard image and a small number of tie points between the adjacent views of side camera as the observation values. Secondly, center camera calibration based on multi-view geometry. The MPC space coordinate system is first established by moving the origin of the rotatable platform coordinate system to the geometric centroid of center of all side cameras. Based on it, the solid multi-view geometry relationship between the center camera and the side camera are used as the constrain as well as a large number of tie points obtained by rotating photography are used as the observation values to optimize the parameters of the center camera through bundle adjustment. The experimental results show that the proposed method can achieve high-precision calibration of MPC, only using one 2D checkerboard and a rotatable platform. It has a good application value because of its high-precision, high-degree of automation and low requirements for environment (independence on 3D control field).

Key words: multi-head panoramic camera, panoramic camera calibration, spherical panoramic imaging, checkerboard calibration, rotating photography

CLC Number:

P233

HUANG Mingyi, WU Jun, PENG Zhiyong. Calibration of multi-head panoramic camera using 2D checkerboard under rotation photography[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(12): 2541-2556.

References

[1] NIELSEN F. Surround video:a multihead camera approach[J]. The Visual Computer, 2005, 21(1-2):92-103.
[2] 季顺平, 史云. 多镜头组合型全景相机两种成像模型的定位精度比较[J]. 测绘学报, 2014, 43(12):1252-1258. JI Shunping, SHI Yun. Comparison of two sensor models for multi-camera rig system in measurements[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(12):1252-1258.
[3] 王冬, 刘凤英, 陈天恩, 等. 一种相机标定参数的柱面全景影像拼接方法[J]. 测绘科学, 2016, 41(7):150-154, 143. WANG Dong,LIU Fengying,CHEN Tianen,et al. The method of cylinder panoramic image mosaic based on camera calibration parameters[J]. Science of Surveying and Mapping, 2016, 41(7):150-154, 143.
[4] 张茂军, 钟力, 孙立峰, 等. HVS:构造一个虚拟实景空间[J]. 自动化学报, 2000, 26(6):736-740. ZHANG Maojun, ZHONG Li, SUN Lifeng, et al. HVS:building a virtual imaging space[J]. Acta Automatica Sinica, 2000, 26(6):736-740.
[5] ALY M, BOUGUET J Y. Street view goes indoors:automatic pose estimation from uncalibrated unordered spherical panoramas[C]//Proceedings of 2012 IEEE Workshop on the Applications of Computer Vision. Breckenridge, CO, USA:IEEE, 2012:1-8.
[6] LEE J, KIM B, KIM K, et al. Rich360:optimized spherical representation from structured panoramic camera arrays[J]. ACM Transactions on Graphics, 2016, 35(4):1-11.
[7] 吴军, 刘祺昌, 黄明益. 城市视频实景地图构建初探[J]. 中国图象图形学报, 2019, 24(9):1409-1415. WU Jun, LIU Qichang, HUANG Mingyi. Preliminary exploration on the construction of urban video real-scene map[J]. Journal of Image and Graphics, 2019, 24(9):1409-1415.
[8] LIN Zongjian, XIE Feifei, SU Guozhong. Accuracy analysis of low altitude photogrammetry with wide-angle camera[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1):30-38.
[9] 郑顺义, 黄荣永, 郭宝云, 等. 附约束条件的立体相机标定方法[J]. 测绘学报, 2012, 41(6):877-885. ZHENG Shunyi, HUANG Rongyong, GUO Baoyun, et al. Stereo-camera calibration with restrictive constraints[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(6):877-885.
[10] 谢文寒, 张祖勋. 基于多像灭点的相机定标[J]. 测绘学报, 2004, 33(4):335-340. XIE Wenhan, ZHANG Zuxun. Camera calibration based on vanishing points of multi-image[J]. Acta Geodaetica et Cartographic Sinica, 2004, 33(4):335-340.
[11] TZAVIDAS S, KATSAGGELOS A K. A multi-camera setup for generating stereo panoramic video[J]. IEEE Transactions on Multimedia, 2005, 7(5):880-890.
[12] 刘凤英, 陈天恩, 王冬. 多影像空三平差辅助下的车载全景相机严密标定方法[J]. 测绘通报, 2017(7):23-28. LIU Fengying, CHEN Tianen, WANG Dong. The calibration method of vehicle-borne panoramic camera based on the aerial triangulation of the multi-direction images[J]. Bulletin of Surveying and Mapping, 2017(7):23-28.
[13] 姜广文, 晁志超, 伏思华, 等. 基于可控旋转的像机高精度标定技术[J]. 光学学报, 2010, 30(5):1308-1314. JIANG Guangwen, CHAO Zhichao, FU Sihua, et al. High-accurate camera calibration technique based on controllable rotation[J]. Acta Optica Sinica, 2010, 30(5):1308-1314.
[14] AMIRI PARIAN J, GRUEN A. Sensor modeling, self-calibration and accuracy testing of panoramic cameras and laser scanners[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2010, 65(1):60-76.
[15] 吴凡路, 刘建军, 任鑫, 等. 基于圆形标志点的深空探测全景相机标定方法[J]. 光学学报, 2013, 33(11):147-153. WU Fanlu, LIU Jianjun, REN Xin, et al. Deep space exploration panoramic camera calibration technique based on circular markers[J]. Acta Optica Sinica, 2013, 33(11):147-153.
[16] KUMAR R K, ILIE A, FRAHM J M, et al. Simple calibration of non-overlapping cameras with a mirror[C]//Proceedings of 2008 IEEE Conference on Computer Vision and Pattern Recognition. Anchorage, AK, USA:IEEE, 2008:1-7.
[17] KAHLESZ F, LILGE C, KLEIN R. Easy-to-use calibration of multiple-camera setups[C]//Proceedings of 2007 International Conference on Computer Vision Systems.[S.l.]:IEEE, 2007.
[18] XING Ziran, YU Jingyi, MA Yi. A new calibration technique for multi-camera systems of limited overlapping field-of-views[C]//Proceedings of 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Vancouver, BC, Canada:IEEE, 2017:5892-5899.
[19] 梅文胜, 胡帅朋, 李谋思, 等. 基于普通数码相机的旋转全景摄影测量方法[J]. 武汉大学学报(信息科学版), 2017, 42(2):243-249. MEI Wensheng, HU Shuaipeng, LI Mousi, et al. Rotation panorama photogrammetry method based on digital camera[J]. Geomatics and Information Science of Wuhan University, 2017, 42(2):243-249.
[20] TZAVIDAS S, KATSAGGELOS A K. Multicamera setup for generating stereo panoramic video[C]//Proceedings of 2002 Three-Dimensional Image Capture and Applications Version.[S.l.]:SPIE,2002:47-58.
[21] 王之卓.摄影测量原理[M].武汉:武汉大学出版社,2007年. WANG Zhizhuo. Principle of photogrammetry[M]. Wuhan:Publishing wuhan university press, 2007.
[22] KANNALA J, BRANDT S S. A generic camera model and calibration method for conventional, wide-angle, and fish-eye lenses[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2006, 28(8):1335-1340.
[23] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[24] 皮英冬, 李欣, 陈智勇, 等. 基于三维控制场的鱼眼影像检校和纠正方法[J]. 光学学报, 2017, 37(1):0115001. PI Yingdong, LI Xin, CHEN Zhiyong, et al. Calibration and rectification of fisheye images based on three-dimensional control field[J]. Acta Optica Sinica, 2017, 37(1):0115001.
[25] 武汉测绘科技大学测量平差教研室. 测量平差基础[M]. 3版. 北京:测绘出版社, 1996. Department of surveying adjustment, Wuhan University of Surveying and Mapping Science and technology. Fundamentals of Surveying Adjustment[M]. 3ed. Beijing:Surveying and Mapping Press, 1996.
[26] KAEHLER A,BRADSKI G.Learning OpenCV 3:computer vision in C++ with the OpenCV library[M].Sebastopol,CA,USA:O'Reilly Media,2017,652-659.
[27] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2):91-110.
[28] 秦绪佳, 王琪, 王慧玲, 等. 基于最佳缝合线的序列遥感图像拼接融合方法[J]. 计算机科学, 2015, 42(10):306-310. QIN Xujia, WANG Qi, WANG Huiling, et al. Image fusion method based on best seam-line for serial remote sensing images mosaic[J]. Computer Science, 2015, 42(10):306-310.
[29] 陈辉, 龙爱群, 彭玉华. 由未标定手持相机拍摄的图片构造全景图[J]. 计算机学报, 2009, 32(2):328-335. CHEN Hui, LONG Aiqun, PENG Yuhua. Building panoramas from photographs taken with an uncalibrated hand-held camera[J]. Chinese Journal of Computers, 2009, 32(2):328-335.