Registration of aerial images and LiDAR point clouds based on distance field and plane constraints

doi:10.11947/j.AGCS.2025.20240122

Abstract

Abstract:

In the field of Earth observation, airborne optical images and airborne light detection and ranging (LiDAR) point clouds are the main data sources for acquiring geo-spatial information. Accurate geometric registration is the prerequisite for the fusion of the two sources of data. In this paper, a modified registration method for airborne LiDAR point clouds and aerial images based on distance field and plane constraints is proposed. This method is divided into two stages: single image registration based on line distance field and bundle block adjustment based on line and plane constraints. In single image registration, line features are extracted from aerial image and airborne LiDAR point cloud respectively, and then distance field is constructed based on the line features of aerial image, and the point cloud line features are projected to the image plane. The global cost of point cloud projection line features in distance field is minimized by progressive robust solution, so that single image can be registered with LiDAR point cloud. In the bundle block adjustment, key frames are selected based on the density of line feature distribution. Subsequently, the features of the conjugate line in key frames are matched to extract the control points as horizontal and elevation constraints. Moreover, the distance between the image tie point and the nearest horizontal plane is used as elevation constraints. The experimental results show that the registration accuracy of the proposed method is close to the average point distance. The proposed method can realize robust registration under the condition of poor initial values, and the registration accuracy is significantly superior to the iterative closest point (ICP) registration method and the registration strategy of cross-modal matching.

Key words: aerial images, LiDAR point clouds, registration, distance field, plane constraints

CLC Number:

P231

Yongjun ZHANG, Changjun ZHU, Siyuan ZOU, Xinyi LIU, Qingzhou MAO, Yi WAN. Registration of aerial images and LiDAR point clouds based on distance field and plane constraints[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(1): 64-74.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Tab. 1

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Tab. 2

Fig. 9

Tab. 3

Fig. 10

References 33

[1]	ZHU Bai, YE Yuanxin, ZHOU Liang, et al. Robust registration of aerial images and LiDAR data using spatial constraints and Gabor structural features[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181: 129-147.
[2]	JONASSEN V O, KJØRSVIK N S, GJEVESTAD J G O. Scalable hybrid adjustment of images and LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 202: 652-662.
[3]	ZHANG Yongjun, XIONG Xiaodong, WANG Mengqiu, et al. A fast aerial image matching method using airborne LiDAR point cloud and POS data[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(1): 26-36.
[4]	ZHANG Yongjun, ZOU Siyuan, LIU Xinyi, et al. LiDAR-guided stereo matching with a spatial consistency constraint[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 183: 164-177.
[5]	ZOU Siyuan, LIU Xinyi, HUANG Xu, et al. Edge-preserving stereo matching using LiDAR points and image line features[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 3239030.
[6]	ZHANG Lei, LI Xu, TANG Kaichen, et al. FS-Net: LiDAR-camera fusion with matched scale for 3D object detection in autonomous driving[J]. IEEE Transactions on Intelligent Transportation Systems, 2023, 24(11): 12154-12165.
[7]	ZHAO Lin, ZHOU Hui, ZHU Xinge, et al. LIF-seg: LiDAR and camera image fusion for 3D LiDAR semantic segmentation[J]. IEEE Transactions on Multimedia, 2024, 26: 1158-1168.
[8]	LI X, ZHANG X, NIU X, et al. Progress and achievements of multi-sensor fusion navigation in China during 2019—2023[J]. Journal of Geodesy and Geoinformation Science, 2023, 6(3): 102-114.
[9]	BALIC N, WEINACKER H, KOCH B. Generation of mosaiced digital true orthophotographs from multispectral and LiDAR data[J]. International Journal of Remote Sensing, 2007, 28(16): 3679-3688.
[10]	陈首彬. 激光LiDAR/视觉融合的SLAM（LV-SLAM）关键技术研究[J]. 测绘学报, 2023, 52(1): 169. DOI:. doi: 10.11947/j.AGCS.2023.20210184
	CHEN Shoubin. Research on SLAM based on LiDAR/visual fusion (LV-SLAM)[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(1): 169. DOI:. doi: 10.11947/j.AGCS.2023.20210184
[11]	REN Siyu, ZENG Yiming, HOU Junhui, et al. CorrI2P: deep image-to-point cloud registration via dense correspondence[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2023, 33(3): 1198-1208.
[12]	WANG Yanjun, LI Shaochun, WANG Mengjie, et al. A simple deep learning network for classification of 3D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3): 49-59.
[13]	LI Jiaxin, HEE LEE G. DeepI2P: image-to-point cloud registration via deep classification[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021.
[14]	ESLAMI M, SAADATSERESHT M. A new tie plane-based method for fine registration of imagery and point cloud dataset[J]. Canadian Journal of Remote Sensing, 2020, 46(3): 295-312.
[15]	YANG Bisheng, CHEN Chi. Automatic registration of UAV-borne sequent images and LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 101: 262-274.
[16]	PANDEY G, MCBRIDE J R, SAVARESE S, et al. Automatic extrinsic calibration of vision and LiDAR by maximizing mutual information[J]. Journal of Field Robotics, 2015, 32(5): 696-722.
[17]	SUN Shaohui, SAVALGGIO C. Complex building roof detection and strict description from LiDAR data and orthorectified aerial imagery[C]//Proceedings of 2012 IEEE International Geoscience and Remote Sensing Symposium. Munich: IEEE, 2012.
[18]	TEO T A, HUANG S H. Automatic co-registration of optical satellite images and airborne LiDAR data using relative and absolute orientations[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2013, 6(5): 2229-2237.
[19]	NAIMAEE R, SAADATSERESHT M, OMIDALIZARANDI M. Automatic extraction of control points from 3D LiDAR mobile mapping and UAV imagery for aerial triangulation[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2023,: 581-588.
[20]	BARSAI G, YILMAZ A, NAGARAJAN S, et al. Registration of images to LiDAR and GIS data without establishing explicit correspondences[J]. Photogrammetric Engineering & Remote Sensing, 2017, 83(10): 705-716.
[21]	ZITOVÁ B, FLUSSER J. Image registration methods: a survey[J]. Image and Vision Computing, 2003, 21(11): 977-1000.
[22]	WONG A, ORCHARD J. Efficient FFT-accelerated approach to invariant optical-LiDAR registration[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008, 46(11): 3917-3925.
[23]	MASTIN A, KEPNER J, FISHER J. Automatic registration of LIDAR and optical images of urban scenes[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami: IEEE, 2009.
[24]	PARMEHR E G, FRASER C S, ZHANG C, et al. Automatic registration of optical imagery with 3D LiDAR data using statistical similarity[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014, 88: 28-40.
[25]	BESL P J, MCKAY N D. Method for registration of 3D shapes[C]//Proceedings of 1992 Sensor Fusion IV: Control Paradigms and Data Structures. Boston: SPIE, 1992.
[26]	LI Jiayuan, HU Qingwu, AI Mingyao. RIFT: multi-modal image matching based on radiation-variation insensitive feature transform[J]. IEEE Transactions on Image Processing, 2020, 29: 3296-3310.
[27]	刘伟玉, 万一, 张永军, 等. 基于相位均匀卷积的LiDAR深度图与航空影像高效匹配方法[J]. 武汉大学学报(信息科学版), 2022, 47(8): 1309-1317.
	LIU Weiyu, WAN Yi, ZHANG Yongjun, et al. An efficient matching method of LiDAR depth map and aerial image based on phase mean convolution[J]. Geomatics and Information Science of Wuhan University, 2022, 47(8): 1309-1317.
[28]	张永军, 洪玮辰, 万一. 利用距离变换模型进行卫星影像与激光点云精配准[J]. 武汉大学学报(信息科学版), 2023, 48(3): 339-348.
	ZHANG Yongjun, HONG Weichen, WAN Yi. Precise registration of satellite images and laser point clouds using distance transformation model[J]. Geomatics and Information Science of Wuhan University, 2023, 48(3): 339-348.
[29]	WAN Y, ZHANG Y, WANG G, et al. Accurate registration of aerial images and als-point cloud via automated junction matching and planar constraints[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020(2): 79-86.
[30]	HUANG Rongyong, ZHENG Shunyi, HU Kun. Registration of aerial optical images with LiDAR data using the closest point principle and collinearity equations[J]. Sensors, 2018, 18(6): 1770.
[31]	VON GIOI R G, JAKUBOWICZ J, MOREL J M, et al. LSD: a fast line segment detector with a false detection control[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 32(4): 722-732.
[32]	张永军, 熊小东, 沈翔. 城区机载LiDAR数据与航空影像的自动配准[J]. 遥感学报, 2012, 16(3): 579-595.
	ZHANG Yongjun, XIONG Xiaodong, SHEN Xiang. Automatic registration of urban aerial imagery with airborne LiDAR data[J]. Journal of Remote Sensing, 2012, 16(3): 579-595.
[33]	LI J, ZHANG Y, HU Q. Robust estimation in robot vision and photogrammetry: a new model and its applications[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021, 51(1): 137-144.

数据集	数据集Ⅰ	数据集Ⅱ
测区位置	浙江省宁波市	武汉大学校园
影像类型	航空多视倾斜影像	航空下视影像
影像数量	1188	50
相机数量	5	1
航高/m	900	250
影像地面分辨率/m	0.051	0.067
LiDAR点云平均点间距/m	0.28	0.20

配准状态	d_XY/m	d_Z/m
配准前	0.75	0.98
DFPC法配准后	0.16	0.19

配准方法	d_XY/m	d_Z/m	配准用时/min	存储开销
HPMEC	0.31	1.04	137	11.9 GB
ICP	0.27	0.32	63	6.2 GB
DFPC	0.06	0.16	12	37 MB