Heterogeneous building point cloud registration via deformable self-attention and density-invariant keypoint detection

doi:10.11947/j.AGCS.2026.20260177

Abstract

Abstract:

To address the problems of insufficient registration accuracy and poor robustness caused by significant differences in acquisition modality, point cloud density, viewing perspective, and spatial coverage between terrestrial laser scanning and unmanned aerial vehicle (UAV) oblique photogrammetry point clouds, a heterogeneous point cloud registration method based on deformable self-attention and density-invariant keypoint detection is proposed. In the feature extraction stage, a spatial deformable self-attention mechanism is introduced to dynamically adjust the receptive field, enabling collaborative modeling of global contextual information and local geometric features, thereby enhancing the discriminative capability of feature representations. In the keypoint detection stage, a density-invariant saliency scoring strategy is designed to effectively suppress the influence of point cloud density variations on keypoint selection, achieving stable keypoint extraction across point clouds with different densities. In the matching stage, an edge-relaxation mechanism is incorporated to optimize the keypoint similarity matrix, reducing mismatches caused by non-overlapping regions, while the rigid transformation is estimated using the RANSAC algorithm. Experimental results demonstrate that the proposed method achieves a feature matching recall of 98.3% and a registration recall of 91.6% on the 3DMatch dataset. On the low-overlap 3DLoMatch dataset, it attains a registration recall of 75.9%, outperforming GeoTransformer by 2.9 percentage points and Predator by 17.8 percentage points. Furthermore, experiments conducted on a self-constructed heterogeneous building scanning dataset demonstrate that the proposed method exhibits robust performance under significant density variations and low-overlap conditions, validating its effectiveness and applicability for practical three-dimensional building reconstruction.

Key words: heterogeneous point clouds, point cloud registration, deformable self-attention, density invariance, building point clouds

CLC Number:

P237

Sihan Liu, Shiwei Shao, Yuefeng Lu, Ziqi Ding, Chunlin Mu, Xiaolei Dou, Chengzhe Lü, Zhirui Jia, Zhenjie Jiao. Heterogeneous building point cloud registration via deformable self-attention and density-invariant keypoint detection[J]. Acta Geodaetica et Cartographica Sinica, 2026, 55(9): 1630-1640.

Add to citation manager EndNote|Reference Manager|ProCite|BibTeX|RefWorks

URL: http://xb.chinasmp.com/EN/10.11947/j.AGCS.2026.20260177

http://xb.chinasmp.com/EN/Y2026/V55/I9/1630

Figures/Tables 8

Fig. 1

Fig. 2

Fig. 3

Tab. 1

Fig. 4

Fig. 5

Tab. 2

Registration errors of heterogeneous building point clouds"

建筑物	方法	各向同性误差		各向异性误差
建筑物	方法	R	t/m	R（x，y，z）	t（x，y，z）/m
1	ICP	178.87°	5.433 6	10.40°，1.51°，176.76°	4.938 7，2.241 8，0.327 0
	D3Feat	0.81°	0.032 1	0.11°，0.69°，0.44°	0.006 3，0.030 8，0.006 6
	Predator	0.46°	0.035 8	0.31°，0.23°，0.32°	0.000 9，0.035 6，0.003 8
	本文方法	0.18°	0.015 1	0.10°，0.13°，0.09°	0.006 4，0.011 9，0.006 7
2	ICP	178.27°	4.160 0	174.85°，36.21°，32.11°	2.679 7，3.110 4，0.671 5
	D3Feat	11.65°	0.567 3	11.62°，1.71°，1.15°	0.032 5，0.039 7，0.565 0
	Predator	0.46°	0.010 3	0.30°，0.35°，0.08°	0.001 3，0.000 6，0.010 2
	本文方法	1.15°	0.051 1	0.33°，0.55°，1.00°	0.041 5，0.005 9，0.029 3
3	ICP	179.99°	2.224 1	0.30°，0.34°，179.86°	1.889 3，1.170 2，0.089 6
	D3Feat	1.44°	0.121 2	1.34°，0.51°，0.03°	0.067 2，0.100 3，0.010 3
	Predator	0.76°	0.063 3	0.00°，0.30°，0.69°	0.006 0，0.062 6，0.007 2
	本文方法	0.29°	0.036 7	0.13°，0.27°，0.02°	0.028 8，0.016 3，0.015 8
4	ICP	175.70°	5.576 7	0.61°，2.68°，175.62°	4.420 2，3.394 9，0.188 5
	D3Feat	13.09°	0.876 5	0.42°，12.52°，3.88°	0.464 3，0.077 7，0.739 3
	Predator	1.85°	0.126 3	1.30°，1.31°，0.34°	0.017 5，0.059 5，0.110 0
	本文方法	1.23°	0.068 4	0.93°，0.54°，0.70°	0.000 1，0.041 0，0.054 8
5	ICP	89.23°	5.160 1	0.12°，1.67°，89.15°	1.083 2，5.042 3，0.169 3
	D3Feat	25.90°	5.198 9	21.93°，7.67°，28.88°	5.143 8，0.250 8，0.711 6
	Predator	0.94°	0.105 6	0.36°，0.86°，0.11°	0.002 5，0.105 5，0.003 6
	本文方法	0.61°	0.042 4	0.26°，0.54°，0.04°	0.001 2，0.042 1，0.004 5
6	ICP	179.19°	4.915 3	22.43°，61.71°，174.72°	4.218 8，2.483 5，0.441 0
	D3Feat	1.19°	0.013 4	1.02°，0.65°，0.68°	0.011 6，0.004 6，0.004 9
	Predator	0.99°	0.050 4	0.66°，0.74°，0.42°	0.048 1，0.005 5，0.013 9
	本文方法	0.52°	0.017 0	0.20°，0.03°，0.41°	0.010 7，0.010 7，0.007 9
7	ICP	124.40°	3.759 6	121.09°，32.56°，22.07°	0.957 1，1.492 6，3.315 3
	D3Feat	1.88°	0.093 4	1.09°，1.26°，0.64°	0.002 1，0.029 6，0.088 5
	Predator	0.96°	0.036 8	0.84°，0.51°，0.14°	0.002 8，0.003 8，0.036 5
	本文方法	1.57°	0.033 5	1.04°，1.10°，0.76°	0.010 8，0.003 4，0.031 5

Tab. 2

Tab. 3

References 33

[1]	程昫, 葛亮, 张帆, 等. 倾斜摄影建筑物模型自动纹理重建[J]. 测绘学报, 2023, 52(9): 1528-1537. DOI: . doi: 10.11947/j.AGCS.2023.20220165
	Cheng Xu, Ge Liang, Zhang Fan, et al. Automated texture mapping of reality CSG building model with oblique aerial imagery[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(9): 1528-1537. DOI: . doi: 10.11947/j.AGCS.2023.20220165
[2]	季顺平, 刘瑾, 高建, 等. 多视影像深度学习密集匹配三维重建智能框架[J]. 测绘学报, 2025, 54(9): 1633-1646. DOI: . doi: 10.11947/j.AGCS.2025.20230306
	Ji Shunping, Liu Jin, Gao Jian, et al. An intelligent 3D reconstruction framework via deep learning based multi-view image matching[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(9): 1633-1646. DOI: . doi: 10.11947/j.AGCS.2025.20230306
[3]	谭金石, 祖为国, 刘丽. 激光扫描与倾斜摄影空地协同的建筑物三维模型构建方法[J]. 应用激光, 2023, 43(7): 107-115.
	Tan Jinshi, Zu Weiguo, Liu Li. 3D construction method of buildings with laser scanning and oblique photography air-ground collaborating[J]. Applied Laser, 2023, 43(7): 107-115.
[4]	Aiger D, Mitra N J, Cohen-Or D.4-points congruent sets for robust pairwise surface registration[J]. ACM Transactions on Graphics, 2008, 27(3): 1-10.
[5]	Fotsing C, Nziengam N, Bobda C. Large common plansets-4-points congruent sets for point cloud registration[J]. ISPRS International Journal of Geo-Information, 2020, 9(11): 647.
[6]	Xu Yusheng, Boerner Richard, Yao Wei, et al. Pairwise coarse registration of point clouds in urban scenes using voxel-based 4-planes congruent sets[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 151: 106-123.
[7]	Biber P, Strasser W. The normal distributions transform: a new approach to laser scan matching[C]//Proceedings of 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems. Las Vegas: IEEE, 2003: 2743-2748.
[8]	Magnusson M, Lilienthal A, Duckett T. Scan registration for autonomous mining vehicles using 3D-NDT[J]. Journal of Field Robotics, 2007, 24(10): 803-827.
[9]	Lu Jun, Liu Wei, Dong Donglai, et al. Point cloud registration algorithm based on NDT with variable size voxel[C]//Proceedings of the 34th Chinese Control Conference. Hangzhou: IEEE, 2015: 3707-3712.
[10]	闫利, 谭骏祥, 刘华, 等. 融合遗传算法和ICP的地面与车载激光点云配准[J]. 测绘学报, 2018, 47(4): 528-536. DOI: . doi: 10.11947/j.AGCS.2018.20170235
	Yan Li, Tan Junxiang, Liu Hua, et al. Registration of TLS and MLS point cloud combining genetic algorithm with ICP[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(4): 528-536. DOI: . doi: 10.11947/j.AGCS.2018.20170235
[11]	Chen Yang, Medioni G. Object modelling by registration of multiple range images[J]. Image and Vision Computing, 1992, 10(3): 145-155.
[12]	Besl P J, McKay N D. Method for registration of 3-D shapes[C]//Proceedings of 1992 Sensor Fusion IV: Control Paradigms and Data Structures. Boston: SPIE, 1992: 586-606.
[13]	Lu Weixin, Wan Guowei, Zhou Yao, et al. DeepVCP: an end-to-end deep neural network for point cloud registration[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 12-21.
[14]	Qi Charleš R, Yi Li, Su Hao, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[J]. Advances in Neural Information Processing Systems, 2017, 30: 5099-5108.
[15]	Aoki Y, Goforth H, Srivatsan R A, et al. PointNetLK: robust & efficient point cloud registration using PointNet[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 7156-7165.
[16]	Qi Charles R, Su Hao, Mo Kaichun, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 652-660.
[17]	Wang Yue, Solomon Justin M. Deep closest point: learning representations for point cloud registration[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul: IEEE, 2019: 3522-3531.
[18]	Wang Yue, Sun Yongbin, Liu Ziwei, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5): 1-12.
[19]	Yew Z J, Lee G H. RPM-net: robust point matching using learned features[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11821-11830.
[20]	Huang Shengyu, Gojcic Zan, Usvyatsov Mikhail, et al. Predator: registration of 3D point clouds with low overlap[C]//Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2021: 4265-4274.
[21]	Qin Zheng, Yu Hao, Wang Changjian, et al. GeoTransformer: fast and robust point cloud registration with geometric transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(8): 9806-9821.
[22]	孔煜, 熊风光, 张志强, 等. 基于深度位置感知Transformer的低重叠点云配准[J]. 计算机科学, 2025, 52(5): 199-211.
	Kong Yu, Xiong Fengguang, Zhang Zhiqiang, et al. Low overlap point cloud registration method based on deep position-aware Transformer[J]. Computer Science, 2025, 52(5): 199-211.
[23]	Xia Zhuofan, Pan Xuran, Song Shiji, et al. Vision transformer with deformable attention[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 4784-4793.
[24]	Khan S, Naseer M, Hayat M, et al. Transformers in vision: a survey[J]. ACM Computing Surveys, 2022, 54(10s): 1-41.
[25]	Xiong Fengguang, Kong Yu, Xie Shuaikang, et al. Spatial deformable transformer for 3D point cloud registration[J]. Scientific Reports, 2024, 14: 5560.
[26]	Bai Xuyang, Luo Zixin, Zhou Lei, et al. D3Feat: joint learning of dense detection and description of 3D local features[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 6358-6366.
[27]	Zeng Andy, Song Shuran, Niebner Matthias, et al. 3DMatch: learning local geometric descriptors from RGB-D reconstructions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1802-1811.
[28]	Liu Ze, Hu Han, Lin Yutong, et al. Swin transformer V2: scaling up capacity and resolution[C]//Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. New Orleans: IEEE, 2022: 12009-12019.
[29]	Fu Kexue, Yuan Mingzhi, Wang Changwei, et al. Dual focus-attention transformer for robust point cloud registration[C]//Proceedings of 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Nashville: IEEE, 2025: 11769-11778.
[30]	杜佳锦, 柏正尧, 刘旭珩, 等. 融合几何注意力和多尺度特征点云配准网络[J]. 计算机工程与应用, 2024, 60(12): 234-244.
	Du Jiajin, Bai Zhengyao, Liu Xuheng, et al. Fusion of geometric attention and multi-scale feature network for point cloud registration[J]. Computer Engineering and Applications, 2024, 60(12): 234-244.
[31]	Liu Quan, Zhu Hongzi, Zhou Yunsong, et al. Density-invariant features for distant point cloud registration[C]//Proceedings of 2023 IEEE/CVF International Conference on Computer Vision. Paris: IEEE, 2023: 18169-18179.
[32]	Gojcic Zan, Zhou Caifa, Wegner Jan D, et al. The perfect match: 3D point cloud matching with smoothed densities[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 5540-5549.
[33]	Yu Hao, Li Fu, Saleh Mahdi, et al. CoFiNet: reliable coarse-to-fine correspondences for robust point cloud registration[C]//Proceedings of 2021 Advances in Neural Information Processing Systems. Virtual: Curran Associates, Inc., 2021: 23872-23884.

方法	FMR		RR		IR
方法	3DMatch	3DLoMatch	3DMatch	3DLoMatch	3DMatch	3DLoMatch
PerfectMatch	82.9	34.2	50.8	11.0	16.4	4.8
D3Feat	93.1	66.5	77.9	39.1	41.8	15.0
CoFiNet	98.3	82.6	87.0	61.0	52.2	26.9
Predator	96.5	75.3	86.6	58.1	49.3	25.8
GeoTransformer	97.8	88.7	91.1	73.0	85.7	58.3
本文方法	98.3	89.1	91.6	75.9	77.9	49.7

方法	3DLoMatch RR/（%）	自建异源建筑数据集
方法	3DLoMatch RR/（%）	R/（°）	t/m
DSA-DI w/o可变形自注意力模块	71.6	0.92	0.044
DSA-DI w/o密度不变模块	69.2	1.15	0.058
DSA-DI w/o边缘松弛模块	72.3	0.87	0.043
DSA-DI（完整模型）	75.9	0.75	0.036