High-precision extraction of building point cloud facade structure based on PCF-Net network

doi:10.11947/j.AGCS.2025.20240225

Abstract

Abstract:

As the application and promotion of digital twin cities and realistic three-dimensional construction progresses, urban high-precision modeling based on 3D point clouds has become one of the important tasks. Building fa?ade structures can serve as prior knowledge to assist in the rapid construction of high-accuracy 3D urban models. Therefore, exploring how to accurately extract building fa?ade structures from point cloud data is a research focus in detailed modeling. Currently, methods based on deep learning can use neural networks to understand complex building fa?ades, but the extraction accuracy for less common structures in fa?ades (such as doors, external air conditioning units) is still not high enough. To address this issue, this paper develops a novel deep learning neural network, position color fusion-net (PCF-Net), focusing on the extraction of small-sample structures in building fa?ades across three aspects: point cloud sampling, feature extraction, and loss function. Initially, during the point cloud sampling process, the proportion of small-sample structure point clouds is increased by attaching weights. Subsequently, a dual-branch network is used to extract spatial features from the colored point clouds and texture features, with an attention mechanism applied to adaptively fuse these two types of features, enhancing the description of key details in building fa?ades. Finally, a loss function that considers both intersection over union (IoU) and extraction accuracy (Acc) constraints is designed to improve the completeness and precision of building fa?ade structure extraction. Experiments show that the proposed PCF-Net network achieves precision metrics of 97.99% OA, 97.80% mAcc, and 95.75% mIoU in extracting fine structures from various types of building fa?ades, demonstrating the network's superior performance in building fa?ade structure extraction (Project address: https://github.com/zangyufus/PCF_net.git).

Key words: 3D modeling, semantic parsing of building facades, point cloud sampling, dual-branch network, attention fusion, loss function

CLC Number:

P237

Yufu ZANG, Shuye WANG, Zhen DONG, Chi CHEN, Ronggang HUANG. High-precision extraction of building point cloud facade structure based on PCF-Net network[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(7): 1243-1253.

Figures/Tables 12

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Tab. 1

Fig. 6

Tab. 2

Fig. 7

Tab. 3

Fig. 8

Tab. 4

References 29

[1]	杨必胜, 陈驰, 董震. 面向智能化测绘的城市地物三维提取[J]. 测绘学报, 2022, 51(7): 1476-1484. DOI: . doi: 10.11947/j.AGCS.2022.20220183
	YANG Bisheng, CHEN Chi, DONG Zhen. 3D geospatial information extraction of urban objects for smart surveying and mapping[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1476-1484. DOI: . doi: 10.11947/j.AGCS.2022.20220183
[2]	CHEN Chi, JIN Ang, WANG Zhiye, et al. SGSR-net: structure semantics guided LiDAR super-resolution network for indoor LiDAR SLAM[J]. IEEE Transactions on Multimedia, 2024, 26: 1842-1854.
[3]	WANG Wei, XU Yuan, REN Yingchao, et al. Parsing of urban facades from 3D point clouds based on a novel multi-view domain[J]. Photogrammetric Engineering & Remote Sensing, 2021, 87(4): 283-293.
[4]	蒋腾平, 王永君, 张林淇, 等. 融合CNN和MRF的激光点云层次化语义分割方法[J]. 测绘学报, 2021, 50(2): 215-225. DOI: . doi: 10.11947/j.AGCS.2021.20200095
	JIANG Tengping, WANG Yongjun, ZHANG Linqi, et al. A LiDAR point cloud hierarchical semantic segmentation method combining CNN and MRF[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 215-225. DOI: . doi: 10.11947/j.AGCS.2021.20200095
[5]	XIE Yuxing, TIAN Jiaojiao, ZHU Xiaoxiang. Linking points with labels in 3D: a review of point cloud semantic segmentation[J]. IEEE Geoscience and remote sensing magazine, 2020, 8(4): 38-59.
[6]	张瑞菊, 郑少开, 尹建英. 建筑物立面点云语义分割方法研究[J]. 城市勘测, 2017(5): 22-25.
	ZHANG Ruiju, ZHENG Shaokai, YIN Jianying. Building facade point cloud semantic segmentation method research[J]. Urban Geotechnical Investigation & Surveying, 2017(5): 22-25.
[7]	费鲜芸, 戴哲, 刘如飞. 三维激光点云中建筑物立面边界特征提取方法[J]. 山东科技大学学报(自然科学版), 2022, 41(2): 21-30.
	FEI Xianyun, DAI Zhe, LIU Rufei. Boundary feature extraction method of building facade in 3D laser point cloud[J]. Journal of Shandong University of Science and Technology (Natural Science), 2022, 41(2): 21-30.
[8]	HU Zengtian, CHEN Chi, YANG Bisheng, et al. Geometric feature enhanced line segment extraction from large-scale point clouds with hierarchical topological optimization[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 112(1): 102858-102868.
[9]	邹纪伟, 刘德儿, 杨鹏, 等. 建筑立面点云对偶深度图像中窗户提取方法[J]. 激光与红外, 2020, 50(7): 875-881.
	ZOU Jiwei, LIU Deer, YANG Peng, et al. Window extraction method in dual depth image of point cloud of building facade[J]. Laser & Infrared, 2020, 50(7): 875-881.
[10]	ZHAO Fuqun, HUANG He, XIAO Nana, et al. A point cloud segmentation algorithm based on multi-feature training and weighted random forest[J]. Measurement Science and Technology, 2024, 36(1): 015407.
[11]	NI Huan, LIN Xiangguo, ZHANG Jixian. Classification of ALS point cloud with improved point cloud segmentation and random forests[J]. Remote Sensing, 2017, 9(3): 288-296.
[12]	CHEN Chao, LI Xiaomin, BELKACEM A N, et al. The mixed kernel function SVM-based point cloud classification[J]. International Journal of Precision Engineering and Manufacturing, 2019, 20: 737-747.
[13]	王丽英, 吴际, 有泽, 等. 多维GMM与邻域约束的多光谱机载LiDAR数据城市地物分类. 测绘学报, 2023, 52(3), 419-431. DOI: . doi: 10.11947/j.AGCS.2023.20210153
	WANG Liying, WU Ji, YOU Ze, et al. Urban object classification of multispectral airborne LiDAR data with multi-dimensional Gauss mixture model and neighborhood constraints[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(3): 419-431. DOI: . doi: 10.11947/j.AGCS.2023.20210153
[14]	BALLOUCH Z, HAJJI R, POUX F, et al. A prior level fusion approach for the semantic segmentation of 3D point clouds using deep learning[J]. Remote Sensing, 2022, 14(14): 3415. DOI: . doi: doi.org/10.3390/rs14143415
[15]	DAI Menglin, WARD W O C, MEYERS G, et al. Residential building facade segmentation in the urban environment[J]. Building and Environment, 2021, 199: 107921.
[16]	WANG Puzuo, YAO Wei. A new weakly supervised approach for ALS point cloud semantic segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188: 237-254. DOI: https://doi.org/10.1016/j.isprsjprs.2022.04.016.
[17]	CHEN Dong, XIANG Guiqiu, PEETHAMBARAN J, et al. AFGL-net: attentive fusion of global and local deep features for building façades parsing[J]. Remote Sensing, 2021, 13(24): 5039.
[18]	HU Qingyong, YANG Bo, XIE Linkai, et al. RanDLA-net: efficient semantic segmentation of large-scale point clouds[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. [EB/OL]: IEEE, 2020: 11108-11117.
[19]	SU Yanfei, LIU Weiquan, YUAN Zhimin, et al. DLA-Net: learning dual local attention features for semantic segmentation of large-scale building facade point clouds[J]. Pattern Recognition, 2022, 123: 108372.
[20]	ZHANG Xinlong, LIN Dong, SOERGEL U. Target-aware attentional network for rare class segmentation in large-scale LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2025, 220(1): 32-50. DOI: https://doi.org/10.1016/j.isprsjprs.2024.11.012.
[21]	方莉娜, 沈贵熙, 游志龙, 等. 融合点云和多视图的车载激光点云路侧多目标识别[J]. 测绘学报, 2021, 50(11): 1558-1573. DOI: . doi: 10.11947/j.AGCS.2021.20210246
	FANG Lina, SHEN Guixi, YOU Zhi, et al. A joint network of point cloud and multiple views for recognition from mobile laser point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1558-1573. DOI: . doi: 10.11947/j.AGCS.2021.20210246
[22]	ZANG Yufu, WANG Shuye, GUAN Haiyan, et al. VAM-Net: vegetation-attentive deep network for multi-modal fusion of visible-light and vegetation-sensitive images[J]. International Journal of Applied Earth Observation and Geoinformation, 2024, 127: 103642.
[23]	YU Bing, HU Jinlong, DONG Xiujun, et al. A robust automatic method to extract building facade maps from 3D point cloud data[J]. Remote Sensing, 2022, 14(16): 3848.
[24]	WANG Y, SUN Y, LIU Z, et al. Dynamic graph cnn for learning on point clouds[J]. ACM Transactions on Graphics (tog), 2019, 38(5), 1-12. DOI: . doi: 10.48550/arXiv.1801.07829
[25]	MORBIDONI C, PIERDICCA R, QUATTRINI R, et al. Graph cnn with radius distance for semantic segmentation of historical buildings TLS point clouds[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, XLIV-4/W1-2020: 95-102.
[26]	陈科, 管海燕, 雷相达, 等. 基于特征增强核点卷积网络的多光谱LiDAR点云分类方法[J]. 地球信息科学学报, 2023, 25(5): 1075-1087.
	CHEN Ke, GUAN Haiyan, LEI Xiangda, et al. A multispectral LiDAR point cloud classification method based on enhanced features kernel point convolutional network[J]. Journal of Geo-information Science, 2023, 25(5): 1075-1087.
[27]	高建文, 管海燕, 彭代锋, 等. 基于局部-全局语义特征增强的遥感影像变化检测网络模型[J]. 地球信息科学学报, 2023, 25(3): 625-637.
	GAO Jianwen, GUAN Haiyan, PENG Daifeng, et al. Local-global semantic feature enhancement model for remote sensing imagery change detection[J]. Journal of Geo-information Science, 2023, 25(3): 625-637.
[28]	QI C R, SU H, MO K, et al. Pointnet: deep learning on point sets for 3D classification and segmentation[J]. Proceedings of the IEEE Conference on Computer Vision and Pttern Recognition, 2017(1): 652-660.
[29]	QI C R, YI Li, SU Hao, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. [2024-10-16]. https://arxiv.org/abs/1706.02413v1.

墙面	点数	立面结构	墙面特点
1	215 465	墙、门、窗户	点云缺失、墙面不完整
2	287 161	墙、门、窗户、空调外机	多样结构，墙面完整
3	297 361	墙、窗户	墙面完整，结构简单
4	221 184	墙、阳台	点云缺失，结构不完整
5	445 910	墙、窗户、门	结构复杂规整，墙面完整

墙面	OA	mAcc	IoU					mIoU
墙面	OA	mAcc	窗户	墙	阳台	空调外机	门	mIoU
1	94.43	94.17	89.38	90.22	—	87.29	—	88.96
2	98.78	95.80	—	98.41	95.21	—	—	97.41
3	95.39	91.20	98.19	98.24	—	—	90.07	95.50
4	98.59	93.22	—	98.16	95.78	—	92.64	96.66
5	99.03	98.90	98.67	97.72	—	—	91.80	96.53

方法	OA	mAcc	IoU					mIoU
方法	OA	mAcc	窗户	墙	阳台	空调外机	门	mIoU
PointNet^［28］	61.25	48.26	69.80	55.60	39.40	6.80	33.40	41.00
PointNet++^［29］	78.58	70.97	86.40	79.70	59.70	5.10	47.90	55.76
R-DGCNN^［25］	97.31	92.25	94.41	96.39	93.35	88.55	70.30	88.60
DLA-Net^［19］	98.72	96.56	95.65	97.51	91.60	87.96	71.13	88.97
PCF-Net（本文方法）	97.99	97.80	92.75	96.88	95.41	95.72	97.99	95.75

方案	Color MLP	点云权重采样	Lovsz-Softmax	加权交叉熵	IoU					mIoU
方案	Color MLP	点云权重采样	Lovsz-Softmax	加权交叉熵	窗户	墙	阳台	空调外机	门	mIoU
1	—	—	—	—	94.21	96.19	93.15	88.35	70.10	87.60
2	√	—	—	—	93.64	95.34	92.33	89.06	73.17	88.71
3	√	√	—	—	93.78	96.72	93.56	90.32	76.52	89.51
4	√	√	√	—	94.80	96.15	92.57	95.49	80.39	91.88
5	√	√	√	√	92.75	96.88	95.41	95.72	97.99	95.75