基于PCF-Net网络的建筑点云立面结构高精度提取

doi:10.11947/j.AGCS.2025.20240225

测绘学报 ›› 2025, Vol. 54 ›› Issue (7): 1243-1253.doi: 10.11947/j.AGCS.2025.20240225

基于PCF-Net网络的建筑点云立面结构高精度提取

臧玉府¹^,²(), 王树野¹, 董震³, 陈驰³, 黄荣刚⁴

^1.南京信息工程大学遥感与测绘工程学院，江苏　南京　210044
^2.自然资源部遥感导航一体化应用工程技术创新中心，江苏　南京　210044
^3.武汉大学精密测量科学与技术创新研究院精密大地测量与定位全国重点实验室，湖北　武汉　430079
^4.中国科学院测量与地球物理研究所大地测量与地球动力学国家重点实验室，湖北　武汉　430077

收稿日期:2024-05-23 修回日期:2025-06-19 出版日期:2025-08-18 发布日期:2025-08-18
作者简介:臧玉府（1987—），男，博士，副教授，研究方向为点云智能处理。E-mail：002767@nuist.edu.cn
基金资助:
国家自然科学基金(42171433);中国科学院精密测量科学与技术创新研究院精密大地测量与定位全国重点实验室开放基金(SKLPG2025-5-4);江苏省研究生科研与实践创新计划(KYCX22_1213)

High-precision extraction of building point cloud facade structure based on PCF-Net network

Yufu ZANG¹^,²(), Shuye WANG¹, Zhen DONG³, Chi CHEN³, Ronggang HUANG⁴

^1.School of Remote Sensing & Geomatics Engineering, Nanjing University of Information Science & Technology, Nanjing 210044, China
^2.Technology Innovation Center for Integrated Applications in Remote Sensing and Navigation, Ministry of Natural Resources, Nanjing 210044, China
^3.State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
^4.State Key Laboratory of Precision Geodesy, Innovation Academy for Precision Measurement Science and Technology, CAS, Wuhan 430077, China

Received:2024-05-23 Revised:2025-06-19 Online:2025-08-18 Published:2025-08-18
About author:ZANG Yufu (1987—), male, PhD, associate professor, majors in point cloud intelligent processing. E-mail: 002767@nuist.edu.cn
Supported by:
The National Natural Science Foundation of China(42171433);Open Fund of State Key Laboratory of Geodesy and Earth's Dynamics, Innovation Academy for Precision Measurement Science and Technology, Chinese Academy of Sciences(SKLPG2025-5-4);Jiangsu Provincial Research and Practice Innovation Program for Graduate Students(KYCX22_1213)

摘要/Abstract

摘要：

随着数字孪生城市、实景三维建设的应用与推广，基于三维点云城市高精度建模已成为重要研究课题，而建筑立面结构信息是辅助构建高精度三维城市模型的必要信息。因此，如何从点云数据中准确地提取建筑立面结构是精细化建模的研究前提。目前，基于深度学习的方法通过神经网络能理解复杂场景、实现目标精准分类，因而得到了广泛应用。然而，在建筑物立面场景中，点云数据存在遮挡严重、噪声极多、点密度差异大等问题，且立面各结构数量比例失衡严重（如门相对窗户的占比极小），使得现有方法难以满足建筑立面结构提取需求。针对该问题，本文围绕建筑立面结构提取在点云采样、特征提取和损失函数3个方面构建了PCF-Net深度学习神经网络。首先，在点云采样的过程中通过附上权重值增加小样本结构点云的比重；然后，设计双分支网络分别提取彩色点云的空间特征和纹理特征，并运用注意力机制自适应融合这两种模态特征，增强对建筑立面复杂场景的描述；最后，设计了顾及交并比（IoU）和提取精度（Acc）的双重约束损失函数以提高建筑立面结构提取的完整度与精确度。试验表明，本文提出的PCF-Net对多种类型的建筑立面提取结构结果分别达到了OA 97.99%，mAcc 97.80%和mIoU 95.75%的精度，而且对于小样本结构提取精度IoU都在90%以上。证明了本文提出的PCF-Net在提取复杂建筑立面结构时的有效性和高精度，为后续高精度三维建模提供了必要的技术支持。

关键词: 三维建模, 建筑立面语义解析, 点云采样, 双分支网络, 注意力融合, 损失函数

Abstract:

As the application and promotion of digital twin cities and realistic three-dimensional construction progresses, urban high-precision modeling based on 3D point clouds has become one of the important tasks. Building fa?ade structures can serve as prior knowledge to assist in the rapid construction of high-accuracy 3D urban models. Therefore, exploring how to accurately extract building fa?ade structures from point cloud data is a research focus in detailed modeling. Currently, methods based on deep learning can use neural networks to understand complex building fa?ades, but the extraction accuracy for less common structures in fa?ades (such as doors, external air conditioning units) is still not high enough. To address this issue, this paper develops a novel deep learning neural network, position color fusion-net (PCF-Net), focusing on the extraction of small-sample structures in building fa?ades across three aspects: point cloud sampling, feature extraction, and loss function. Initially, during the point cloud sampling process, the proportion of small-sample structure point clouds is increased by attaching weights. Subsequently, a dual-branch network is used to extract spatial features from the colored point clouds and texture features, with an attention mechanism applied to adaptively fuse these two types of features, enhancing the description of key details in building fa?ades. Finally, a loss function that considers both intersection over union (IoU) and extraction accuracy (Acc) constraints is designed to improve the completeness and precision of building fa?ade structure extraction. Experiments show that the proposed PCF-Net network achieves precision metrics of 97.99% OA, 97.80% mAcc, and 95.75% mIoU in extracting fine structures from various types of building fa?ades, demonstrating the network's superior performance in building fa?ade structure extraction (Project address: https://github.com/zangyufus/PCF_net.git).

Key words: 3D modeling, semantic parsing of building facades, point cloud sampling, dual-branch network, attention fusion, loss function

中图分类号:

P237

臧玉府, 王树野, 董震, 陈驰, 黄荣刚. 基于PCF-Net网络的建筑点云立面结构高精度提取[J]. 测绘学报, 2025, 54(7): 1243-1253.

Yufu ZANG, Shuye WANG, Zhen DONG, Chi CHEN, Ronggang HUANG. High-precision extraction of building point cloud facade structure based on PCF-Net network[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(7): 1243-1253.

图/表 12

图1

图2

图3

图4

图5

表1

图6

表2

图7

表3

图8

表4

参考文献 29

[1]	杨必胜, 陈驰, 董震. 面向智能化测绘的城市地物三维提取[J]. 测绘学报, 2022, 51(7): 1476-1484. DOI: . doi: 10.11947/j.AGCS.2022.20220183
	YANG Bisheng, CHEN Chi, DONG Zhen. 3D geospatial information extraction of urban objects for smart surveying and mapping[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(7): 1476-1484. DOI: . doi: 10.11947/j.AGCS.2022.20220183
[2]	CHEN Chi, JIN Ang, WANG Zhiye, et al. SGSR-net: structure semantics guided LiDAR super-resolution network for indoor LiDAR SLAM[J]. IEEE Transactions on Multimedia, 2024, 26: 1842-1854.
[3]	WANG Wei, XU Yuan, REN Yingchao, et al. Parsing of urban facades from 3D point clouds based on a novel multi-view domain[J]. Photogrammetric Engineering & Remote Sensing, 2021, 87(4): 283-293.
[4]	蒋腾平, 王永君, 张林淇, 等. 融合CNN和MRF的激光点云层次化语义分割方法[J]. 测绘学报, 2021, 50(2): 215-225. DOI: . doi: 10.11947/j.AGCS.2021.20200095
	JIANG Tengping, WANG Yongjun, ZHANG Linqi, et al. A LiDAR point cloud hierarchical semantic segmentation method combining CNN and MRF[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 215-225. DOI: . doi: 10.11947/j.AGCS.2021.20200095
[5]	XIE Yuxing, TIAN Jiaojiao, ZHU Xiaoxiang. Linking points with labels in 3D: a review of point cloud semantic segmentation[J]. IEEE Geoscience and remote sensing magazine, 2020, 8(4): 38-59.
[6]	张瑞菊, 郑少开, 尹建英. 建筑物立面点云语义分割方法研究[J]. 城市勘测, 2017(5): 22-25.
	ZHANG Ruiju, ZHENG Shaokai, YIN Jianying. Building facade point cloud semantic segmentation method research[J]. Urban Geotechnical Investigation & Surveying, 2017(5): 22-25.
[7]	费鲜芸, 戴哲, 刘如飞. 三维激光点云中建筑物立面边界特征提取方法[J]. 山东科技大学学报(自然科学版), 2022, 41(2): 21-30.
	FEI Xianyun, DAI Zhe, LIU Rufei. Boundary feature extraction method of building facade in 3D laser point cloud[J]. Journal of Shandong University of Science and Technology (Natural Science), 2022, 41(2): 21-30.
[8]	HU Zengtian, CHEN Chi, YANG Bisheng, et al. Geometric feature enhanced line segment extraction from large-scale point clouds with hierarchical topological optimization[J]. International Journal of Applied Earth Observation and Geoinformation, 2022, 112(1): 102858-102868.
[9]	邹纪伟, 刘德儿, 杨鹏, 等. 建筑立面点云对偶深度图像中窗户提取方法[J]. 激光与红外, 2020, 50(7): 875-881.
	ZOU Jiwei, LIU Deer, YANG Peng, et al. Window extraction method in dual depth image of point cloud of building facade[J]. Laser & Infrared, 2020, 50(7): 875-881.
[10]	ZHAO Fuqun, HUANG He, XIAO Nana, et al. A point cloud segmentation algorithm based on multi-feature training and weighted random forest[J]. Measurement Science and Technology, 2024, 36(1): 015407.
[11]	NI Huan, LIN Xiangguo, ZHANG Jixian. Classification of ALS point cloud with improved point cloud segmentation and random forests[J]. Remote Sensing, 2017, 9(3): 288-296.
[12]	CHEN Chao, LI Xiaomin, BELKACEM A N, et al. The mixed kernel function SVM-based point cloud classification[J]. International Journal of Precision Engineering and Manufacturing, 2019, 20: 737-747.
[13]	王丽英, 吴际, 有泽, 等. 多维GMM与邻域约束的多光谱机载LiDAR数据城市地物分类. 测绘学报, 2023, 52(3), 419-431. DOI: . doi: 10.11947/j.AGCS.2023.20210153
	WANG Liying, WU Ji, YOU Ze, et al. Urban object classification of multispectral airborne LiDAR data with multi-dimensional Gauss mixture model and neighborhood constraints[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(3): 419-431. DOI: . doi: 10.11947/j.AGCS.2023.20210153
[14]	BALLOUCH Z, HAJJI R, POUX F, et al. A prior level fusion approach for the semantic segmentation of 3D point clouds using deep learning[J]. Remote Sensing, 2022, 14(14): 3415. DOI: . doi: doi.org/10.3390/rs14143415
[15]	DAI Menglin, WARD W O C, MEYERS G, et al. Residential building facade segmentation in the urban environment[J]. Building and Environment, 2021, 199: 107921.
[16]	WANG Puzuo, YAO Wei. A new weakly supervised approach for ALS point cloud semantic segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188: 237-254. DOI: https://doi.org/10.1016/j.isprsjprs.2022.04.016.
[17]	CHEN Dong, XIANG Guiqiu, PEETHAMBARAN J, et al. AFGL-net: attentive fusion of global and local deep features for building façades parsing[J]. Remote Sensing, 2021, 13(24): 5039.
[18]	HU Qingyong, YANG Bo, XIE Linkai, et al. RanDLA-net: efficient semantic segmentation of large-scale point clouds[C]//Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. [EB/OL]: IEEE, 2020: 11108-11117.
[19]	SU Yanfei, LIU Weiquan, YUAN Zhimin, et al. DLA-Net: learning dual local attention features for semantic segmentation of large-scale building facade point clouds[J]. Pattern Recognition, 2022, 123: 108372.
[20]	ZHANG Xinlong, LIN Dong, SOERGEL U. Target-aware attentional network for rare class segmentation in large-scale LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2025, 220(1): 32-50. DOI: https://doi.org/10.1016/j.isprsjprs.2024.11.012.
[21]	方莉娜, 沈贵熙, 游志龙, 等. 融合点云和多视图的车载激光点云路侧多目标识别[J]. 测绘学报, 2021, 50(11): 1558-1573. DOI: . doi: 10.11947/j.AGCS.2021.20210246
	FANG Lina, SHEN Guixi, YOU Zhi, et al. A joint network of point cloud and multiple views for recognition from mobile laser point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(11): 1558-1573. DOI: . doi: 10.11947/j.AGCS.2021.20210246
[22]	ZANG Yufu, WANG Shuye, GUAN Haiyan, et al. VAM-Net: vegetation-attentive deep network for multi-modal fusion of visible-light and vegetation-sensitive images[J]. International Journal of Applied Earth Observation and Geoinformation, 2024, 127: 103642.
[23]	YU Bing, HU Jinlong, DONG Xiujun, et al. A robust automatic method to extract building facade maps from 3D point cloud data[J]. Remote Sensing, 2022, 14(16): 3848.
[24]	WANG Y, SUN Y, LIU Z, et al. Dynamic graph cnn for learning on point clouds[J]. ACM Transactions on Graphics (tog), 2019, 38(5), 1-12. DOI: . doi: 10.48550/arXiv.1801.07829
[25]	MORBIDONI C, PIERDICCA R, QUATTRINI R, et al. Graph cnn with radius distance for semantic segmentation of historical buildings TLS point clouds[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, XLIV-4/W1-2020: 95-102.
[26]	陈科, 管海燕, 雷相达, 等. 基于特征增强核点卷积网络的多光谱LiDAR点云分类方法[J]. 地球信息科学学报, 2023, 25(5): 1075-1087.
	CHEN Ke, GUAN Haiyan, LEI Xiangda, et al. A multispectral LiDAR point cloud classification method based on enhanced features kernel point convolutional network[J]. Journal of Geo-information Science, 2023, 25(5): 1075-1087.
[27]	高建文, 管海燕, 彭代锋, 等. 基于局部-全局语义特征增强的遥感影像变化检测网络模型[J]. 地球信息科学学报, 2023, 25(3): 625-637.
	GAO Jianwen, GUAN Haiyan, PENG Daifeng, et al. Local-global semantic feature enhancement model for remote sensing imagery change detection[J]. Journal of Geo-information Science, 2023, 25(3): 625-637.
[28]	QI C R, SU H, MO K, et al. Pointnet: deep learning on point sets for 3D classification and segmentation[J]. Proceedings of the IEEE Conference on Computer Vision and Pttern Recognition, 2017(1): 652-660.
[29]	QI C R, YI Li, SU Hao, et al. PointNet++: deep hierarchical feature learning on point sets in a metric space[EB/OL]. [2024-10-16]. https://arxiv.org/abs/1706.02413v1.

墙面	点数	立面结构	墙面特点
1	215 465	墙、门、窗户	点云缺失、墙面不完整
2	287 161	墙、门、窗户、空调外机	多样结构，墙面完整
3	297 361	墙、窗户	墙面完整，结构简单
4	221 184	墙、阳台	点云缺失，结构不完整
5	445 910	墙、窗户、门	结构复杂规整，墙面完整

墙面	OA	mAcc	IoU					mIoU
墙面	OA	mAcc	窗户	墙	阳台	空调外机	门	mIoU
1	94.43	94.17	89.38	90.22	—	87.29	—	88.96
2	98.78	95.80	—	98.41	95.21	—	—	97.41
3	95.39	91.20	98.19	98.24	—	—	90.07	95.50
4	98.59	93.22	—	98.16	95.78	—	92.64	96.66
5	99.03	98.90	98.67	97.72	—	—	91.80	96.53

方法	OA	mAcc	IoU					mIoU
方法	OA	mAcc	窗户	墙	阳台	空调外机	门	mIoU
PointNet^［28］	61.25	48.26	69.80	55.60	39.40	6.80	33.40	41.00
PointNet++^［29］	78.58	70.97	86.40	79.70	59.70	5.10	47.90	55.76
R-DGCNN^［25］	97.31	92.25	94.41	96.39	93.35	88.55	70.30	88.60
DLA-Net^［19］	98.72	96.56	95.65	97.51	91.60	87.96	71.13	88.97
PCF-Net（本文方法）	97.99	97.80	92.75	96.88	95.41	95.72	97.99	95.75

方案	Color MLP	点云权重采样	Lovsz-Softmax	加权交叉熵	IoU					mIoU
方案	Color MLP	点云权重采样	Lovsz-Softmax	加权交叉熵	窗户	墙	阳台	空调外机	门	mIoU
1	—	—	—	—	94.21	96.19	93.15	88.35	70.10	87.60
2	√	—	—	—	93.64	95.34	92.33	89.06	73.17	88.71
3	√	√	—	—	93.78	96.72	93.56	90.32	76.52	89.51
4	√	√	√	—	94.80	96.15	92.57	95.49	80.39	91.88
5	√	√	√	√	92.75	96.88	95.41	95.72	97.99	95.75

基于PCF-Net网络的建筑点云立面结构高精度提取

High-precision extraction of building point cloud facade structure based on PCF-Net network

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 12

参考文献 29

相关文章 8

编辑推荐

Metrics

本文评价

[1]	董子博, 王竞雪, 卜丽静, 房琳, 许峥辉. MAFNet：基于多尺度空洞融合网络的遥感影像建筑物提取方法[J]. 测绘学报, 2025, 54(6): 1094-1106.
[2]	邢广澳, 卢官明, 韩斌. MAFUNet：结合注意力机制和主动轮廓损失的SAR图像水体分割算法[J]. 测绘学报, 2025, 54(5): 924-936.
[3]	李维炼, 朱军, 朱庆, 李佳洛. 耦合数值仿真与时空插值融合的山区桥梁温度场三维建模方法[J]. 测绘学报, 2025, 54(4): 749-759.
[4]	梁哲恒, 黎宵, 邓鹏, 盛森, 姜福泉. 融合多尺度特征注意力的遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5): 668-676.
[5]	贾军辉, 黄明, 刘祥磊. 基于三维狄洛尼三角网的曲面重建算法[J]. 测绘学报, 2018, 47(2): 281-290.
[6]	李卉,钟成,黄先锋,李德仁. 集成激光雷达数据和遥感影像的立交桥自动检测方法[J]. 测绘学报, 2012, 41(3): 0-416.
[7]	孙敏,胡争. 基于向量匹配的稀疏深度图生成算法[J]. 测绘学报, 2011, 40(1): 90-95.
[8]	孙中昶; 卢秀山; 田茂义. 矿山巷道三维建模算法研究及实现[J]. 测绘学报, 2009, 38(3): 0-201.