适用于城市场景大规模点云语义标识的深度学习网络

doi:10.11947/j.AGCS.2021.20210093

摘要/Abstract

摘要： 近年来，以点云为代表的三维数据不断涌现，如何利用人工智能手段，高度提升点云的解译能力，实现城市地物目标的语义标识、三维精准提取等成为亟待攻克的难题。为此，本文提出了一种端到端的三维点云深度学习网络，通过构建不规则分布点云的上下采样策略、特征多层聚合与传播，以及顾及样本不均的损失函数，有效保障了点云采样的高效性、特征提取的准确性及网络整体性能的最优性。三维点云大规模数据集上测试表明，该深度学习网络在城市场景的语义标识正确性方面取得了优异的结果，性能优于当前的主流网络，为三维地理信息的高性能提取提供了有力支撑。

关键词: 深度学习, 人工智能, 点云, 语义标识, 特征提取

Abstract: In recent years, point cloud has become an important type of 3D spatial data. How to improve the understanding abilities of point cloud using artificial intelligence for correct semantic labeling and accurate detection of objects is an urgent and difficult problem. This paper hence proposes an end-to-end 3D point cloud deep learning network, which effectively guarantees the efficiencies of point cloud sampling, the accuracy of feature extraction and the optimization of the overall network performance by the up-down sampling strategy of irregular distribution point cloud, multi-layer aggregation and propagation of features and the loss function for uneven samples. The studies on the large-scale 3D point cloud benchmark data show that it achieves excellent performance in semantic labeling for large-scale outdoor scenes of point clouds, better than those of the state-of-art deep learning networks of point cloud, providing a strong support for the high-performance extraction of 3D geospatial information.

Key words: Deep learning, artificial intelligence, point cloud, semantic labeling, feature extraction

中图分类号:

P207

杨必胜, 韩旭, 董震. 适用于城市场景大规模点云语义标识的深度学习网络[J]. 测绘学报, 2021, 50(8): 1059-1067.

YANG Bisheng, HAN Xu, DONG Zhen. A deep learning network for semantic labeling of large-scale urban point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8): 1059-1067.

参考文献

[1] 杨必胜, 梁福逊, 黄荣刚. 三维激光扫描点云数据处理研究进展、挑战与趋势[J]. 测绘学报, 2017, 46(10):1509-1516.DOI:10.11947/j.AGCS.2017.20170351. YANG Bisheng, LIANG Fuxun, HUANG Ronggang. Progress, challenges and perspectives of 3D LiDAR point cloud processing[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1509-1516.DOI:10.11947/j.AGCS.2017.20170351.
[2] 廖小罕. 中国对地观测20年科技进步和发展[J]. 遥感学报, 2021, 25(1):267-275. LIAO Xiaohan. Scientific and technological progress and development prospect of the earth observation in China in the past 20 years[J]. National Remote Sensing Bulletin, 2021, 25(1):267-275.
[3] 杨必胜, 董震. 点云智能研究进展与趋势[J]. 测绘学报, 2019, 48(12):1575-1585. DOI:10.11947/j.AGCS.2019.20190465. YANG Bisheng, DONG Zhen. Progress and perspective of point cloud intelligence[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1575-1585. DOI:10.11947/j.AGCS.2019.20190465.
[4] 杨必胜, 董震. 点云智能处理[M]. 北京:科学出版社, 2020. YANG Bisheng, DONG Zhen. Intelligent processing of point cloud[M]. Beijing:Science Press, 2020.
[5] MA J W, CZERNIAWSKI T, LEITE F. Semantic segmentation of point clouds of building interiors with deep learning:Augmenting training datasets with synthetic BIM-based point clouds[J]. Automation in Construction, 2020, 113:103144.
[6] DONG Zhen, YANG Bisheng, HU Pingbo, et al. An efficient global energy optimization approach for robust 3D plane segmentation of point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 137:112-133.
[7] YI Li, SU Hao, GUO Xingwen, et al. SyncSpecCNN:synchronized spectral CNN for 3D shape segmentation[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu, HI, USA:IEEE, 2017:6584-6592.
[8] WU Zhirong, SONG Shuran, KHOSLA A, et al. 3D ShapeNets:a deep representation for volumetric shapes[C]//2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA:IEEE, 2015:1912-1920.
[9] DAI A, CHANG A X, SAVVA M, et al. ScanNet:richly-annotated 3D reconstructions of indoor scenes[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:2432-2443.
[10] HACKEL T, SAVINOV N, LADICKY L, et al. Semantic3d.net:a new large-scale point cloud classification benchmark[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017, IV-1/W1:91-98.
[11] GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2012:3354-3361.
[12] DONG Zhen, LIANG Fuxun, YANG Bisheng, et al. Registration of large-scale terrestrial laser scanner point clouds:a review and benchmark[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 163:327-342.
[13] 杨必胜, 韩旭, 董震. 点云深度学习基准数据集[J]. 遥感学报, 2021, 25(1):231-240. YANG Bisheng, HAN Xu, DONG Zhen. Point Cloud Benchmark Dataset WHU-TLS and WHU-MLS for Deep Learning[J]. Journal of Remote Sensing, 2021, 25(1):231-240.
[14] FANG Hao, LAFARGE F. Pyramid scene parsing network in 3D:Improving semantic segmentation of point clouds with multi-scale contextual information[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 154:246-258.
[15] HU Qingyong, YANG Bo, XIE Linhai, et al. RandLA-net:efficient semantic segmentation of large-scale point clouds[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA:IEEE, 2020:11105-11114.
[16] CHEN Xiaozhi, MA Huimin, WAN Ji, et al. Multi-view 3D object detection network for autonomous driving[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:6526-6534.
[17] YANG Bin, LUO Wenjie, URTASUN R. PIXOR:real-time 3D object detection from point clouds[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA:IEEE, 2018:7652-7660.
[18] LANG A H, VORA S, CAESAR H, et al. PointPillars:fast encoders for object detection from point clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA:IEEE, 2019:12689-12697.
[19] GRAHAM B, ENGELCKE M, MAATEN L V D. 3D semantic segmentation with submanifold sparse convolutional networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT, USA:IEEE, 2018:9224-9232.
[20] MENG H Y, GAO Lin, LAI Yukun, et al. VV-net:voxel VAE net with group convolutions for point cloud segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South):IEEE, 2019:8499-8507.
[21] CHARLES R, SU H, KAICHUN M, et al. PointNet:Deep Learning on Point Sets for 3D Classification and Segmentation[C/OL]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu, HI, USA:IEEE,2017.
[22] QI C R, YI Li, SU Hao, et al. PointNet++:deep hierarchical feature learning on point sets in a metric space[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook, NY, USA:Curran Associates Inc., 2017:5105-5114.
[23] WANG Yue, SUN Yongbin, LIU Ziwei, et al. Dynamic graph CNN for learning on point clouds[J]. ACM Transactions on Graphics, 2019, 38(5):1-12.
[24] JIANG Li, ZHAO Hengshuang, LIU Shu, et al. Hierar chical point-edge interaction network for point cloud semantic segmentation[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Seoul, Korea (South):IEEE, 2019:10432-10440. DOI:10.1109/ICCV.2019.01053.
[25] HERMOSILLA P, RITSCHEL T, VÁZQUEZ P P, et al. Monte Carlo convolution for learning on non-uniformly sampled point clouds[J]. ACM Transactions on Graphics, 2019, 37(6):1-12.
[26] XU Yifan, FAN Tianqi, XU Mingye, et al. SpiderCNN:deep learning on point sets with parameterized convolutional filters[M]//FERRARI V, HEBERT M, SMINCHISESCU C, et al.Computer Vision-ECCV 2018. Cham:Springer International Publishing, 2018:90-105. DOI:10.1007/978-3-030-01237-3_6.
[27] WU Wenxuan, QI Z, FUXIN Li. PointConv:deep convolutional networks on 3D point clouds[C]//2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA:IEEE, 2019:9613-9622.
[28] ZHANG Rui, LI Guangyun, LI Minglei, et al. Fusion of images and point clouds for the semantic segmentation of large-scale 3D scenes based on deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 143:85-96.
[29] LUO Zhipeng, LIU Di, LI J, et al. Learning sequential slice representation with an attention-embedding network for 3D shape recognition and retrieval in MLS point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 161:147-163.