基于深度残差网络的机载LiDAR点云分类

doi:10.11947/j.AGCS.2020.20190004

摘要/Abstract

摘要： 机载LiDAR点云的分类是利用其进行城市场景三维重建的关键步骤之一。为充分利用现有的图像领域性能较好的深度学习网络模型，提高点云分类精度，并降低训练时间和对训练样本数量的要求，本文提出一种基于深度残差网络的机载LiDAR点云分类方法。首先提取归一化高程、表面变化率、强度和归一化植被指数4种具有较高区分度的点云低层次特征；然后通过设置不同的邻域大小和视角，利用所提出的点云特征图生成策略，得到多尺度和多视角点云特征图；再将点云特征图输入到预训练的深度残差网络，提取多尺度和多视角深层次特征；最后构建并训练神经网络分类器，利用训练的模型对待分类点云进行预测，经后处理得到分类结果。利用ISPRS三维语义标记竞赛的公开标准数据集进行试验，结果表明，本文方法可有效区分建筑物、地面、车辆等8类地物，分类结果的总体精度为87.1%，可为城市场景三维重建提供可靠的信息。

关键词: 点云分类, 深层次特征, 多尺度和多视角, 迁移学习, 深度残差网络, 机载LiDAR

Abstract: Airborne LiDAR point cloud classification is one of the key steps for three-dimensional reconstruction of urban scenes. To leverage the existing high-performing deep learning network model in image field of image processing, improve classification accuracy and reduce training time and demand for training samples simultaneously, an airborne LiDAR point cloud classification method based on deep residual network is proposed in this paper. Firstly, high discriminative low-level features, i.e. normalized height, point cloud normal vector, intensity and normalized differential vegetation index, are extracted. Secondly, by setting different neighborhood sizes and perspectives, multi-scale and multi-view point cloud feature images are generated via using the proposed point cloud feature image generation strategy. Then, point cloud feature images are input into the pre-trained deep residual network to extract multi-scale and multi-view deep features. Finally, a neural network classifier is constructed and trained, point cloud classification results are obtained by utilizing the trained classifier and postprocessing. Benchmark datasets of ISPRS 3D Semantic Labeling Contest are used, the experimental results show that the proposed method can effectively distinguish 8 types ground objects such as buildings, ground and vehicles etc., and the overall accuracy of the classification result is 87.1%, which can provide reliable information for three-dimensional reconstruction of urban scenes.

Key words: point cloud classification, deep feature, multi-scale and multi-view, transfer learning, deep residual network, airborne LiDAR

中图分类号:

P237

赵传, 郭海涛, 卢俊, 余东行, 张保明. 基于深度残差网络的机载LiDAR点云分类[J]. 测绘学报, 2020, 49(2): 202-213.

ZHAO Chuan, GUO Haitao, LU Jun, YU Donghang, ZHANG Baoming. Airborne LiDAR point cloud classification based on deep residual network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2): 202-213.

参考文献

[1] ZHANG Liqiang, LI Zhuqiang, LI Anjian, et al. Large-scale urban point cloud labeling and reconstruction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018(138):86-100.
[2] JARZBEK-RYCHARD M, MAAS H G. Geometric refinement of ALS-data derived building models using monoscopic aerial images[J]. Remote Sensing, 2017, 9(3):282.
[3] 杨俊涛, 康志忠. 多尺度特征和马尔可夫随机场模型的电力线场景点云分类法[J]. 测绘学报, 2018, 47(2):188-197. DOI:10.11947/j.AGCS.2018.20170556. YANG Juntao, KANG Zhizhong. Multi-scale features and markov random field model for powerline scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2):188-197. DOI:10.11947/j.AGCS.2018.20170556.
[4] 张继贤, 段敏燕, 林祥国, 等. 激光雷达点云电力线三维重建模型的对比与分析[J]. 武汉大学学报(信息科学版), 2017, 42(11):1565-1772. ZHANG Jixian, DUAN Minyan, LIN Xiangguo, et al. Comparison and analysis of models for 3D power line reconstruction using LiDAR point cloud[J]. Geomatics and Information Science of Wuhan University, 2017, 42(11):1565-1772.
[5] 王丹菂, 徐青, 邢帅, 等. 一种由粗到精的机载激光测深信号检测方法[J]. 测绘学报, 2018, 47(8):1148-1159. DOI:10.11947/j.AGCS.2018.20170466. WANG Dandi, XU Qing, XING Shuai, et al. A coarse-to-fine signal detection method for airborne LiDAR bathymetry[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(8):1148-1159. DOI:10.11947/j.AGCS.2018.20170466.
[6] WANG Cheng, NIE Sheng, XI Xiaohuan, et al. Estimating the biomass of maize with hyperspectral and LiDAR data[J]. Remote Sensing, 2017, 9(1):11.
[7] 谢瑞, 程效军, 管海燕. 机载激光扫描与航空影像的融合分类与精度分析[J]. 同济大学学报(自然科学版), 2013, 41(4):607-613. XIE Rui, CHENG Xiaojun, GUAN Haiyan, Classification and accuracy analysis of LiDAR and aerial images[J]. Journal of Tongji University (Natural Science), 2013, 41(4):607-613.
[8] 熊艳, 高仁强, 徐战亚. 机载LiDAR点云数据降维与分类的随机森林方法[J]. 测绘学报, 2018, 47(4):508-518. DOI:10.11947/j.AGCS.2018.20170417. XIONG Yan, GAO Renqiang, XU Zhanya. Random forest method for dimension reduction and point cloud classification based on airborne LiDAR[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(4):508-518. DOI:10.11947/j.AGCS.2018.20170417.
[9] ZHU Qing, LI Yuan, HU Ho, et al. Robust point cloud classification based on multi-level semantic relationships for urban scenes[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017(129):86-102.
[10] 张继贤, 林祥国, 梁欣廉. 点云信息提取研究进展和展望[J]. 测绘学报, 2017, 46(10):1460-1469. DOI:10.11947/j.AGCS.2017.20170345. ZHANG Jixian, LIN Xiangguo, LIANG Xinlian. Advances and prospects of information extraction from point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1460-1469. DOI:10.11947/j.AGCS.2017.20170345.
[11] 潘锁艳, 管海燕. 机载多光谱LiDAR数据的地物分类方法[J]. 测绘学报, 2018, 47(2):198-207. DOI:10.11947/j.AGCS.2018.20170512. PAN Suoyan, GUAN Haiyan. Object classification using airborne multispectral LiDAR data[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2):198-207. DOI:10.11947/j.AGCS.2018.20170512.
[12] 刘志青, 李鹏程, 陈小卫, 等. 基于信息向量机的机载激光雷达点云数据分类[J]. 光学精密工程, 2016, 24(1):210-219. LIU Zhiqing, LI Pengcheng, CHEN Xiaowei, et al. Classification of airborne LiDAR point cloud data based on information vector machine[J]. Optics and Precision Engineering, 2016, 24(1):210-219.
[13] 朱江涛, 黄睿. 基于Adaboost的高光谱与LiDAR数据特征选择与分类[J]. 遥感信息, 2014, 29(6):68-72. ZHU Jiangtao, HUANG Rui. Feature selection and classification of hyperspectral data and LiDAR data based on Adaboost[J]. Remote Sensing Information, 2014, 29(6):68-72.
[14] ZHANG Zhenxin, ZHANG Liqiang, TONG Xiaohua, et al. Discriminative-dictionary-learning-based multilevel point-cluster features for ALS point-cloud classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(12):7309-7322.
[15] GUO Bo, HUANG Xianfeng, ZHANG Fan, et al. Classification of airborne laser scanning data using JointBoost[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015(100):71-83.
[16] NIEMEYER J, ROTTENSTEINER F, SOERGEL U. Contextual classification of LiDAR data and building object detection in urban areas[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014(87):152-165.
[17] XU S, VOSSELMAN G, ELBERINK S O. Multiple-entity based classification of airborne laser scanning data in urban areas[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2014(88):1-15.
[18] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[19] YOUSEFHUSSIEN M, KELBE D J, IENTILUCCI E J, et al. A multi-scale fully convolutional network for semantic labeling of 3D point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018(143):191-204.
[20] YANG Zhishuang, JIANG Wanshou, XU Bo, et al. A convolutional neural network-based 3D semantic labeling method for ALS point clouds[J]. Remote Sensing, 2017, 9(9):936.
[21] YANG Zhishuang, TAN Bo, PEI Huikun, et al. Segmentation and multi-scale convolutional neural network-based classification of airborne laser scanner data[J]. Sensors, 2018, 18(10):3347.
[22] ZHAO Ruibin, PANG Mingyong, WANG Jidong. Classifying airborne LiDAR point clouds via deep features learned by a multi-scale convolutional neural network[J]. International Journal of Geographical Information Science, 2018, 32(5):960-979.
[23] ISPRS. ISPRS semantic labeling contest (3D):results[EB/OL]. (2018-11-6).[2018-11-13]. http://www2.isprs.org/commissions/comm2/wg4/vaihingen-3d-semantic-labeling.html.
[24] 姚登举, 杨静, 詹晓娟. 基于随机森林的特征选择算法[J]. 吉林大学学报(工学版), 2014, 44(1):137-141. YAO Dengju, YANG Jing, ZHAN Xiaojuan. Feature selection algorithm based on random forest[J]. Journal of Jilin University (Engineering and Technology Edition), 2014, 44(1):137-141.
[25] YAN W Y, SHAKER A, EL-ASHMAWY N. Urban land cover classification using airborne LiDAR data:a review[J]. Remote Sensing of Environment, 2015(158):295-310.
[26] TAN Chuanqi, SUN Fuchun, KONG Tao, et al. A survey on deep transfer learning[C]//Proceedings of the 27th International Conference on Artificial Neural Networks. Rhodes, Greece:Springer, 2018:270-279.
[27] RUSSAKOVSKY O, DENG Jia, SU Hao, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3):211-252.
[28] KORNBLITH S, SHLENS J, LE Q V. Do Better imagenet models transfer better?[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Los Angeles CA, United States:IEEE, 2019:2661-2671.
[29] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016:770-778.
[30] BOYKOV Y, KOLMOGOROV V. An experimental comparison of min-cut/max-flow algorithms for energy minimization in vision[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2004, 26(9):1124-1137.
[31] NIEMEYER J, ROTTENSTEINER F, SOERGEL U, et al. Hierarchical higher order crf for the classification of airborne LiDAR point clouds in urban areas[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016(XLI-B3):655-662.
[32] QI CHARLES R, SU Hao, MO Kaichun, et al. PointNet:deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI:IEEE, 2017:77-85.