Urban object classification of multispectral airborne LiDAR data with multidimensional Gauss mixture model and neighborhood constraints

doi:10.11947/j.AGCS.2023.20210153

Abstract

Abstract: Making full use of the spatial location and multispectral information of multispectral airborne LiDAR point cloud to completely and accurately segment objects is an important prerequisite for the subsequent application of the data.However, the traditional point-based classification algorithms are difficult to design and inefficient to execute due to the limitations of point cloud that cannot clearly represent topology and neighborhood information.Moreover, the classification algorithms which interpolate point cloud into image is limited by the information and precision loss of image, which leads to low classification accuracy. In addition, neither point cloud nor image can directly represent the three-dimensional geometry of objects, which is not conducive to the subsequent three-dimensional modeling and analysis of objects.Therefore, this paper proposes a classification algorithm based on multi-dimensional Gauss mixture model (GMM) and neighborhood constraints for multispectral airborne LiDAR data. The multispectral airborne LiDAR data are firstly voxelized into a multi-valued virtual voxel structure based on the principle of no information loss and representing neighborhood information explicitly, in which the virtual voxel is the combination of the voxel and its internal laser points, and the virtual voxel value is a feature vector composed of the spectrum, elevation, local normal vector distribution and point density features of the laser points in the voxel.Then, a fuzzy clustering segmentation model is constructed to segment the virtual voxel structure, and the fuzzy membership matrix of each virtual voxel is obtained.Among them, the multi-peak distribution of objects in feature space is fitted by multi-dimensional GMM to establish the label field, and the probability distribution of multi-dimensional GMM is used as the dissimilarity metric.The spatial correlation of classes for the adjacent voxels in the label field is modeled by the hidden Markov random field (HMARF) model, and the prior probability under the neighborhood constraint is established, which is used as the parameter to control the clustering scale.An objective function with a regularization term is used to solve the scaling sensitivity problem of clustering.Finally, the membership matrix is defuzzified to determine the classification result.The effectiveness and feasibility of the proposed algorithm are quantitatively evaluated based on Optech Titan multispectral airborne LiDAR data of different complexity and different data volume.The experimental results show that the average overall accuracy and Kappa coefficient of the algorithm is 91.32% and 0.872, respectively, which can effectively realize the classification of various urban objects.The proposed algorithm makes comprehensive use of the spectral, geometric and neighborhood consistency information of objects, expands the types of information utilization, and provides a feasible scheme for the comprehensive utilization of the spatial location and multispectral information of multispectral airborne LiDAR data.

Key words: multispectral airborne LiDAR, virtual voxel, voxel model, hidden Markov random field, multi-dimensional Gaussian mixture model, fuzzy clustering

CLC Number:

P236

WANG Liying, WU Ji, YOU Ze, LI Yu, CAMARA Mahamadou. Urban object classification of multispectral airborne LiDAR data with multidimensional Gauss mixture model and neighborhood constraints[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(3): 419-431.

References

[1] VAN REES E. The first multispectral airborne LiDAR sensor[J]. GeoInformatics, 2015, 18(1):10.
[2] TEO T A, WU H M. Analysis of land cover classification using multi-wavelength LiDAR system[J]. Applied Sciences, 2017, 7(7):663.
[3] 杜建丽, 陈动, 张振鑫, 等. 建筑点云几何模型重建方法研究进展[J]. 遥感学报, 2019, 23(3):374-391. DU Jianli, CHEN Dong, ZHANG Zhenxin, et al. Research progress of building reconstruction via airborne point clouds[J]. Journal of Remote Sensing, 2019, 23(3):374-391.
[4] FERNANDEZ-DIAZ J, CARTER W, GLENNIE C, et al. Capability assessment and performance metrics for the Titan multispectral mapping LiDAR[J]. Remote Sensing, 2016, 8(11):936.
[5] BAKULA K,KUPIDURA P, JELOWICKI L. Testing of land cover classification from multispectral airborne laser scanning data[J]. ISPRS-International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, XLI-B7:161-169.
[6] HOPKINSON C, CHASMER L, GYNAN C, et al. Multisensor and multispectral LiDAR characterization and classification of a forest environment[J]. Canadian Journal of Remote Sensing, 2016, 42(5):501-520.
[7] MORSY S, SHAKER A,EI-RABBANY A. Multispectral LiDAR data for land coverclassification of urban areas[J]. Sensors, 2017, 17(5):958.
[8] MILLER C I, THOMAS J J, KIM A M, et al. Application of image classification techniques to multispectral LiDAR point cloud data[C]//Proceedings of 2016 SPIE 9832, Laser Radar Technology and Applications XXI. Baltimore, MD, USA:SPIE, 2016:98320.
[9] PAN Suoyan, GUAN Haiyan, YU Yongtao, et al. A comparative land-cover classification feature study of learning algorithms:DBM, PCA, and RF using multispectral LiDAR data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2019, 12(4):1314-1326.
[10] WANG Chisheng, SHU Qiqi, WANG Xinyu, et al. A random forest classifier based on pixel comparison features for urban LiDAR data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2019, 148:75-86.
[11] 曹爽, 潘锁艳, 管海燕. 机载多光谱LiDAR的随机森林地物分类[J]. 测绘通报, 2019(11):79-84. CAO Shuang, PAN Suoyan, GUAN Haiyan. Random forest-based land-use classification using multispectral LiDAR data[J]. Bulletin of Surveying and Mapping, 2019(11):79-84.
[12] JIA Sen, ZHAN Zhangwei, ZHANG Meng, et al. Multiple feature-based superpixel-level decision fusion for hyperspectral and LiDAR data classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(2):1437-1452.
[13] WANG C K,TSENG Y, CHU H J. Airborne dual-wavelength LiDAR data for classifying land cover[J]. Remote Sensing,2013,6(1):700-715.
[14] HUO L Z, SILVA C A, KLAUBERG C, et al. Supervised spatial classification of multispectral LiDAR data in urban areas[J]. PLoS One,2018,13(10):e0206185.
[15] 潘锁艳,管海燕.机载多光谱LiDAR数据的地物分类方法[J].测绘学报,2018,47(2):198-207.DOI:10.11947/j.AGCS.2018.20170512. PAN Suoyan, GUAN Haiyan. Object classification using airborne multispectral LiDAR data[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2):198-207.DOI:10.11947/j.AGCS.2018.20170512.
[16] PAN Suoyan, GUAN Haiyan, CHEN Yating, et al. Land-cover classification of multispectral LiDAR data using CNN with optimized hyper-parameters[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 166:241-254.
[17] YU Yongtao, GUAN Haiyan, LI Dilong, et al. A hybrid capsule network for land cover classification using multispectral LiDAR data[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 17(7):1263-1267.
[18] MATIKAINEN L, KARILA K, HYYPPÄ J, et al. Object-based analysis of multispectral airborne laser scanner data for land cover classification and map updating[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 128:298-313.
[19] LEIGH H W, MAGRUDER L A. Using dual-wavelength, full-waveform airborne LiDAR for surface classification and vegetation characterization[J]. Journal of Applied Remote Sensing, 2016, 10(4):045001.
[20] ZOU X L, ZHAO G H, JONATHAN L, et al. 3D land cover classification based on multispectral LiDAR point clouds[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, XLI-B1:741-747.
[21] MORSY S, SHAKER A, EI-RABBANY A, et al. Airborne multispectral LiDAR data for land-cover classification and land/water mapping using different spectral indexes[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2016, Ⅲ-3:217-224.
[22] WICHMANN V, BREMER M, LINDENBERGER J, et al. Evaluating the potential of multispectral airborne LiDAR for topographic mapping and land cover classification[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2015, Ⅱ-3/W5(1):113-119.
[23] GANDI N M I. Classification airborne LiDAR point cloud data[J]. International Journal for Modern Trends in Science and Technology, 2021, 7(1):36-39.
[24] JÄRVINEN A. Airborne laser scanning data comparison based on roof features[D]. Helsinki:Aalto University, 2019.
[25] MORSY S, SHAKER A, EI-RABBANY A. Using multispectral airborne LiDAR data for land/water discrimination:a case study at Lake Ontario, Canada[J]. Applied Sciences, 2018, 8(3):349.
[26] ARTTU S. TerraScan user guide[EB/OL].(2021-07-15)[2022-05-08]. https://www.terrasolid.com/guides/tscan/index.html.
[27] DAI Wenxia, YANG Bisheng, DONG Zhen, et al. A new method for 3D individual tree extraction using multispectral airborne LiDAR point clouds[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2018, 144:400-411.
[28] YU Xiaowei, HYYPPÄ J, LITKEY P, et al. Single-sensor solution to tree species classification using multispectral airborne laser scanning[J]. Remote Sensing, 2017, 9(2):108.
[29] KUKKONEN M, MALTAMO M, KORHONEN L, et al. Comparison of multispectral airborne laser scanning and stereo matching of aerial images as a single sensor solution to forest inventories by tree species[J]. Remote Sensing of Environment, 2019, 231:111208.
[30] KUKKONEN M, MALTAMO M, KORHONEN L, et al. Multispectral airborne LiDAR data in the prediction of boreal tree species composition[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6):3462-3471.
[31] AMIRI N, HEURICH M, KRZYSTEK P, et al. Feature relevance assessment of multispectral airborne LiDAR data for tree species classification[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, XLII-3:31-34.
[32] 赵沛冉, 管海燕, 李迪龙, 等. 利用样本生成方法进行机载多光谱LiDAR数据深度学习分类[J]. 测绘通报, 2021(12):16-21. ZHAO Peiran, GUAN Haiyan, LI Dilong, et al. Deep learning classification of airborne multispectral LiDAR data using sample generation method[J]. Bulletin of Surveying and Mapping, 2021(12):16-21.
[33] 景庄伟. 基于深度学习的多光谱LiDAR点云数据地物分类研究[D]. 南京:南京信息工程大学, 2021. JING Zhuangwei. Research on classification of ground objects based on multi-spectral LiDAR point cloud data based on deep learning[D]. Nanjing:Nanjing University of Information Science & Technology, 2021.
[34] WANG Qingwang, GU Yanfeng. A discriminative tensor representation model for feature extraction and classification of multispectral LiDAR data[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(3):1568-1586.
[35] SUN Jia, SHI Shuo, CHEN Biwu, et al. Combined application of 3D spectral features from multispectral LiDAR for classification[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium. Fort Worth, TX, USA:IEEE, 2017:5264-5267.
[36] EKHTARI N,GLENNIE C,FERNANDEZ-DIAZ J C.Classification of multispectral LiDAR point clouds[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium.Fort Worth,TX, USA:IEEE, 2017:2756-2759.
[37] EKHTARI N,GLENNIE C,FERNANDEZ-DIAZ J C.Classification of airborne multispectral LiDAR point clouds for land cover mapping[J].IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing,2018,11(6):2068-2078.
[38] CHEN Biwu, SHI Shuo, GONG Wei, et al. Multispectral LiDAR point cloud classification:a two-step approach[J]. Remote Sensing, 2017, 9(4):373.
[39] 袁鹏飞,黄荣刚,胡平波,等. 基于多光谱LiDAR数据的道路中心线提取[J].地球信息科学学报,2018, 20(4):452-461. YUAN Pengfei,HUANG Ronggang,HU Pingbo, et al.Road axis extraction method based on multi-spectral LiDAR data[J].Journal of Geo-information Science,2018,20(4):452-461.
[40] CHATZIS S P,VARVARIGOU T A.A fuzzy clustering approach toward hidden Markov random field models for enhanced spatially constrained image segmentation[J].IEEE Transactions on Fuzzy Systems,2008,16(5):1351-1361.
[41] 李玉,徐艳,赵雪梅,等.利用高斯混合模型的多光谱图像模糊聚类分割[J].光学精密工程,2017,25(2):509-518. LI Yu,XU Yan,ZHAO Xuemei,et al.Fuzzy clustering segmentation of multispectral images using Gaussian mixture model[J].Optical precision engineering,2017,25(2):509-518.
[42] 杨俊,李娜,李迟迟,等.基于高斯混合模型和马尔科夫随机场的脑MR图像分割[J].解剖学研究, 2018, 40(5):59-63. YANG Jun, LI Na,LI Chichi,et al. Brain MR image segmentation based on Gaussian mixture model and Markov random field[J].Anatomical study,2018,40(5):59-63.
[43] SHAPOVALOV R, VELIZHEV A, BARINOVA O. Non-associative Markov networks for 3D point cloud classification[C]//Proceedings of 2010 European Conference on Computer Vision.Zurich, Switzerland:Springer,2010:500-515.
[44] NIEMEYER J, ROTTENSTEINER F, SOERGEL U. Classification of urban LiDAR data using conditional random field and random forests[C]//Proceedings of 2013 Joint Urban Remote Sensing Event. Sao Paulo, Brazil:IEEE, 2013:139-142.
[45] NIEMEYER J,ROTTENSTEINER F,SOERGEL U.Conditional random fields for LiDAR point cloud classification in complex urban areas[J].ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences,2012,1(3):263-268.
[46] NIEMEYER J, ROTTENSTEINER F, SOERGEL U, et al. Hierarchical higher order CRF for the classification of airborne LiDAR point clouds in urban areas[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences,2016,XLI-B3:655-662.
[47] 王丽英,王圣,李玉.强度体元基元下的机载LiDAR 3D滤波[J].地球信息科学学报,2019,21(12):1945-1954. WANG Liying, WANG Sheng, LI Yu. Airborne LiDAR 3D filtering under intensity voxels[J].Journal of Earth Information Sciences,2019,21(12):1945-1954.
[48] 袁夏,赵春霞,张浩峰,等.基于点云数据的自然地形分类算法[J].南京理工大学学报(自然科学版), 2010,34(2):222-226, 237. YUAN Xia, ZHAO Chunxia, ZHANG Haofeng, et al. Nature terrain classification using point cloud data[J].Journal of Nanjing University of Science and Technology (Natural Science),2010,34(2):222-226, 237.
[49] LI Chunzhong,XU Zongben.Structure identification-based clustering according to density consistency[J].Mathematical Problems in Engineering,2011,1:890901.
[50] ICHIHASHI H,MIYAGISHI K,HONDA K.Fuzzy C-means clustering with regularization by K-L information[C]//Proceedings of the 10th IEEE International Conference on Fuzzy Systems. Melbourne, Australia:IEEE, 2001:924-927.