Roadside Multiple Objects Extraction from Mobile Laser Scanning Point Cloud Based on DBN

doi:10.11947/j.AGCS.2018.20170524

Abstract

Abstract: This paper proposed an novel algorithm for exploring deep belief network (DBN) architectures to extract and recognize roadside facilities (trees,cars and traffic poles) from mobile laser scanning (MLS) point cloud.The proposed methods firstly partitioned the raw MLS point cloud into blocks and then removed the ground and building points.In order to partition the off-ground objects into individual objects,off-ground points were organized into an Octree structure and clustered into candidate objects based on connected component.To improve segmentation performance on clusters containing overlapped objects,a refining processing using a voxel-based normalized cut was then implemented.In addition,multi-view features descriptor was generated for each independent roadside facilities based on binary images.Finally,a deep belief network (DBN) was trained to extract trees,cars and traffic pole objects.Experiments are undertaken to evaluate the validities of the proposed method with two datasets acquired by Lynx Mobile Mapper System.The precision of trees,cars and traffic poles objects extraction results respectively was 97.31%,97.79% and 92.78%.The recall was 98.30%,98.75% and 96.77% respectively.The quality is 95.70%,93.81% and 90.00%.And the F1 measure was 97.80%,96.81% and 94.73%.

Key words: MLS point cloud, deep belief network (DBN), deep learning, point cloud segmentation, road side objects extraction

CLC Number:

P237

LUO Haifeng, FANG Lina, CHEN Chongcheng, Huang Zhiwen. Roadside Multiple Objects Extraction from Mobile Laser Scanning Point Cloud Based on DBN[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(2): 234-246.

References

[1] YANG Bisheng, WEI Zhen, LI Qingquan, et al. Automated Extraction of Street-scene Objects from Mobile LiDAR Point Clouds[J]. International Journal of Remote Sensing, 2012, 33(18):5839-5861.
[2] LIN Yi, HYYPPA J. k-segments-based Geometric Modeling of VLS Scan Lines[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 8(1):93-97.
[3] 李婷, 詹庆明, 喻亮. 基于地物特征提取的车载激光点云数据分类方法[J]. 国土资源遥感, 2012, 92(1):17-21. LI Ting, ZHAN Qingming, YU Liang. A Classification Method for Mobile Laser Scanning Data Based on Object Feature Extraction[J]. Remote Sensing for Land & Resources, 2012, 92(1):17-21.
[4] 董震, 杨必胜. 车载激光扫描数据中多类目标的层次化提取方法[J]. 测绘学报, 2015, 44(9):980-987. DOI:10.11947/j.AGCS.2015.20140339. DONG Zhen, YANG Bisheng. Hierarchical Extraction of Multiple Objects from Mobile Laser Scanning Data[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(9):980-987. DOI:10.11947/j.AGCS.2015.20140339.
[5] PU Shi, RUTZINGER M, VOSSELMAN G, et al. Recognizing Basic Structures from Mobile Laser Scanning Data for Road Inventory Studies[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(S6):S28-S39.
[6] ZAI Dawei, CHEN Yiping, LI J, et al. Inventory of 3D Street Lighting Poles Using Mobile Laser Scanning Point Clouds[C]//Proceedings of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Milan:IEEE, 2015:573-576.
[7] HUANG Pengdi, CHEN Yiping, LI J, et al. Extraction of Street Trees from Mobile Laser Scanning Point Clouds Based on Subdivided Dimensional Features[C]//Proceedings of 2015 IEEE International Geoscience and Remote Sensing Symposium. Milan:IEEE, 2015:557-560.
[8] YU Yongtao, LI J, LI Jun, et al. Pairwise Three-dimensional Shape Context for Partial Object Matching and Retrieval on Mobile Laser Scanning Data[J]. IEEE Geoscience and Remote Sensing Letters, 2014, 11(5):1019-1023.
[9] YU Yongtao, LI Jun, GUAN Haiyan, et al. Automated Extraction of 3D Trees from Mobile LiDAR Point Clouds[C]//ISPRS International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences. Riva del Garda, Italy:ISPRS, 2014:629-632.
[10] BASU S, GANGULY S, MUKHOPADHYAY S, et al. DeepSat:A Learning Framework for Satellite Imagery[C]//Proceedings of the 23rd SIGSPATIAL International Conference on Advances in Geographic Information Systems. Seattle, Washington:ACM, 2015:37.
[11] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet Classification with Deep Convolutional Neural Networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:Curran Associates Inc., 2012:1097-1105.
[12] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH:IEEE, 2013:580-587.
[13] SUN Yi, WANG Xiaogang, TANG Xiaoou. Deep Learning Face Representation from Predicting 10,000 Classes[C]//Proceedings of 2014 IEEE Conference on Computer Vision and Pattern Recognition. Columbus, OH:IEEE, 2014:1891-1898.
[14] SARIKAYA R, HINTON G E, DEORAS A. Application of Deep Belief Networks for Natural Language Understanding[J]. Proceedings of IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2014, 22(4):778-784.
[15] GRAVES A, MOHAMED A R, HINTON G. Speech Recognition with Deep Recurrent Neural Networks[C]//Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC:IEEE, 2013:6645-6649.
[16] DENG Li, HINTON G, KINGSBURY B. New Types of Deep Neural Network Learning for Speech Recognition and Related Applications:An Overview[C]//Proceedings of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC:IEEE, 2013:8599-8603.
[17] WU Zhirong, SONG Shuran, KHOSLA A, et al. 3D ShapeNets:A Deep Representation for Volumetric Shapes[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:1912-1920.
[18] GARCIA-GARCIA A, GOMEZ-DONOSO F, GARCIA-RODRIGUEZ J, et al. PointNet:A 3D Convolutional Neural Network for Real-time Object Class Recognition[C]//Proceedings of 2016 International Joint Conference on Neural Networks. Vancouver, BC:IEEE, 2016:1578-1584.
[19] SU Hang, MAJI S, KALOGERAKIS E, et al. Multi-View Convolutional Neural Networks for 3D Shape Recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago:IEEE, 2015:945-953.
[20] SONG Shuran, XIAO Jianxiong. Deep Sliding Shapes for Amodal 3D Object Detection in RGB-D Images[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV:IEEE, 2016:808-816.
[21] YU Yongtao, GUAN Haiyan, JI Zheng. Automated Detection of Urban Road Manhole Covers Using Mobile Laser Scanning Data[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(6):3258-3269.
[22] YU Yongtao, LI J, GUAN Haiyan, et al. Automated Detection of Three-dimensional Cars in Mobile Laser Scanning Point Clouds Using DBM-hough-forests[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016, 54(7):4130-4142.
[23] YU Yongtao, LI J, WEN Chenglu, et al. Bag-of-visual-phrases and Hierarchical Deep Models for Traffic Sign Detection and Recognition in Mobile Laser Scanning Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 113:106-123.
[24] HINTON G E, OSINDERO S, TEH Y W. A Fast Learning Algorithm for Deep Belief Nets[J]. Neural Computation, 2016, 18(7):1527-1554.
[25] CHO K H, ILIN A, RAIKO T. Improved Learning of Gaussian-Bernoulli Restricted Boltzmann Machines[C]//Proceedings of the 21th International Conference on Artificial Neural Networks. Espoo, Finland:Springer, 2011:10-17.
[26] HINTON G E. Training Products of Experts by Minimizing Contrastive Divergence[J]. Neural Computation, 2002, 14(8):1771-1800.
[27] 方莉娜, 杨必胜. 车载激光扫描数据的结构化道路自动提取方法[J]. 测绘学报, 2013, 42(2):260-267. FANG Li'na, YANG Bisheng. Automated Extracting Structural Roads from Mobile Laser Scanning Point Clouds[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(2):260-267.
[28] YU Yongtao, LI J, GUAN Haiyan, et al. Automated Extraction of Urban Road Facilities Using Mobile Laser Scanning Data[J]. IEEE Transactions on Intelligent Transportation Systems, 2015, 16(4):2167-2181.
[29] REINGOLD O. Undirected Connectivity in Log-space[J]. Journal of the ACM, 2008, 55(4):17.
[30] SHI Jianbo, MALIK J. Normalized Cuts and Image Segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8):888-905.
[31] ZHONG Lishan, CHENG Liang, XU Hao, et al. Segmentation of Individual Trees From TLS and MLS Data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(2):774-787.
[32] REITBERGER J, SCHNÖRR C, KRZYSTEK P, et al. 3D Segmentation of Single Trees Exploiting Full Waveform LIDAR Data[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2009, 64(6):561-574.
[33] GUAN Haiyan, YU Yongtao, LI J, et al. Pole-like Road Object Detection in Mobile LiDAR Data via Supervoxel and Bag-of-contextual-visual-words Representation[J]. IEEE Geoscience and Remote Sensing Letters, 2016, 13(4):520-524.