Ranked batch-mode active learning method for semantic annotation of point cloud scene

doi:10.11947/j.AGCS.2023.20210332

Abstract

Abstract: Due to the semantic annotation of point cloud scene by manual annotation is time-consuming and label cost process, and the annotation accuracy is not high, the point cloud processing is not suitable for large-scale scenes, this paper proposed an active learning annotation method of point cloud based on ranked batch-mode. This method firstly downsampling the original point cloud, then an improved recursive feature addition method is used to filter out the optimal feature subset from a huge feature set, and a ranked batch-mode sampling algorithm is adopted to iteratively select and manually label fraction of unlabeled points. The semantic annotation of the down-sampled point cloud is completed by creating a minimum manual annotation training set, and finally the original point cloud data is annotated using the neighborhood equal-weight label propagation algorithm. Experiments on three outdoor large scene point clouds show that the method in this paper only needs to manually label 7.50%, 7.35%, and 5.83% of the point clouds to complete the labeling of the down-sampled point clouds. In addition, comparative experiments show that this method is superior to other methods in labeling accuracy and reducing labor costs, and save a lot of labor costs for point cloud semantic annotation work.

Key words: point cloud scene, ranked batch-mode, feature selection, active learning, semantic annotation

CLC Number:

P237

ZOU Lujie, HUA Xianghong, ZHAO Bufan, TAO Wuyong, LI Qiqi. Ranked batch-mode active learning method for semantic annotation of point cloud scene[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 260-271.

References

[1] 李德仁, 邵振峰, 于文博, 等. 基于时空位置大数据的公共疫情防控服务让城市更智慧[J]. 武汉大学学报(信息科学版), 2020, 45(4):475-487, 556. LI Deren, SHAO Zhenfeng, YU Wenbo, et al. Public epidemic prevention and control services based on big data of spatio-temporal location make cities more smart[J]. Geomatics and Information Science of Wuhan University, 2020, 45(4):475-487, 556.
[2] 刘经南, 詹骄, 郭迟,等.智能高精地图数据逻辑结构与关键技术[J]. 测绘学报, 2019, 48(8):939-953. DOI:10.11947/j.AGCS.2019.20190125. LIU Jingnan, ZHAN Jiao, GUO Chi, et al. Data logic structure and key technologies on intelligent high-precision map[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(8):939-953. DOI:10.11947/j.AGCS.2019.20190125.
[3] 李德仁, 姚远, 邵振峰. 智慧城市中的大数据[J]. 武汉大学学报(信息科学版), 2014, 39(6):631-640. LI Deren, YAO Yuan, SHAO Zhenfeng. Big data in smart city[J]. Geomatics and Information Science of Wuhan University, 2014, 39(6):631-640.
[4] 杨必胜, 董震. 点云智能研究进展与趋势[J]. 测绘学报, 2019, 48(12):1575-1585. DOI:10.11947/j.AGCS.2019.20190465. YANG Bisheng, DONG Zhen. Progress and perspective of point cloud intelligence[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1575-1585. DOI:10.11947/j.AGCS.2019.20190465.
[5] ZHAO Bufan, HUA Xianghong, YU Kegen, et al. Indoor point cloud segmentation using iterative Gaussian mapping and improved model fitting[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(11):7890-7907.
[6] ZHANG Liangpei, ZHANG Yun, CHEN Zhenzhong, et al. Splitting and merging based multi-model fitting for point cloud segmentation[J]. Journal of Geodesy and Geoinformation Science, 2020, 2(2):78-89.
[7] 蒋腾平, 王永君, 张林淇, 等.融合CNN和MRF的激光点云层次化语义分割方法[J]. 测绘学报, 2021, 50(2):215-225. DOI:10.11947/j.AGCS.2021.20220095. JIANG Tengping, WANG Yongjun, ZHANG Linqi, et al. A LiDAR point cloud hierarchical semantic segmentation method combining CNN and MRF[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2):215-225. DOI:10.11947/j.AGCS.2021.20220095.
[8] LI S Z. Markov random field modeling in image analysis[M]. London:Springer Science & Business Media, 2009.
[9] LI Yan, DAI Jicheng, TAN Junxiang, et al. Global fine registration of point cloud in LiDAR SLAM based on pose graph[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):26-35.
[10] ANAND A, KOPPULA H S, JOACHIMS T, et al. Contextually guided semantic labeling and search for 3D point clouds[J]. International Journal of Robotics Research, 2011, 32(1):19-34.
[11] MUNOZ D, VANDAPEL N, HEBERT M. Onboard contextual classification of 3D point clouds with learned high-order Markov random fields[C]//Proceedings of 2009 IEEE International Conference on Robotics and Automation. Kobe, Japan:IEEE, 2009.
[12] LI Zhuqiang, ZHANG Liqiang, ZHONG Ruofei, et al. Classification of urban point clouds:a robust supervised approach with automatically generating training data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2016, 10(3):1207-1220.
[13] WANG Yan, JI Rongrong, CHANG Shifu. Label propagation from imagenet to 3D point clouds[C]//Proceedings of 2013 IEEE conference on computer vision and pattern recognition. Portland, OR, USA:IEEE, 2013:3135-3142.
[14] FLOROS G, LEIBE B. Joint 2D-3D temporally consistent semantic segmentation of street scenes[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2012:2823-2830.
[15] FU Yifan, ZHU Xingquan, LI Bin. A survey on instance selection for active learning[J]. Knowledge and Information Systems, 2013, 35(2):249-283.
[16] WANG Yu, MENDEZ A E M, CARTWRIGHT M, et al. Active learning for efficient audio annotation and classification with a large amount of unlabeled data[C]//Proceedings of 2019 ICASSP International Conference on Acoustics, Speech and Signal Processing. Brighton, UK:IEEE, 2019:880-884.
[17] THOMAS H, QI C R, DESCHAUD J E, et al. Kpconv:flexible and deformable convolution for point clouds[C]//Proceedings of 2019 IEEE/CVF International Conference on Computer Vision. Seoul, South Korea:IEEE, 2019:6411-6420.
[18] YANG L, MACEACHREN A M, MITRA P, et al. Visually-enabled active deep learning for (geo) text and image classification:a review[J]. ISPRS International Journal of Geo-Information, 2018, 7(2):65.
[19] VEZHNEVETS A, BUHMANN J M, FERRARI V. Active learning for semantic segmentation with expected change[C]//Proceedings of 2012 IEEE conference on computer vision and pattern recognition. Providence, RI, USA:IEEE, 2012:3162-3169.
[20] LIN Y, VOSSELMAN G, CAO Y, et al. Active and incremental learning for semantic ALS point cloud segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 169:73-92.
[21] 罗欢. 高分辨率激光扫描点云语义标注研究[D].厦门:厦门大学,2017. LUO Huan. Research on semantic labeling of high-resolution laser scanning point clouds[D]. Xiamen:Xiamen University,2017.
[22] LUO H, WANG C, Wen C, et al. Semantic labeling of mobile LiDAR point clouds via active learning and higher order MRF[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(7):3631-3644.
[23] ADAMS A, GELFAND N, DOLSON J, et al. Gaussian kd-trees for fast high-dimensional filtering[M]. New York:ACM press. 2009:1-12.
[24] WEINMANN M, JUTZI B, HINZ S, et al. Semantic point cloud interpretation based on optimal neighborhoods, relevant features and efficient classifiers[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2015, 105:286-304.
[25] MALLET C, BRETAR F, ROUX M, et al. Relevance assessment of full-waveform LiDAR data for urban area classification[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2011, 66(6):S71-S84.
[26] TAO Wuyong, HUA Xianghong, WANG Ruisheng, et al. Quintuple local coordinate images for local shape description[J]. Photogrammetric Engineering & Remote Sensing, 2020, 86(2):121-132.
[27] TAO Wuyong, HUA Xianghong, YU Kegen, et al. A pipeline for 3D object recognition based on local shape description in cluttered scenes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(1):801-816.
[28] WEINMANN M, JUTZI B, MALLET C. Feature relevance assessment for the semantic interpretation of 3D point cloud data[J]. ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2013, 5(W2):1.
[29] GUYON I, WESTON J, BARNHILL S, et al. Gene selection for cancer classification using support vector machines[J]. Machine learning, 2002, 46(1):389-422.
[30] GR MPING U. Variable importance assessment in regression:linear regression versus random forest[J]. The American Statistician, 2009, 63(4):308-319.
[31] CARDOSO T N C, SILVA R M, CANTUO S, et al. Ranked batch-mode active learning[J]. Information Sciences, 2017, 379:313-337.
[32] LEWIS D D, GALE W A. A sequential algorithm for training text classifiers[C]//Proceedings of 1994 SIGIR. London, UK:Springer,1994:3-12.
[33] BRANDT J. Transform coding for fast approximate nearest neighbor search in high dimensions[C]//Proceedings of 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Francisco, CA, USA:IEEE, 2010:1815-1822.
[34] XU S, WANG R, WANG H, et al. An optimal hierarchical clustering approach to mobile LiDAR point clouds[J]. IEEE Transactions on Intelligent Transportation Systems, 2019, 21(7):2765-2776.
[35] HACKEL T, SAVINOV N, LADICKY L, et al. Semantic 3D.net:a new large-scale point cloud classification benchmark[J]. ISPRS Annals of Photogrammetry, Remote Sensing and Spatial Information Sciences, 2017(IV-1/W1):91-98.
[36] PATRA S, BRUZZONE L. A batch-mode active learning technique based on multiple uncertainty for SVM classifier[J]. IEEE Geoscience and Remote Sensing Letters, 2011, 9(3):497-501.