测绘学报 ›› 2024, Vol. 53 ›› Issue (7): 1355-1370.doi: 10.11947/j.AGCS.2024.20230482
汤圣君1,2(), 杜思齐2, 王伟玺1,2(), 郭仁忠1,2
收稿日期:
2023-10-20
发布日期:
2024-08-12
通讯作者:
王伟玺
E-mail:shengjuntang@szu.edu.cn;wangwx@szu.edu.cn
作者简介:
汤圣君(1991—),男,博士,副研究员,主要研究方向为城市三维要素结构化重建、多传感器融合测图等。E-mail:shengjuntang@szu.edu.cn
基金资助:
Shengjun TANG1,2(), Siqi DU2, Weixi WANG1,2(), Renzhong GUO1,2
Received:
2023-10-20
Published:
2024-08-12
Contact:
Weixi WANG
E-mail:shengjuntang@szu.edu.cn;wangwx@szu.edu.cn
About author:
TANG Shengjun (1991—), male, PhD, associate researcher, majors in urban 3D element structured reconstruction and multi-sensor fusion mapping. E-mail: shengjuntang@szu.edu.cn
Supported by:
摘要:
现有室内三维场景表达聚焦于对象化的描述方法,其要素表达还停留在对象级语义理解层面,欠缺对室内场景复杂关系信息的显示表达。面向室内空间智能任务需求,亟需一个能够完整、准确描述室内要素几何、语义及关系,且具备语义检索和分析推理能力的结构化模型支撑。基于三维场景图基础理论,本文创新性地提出面向室内空间智能的三维场景图表达模型,系统性介绍了室内三维场景图要素层级化组织、几何表达、语义描述、关系描述方法,建立了一个可统一描述室内要素几何、语义及关系的室内三维场景图概念模型。同时,该图模型可以与现有的三维场景表达方法进行融合表达,具有良好的数据兼容性。最终,本文基于公开的IFC模型构建了完整且具有多层级关系信息的三维场景图模型,并且结合大语言模型通过复杂场景检索、拓扑分析等应用对该模型的应用能力、潜力及局限性进行了系统性探讨和分析。结果表明,室内三维场景图模型具备复杂计算和分析能力,可直接与大语言模型集成,通过简单的自然语言提示实现复杂的场景分析应用。
中图分类号:
汤圣君, 杜思齐, 王伟玺, 郭仁忠. 面向室内空间智能的三维场景图表达与应用[J]. 测绘学报, 2024, 53(7): 1355-1370.
Shengjun TANG, Siqi DU, Weixi WANG, Renzhong GUO. 3D scene graph representation and application for intelligent indoor spaces[J]. Acta Geodaetica et Cartographica Sinica, 2024, 53(7): 1355-1370.
表1
各层级要素类型描述"
级别 | 描述 |
---|---|
复杂建筑要素 | 用于描述一个建筑群,一座建筑可以跨越几个相连或不相连的建筑,组成一个建筑群,它是由一个以上的独立建筑组成的。 |
单体建筑要素 | 用于描述一个单体建筑,一个单体建筑至少包含一个楼层要素,建筑中不同楼层可能具有特定用途或所有权。 |
楼层要素 | 用于描述单个楼层空间,每个楼层由各种室内空间要素和室内实体要素组成。 |
功能空间要素 | 用于描述具有室内空间功能属性的结构信息,可以包含房间、阳台、走廊等功能空间。 |
连接空间要素 | 用于描述连接不同楼层的要素信息,可以包含楼梯、电梯等。 |
构造实体要素 | 用于描述建筑的固有组成部分的要素信息,包含墙、楼板、梁、柱、门、窗等要素。 |
附属实体要素 | 用于描述附属于建筑物内特定空间的要素信息,包含家具、设施、传感器等要素。 |
[1] | 应申, 朱利平, 李霖, 等. 基于室内空间特征的室内地图表达[J]. 导航定位学报, 2015, 3(4):74-78, 91. |
YING Shen, ZHU Liping, LI Lin, et al. Representation of indoor maps with the analysis of indoor space[J]. Journal of Navigation and Positioning, 2015, 3(4):74-78, 91. | |
[2] | 高伟, 吴毅红, 申抒含, 等. 视觉主导的多传感器融合机器地图构建与定位技术[M]//卫星导航定位与北斗系统应用. 北京: 测绘出版社, 2019: 138-143. |
GAO Wei, WU Yihong, SHEN Shuhan, et al. Vision led multi-sensor fusion machine map construction and positioning technology [M]//Satellite navigation positioning and Beidou system application. Beijing: Surveying and Mapping Press, 2019: 138-143 | |
[3] | 徐凯, 胡瑞珍, 杨鑫. 几何引导的主动式三维感知与交互[J]. 图学学报, 2022, 43(6):1049-1056. |
XU Kai, HU Ruizhen, YANG Xin. Geometry-guided active 3D perception and interaction[J]. Journal of Graphics, 2022, 43(6):1049-1056. | |
[4] | LAAKSO M, KIVINIEMI A. The IFC standard: a review of history, development, and standardization[J]. Electronic Journal of Information Technology in Construction, 2012, 17:134-161. |
[5] | GRÖGER G, PLÜMER L. CityGML: interoperable semantic 3D city models[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2012, 71:12-33. |
[6] | LI K J, CONTI G, KONSTANTINIDIS E, et al. OGC IndoorGML: a standard approach for indoor maps[M]//Geographical and fingerprinting data to create systems for indoor positioning and indoor/outdoor navigation. Amsterdam: Elsevier, 2019: 187-207. |
[7] | 朱庆, 张利国, 丁雨淋, 等. 从实景三维建模到数字孪生建模[J]. 测绘学报, 2022, 51(6):1040-1049. DOI: 10.11947/j.AGCS.2022.20210640. |
ZHU Qing, ZHANG Liguo, DING Yulin, et al. From real 3D modeling to digital twin modeling[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(6):1040-1049. DOI: 10.11947/j.AGCS.2022.20210640. | |
[8] | 游天, 周成虎, 陈曦. 室内地图表示方法研究与实践[J]. 测绘科学技术学报, 2014, 31(6):635-640. |
YOU Tian, ZHOU Chenghu, CHEN Xi. The research and practice of indoor map representation[J]. Journal of Geomatics Science and Technology, 2014, 31(6):635-640. | |
[9] | 危双丰, 刘振彬, 赵江洪, 等. SLAM室内三维重建技术综述[J]. 测绘科学, 2018, 43(7):15-26. |
WEI Shuangfeng, LIU Zhenbin, ZHAO Jianghong, et al. A review of indoor 3D reconstruction with SLAM[J]. Science of Surveying and Mapping, 2018, 43(7):15-26. | |
[10] | 崔扬. 移动LiDAR点云室内三维结构化重建方法和关键技术研究[J]. 测绘学报, 2021, 50(7):990. DOI: 10.11947/j.AGCS.2021.20200592. |
CUI Yang. Research on methodology and the key technology of indoor 3D structured reconstruction from mobile LiDAR point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(7):990. DOI: 10.11947/j.AGCS.2021.20200592. | |
[11] | 付燕平, 严庆安, 廖杰, 等. 基于彩色图像引导的RGB-D相机追踪与三维重建[J]. 武汉大学学报(工学版), 2022, 55(1):92-100. |
FU Yanping, YAN Qing'an, LIAO Jie, et al. RGB-D camera tracking and 3D reconstruction via color image guiding[J]. Engineering Journal of Wuhan University, 2022, 55(1):92-100. | |
[12] | WALD J, DHAMO H, NAVAB N, et al. Learning 3D semantic scene graphs from 3D indoor reconstructions[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 3961-3970. |
[13] | 王玮琦, 游雄, 杨剑, 等. 一种改进匹配点对选取策略的ElasticFusion室内三维重建算法[J]. 武汉大学学报(信息科学版), 2020, 45(9):1469-1477. |
WANG Weiqi, YOU Xiong, YANG Jian, et al. Elastic fusion for indoor 3D reconstruction with an improved matching points selection strategy[J]. Geomatics and Information Science of Wuhan University, 2020, 45(9):1469-1477. | |
[14] | HOU Ji, DAI A, NIEBNER M. 3D-SIS: 3D semantic instance segmentation of RGB-D scans[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 4421-4430. |
[15] | SU Hang, MAJI S, KALOGERAKIS E, et al. Multi-view convolutional neural networks for 3D shape recognition[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago: IEEE, 2015: 945-953. |
[16] | SONG Shuran, LICHTENBERG S P, XIAO Jianxiong. SUN RGB-D: a RGB-D scene understanding benchmark suite[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 567-576. |
[17] | NAJIBI M, LAI Guangda, KUNDU A, et al. DOPS: learning to detect 3D objects and predict their 3D shapes[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11913-11922. |
[18] | 熊汉江, 郑先伟, 丁友丽, 等. 基于2D-3D语义传递的室内三维点云模型语义分割[J]. 武汉大学学报(信息科学版), 2018, 43(12):2303-2309. |
XIONG Hanjiang, ZHENG Xianwei, DING Youli, et al. Semantic segmentation of indoor 3D point cloud model based on 2D-3D semantic transfer[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):2303-2309. | |
[19] | YI Li, ZHAO Wang, WANG He, et al. GSPN: generative shape proposal network for 3D instance segmentation in point cloud[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 3947-3956. |
[20] | LIU Yunze, YI Li, ZHANG Shanghang, et al. P4Contrast: contrastive learning with pairs of point-pixel pairs for RGB-D scene understanding[EB/OL]. [2023-12-01]. http://arxiv.org/pdf/1907.11692. |
[21] | CHARLES R Q, HAO Su, MO Kaichun, et al. PointNet: deep learning on point sets for 3D classification and segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu: IEEE, 2017: 652-660. |
[22] | 邓林涛, 方志军. 基于特征负反馈卷积的点云分析方法[J]. 激光与光电子学进展, 2022, 59(12):1210006. |
DENG Lintao, FANG Zhijun. Point cloud analysis method based on feature negative feedback convolution[J]. Laser & Optoelectronics Progress, 2022, 59(12):1210006. | |
[23] | CHEN X, WU H, LICHTI D, et al. Extraction of indoor objects based on the exponential function density clustering model[J]. Information Sciences, 2022, 607:1111-1135. |
[24] | NING Xiaojuan, WANG Yinghui, HAO Wen, et al. Structure-based object classification and recognition for 3D scenes in point clouds[C]//Proceedings of 2014 International Conference on Virtual Reality and Visualization. Shenyang: IEEE, 2014: 166-173. |
[25] | KIRILLOV A, HE Kaiming, GIRSHICK R, et al. Panoptic segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach: IEEE, 2019: 9404-9413. |
[26] | REN Dayong, WU Zhengyi, LI Jiawei, et al. Point attention network for point cloud semantic segmentation[J]. Science China Information Sciences, 2022, 65(9):192104. |
[27] | CHEN Kai, OUYANG W, LOY C C, et al. Hybrid task cascade for instance segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recgnition. Long Beach: IEEE, 2019: 4974-4983. |
[28] | RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference of Medical Image Computing and Computer. Cham: Springer, 2015: 234-241. |
[29] | FANG Hao, LAFARGE F, PAN Cihui, et al. Floorplan generation from 3D point clouds: a space partitioning approach[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175:44-55. |
[30] | LIU Chen, WU Jiajun, KOHLI P, et al. Raster-to-vector: revisiting floorplan transformation[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 2195-2203. |
[31] | 张蔚, 王腾, 卢政达, 等. 基于CNN的住宅平面图元素识别与布局语义分析[J]. 中国体视学与图像分析, 2020, 25(2):174-182. |
ZHANG Wei, WANG Teng, LU Zhengda, et al. Floor plan recognition and semantic layout analysis based on a convolutional neural network[J]. Chinese Journal of Stereology and Image Analysis, 2020, 25(2):174-182. | |
[32] | FANG Hao, PAN Cihui, HUANG Hui. Structure-aware indoor scene reconstruction via two levels of abstraction[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 178:155-170. |
[33] | HAN Jiali, RONG Mengqi, JIANG Hanqing, et al. Vectorized indoor surface reconstruction from 3D point cloud with multistep 2D optimization[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177:57-74. |
[34] | TRAN H, KHOSHELHAM K. Procedural reconstruction of 3D indoor models from lidar data using reversible jump Markov chain Monte Carlo[J]. Remote Sensing, 2020, 12(5):838. |
[35] | HAN Jiali, LIU Yuzhou, RONG Mengqi, et al. FloorUSG: indoor floorplan reconstruction by unifying 2D semantics and 3D geometry[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2023, 196:490-501. |
[36] | JAYARAJ P, RAMIYA A M. D citygml building modelling from LiDAR point cloud data[J]. International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, 425:175-180. |
[37] | 王敬淳, 汤圣君, 王伟玺, 等. 室内三维点云空间自动划分与规则化方法[J]. 测绘通报, 2022(8):93-97, 138. |
WANG Jingchun, TANG Shengjun, WANG Weixi, et al. Automatic division and regularization of indoor 3D point cloud space[J]. Bulletin of Surveying and Mapping, 2022(8):93-97, 138. | |
[38] | TANG Shengjun, LI Xiaoming, ZHENG Xianwei, et al. BIM generation from 3D point clouds by combining 3D deep learning and improved morphological approach[J]. Automation in Construction, 2022, 141:104422. |
[39] | 丁雨淋, 何小波, 朱庆, 等. 实时威胁态势感知的室内火灾疏散路径动态优化方法[J]. 测绘学报, 2016, 45(12):1464-1475. DOI: 10.11947/j.AGCS.2016.20160053. |
DING Yulin, HE Xiaobo, ZHU Qing, et al. A dynamic optimization method of indoor fire evacuation route based on realtime situation awareness[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(12):1464-1475. DOI: 10.11947/j.AGCS.2016.20160053. | |
[40] | 邓中亮, 王小恒. 一种用于大型建筑火灾中的应急疏散算法[J]. 软件, 2011, 32(2):112-114. |
DENG Zhongliang, WANG Xiaoheng. An emergency evacuation algorithm in large building fire[J]. Computer Engineering & Software, 2011, 32(2):112-114. | |
[41] | 朱庆, 胡明远, 许伟平, 等. 面向火灾动态疏散的三维建筑信息模型[J]. 武汉大学学报(信息科学版), 2014, 39(7):762-766, 872. |
ZHU Qing, HU Mingyuan, XU Weiping, et al. 3D building information model for facilitating dynamic analysis of indoor fire emergency[J]. Geomatics and Information Science of Wuhan University, 2014, 39(7):762-766, 872. | |
[42] | 韩李涛, 郭欢, 张海思. 一种多出口室内应急疏散路径规划算法[J]. 测绘科学, 2018, 43(12):105-110. |
HAN Litao, GUO Huan, ZHANG Haisi. An algorithm for route planning applied in multi-exit indoor emergency evacuation[J]. Science of Surveying and Mapping, 2018, 43(12):105-110. | |
[43] | 李清泉, 周宝定, 马威, 等. GIS辅助的室内定位技术研究进展[J]. 测绘学报, 2019, 48(12):1498-1506. DOI: 10.11947/j.AGCS.2019.20190455. |
LI Qingquan, ZHOU Baoding, MA Wei, et al. Research process of GIS-aided indoor localization[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1498-1506. DOI: 10.11947/j.AGCS.2019.20190455. | |
[44] | 刘涛, 张星, 李清泉, 等. 顾及地标可视性的室内导航路径优化算法[J]. 武汉大学学报(信息科学版), 2017, 42(1):43-48. |
LIU Tao, ZHANG Xing, LI Qingquan, et al. An indoor pedestrian route planning algorithm based on landmark visibility[J]. Geomatics and Information Science of Wuhan University, 2017, 42(1):43-48. | |
[45] | 陈锐志, 陈亮. 基于智能手机的室内定位技术的发展现状和挑战[J]. 测绘学报, 2017, 46(10):1316-1326. DOI: 10.11947/j.AGCS.2017.20170383. |
CHEN Ruizhi, CHEN Liang. Indoor positioning with smartphones: the state-of-the-art and the challenges[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(10):1316-1326. DOI: 10.11947/j.AGCS.2017.20170383. | |
[46] | WALD J, NAVAB N, TOMBARI F. Learning 3D semantic scene graphs with instance embeddings[J]. International Journal of Computer Vision, 2022, 130(3):630-651. |
[47] | ZHU G, ZHANG L, JIANG Y, et al. Scene graph generation: a comprehensive survey[EB/OL]. [2024-01-25]. http://arxiv.org/abs/2201.00443. |
[48] | ZLATANOVA S. On 3D topological relationships[C]//Proceedings of 2000 International Workshop on Database & Expert Systems Applications. London: IEEE, 2000. |
[49] | IFC Wiki. KIT IFC Examples[EB/OL]. [2024-02-15]. https://www.ifcwiki.org/index.php?title=KIT_IFC_Examples. |
[1] | 徐龙威, 吴忠望, 董绪荣. GLONASS频间码偏差实时估计方法及其在RTK定位中的应用[J]. 测绘学报, 2022, 51(2): 169-181. |
[2] | 王永君, 陈青燕, 杨玉娇, 陈学业, 孙剑. 语义辅助的CityGML模型一致性检测方法[J]. 测绘学报, 2021, 50(5): 664-674. |
[3] | 朱庆, 冯斌, 李茂粟, 陈媚特, 徐肇文, 谢潇, 张叶廷, 刘铭崴, 黄志勤, 冯义从. 面向动态关联数据的高效稀疏图索引方法[J]. 测绘学报, 2020, 49(6): 681-691. |
[4] | 黄鸿, 陈美利, 王丽华, 李政英. 空-谱协同正则化稀疏超图嵌入的高光谱图像分类[J]. 测绘学报, 2019, 48(6): 676-687. |
[5] | 徐龙威, 刘晖, 舒宝, 郑福, 温景仁. GLONASS频间码偏差特性分析及其在宽巷模糊度固定中的应用[J]. 测绘学报, 2018, 47(4): 465-472. |
[6] | 赵飞. 指标驱动下以任务流为中心的在线交互制图模型[J]. 测绘学报, 2011, 40(5): 0-0. |
[7] | 李霖,朱海红,贺 彪,王红,邱俊武,于忠海. 基于代数结构的地形图制图模型[J]. 测绘学报, 2011, 40(3): 373-378. |
[8] | 谢忠; 魏东琦; 吴亮; 郭际元
|
阅读次数 | ||||||
全文 |
|
|||||
摘要 |
|
|||||