From real 3D modeling to digital twin modeling

doi:10.11947/j.AGCS.2022.20210640

Abstract

Abstract: From real 3D modeling to digital twin modeling is the basic requirement for the construction and development of the country's digital economy and smart society. This paper discusses the connotations and key technologies of real 3D modeling and digital twin modeling, and introduces three typical application scenarios: digital villages, future communities and intelligent railways. Wide-area real 3D modeling is facing challenges in low-cost and efficient data collection and intelligent automated modeling and dynamic updating. There are still key technical bottlenecks in the integrated expression of all-element holistic representation data and mechanism models for digital twin modeling at the city level or major engineering level. Surveying and mapping technology is in urgent need of multi-disciplinary cross-integration innovation, breakthroughs in organic collaboration between the sky and the ground, real-time and dynamic acquisition of multi-detailed real 3D data, automated intelligent processing of multi-specialty, multi-scale and multi-modal spatio-temporal data, and detailed 3D construction of complex scenes under incomplete data conditions. Core key technologies such as dynamic construction of full-life-cycle digital twin models that combine characterization data and mechanism models to form general geospatial intelligence to achieve high-quality development of surveying and mapping technology and an irreplaceable and more powerful basic support for economic and social development.

Key words: real 3D model, digital twin model, digital China, smart society, general geospatial intelligence

CLC Number:

P208

ZHU Qing, ZHANG Liguo, DING Yulin, HU Han, GE Xuming, LIU Mingwei, WANG Wei. From real 3D modeling to digital twin modeling[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(6): 1040-1049.

References

[1] 张广运, 张荣庭, 戴琼海, 等. 测绘地理信息与人工智能2.0融合发展的方向[J]. 测绘学报, 2021, 50(8):1096-1108.DOI:10.11947/j.AGCS.2021.20210200. ZHANG Guangyun, ZHANG Rongting, DAI Qionghai, et al. The direction of integration surveying and mapping geographic information and artificial intelligence 2.0[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1096-1108.DOI:10.11947/j.AGCS.2021.20210200.
[2] 张永军, 张祖勋, 龚健雅. 天空地多源遥感数据的广义摄影测量学[J]. 测绘学报, 2021, 50(1):1-11.DOI:10.11947/j.AGCS.2021.20200245. ZHANG Yongjun, ZHANG Zuxun, GONG Jianya. Generalized photogrammetry of spaceborne, airborne and terrestrial multi-source remote sensing datasets[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1):1-11.DOI:10.11947/j.AGCS.2021.20200245.
[3] 陶超, 阴紫薇, 朱庆, 等. 遥感影像智能解译:从监督学习到自监督学习[J]. 测绘学报, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089. TAO Chao, YIN Ziwei, ZHU Qing, et al. Remote sensing image intelligent interpretation:from supervised learning to self-supervised learning[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1122-1134.DOI:10.11947/j.AGCS.2021.20210089.
[4] TAO Fei, ZHANG He, LIU Ang, et al. Digital twin in industry:state-of-the-art[J]. IEEE Transactions on Industrial Informatics, 2019, 15(4):2405-2415.
[5] BOSCHERT S, ROSEN R. Digital twin-the simulation aspect[M].Berlin:Springer, 2016:59-74.
[6] 陈宝权. 从数字城市到数字孪生城市[J]. 中国计算机学会通讯, 2021, 17(7):22-29. CHEN Baoquan. From Digital City to Digital Twin City[J]. Communications of the CCF, 2021, 17(7):22-29.
[7] 陈军, 刘万增, 武昊, 等. 智能化测绘的基本问题与发展方向[J]. 测绘学报, 2021, 50(8):995-1005.DOI:10.11947/j.AGCS.2021.20210235. CHEN Jun, LIU Wanzeng, WU Hao, et al. Smart surveying and mapping:fundamental issues and research agenda[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):995-1005. DOI:10.11947/j.AGCS.2021.20210235.
[8] 顾建祥, 杨必胜, 董震, 等. 面向数字孪生城市的智能化全息测绘[J]. 测绘通报, 2020(6):134-140. GU Jianxiang, YANG Bisheng, DONG Zhen, et al. Intelligent perfect-information surveying and mapping for digital twin cities[J]. Bulletin of Surveying and Mapping, 2020(6):134-140.
[9] 中华人民共和国住房和城乡建设部.城市三维建模技术规范:CJJ/T 157-2010[S].北京:中国建筑工业出版社,2010. Ministry of Housing and Urban-Rural Development of the People's Republic of China. Technical specification for three dimentional city modeling:CJJ/T 157-2010[S].Beijing:China Architecture & Building Press, 2010.
[10] 王密, 龚健雅, 李德仁. 大型无缝影像数据库管理系统的设计与实现[J]. 武汉大学学报(信息科学版), 2003, 28(3):294-300. WANG Mi, GONG Jianya, LI Deren. Design and implementation of large-scale image database management system[J]. Geomatics and Information Science of Wuhan University, 2003, 28(3):294-300.
[11] ZHU Qing, LI Deren, ZHANG Yeting, et al. CyberCity GIS (CCGIS):integration of DEMs, images, and 3D models[J]. Photogrammetric engineering and remote sensing, 2002,68(4):361-368.
[12] 中华人民共和国自然资源部. 实景三维地理信息数据激光雷达测量技术规程:CH/T 3020-2018[S]. 北京:测绘出版社, 2019. Ministry of Natural Resources of the People's Republic of China. Technical regulations for real scene 3D GIS data collection using LiDAR(CH/T 3020-2018)[S].Beijing:Surveying And Mapping Press,2019.
[13] 杨必胜, 韩旭, 董震. 适用于城市场景大规模点云语义标识的深度学习网络[J]. 测绘学报, 2021, 50(8):1059-1067.DOI:10.11947/j.AGCS.2021.20210093. YANG Bisheng, HAN Xu, DONG Zhen. A deep learning network for semantic labeling of large-scale urban point clouds[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(8):1059-1067.DOI:10.11947/j.AGCS.2021.20210093.
[14] 朱庆, 李世明, 胡翰, 等. 面向三维城市建模的多点云数据融合方法综述[J]. 武汉大学学报(信息科学版), 2018, 43(12):1962-1971. ZHU Qing, LI Shiming, HU Han, et al. Multiple point clouds data fusion method for 3D city modeling[J]. Geomatics and Information Science of Wuhan University, 2018, 43(12):1962-1971.
[15] RIZZOLI P, MARTONE M, GONZALEZ C, et al. Generation and performance assessment of the global TanDEM-X digital elevation model[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017, 132:119-139.
[16] DONG Yuting, ZHANG Lu, JIANG Houjun, et al. Cascaded multi-baseline interferometry with bistatic TerraSAR-X/TanDEM-X observations for DEM generation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:224-237.
[17] 楼良盛, 刘志铭, 张昊, 等. 天绘二号卫星工程设计与实现[J]. 测绘学报, 2020, 49(10):1252-1264.DOI:10.11947/j.AGCS.2020.20200175. LOU Liangsheng, LIU Zhiming, ZHANG Hao, et al. TH-2 satellite engineering design and implementation[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(10):1252-1264.DOI:10.11947/j.AGCS.2020.20200175.
[18] BROWN T B, MANN B, RYDER N, et al. Language models are few-shot learners[EB/OL].[2022-01-15].https://arxiv.org/abs/2005.14165.
[19] STOKER J, ABDULLAH Q, NAYEGANDHI A, et al. Evaluation of single photon and geiger mode LiDAR for the 3D elevation program[J]. Remote Sensing, 2016, 8(9):767.
[20] LEICA. Single photon LiDAR:the logical next step in airborne LiDAR mapping[Z]. 2019.
[21] BOSCH M, KURTZ Z, HAGSTROM S, et al. A multiple view stereo benchmark for satellite imagery[C]//Proceedings of 2016 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). Washington, DC, USA:IEEE, 2016:1-9.
[22] SONALI P, BHARATH C, TANMAY P, et al. A new stereo benchmarking dataset for satellite images[EB/OL]. https://arxiv.org/abs/1907.04404.
[23] LETORTU P, TAOUKI R, JAUD M, et al. Three-dimensional (3D) reconstructions of the coastal cliff face in Normandy (France) based on oblique Pléiades imagery:assessment of Ames Stereo Pipeline^® (ASP^®) and MicMac^® processing chains[J]. International Journal of Remote Sensing, 2021, 42(12):4558-4578.
[24] AIRBUS. 3D textured models[EB/OL].[2021-12-28]. https://www.intelligence-airbusds.com/imagery/reference-layers/3d-textured-model/.
[25] CHOY C, DONG Wei, KOLTUN V. Deep global registration[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA:IEEE, 2020:2514-2523.
[26] SARLIN Paul-Edouard, DETONE Daniel, MALISIEWICZ Tomasz, et al SuperGlue:learning feature matching with grath neural Network[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Seattle,State of Washington,USA:CVPR,2020:4938-4947.
[27] LIN C, MA W, TORRALBA A, et al.BARF:Bundle-adjusting neural radiance fields[EB/OL].[2022-01-11]. https://arxiv.org/abs/2104.06405v2.
[28] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651.
[29] SOTO VEGA P J, DA COSTA G A O P, FEITOSA R Q, et al. An unsupervised domain adaptation approach for change detection and its application to deforestation mapping in tropical biomes[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 181:113-128.
[30] 胡翰, 丁雨淋, 朱庆, 等. 面向全球DEM生产的点云智能滤波与DEM泊松编辑方法[J]. 测绘学报, 2019, 48(3):374-383.DOI:10.11947/j.AGCS.2019.20180010. HU Han, DING Yulin, ZHU Qing, et al. Precision global DEM generation based on adaptive surface filter and Poisson terrain editing[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(3):374-383.DOI:10.11947/j.AGCS.2019.20180010.
[31] XU Bo, HU Han, ZHU Qing, et al. Efficient interactions for reconstructing complex buildings via joint photometric and geometric saliency segmentation[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175:416-430.
[32] YU Dawen, JI Shunping, LIU Jin, et al. Automatic 3D building reconstruction from multi-view aerial images with deep learning[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 171:155-170.
[33] KELLY T, GUERRERO P, STEED A, et al. FrankenGAN:guided detail synthesis for building mass-models using style-synchonized GANs[J]. ACM Transactions on Graphics, 2018, 37(6):216.1-216.14.
[34] MVLLER P, ZENG Gang, WONKA P, et al. Image-based procedural modeling of facades[J]. ACM Transactions on Graphics, 2007, 26(3):85.
[35] ZHU Qing, SHANG Qisen, HU Han, et al. Structure-aware completion of photogrammetric meshes in urban road environment[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 175:56-70.
[36] BOSCHERT Stefan, ROSEN Roland. Digital twin-the simulation aspect[M].Mechatronic Futures:Springer, 2016:59-74.
[37] 朱庆, 曾浩炜, 丁雨淋, 等. 重大滑坡隐患分析方法综述[J]. 测绘学报, 2019, 48(12):1551-1561.DOI:10.11947/j.AGCS.2019.20190452. ZHU Qing, ZENG Haowei, DING Yulin, et al. A review of major potential landslide hazards analysis[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(12):1551-1561.DOI:10.11947/j.AGCS.2019.20190452.
[38] 朱庆, 朱军, 黄华平, 等. 实景三维空间信息平台与数字孪生川藏铁路[J]. 高速铁路技术, 2020, 11(2):46-53. ZHU Qing, ZHU Jun, HUANG Huaping, et al. Real 3D spatial information platform and digital twin Sichuan-Tibet railway[J]. High Speed Railway Technology, 2020, 11(2):46-53.
[39] 朱庆,王所智,丁雨淋,等.铁路隧道钻爆法施工智能管理的安全质量进度知识图谱构建方法[J/OL].武汉大学学报(信息科学版):1-16[2022-04-28].http://kns.cnki.net/kcms/detail/42.1676.TN.20211028.1223.004.html. ZHU Qing,WANG Suozhi,DING Yulin,et al.Construction method of "safety-quality-schedule"knowledge graph for intelligent management of drilling and blasting construction of railway tunnels[J].Geomaticsand Information Science of Wuhan University:1-16[2022-04-28].http://kns.cnki.net/kcms/detail/42.1676.TN.20211028.1223.004.html.
[40] 林珲, 朱庆, 陈旻. 有无相生虚实互济:虚拟地理环境研究20周年综述[J]. 测绘学报, 2018, 47(8):1027-1030.DOI:DOI:10.11947/j.AGCS.2018.20180202. LIN Hui, ZHU Qing, CHEN Min. The being and non-being generate each other, and the virtual and the real are mutually interactive-the progress of virtual geographic environments (VGE) studies in last 20 years[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(8):1027-1030.DOI:DOI:10.11947/j.AGCS.2018.20180202.