Detection of damaged buildings based on generative adversarial networks

doi:10.11947/j.AGCS.2022.20200318

Abstract

Abstract: As one of the most affected hazard-affected bodies in natural disasters, accurate damage information extraction of buildings plays a significant role in post-disaster emergency rescue. Referring to the idea of multi-mode fusion technique, a recursive generative adversarial networks (RS-GAN) method is proposed to automatically detect damaged buildings. In RS-GAN, the workflow of damaged buildings detection is composed of two sub-tasks as follows:building identification before disasters as well as damaged building detection after disasters, which are completed in two GAN branches respectively. Specifically, RS-GAN adds a joint loss function to connect the two GAN branches, making full use of the potential mutual benefit between the two tasks to improve the detection accuracy. In addition, the results of building identification are added to the damaged building detection task to locate potential damaged areas. The method proposed in this paper is an end-to-end model, which can automatically detect damaged buildings without excessive manual intervention. To verify the effect of the RS-GAN model, in this paper, two experiments were set with the Santa Rosa dataset and Missouri respectively. Experimental results show that RS-GAN method has better detection performance compared to other competitive methods, and the overall accuracy and average accuracy on the Santa Rosa data set are 0.90 and 0.86, respectively.

Key words: generative adversarial networks, pre-disaster and post-disaster bitemporal images, building contour extraction, damaged building detection, step-by-step learning

CLC Number:

P237

GE Xiaosan, CHEN Xi, ZHAO Wenzhi, LI Ruixiang. Detection of damaged buildings based on generative adversarial networks[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(2): 238-247.

References

[1] 黄露. 基于机器学习的汶川震区滑坡灾害气象预警模型研究[J]. 测绘学报, 2020, 49(2):267.DOI:10.11947/j.AGCS.2020.20190061. HUANG Lu. Research on meteorological early-warning model of landslides in Wenchuan earthquake area based on machine learning[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2):267.DOI:10.11947/j.AGCS.2020.20190061.
[2] DONG Laigen, SHAN Jie. A comprehensive review of earthquake-induced building damage detection with remote sensing techniques[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 84:85-99.
[3] 陶和平, 刘斌涛, 刘淑珍, 等. 遥感在重大自然灾害监测中的应用前景:以5·12汶川地震为例[J]. 山地学报, 2008, 26(3):276-279. TAO Heping, LIU Bintao, LIU Shuzhen, et al. Natural hazards monitoring using remote sensing-a case study of 5·12 Wenchuan earthquake[J]. Journal of Mountain Science, 2008, 26(3):276-279.
[4] 眭海刚, 刘超贤, 黄立洪, 等. 遥感技术在震后建筑物损毁检测中的应用[J]. 武汉大学学报(信息科学版), 2019, 44(7):1008-1019. SUI Haigang, LIU Chaoxian, HUANG Lihong, et al. Application of remote sensing technology in earthquake-induced building damage detection[J]. Geomatics and Information Science of Wuhan University, 2019, 44(7):1008-1019.
[5] YAMAZAKI F, SUZUKI D, MARUYAMA Y. Detection of damages due to earthquakes using digital aerial images[C]//Proceedings of the 6th International Workshop on Remote Sensing for Disaster Applications, Pavia, Italy:[s.n.],2008.
[6] 谢嘉丽, 李永树, 李何超, 等. 利用灰度共生矩阵纹理特征识别空心村损毁建筑物的方法[J]. 测绘通报, 2017(12):90-93, 102. XIE Jiali, LI Yongshu, LI Hechao, et al. Recognition of damage buildings in hollow village based on texture feature of gray level co-occurrence matrix[J]. Bulletin of Surveying and Mapping, 2017(12):90-93, 102.
[7] 何浩, 刘修国, 沈永林. 基于视差的高分辨率遥感影像建筑物变化检测[J]. 遥感技术与应用, 2019, 34(6):1315-1323. HE Hao, LIU Xiuguo, SHEN Yonglin. Building change detection method considering the parallax for high resolution remote sensing image[J]. Remote Sensing Technology and Application, 2019, 34(6):1315-1323.
[8] 刘莹, 陶超, 闫培, 等. 图割能量驱动的高分辨率遥感影像震害损毁建筑物检测[J]. 测绘学报, 2017, 46(7):910-917.DOI::10.11947/j.AGCS.2017.20170035. LIU Ying, TAO Chao, YAN Pei, et al. Graph cut energy driven earthquake-damaged building detection from high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(7):910-917.DOI::10.11947/j.AGCS.2017.20170035.
[9] 刘莹, 李强. 融合多特征的高分辨率遥感影像震害损毁建筑物检测[J]. 测绘与空间地理信息, 2018, 41(6):61-64. LIU Ying, LI Qiang. Damaged building detection from high resolution remote sensing images by integrating multiple features[J]. Geomatics & Spatial Information Technology, 2018, 41(6):61-64.
[10] 李强, 张景发. 不同特征融合的震后损毁建筑物识别研究[J]. 地震研究, 2016,39(3):486-493, 527. LI Qiang, ZHANG Jingfa. Research on earthquake damaged building extraction by different features fusion[J]. Journal of Seismological Research, 2016, 39(3):486-493, 527.
[11] SONOBE M. Characteristics of texture index of damaged buildings using time-series high-resolution optical satellite images[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, 43:1709-1714.
[12] 袁玉, 申旭辉, 薛腾飞, 等. SAR图像变化检测提取建筑物震害信息的研究综述[J]. 地壳构造与地壳应力文集, 2017(1):131-143. YUAN Yu, SHEN Xuhui, XUE Tengfei, et al. Review of researching for building earthquake damage information extraction based on SAR images change detection[J]. Bulletin of theInstituteof Crustal Dynamics, 2017(1):131-143.
[13] 许志华. 基于低空影像和地面LiDAR数据的建筑物损毁评估方法研究[J]. 地理与地理信息科学, 2018, 34(4):126. XU Zhihua. Building damage assessment from unmanned aerial vehicle images and terrestrial LiDAR data[J]. Geography and Geo-Information Science, 2018, 34(4):126.
[14] SUN Long, WU Tao, SUN Guangcai, et al. Object detection research of SAR image using improved faster RegionBasedconvolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):18-28.
[15] 叶昕, 王俊, 秦其明. 基于高分一号卫星遥感图像的建筑物震害损毁检测研究:以2015年尼泊尔MS8.1地震为例[J]. 地震学报, 2016,38(3):477-485, 509. YE Xin, WANG Jun, QIN Qiming. Damaged building detection based on GF-1 satellite remote sensing image:a case study for Nepal MS8.1 earthquake[J]. Acta Seismologica Sinica, 2016,38(3):477-485, 509.
[16] SAITO K, SPENCE R J S, GOING C, et al. Using high-resolution satellite images for post-earthquake building damage assessment:a study following the 26 January 2001 gujarat earthquake[J]. Earthquake Spectra, 2004, 20(1):145-169.
[17] LIU Sicong, DU Qian, TONG Xiaohua, et al. Multiscale morphological compressed change vector analysis for unsupervised multiple change detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(9):4124-4137.
[18] NEETI N, EASTMAN J R. Novel approaches in extended principal component analysis to compare spatio-temporal patterns among multiple image time series[J]. Remote Sensing of Environment, 2014, 148:84-96.
[19] 马国锐, 吴娇, 姚聪. 多尺度特征融合的建筑物及损毁语义分割[J]. 测绘工程, 2020, 29(4):1-6. MA Guorui, WU Jiao, YAO Cong. Semantic segmentation of building and damage based on multi-scale feature fusion[J]. Engineering of Surveying and Mapping, 2020, 29(4):1-6.
[20] MIURA H, ARIDOME T, MATSUOKA M. Deep learning-based identification of collapsed, non-collapsed and blue tarp-covered buildings from post-disaster aerial images[J]. Remote Sensing, 2020, 12(12):1924.
[21] 左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报,2019,48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740. ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740.
[22] DAI Yuchao,ZHANG Jing,HE Mingyi,et al.Salient object detection from multi-spectral remote sensing images with deep residual network[J].Journal of Geodesy and Geoinformation Science,2019,2(2):101-110.
[23] 周阳, 张云生, 陈斯飏, 等. 基于DCNN特征的建筑物震害损毁区域检测[J]. 国土资源遥感, 2019, 31(2):44-50. ZHOU Yang, ZHANG Yunsheng, CHEN Siyang, et al. Disaster damage detection in building areas based on DCNN features[J]. Remote Sensing for Land & Resources, 2019, 31(2):44-50.
[24] LI Yundong, YE Shi, BARTOLI I. Semisupervised classification of hurricane damage from postevent aerial imagery using deep learning[J]. Journal of Applied Remote Sensing, 2018, 12(4):045008.
[25] HEZAVEH M M, KANAN C, SALVAGGIO C. Roof damage assessment using deep learning[C]//Proceedings of 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). IEEE, 2017:6403-6408.
[26] HEZAVEH M M, KANAN C, SALVAGGIO C. Roof damage assessment using deep learning[C]//Proceedings of 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). Washington, DC, USA:IEEE, 2017:6403-6408.
[27] GONG Maoguo, NIU Xudong, ZHANG Puzhao, et al. Generative adversarial networks for change detection in multispectral imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12):2310-2314.
[28] LEBEDEV M A, VIZILTER Y V, VYGOLOV O V, et al. Change detection in remote sensing images using conditional adversarial networks[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, XLII-2:565-571.
[29] ZHAO Wenzhi, MOU Lichao, CHEN Jiage, et al. Incorporating metric learning and adversarial network for seasonal invariant change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2720-2731.
[30] GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD:a dataset for assessing building damage from satellite imagery[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, CA, USA:IEE,2019:10-17.
[31] VICKERY P J, SKERLJ P F, LIN J, et al. HAZUS-MH hurricane model methodology. Ⅱ:damage and loss estimation[J]. Natural Hazards Review, 2006, 7(2):94-103.
[32] KELMAN I. Physical flood vulnerability of residential properties in coastal, eastern England[D]. Cambridge,UK:University of Cambridge, 2003.
[33] BERNARDINI A, GIOVINAZZI S, LAGOMARSINO S, et al. The vulnerability assessment of current buildings by a macroseismic approach derived from the EMS-98 scale[C]//Proceedings of the 3rd International Congress of Seismic Engineering.Girona, Spain:[s.n.],2007.