基于生成对抗网络的建筑物损毁检测

doi:10.11947/j.AGCS.2022.20200318

测绘学报 ›› 2022, Vol. 51 ›› Issue (2): 238-247.doi: 10.11947/j.AGCS.2022.20200318

基于生成对抗网络的建筑物损毁检测

葛小三^1,2, 陈曦^1,2,3, 赵文智³, 李瑞祥^1,2

1. 河南理工大学测绘与国土信息工程学院, 河南焦作 454003;
2. 河南理工大学自然资源部矿山时空信息与生态修复重点实验室, 河南焦作 454003;
3. 北京师范大学地理科学学部遥感科学与工程研究院, 北京 100875

收稿日期:2020-07-17 修回日期:2021-08-03 发布日期:2022-02-28
通讯作者: 赵文智 E-mail:wenzhi.zhao@bnu.edu.cn
作者简介:葛小三(1971-),男,博士,教授。E-mail:hzhang@re.ecnu.edu.cn
基金资助:
北京市自然科学基金（4214065）；国家自然科学基金（41572341）；河南省自然科学基金项目（222300420450）；国家重点研发计划（2018YFC1508903）

Detection of damaged buildings based on generative adversarial networks

GE Xiaosan^1,2, CHEN Xi^1,2,3, ZHAO Wenzhi³, LI Ruixiang^1,2

1. School of Surveying and Land Information Engineering, Henan Polytechnic University, Jiaozuo 454003, China;
2. Key Laboratory of Spatio-temporal Information and Ecological Restoration of Mines(MNR), Henan Polytechnic University, Jiaozuo 454003, China;
3. Institute of Remote Sensing Science and Engineering, Faculty of Geographical Science, Beijing Normal University, Beijing 100875, China

Received:2020-07-17 Revised:2021-08-03 Published:2022-02-28
Supported by:
Beijing Municipal Natural Science Foundation (No. 4214065); The National Natural Science Foundation of China (No. 41572341); Natural Science Foundation of Henan, China (222300420450); The National Key Research and Development Program of China (No. 2018YFC1508903)

摘要/Abstract

摘要： 建筑物作为自然灾害中最受影响的承灾体之一，其损毁信息的准确提取对灾后应急救援具有十分重要的意义。本文借鉴多模态的思想，提出了一种自动检测损毁建筑物的recursive-generative adversarial networks（RS-GAN）方法，将损毁建筑物检测分为灾前建筑物识别和灾后损毁建筑物检测两个任务，且分别在两个GAN分支中完成。RS-GAN加入联合损失函数将两个GAN分支进行连接，充分利用两个任务之间的潜在互利性提升检测效果。RS-GAN利用第1条GAN分支识别建筑物灾前形状与位置，并将识别结果作为第2条GAN分支的输入进行损毁建筑物检测任务，从而使检测结果具有更清晰的轮廓。该方法为端到端模型，在不需要过多的人工干预情形下，实现了损毁建筑物的自动检测。为了验证RS-GAN模型的效果，在圣罗莎和密苏里两个数据集上进行了测试。试验结果表明，RS-GAN方法拥有更好的检测性能，在圣罗莎数据集上的总体精度和平均精度分别达到了0.90和0.86。

关键词: 生成对抗网络(GAN), 灾前灾后双时相遥感影像, 建筑物轮廓提取, 损毁建筑物检测, 分步学习

Abstract: As one of the most affected hazard-affected bodies in natural disasters, accurate damage information extraction of buildings plays a significant role in post-disaster emergency rescue. Referring to the idea of multi-mode fusion technique, a recursive generative adversarial networks (RS-GAN) method is proposed to automatically detect damaged buildings. In RS-GAN, the workflow of damaged buildings detection is composed of two sub-tasks as follows:building identification before disasters as well as damaged building detection after disasters, which are completed in two GAN branches respectively. Specifically, RS-GAN adds a joint loss function to connect the two GAN branches, making full use of the potential mutual benefit between the two tasks to improve the detection accuracy. In addition, the results of building identification are added to the damaged building detection task to locate potential damaged areas. The method proposed in this paper is an end-to-end model, which can automatically detect damaged buildings without excessive manual intervention. To verify the effect of the RS-GAN model, in this paper, two experiments were set with the Santa Rosa dataset and Missouri respectively. Experimental results show that RS-GAN method has better detection performance compared to other competitive methods, and the overall accuracy and average accuracy on the Santa Rosa data set are 0.90 and 0.86, respectively.

Key words: generative adversarial networks, pre-disaster and post-disaster bitemporal images, building contour extraction, damaged building detection, step-by-step learning

中图分类号:

P237

葛小三, 陈曦, 赵文智, 李瑞祥. 基于生成对抗网络的建筑物损毁检测[J]. 测绘学报, 2022, 51(2): 238-247.

GE Xiaosan, CHEN Xi, ZHAO Wenzhi, LI Ruixiang. Detection of damaged buildings based on generative adversarial networks[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(2): 238-247.

参考文献

[1] 黄露. 基于机器学习的汶川震区滑坡灾害气象预警模型研究[J]. 测绘学报, 2020, 49(2):267.DOI:10.11947/j.AGCS.2020.20190061. HUANG Lu. Research on meteorological early-warning model of landslides in Wenchuan earthquake area based on machine learning[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2):267.DOI:10.11947/j.AGCS.2020.20190061.
[2] DONG Laigen, SHAN Jie. A comprehensive review of earthquake-induced building damage detection with remote sensing techniques[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 84:85-99.
[3] 陶和平, 刘斌涛, 刘淑珍, 等. 遥感在重大自然灾害监测中的应用前景:以5·12汶川地震为例[J]. 山地学报, 2008, 26(3):276-279. TAO Heping, LIU Bintao, LIU Shuzhen, et al. Natural hazards monitoring using remote sensing-a case study of 5·12 Wenchuan earthquake[J]. Journal of Mountain Science, 2008, 26(3):276-279.
[4] 眭海刚, 刘超贤, 黄立洪, 等. 遥感技术在震后建筑物损毁检测中的应用[J]. 武汉大学学报(信息科学版), 2019, 44(7):1008-1019. SUI Haigang, LIU Chaoxian, HUANG Lihong, et al. Application of remote sensing technology in earthquake-induced building damage detection[J]. Geomatics and Information Science of Wuhan University, 2019, 44(7):1008-1019.
[5] YAMAZAKI F, SUZUKI D, MARUYAMA Y. Detection of damages due to earthquakes using digital aerial images[C]//Proceedings of the 6th International Workshop on Remote Sensing for Disaster Applications, Pavia, Italy:[s.n.],2008.
[6] 谢嘉丽, 李永树, 李何超, 等. 利用灰度共生矩阵纹理特征识别空心村损毁建筑物的方法[J]. 测绘通报, 2017(12):90-93, 102. XIE Jiali, LI Yongshu, LI Hechao, et al. Recognition of damage buildings in hollow village based on texture feature of gray level co-occurrence matrix[J]. Bulletin of Surveying and Mapping, 2017(12):90-93, 102.
[7] 何浩, 刘修国, 沈永林. 基于视差的高分辨率遥感影像建筑物变化检测[J]. 遥感技术与应用, 2019, 34(6):1315-1323. HE Hao, LIU Xiuguo, SHEN Yonglin. Building change detection method considering the parallax for high resolution remote sensing image[J]. Remote Sensing Technology and Application, 2019, 34(6):1315-1323.
[8] 刘莹, 陶超, 闫培, 等. 图割能量驱动的高分辨率遥感影像震害损毁建筑物检测[J]. 测绘学报, 2017, 46(7):910-917.DOI::10.11947/j.AGCS.2017.20170035. LIU Ying, TAO Chao, YAN Pei, et al. Graph cut energy driven earthquake-damaged building detection from high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(7):910-917.DOI::10.11947/j.AGCS.2017.20170035.
[9] 刘莹, 李强. 融合多特征的高分辨率遥感影像震害损毁建筑物检测[J]. 测绘与空间地理信息, 2018, 41(6):61-64. LIU Ying, LI Qiang. Damaged building detection from high resolution remote sensing images by integrating multiple features[J]. Geomatics & Spatial Information Technology, 2018, 41(6):61-64.
[10] 李强, 张景发. 不同特征融合的震后损毁建筑物识别研究[J]. 地震研究, 2016,39(3):486-493, 527. LI Qiang, ZHANG Jingfa. Research on earthquake damaged building extraction by different features fusion[J]. Journal of Seismological Research, 2016, 39(3):486-493, 527.
[11] SONOBE M. Characteristics of texture index of damaged buildings using time-series high-resolution optical satellite images[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2020, 43:1709-1714.
[12] 袁玉, 申旭辉, 薛腾飞, 等. SAR图像变化检测提取建筑物震害信息的研究综述[J]. 地壳构造与地壳应力文集, 2017(1):131-143. YUAN Yu, SHEN Xuhui, XUE Tengfei, et al. Review of researching for building earthquake damage information extraction based on SAR images change detection[J]. Bulletin of theInstituteof Crustal Dynamics, 2017(1):131-143.
[13] 许志华. 基于低空影像和地面LiDAR数据的建筑物损毁评估方法研究[J]. 地理与地理信息科学, 2018, 34(4):126. XU Zhihua. Building damage assessment from unmanned aerial vehicle images and terrestrial LiDAR data[J]. Geography and Geo-Information Science, 2018, 34(4):126.
[14] SUN Long, WU Tao, SUN Guangcai, et al. Object detection research of SAR image using improved faster RegionBasedconvolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):18-28.
[15] 叶昕, 王俊, 秦其明. 基于高分一号卫星遥感图像的建筑物震害损毁检测研究:以2015年尼泊尔MS8.1地震为例[J]. 地震学报, 2016,38(3):477-485, 509. YE Xin, WANG Jun, QIN Qiming. Damaged building detection based on GF-1 satellite remote sensing image:a case study for Nepal MS8.1 earthquake[J]. Acta Seismologica Sinica, 2016,38(3):477-485, 509.
[16] SAITO K, SPENCE R J S, GOING C, et al. Using high-resolution satellite images for post-earthquake building damage assessment:a study following the 26 January 2001 gujarat earthquake[J]. Earthquake Spectra, 2004, 20(1):145-169.
[17] LIU Sicong, DU Qian, TONG Xiaohua, et al. Multiscale morphological compressed change vector analysis for unsupervised multiple change detection[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017, 10(9):4124-4137.
[18] NEETI N, EASTMAN J R. Novel approaches in extended principal component analysis to compare spatio-temporal patterns among multiple image time series[J]. Remote Sensing of Environment, 2014, 148:84-96.
[19] 马国锐, 吴娇, 姚聪. 多尺度特征融合的建筑物及损毁语义分割[J]. 测绘工程, 2020, 29(4):1-6. MA Guorui, WU Jiao, YAO Cong. Semantic segmentation of building and damage based on multi-scale feature fusion[J]. Engineering of Surveying and Mapping, 2020, 29(4):1-6.
[20] MIURA H, ARIDOME T, MATSUOKA M. Deep learning-based identification of collapsed, non-collapsed and blue tarp-covered buildings from post-disaster aerial images[J]. Remote Sensing, 2020, 12(12):1924.
[21] 左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报,2019,48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740. ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6):718-726. DOI:10.11947/j.AGCS.2019.20170740.
[22] DAI Yuchao,ZHANG Jing,HE Mingyi,et al.Salient object detection from multi-spectral remote sensing images with deep residual network[J].Journal of Geodesy and Geoinformation Science,2019,2(2):101-110.
[23] 周阳, 张云生, 陈斯飏, 等. 基于DCNN特征的建筑物震害损毁区域检测[J]. 国土资源遥感, 2019, 31(2):44-50. ZHOU Yang, ZHANG Yunsheng, CHEN Siyang, et al. Disaster damage detection in building areas based on DCNN features[J]. Remote Sensing for Land & Resources, 2019, 31(2):44-50.
[24] LI Yundong, YE Shi, BARTOLI I. Semisupervised classification of hurricane damage from postevent aerial imagery using deep learning[J]. Journal of Applied Remote Sensing, 2018, 12(4):045008.
[25] HEZAVEH M M, KANAN C, SALVAGGIO C. Roof damage assessment using deep learning[C]//Proceedings of 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). IEEE, 2017:6403-6408.
[26] HEZAVEH M M, KANAN C, SALVAGGIO C. Roof damage assessment using deep learning[C]//Proceedings of 2017 IEEE Applied Imagery Pattern Recognition Workshop (AIPR). Washington, DC, USA:IEEE, 2017:6403-6408.
[27] GONG Maoguo, NIU Xudong, ZHANG Puzhao, et al. Generative adversarial networks for change detection in multispectral imagery[J]. IEEE Geoscience and Remote Sensing Letters, 2017, 14(12):2310-2314.
[28] LEBEDEV M A, VIZILTER Y V, VYGOLOV O V, et al. Change detection in remote sensing images using conditional adversarial networks[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2018, XLII-2:565-571.
[29] ZHAO Wenzhi, MOU Lichao, CHEN Jiage, et al. Incorporating metric learning and adversarial network for seasonal invariant change detection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(4):2720-2731.
[30] GUPTA R, GOODMAN B, PATEL N, et al. Creating xBD:a dataset for assessing building damage from satellite imagery[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. Long Beach, CA, USA:IEE,2019:10-17.
[31] VICKERY P J, SKERLJ P F, LIN J, et al. HAZUS-MH hurricane model methodology. Ⅱ:damage and loss estimation[J]. Natural Hazards Review, 2006, 7(2):94-103.
[32] KELMAN I. Physical flood vulnerability of residential properties in coastal, eastern England[D]. Cambridge,UK:University of Cambridge, 2003.
[33] BERNARDINI A, GIOVINAZZI S, LAGOMARSINO S, et al. The vulnerability assessment of current buildings by a macroseismic approach derived from the EMS-98 scale[C]//Proceedings of the 3rd International Congress of Seismic Engineering.Girona, Spain:[s.n.],2007.

基于生成对抗网络的建筑物损毁检测

Detection of damaged buildings based on generative adversarial networks

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	程结海, 黄中意, 王建如, 何湜. 高空间分辨率遥感影像最优分割结果自动确定方法[J]. 测绘学报, 2022, 51(5): 658-667.
[2]	梁哲恒, 黎宵, 邓鹏, 盛森, 姜福泉. 融合多尺度特征注意力的遥感影像变化检测方法[J]. 测绘学报, 2022, 51(5): 668-676.
[3]	白坤, 慕晓冬, 陈雪冰, 朱永清, 尤轩昂. 融合半监督学习的无监督遥感影像场景分类[J]. 测绘学报, 2022, 51(5): 691-702.
[4]	黄明益, 吴军, 高炯笠. 多镜头全景摄像机球面视频无缝生成[J]. 测绘学报, 2022, 51(5): 703-717.
[5]	王丹菂, 邢帅, 徐青, 林雨准, 李鹏程. 单频机载激光测深海陆回波自动分类方法[J]. 测绘学报, 2022, 51(5): 750-761.
[6]	张志敏. 基于遥感反照率的青藏高原冰川年际物质平衡估算研究[J]. 测绘学报, 2022, 51(5): 781-781.
[7]	李永强, 李鹏鹏, 董亚涵, 范辉龙. 车载LiDAR点云数据中杆状地物自动提取与分类[J]. 测绘学报, 2020, 49(6): 724-735.
[8]	王竞雪, 刘肃艳, 王伟玺. 联合共线约束与匹配冗余的组直线匹配结果检核算法[J]. 测绘学报, 2020, 49(6): 746-756.
[9]	詹总谦, 胡孟琦, 满益云. 多尺度区域生长点云滤波地表拟合法[J]. 测绘学报, 2020, 49(6): 757-766.
[10]	韩斌, 吴一全. SAR图像河流提取的主动轮廓模型的稳健估计算法[J]. 测绘学报, 2020, 49(6): 777-786.
[11]	邓睿哲, 陈启浩, 陈奇, 刘修国. 遥感影像船舶检测的特征金字塔网络建模方法[J]. 测绘学报, 2020, 49(6): 787-797.
[12]	黄亮. 多时相遥感影像变化检测技术研究[J]. 测绘学报, 2020, 49(6): 801-801.
[13]	吴文豪, 张磊, 李陶, 龙四春, 段梦, 周志伟, 祝传广, 蒋廷臣. 基于几何配准的多模式SAR影像配准及其误差分析[J]. 测绘学报, 2019, 48(11): 1439-1451.
[14]	赵生银, 安如, 朱美如. 联合像元-深度-对象特征的遥感图像城市变化检测[J]. 测绘学报, 2019, 48(11): 1452-1463.
[15]	刘照欣, 赵辽英, 厉小润, 陈淑涵. 高光谱亚像元定位的线特征探测法[J]. 测绘学报, 2019, 48(11): 1464-1474.