高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法

doi:10.11947/j.AGCS.2022.20200601

摘要/Abstract

摘要： 利用高分辨率遥感影像提取建筑物是目前研究热点之一，但由于建筑物颜色各异、形状大小不同、细节繁多，提取结果普遍存在边缘模糊、转角圆滑和细节丢失等问题。本文提出一种基于空洞卷积的E-Unet深度学习网络。在E-Unet的结构设计中，引入跳跃连接以减少边缘和转角的细节损失；采用新设计的卷积模块，使其扩大感受野的同时减少参数量；底层增加Dropout模块避免网络发生过拟合现象；遥感影像输入网络前先进行直方图均衡化、高斯双边滤波和波段间比值运算，然后合并为多波段张量输入模型（不转换为灰度图像）。为验证网络性能、明确性能提升的原因，本文在Massachusetts和WHU建筑物数据集上设计了两组试验。第1组是E-Unet、Unet和Res-net 3种网络的对比试验，结果表明E-Unet不仅精度评价结果优于Unet和Res-net，而且建筑物边角的细节被完整提取。第2组是消融试验，目的是明确预处理模块对提取精度的提升效果，结果表明预处理模块能提升不同网络提取精度。通过这两组试验证明了预处理模块的有效性和本文提出网络的优越性。

关键词: 深度学习, 建筑物提取, 空洞卷积, 高分辨率遥感影像, E-Unet

Abstract: The utilization of high-resolution remote sensing images to extract urban buildings is one of the current research hotspots, but owing to the different colors, shapes and sizes of buildings, and a wide range of details, the extraction results generally suffer from blurred edges, rounded corners and loss of details. For this reason, this study proposes an E-Unet deep learning network based on cavity convolution. In the structural design, jump connections are introduced to reduce the detail loss of edges and corners; a newly designed convolution module is adopted to expand the perceptual field while reducing the number of parameters; a Dropout module is added to the bottom layer of the network to avoid overfitting; histogram equalization, Gaussian bilateral filtering and inter-band ratio operations are performed on the raw data, which are then combined into a multi-band tensor input network(without conversion to grey-scale images). To validate the network performance and clarify the reasons for the performance improvement, two sets of experiments were designed in this study on the Massachusetts and WHU building datasets. The first set of experiments is a comparison experiment between the E-Unet, Unet and Res-net networks. The results show that E-Unet not only outperforms Unet and ResNet in all accuracy evaluation metrics, but also has high fidelity in the details of the extraction results. The second set of experiments are pre-processing stripping experiments to clarify the performance improvement of the network itself and the pre-processing module. The effectiveness of the pre-processing module and the superiority of the proposed network in this research are demonstrated by the two sets of experiments.

Key words: deep learning, building extraction, atrous convolution, high resolution remote sensing image, E-Unet

中图分类号:

P231

何直蒙, 丁海勇, 安炳琪. 高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法[J]. 测绘学报, 2022, 51(3): 457-467.

HE Zhimeng, DING Haiyong, AN Bingqi. E-Unet: a atrous convolution-based neural network for building extraction from high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(3): 457-467.

参考文献

[1] 陈洋, 范荣双, 王竞雪, 等. 基于深度学习的资源三号卫星遥感影像云检测方法[J]. 光学学报, 2018, 38(1):0128005. CHEN Yang, FAN Rongshuang, WANG Jingxue, et al. Cloud detection of ZY-3 satellite remote sensing images based on deep learning[J]. Acta Optica Sinica, 2018, 38(1):0128005.
[2] 龚健雅, 季顺平. 摄影测量与深度学习[J]. 测绘学报, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640. GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):693-704. DOI:10.11947/j.AGCS.2018.20170640.
[3] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA,USA:IEEE, 2015:1-9.
[4] KHRYASHCHEV V, IVANOVSKY L, PAVLOV V, et al. Comparison of different convolutional neural network architectures for satellite image segmentation[C]//Proceedings of the 23rd Conference of Open Innovations Association (FRUCT). Bologna, Italy:IEEE, 2018.
[5] 崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019, 48(5):597-608. DOI:10.11947/j.AGCS.2019.20180062. CUI Weihong, XIONG Baoyu, ZHANG Liyao. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):597-608. DOI:10.11947/j.AGCS.2019.20180062.
[6] 贺浩, 王仕成, 杨东方, 等. 基于Encoder-Decoder网络的遥感影像道路提取方法[J]. 测绘学报, 2019, 48(3):330-338. DOI:10.11947/j.AGCS.2019.20180005. HE Hao, WANG Shicheng, YANG Dongfang, et al. An road extraction method for remote sensing image based on Encoder-Decoder network[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(3):330-338. DOI:10.11947/j.AGCS.2019.20180005.
[7] 范荣双, 陈洋, 徐启恒, 等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1):34-41. DOI:10.11947/j.AGCS.2019.20170638. FAN Rongshuang, CHEN Yang, XU Qiheng, et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):34-41. DOI:10.11947/j.AGCS.2019.20170638.
[8] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[9] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[10] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
[11] WENG Weihao, ZHU Xin. INet:convolutional networks for biomedical image segmentation[J]. IEEE Access, 2021, 9:16591-16603.
[12] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:770-778.
[13] ALOM M Z, HASAN M, YAKOPCIC C, et al. Recurrent residual convolutional neural network based on U-net (R2U-net) for medical image segmentation[EB/OL].[2021-12-24]. DOI:https://arxiv.org/abs/1802.06955.
[14] 张翠军, 安冉, 马丽. 改进U-Net的遥感图像中建筑物变化检测[J]. 计算机工程与应用, 2021, 57(3):239-246. ZHANG Cuijun, AN Ran, MA Li. Building change detection in remote sensing image based on improved U-net[J]. Computer Engineering and Applications, 2021, 57(3):239-246.
[15] 刘浩, 骆剑承, 黄波, 等. 基于特征压缩激活Unet网络的建筑物提取[J]. 地球信息科学学报, 2019, 21(11):1779-1789. LIU Hao, LUO Jiancheng, HUANG Bo, et al. Building extraction based on SE-unet[J]. Journal of Geo-Information Science, 2019, 21(11):1779-1789.
[16] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459. DOI:10.11947/j. agcs.2019.20180206. JI Shunping, WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459. DOI:10.11947/j. agcs.2019.20180206.
[17] 何代毅, 施文灶, 林志斌, 等. 基于改进Mask-RCNN的遥感影像建筑物提取[J]. 计算机系统应用, 2020, 29(9):156-163. HE Daiyi, SHI Wenzao, LIN Zhibin, et al. Building extraction from remote sensing image based on improved mask-RCNN[J]. Computer Systems & Applications, 2020, 29(9):156-163.
[18] 汪志文. 基于深度学习的高分辨率遥感影像语义分割的研究与应用[D]. 北京:北京邮电大学, 2019. WANG Zhiwen. Research and application of semantic segmentation of high-resolution remote sensing image based on deep learning[D]. Beijing:Beijing University of Posts and Telecommunications, 2019.
[19] 吴柳青, 胡翔云. 基于多尺度多特征的高空间分辨率遥感影像建筑物自动化检测[J]. 国土资源遥感, 2019, 31(1):71-78. WU Liuqing, HU Xiangyun. Automatic building detection of high-resolution remote sensing images based on multi-scale and multi-feature[J]. Remote Sensing for Land & Resources, 2019, 31(1):71-78.
[20] 杨乐, 王慧, 李烁, 等. 结合DeepLabv3架构的多源数据建筑物提取方法[J]. 测绘与空间地理信息, 2020, 43(6):62-66. YANG Le, WANG Hui, LI Shuo, et al. Multi-source data building extraction method combined with DeepLabv3 architecture[J]. Geomatics & Spatial Information Technology, 2020, 43(6):62-66.
[21] 王宇, 杨艺, 王宝山, 等. 深度神经网络条件随机场高分辨率遥感图像建筑物分割[J]. 遥感学报, 2019, 23(6):1194-1208. WANG Yu, YANG Yi, WANG Baoshan, et al. Building segmentation in high-resolution remote sensing image through deep neural network and conditional random fields[J]. Journal of Remote Sensing, 2019, 23(6):1194-1208.
[22] VAKALOPOULOU M, KARANTZALOS K, KOMODAKIS N, et al. Building detection in very high resolution multispectral data with deep learning features[C]//Proceedings of 2015 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Milan, Italy:IEEE, 2015:1873-1876. DOI:10.1109/IGARSS.2015.7326158.
[23] YUAN Jiangye. Automatic building extraction in aerial scenes using convolutional networks[EB/OL].[2021-09-30].https://arxiv.org/abs/1602.06564.
[24] 孙文荣, 周先春, 嵇亚婷. 基于直方图均衡化、PCA和SVM算法的人脸识别[J]. 软件, 2014, 35(8):11-15. SUN Wenrong, ZHOU Xianchun, JI Yating. Face recognition based on histogram equalization, Principal Components Analysis and Support Vector Machine algorithms[J]. Computer Engineering & Software, 2014, 35(8):11-15.
[25] 黄昕. 高分辨率遥感影像多尺度纹理、形状特征提取与面向对象分类研究[D]. 武汉:武汉大学, 2009:71-76. HUANG Xin. Multiscale texture and shape feature extraction and object-oriented classification for very high resolution remotely sensed imagery[D]. Wuhan:Wuhan University, 2009:71-76.
[26] HE Hao, WANG Shuyang, WANG Shicheng, et al. A road extraction method for remote sensing image based on encoder-decoder network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):16-25.
[27] HINTON G E, SRIVASTAVA N, KRIZHEVSKY A, et al. Improving neural networks by preventing co-adaptation of feature detectors[EB/OL].[2021-09-30].http://dx.doi.org/10.9774/GLEAF.978-1-909493-38-4_2.
[28] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:2818-2826.
[29] JI Shunping, WEI Shiqing, LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1):574-586.
[30] MNIH V. Machine learning for aerial image labeling.[D]. Toronto,Canada:University of Toronto. 2013.