结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法

doi:10.11947/j.AGCS.2023.20210691

摘要/Abstract

摘要： 针对深层卷积较难兼顾全局特征与局部特征从而导致提取建筑物边缘不准确和微小建筑物丢失的问题,以注意力机制和跳跃连接为基础提出SER-UNet算法。SER-UNet算法在编码器阶段耦合SE-ResNet和最大池化层,在解码器阶段关联SE-ResNet与反卷积层,通过跳跃连接将编码器提取的浅层特征和解码器提取的深层特征进行融合后输出特征图。验证SER-UNet算法的有效性,在MAP-Net网络并行多路径特征提取阶段使用SER-UNet算法替换原网络中的特征提取结构,分别在WHU数据集和Inria数据集上进行评估,IoU与精度分别达91.46%、82.61%和95.67%、92.75%,对比UNet、PSPNet、ResNet101、MAP-Net网络,IoU分别提高0.49%、0.14%、1.89%、1.57%,精度分别提高0.14%、1.06%、2.42%、1.09%。分析SER-UNet算法的泛化能力,将级联SER-UNet的MAP-Net网络在AerialImage数据集上进行提取验证,IoU与精度分别达85.32%和94.13%。结果表明,结合SER-UNet算法的MAP-Net并行多路径网络表现出较好的提取精度与泛化能力。此外,SER-UNet算法可以有效地嵌入PSPNet、ResNet101、HRNetv2等网络中,提升网络特征表示能力。

关键词: 高分辨率遥感影像, 建筑物提取, 并行多路径, 注意力机制, 跳跃连接

Abstract: Aiming at the problems of inaccurate edges and loss of small buildings in the extracted buildings due to the inability of deep convolution to take into account global features and local features, the SER-UNet algorithm is proposed based on attention mechanism and skip connection. SER-UNet algorithm couples SE_ResNet and max pooling layers in the encoder stage, and the SE_ResNet structure and deconvolution are used in the decoder stage. The feature map is output after fusing the shallow features extracted by the encoder and the deep features extracted by the decoder through skip connections. In order to analyze the effectiveness of the method, the SER-UNet is used to replace the feature extraction structure in the original network in the parallel multi-path feature extraction stage of the MAP-Net network. Finally, the method proposed is experimentally evaluated on the WHU dataset and the Inria dataset, and the IoU and precision reach 91.46%, 82.61% and 95.67%, 92.75%, compared with UNet, PSPNet, ResNet101, and MAP-Net Networks, the IoU is increased by 0.49%, 0.14%, 1.89%, and 1.57%, and the precision is increased by 0.14%, 1.06%, 2.42% and 1.09%, respectively. To further analyze the validity of the SER-UNet algorithm, the edge integrity and small extraction verification IoU and precision reached 85.32% and 94.13% on the AerialImage dataset. The experiment results show that the MAP-Net parallel multipath network combined with SER-UNet algorithm shows good generalization ability. In addition, the SER-UNet algorithm can be effectively embedded in PSPNet, ResNet101, HRNetv2 and other Networks to improve the ability of Network feature representation.

Key words: high-resolution remote sensing image, building extraction, parallel multipath, attention mechanism, skip connection

中图分类号:

P227

胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817.

HU Minghong, LI Jiatian, YAO Yanji, A Xiaohui, LU Mei, LI Wen. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 808-817.

参考文献

[1] MAYER H. Automatic object extraction from aerial imagery-a survey focusing on buildings[J]. Computer Vision and Image Understanding, 1999, 74(2):138-149.
[2] LIN C, NEVATIA R. Building detection and description from a single intensity image[J]. Computer vision and image understanding, 1998, 72(2):101-121.
[3] DONG Yanni, DU Bo, ZHANG Liangpei. Target detection based on random forest metric learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(4):1830-1838.
[4] WEI Yanfeng, ZHAO Zhongming, SONG Jianghong. Urban building extraction from high-resolution satellite panchromatic image using clustering and edge detection[C]//Proceedings of 2004 IEEE International Geoscience and Remote Sensing Symposium. Anchorage:IEEE, 2004:2008-2010.
[5] DU Jianli, CHEN Dong, WANG Ruisheng, et al. A novel framework for 2.5-D building contouring from large-scale residential scenes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6):4121-4145.
[6] SHACKELFORD A K, DAVIS C H, WANG Xiangyun. Automated 2D building footprint extraction from high-resolution satellite multispectral imagery[C]//Proceedings of 2004 IEEE International Geoscience and Remote Sensing Symposium. Anchorage:IEEE, 2004:1996-1999.
[7] Mohammad, Awrangjeb. Automatic extraction of building roofs using LiDAR data and multispectral imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 83:1-18.
[8] CUI Weihong, ZHANG Yi. An effective graph-based hierarchy image segmentation[J]. Intelligent Automation & Soft Computing, 2011, 17(7):969-981.
[9] 范荣双, 陈洋, 徐启恒, 等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1):34-41.DOI:10.11947/j.AGCS.2019.20170638. FAN Rongshuang, CHEN Yang, XU Qiheng, et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):34-41.DOI:10.11947/j.AGCS.2019.20170638.
[10] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018(1):1-15.
[11] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
[12] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[13] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition"[EB/OL].[2021-07-10]. https://arxiv.org/abs/1409.1556".
[14] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston:IEEE, 2015:1-9.
[15] LIU, LUO, HUANG, et al. DE-net:deep encoding network for building extraction from high-resolution remote sensing imagery[J]. Remote Sensing, 2019, 11(20):2380-2399.
[16] HUANG Zuming, CHENG Guangliang, WANG Hongzhen, et al. Building extraction from multi-source remote sensing images via deep deconvolution neural networks[C]//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Beijing:IEEE, 2016:1835-1838.
[17] MALTEZOS E, DOULAMIS A, DOULAMIS N, et al. Building extraction from LiDAR data applying deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(1):155-159.
[18] RONNEBERGER O, FISCHER P, BROX T. U-net:convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 International Conference on Medical image computing and computer-assisted intervention. Cham:Springer,2015:234-241.
[19] 崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019, 48(5):597-608. CUI Weihong, XIONG Baoyu, ZHANG Liyao. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):597-608.
[20] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[21] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas:IEEE, 2016:770-778.
[22] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017:6230-6239.
[23] CAI Yong, CHEN Dingyuan, TANG Yuanzhe, et al. Multi-scale building instance extraction framework in high resolution remote sensing imagery based on feature pyramid object-aware convolution neural network[C]//Proceedings of 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS.Brussels:IEEE, 2021:2779-2782.
[24] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:7132-7141.
[25] ZHU Qing, LIAO Cheng, HU Han, et al. MAP-net:multiple attending path neural network for building footprint extraction from remote sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):6169-6181.
[26] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206. JI Shunping, WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
[27] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth:IEEE, 2017:3226-3229.
[28] KYRKOU C, THEOCHARIDES T. Deep-learning-based aerial image classification for emergency response applications using unmanned aerial vehicles[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Long Beach:IEEE, 2020:517-525.