SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath

doi:10.11947/j.AGCS.2023.20210691

Abstract

Abstract: Aiming at the problems of inaccurate edges and loss of small buildings in the extracted buildings due to the inability of deep convolution to take into account global features and local features, the SER-UNet algorithm is proposed based on attention mechanism and skip connection. SER-UNet algorithm couples SE_ResNet and max pooling layers in the encoder stage, and the SE_ResNet structure and deconvolution are used in the decoder stage. The feature map is output after fusing the shallow features extracted by the encoder and the deep features extracted by the decoder through skip connections. In order to analyze the effectiveness of the method, the SER-UNet is used to replace the feature extraction structure in the original network in the parallel multi-path feature extraction stage of the MAP-Net network. Finally, the method proposed is experimentally evaluated on the WHU dataset and the Inria dataset, and the IoU and precision reach 91.46%, 82.61% and 95.67%, 92.75%, compared with UNet, PSPNet, ResNet101, and MAP-Net Networks, the IoU is increased by 0.49%, 0.14%, 1.89%, and 1.57%, and the precision is increased by 0.14%, 1.06%, 2.42% and 1.09%, respectively. To further analyze the validity of the SER-UNet algorithm, the edge integrity and small extraction verification IoU and precision reached 85.32% and 94.13% on the AerialImage dataset. The experiment results show that the MAP-Net parallel multipath network combined with SER-UNet algorithm shows good generalization ability. In addition, the SER-UNet algorithm can be effectively embedded in PSPNet, ResNet101, HRNetv2 and other Networks to improve the ability of Network feature representation.

Key words: high-resolution remote sensing image, building extraction, parallel multipath, attention mechanism, skip connection

CLC Number:

P227

HU Minghong, LI Jiatian, YAO Yanji, A Xiaohui, LU Mei, LI Wen. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 808-817.

References

[1] MAYER H. Automatic object extraction from aerial imagery-a survey focusing on buildings[J]. Computer Vision and Image Understanding, 1999, 74(2):138-149.
[2] LIN C, NEVATIA R. Building detection and description from a single intensity image[J]. Computer vision and image understanding, 1998, 72(2):101-121.
[3] DONG Yanni, DU Bo, ZHANG Liangpei. Target detection based on random forest metric learning[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2015, 8(4):1830-1838.
[4] WEI Yanfeng, ZHAO Zhongming, SONG Jianghong. Urban building extraction from high-resolution satellite panchromatic image using clustering and edge detection[C]//Proceedings of 2004 IEEE International Geoscience and Remote Sensing Symposium. Anchorage:IEEE, 2004:2008-2010.
[5] DU Jianli, CHEN Dong, WANG Ruisheng, et al. A novel framework for 2.5-D building contouring from large-scale residential scenes[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(6):4121-4145.
[6] SHACKELFORD A K, DAVIS C H, WANG Xiangyun. Automated 2D building footprint extraction from high-resolution satellite multispectral imagery[C]//Proceedings of 2004 IEEE International Geoscience and Remote Sensing Symposium. Anchorage:IEEE, 2004:1996-1999.
[7] Mohammad, Awrangjeb. Automatic extraction of building roofs using LiDAR data and multispectral imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2013, 83:1-18.
[8] CUI Weihong, ZHANG Yi. An effective graph-based hierarchy image segmentation[J]. Intelligent Automation & Soft Computing, 2011, 17(7):969-981.
[9] 范荣双, 陈洋, 徐启恒, 等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1):34-41.DOI:10.11947/j.AGCS.2019.20170638. FAN Rongshuang, CHEN Yang, XU Qiheng, et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):34-41.DOI:10.11947/j.AGCS.2019.20170638.
[10] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018(1):1-15.
[11] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4):640-651.
[12] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[13] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition"[EB/OL].[2021-07-10]. https://arxiv.org/abs/1409.1556".
[14] SZEGEDY C, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston:IEEE, 2015:1-9.
[15] LIU, LUO, HUANG, et al. DE-net:deep encoding network for building extraction from high-resolution remote sensing imagery[J]. Remote Sensing, 2019, 11(20):2380-2399.
[16] HUANG Zuming, CHENG Guangliang, WANG Hongzhen, et al. Building extraction from multi-source remote sensing images via deep deconvolution neural networks[C]//Proceedings of 2016 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Beijing:IEEE, 2016:1835-1838.
[17] MALTEZOS E, DOULAMIS A, DOULAMIS N, et al. Building extraction from LiDAR data applying deep convolutional neural networks[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(1):155-159.
[18] RONNEBERGER O, FISCHER P, BROX T. U-net:convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 International Conference on Medical image computing and computer-assisted intervention. Cham:Springer,2015:234-241.
[19] 崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019, 48(5):597-608. CUI Weihong, XIONG Baoyu, ZHANG Liyao. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):597-608.
[20] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet:a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12):2481-2495.
[21] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas:IEEE, 2016:770-778.
[22] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017:6230-6239.
[23] CAI Yong, CHEN Dingyuan, TANG Yuanzhe, et al. Multi-scale building instance extraction framework in high resolution remote sensing imagery based on feature pyramid object-aware convolution neural network[C]//Proceedings of 2021 IEEE International Geoscience and Remote Sensing Symposium IGARSS.Brussels:IEEE, 2021:2779-2782.
[24] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:7132-7141.
[25] ZHU Qing, LIAO Cheng, HU Han, et al. MAP-net:multiple attending path neural network for building footprint extraction from remote sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):6169-6181.
[26] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206. JI Shunping, WEI Shiqing. Building extraction via convolutional neural networks from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206.
[27] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth:IEEE, 2017:3226-3229.
[28] KYRKOU C, THEOCHARIDES T. Deep-learning-based aerial image classification for emergency response applications using unmanned aerial vehicles[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Long Beach:IEEE, 2020:517-525.