Multi-path RSU network method for high-resolution remote sensing image building extraction

doi:10.11947/j.AGCS.2021.20200508

Abstract

Abstract: Inaccurate boundaries and holes are two major problems when extracting buildings from high-resolution remote sensing images by a convolution network. To solve these problems, we proposed the MPRSU-Net (multi-path residual U-block network), which is based on the RSU (residual U-block). The RSU is able to fuse local features and multi-scale features, with the help of the encoder-decoder structure and the residual connection. However, a single RSU is not enough to gather enough information, MPRSU-Net parallels RSU blocks of different scales by the multi-path structure and exchanges information among these blocks to further enhance the feature aggregation efficiency. Experimental results showed that the MPRSU-Net achieved 95.65%,88.63% precision, and 91.17%,79.31% IoU on 0.3 m resolution WHU and Inria building datasets, which showed the effectiveness of the proposed method. In addition, compared with the U² Net, MPRSU-Net is much lighter in computation and reduces the amount of model parameters by 68.63%, demonstrating that the method has some application value.

Key words: high-resolution remote sensing image, building extraction, multi-scale, convolutional neural networks, multi-path

CLC Number:

P237

ZHANG Yuxin, YAN Qingsong, DENG Fei. Multi-path RSU network method for high-resolution remote sensing image building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(1): 135-144.

References

[1] PESARESI M, GERHARDINGER A, KAYITAKIRE F. A robust built-up area presence index by anisotropic rotation-invariant textural measure[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2008, 1(3):180-192.
[2] JIN Xiaoying, DAVIS C H. Automated building extraction from high-resolution satellite imagery in urban areas using structural, contextual, and spectral information[J]. EURASIP Journal on Advances in Signal Processing, 2005,2005(14):2196-2206.
[3] HUANG Xin, ZHANG Liangpei. Morphological building/shadow index for building extraction from high-resolution imagery over urban areas[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2012, 5(1):161-172.
[4] JUNG C, SCHRAMM R. Rectangle detection based on a windowed hough transform[J]. Brazilian Symposium of Computer Graphic and Image Processing. 2004, 2004(4):113-120.
[5] OK A O, SENARAS C, YUKSEL B. Automated detection of arbitrarily shaped buildings in complex environments from monocular VHR optical satellite imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(3):1701-1717.
[6] 张亚一, 费鲜芸, 王健, 等. 基于高分辨率遥感影像的建筑物提取方法综述[J]. 测绘与空间地理信息, 2020, 43(4):76-79. ZHANG Yayi, FEI Xianyun, WANG Jian, et al. Survey of building extraction methods based on high resolution remote sensing images[J]. Geomatics & Spatial Information Technology, 2020, 43(4):76-79.
[7] 季顺平, 魏世清. 遥感影像建筑物提取的卷积神经元网络与开源数据集方法[J]. 测绘学报, 2019, 48(4):448-459.DOI:10.11947/j.AGCS.2019.20180206. JI Shunping, WEI Shiqing. Building extraction via convolutional neural network from an open remote sensing building dataset[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):448-459. DOI:10.11947/j.AGCS.2019.20180206.
[8] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science. 2018, 1(1):1-15.
[9] DAI Yuchao, ZHANG Jing, HE Mingyi, et al. Salient object detection from multi-spectral remote sensing images with deep residual network[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):101-110.
[10] SUN Long, WU Tao, SUN Guangcai, et al. Object detection research of SAR image using improved faster region-based convolutional neural network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(3):18-28.
[11] HE Hao, WANG Shuyang, WANG Shicheng, et al. A road extraction method for remote sensing image based on encoder-decoder network[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):16-25.
[12] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 39(4):640-651.
[13] RONNEBERGER O, FISCHER P, BROX T. U-Net:Convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th International Conference on Medical Image Computing and Computer-Assisted Intervention. Munich, Germany:Springer, 2015:234-241.
[14] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:6230-6239.
[15] LIN Guosheng, MILAN A, SHEN Chunhua, et al. RefineNet:multi-path refinement networks for high-resolution semantic segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:5168-5177.
[16] 崔卫红, 熊宝玉, 张丽瑶. 多尺度全卷积神经网络建筑物提取[J]. 测绘学报, 2019, 48(5):597-608. DOI:10.11947/j.AGCS.2019.20180062. CUI Weihong, XIONG Baoyu, ZHANG Liyao. Multi-scale fully convolutional neural network for building extraction[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(5):597-608.DOI:10.11947/j.AGCS.2019.20180062.
[17] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of 2017 IEEE Conference on Neural Information Processing Systems.[S.l.]:IEEE,2017.
[18] WANG Xiaolong, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA:IEEE, 2018:7794-7803.
[19] FU Jun, LIU Jing, TIAN Haijie, et al. Dual attention network for scene segmentation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).Long Beach, CA, USA:IEEE, 2019:3141-3149.
[20] TAO A, SAPRA K, CATANZARO B. Hierarchical multi-scale attention for semantic segmentation[C]//Proceedings of 2020 IEEE Conference on Computer Vision and Pattern Recognition.[S.l.]:IEEE,2020.
[21] SUN Ke, XIAO Bin, LIU Dong, et al. Deep high-resolution representation learning for human pose estimation[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA:IEEE, 2019:5686-5696.
[22] QIN Xuebin, ZHANG Zichen, HUANG Chenyang, et al. U2-Net:Going deeper with nested U-structure for salient object detection[J]. Pattern Recognition, 2020, 106:107404.
[23] LIU Penghua, LIU Xiaoping, LIU Mengxi, et al. Building footprint extraction from high-resolution images via spatial residual inception convolutional neural network[J]. Remote Sensing, 2019, 11(7):830.
[24] FENG Wenqing, SUI Haigang, HUA Li, et al. Building extraction from VHR remote sensing imagery by combining an improved deep convolutional encoder-decoder architecture and historical land use vector map[J]. International Journal of Remote Sensing, 2020, 41(17):6595-6617.
[25] ZHU Qing, LIAO Cheng, HU Han, et al. MAP-net:multiple attending path neural network for building footprint extraction from remote sensed imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7):6169-6181.
[26] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016:770-778.
[27] BISHOP C M, HINTON G. Neural networks for pattern recognition[M]. Oxford:Oxford University Press, 1995.
[28] MAGGIORI E, TARABALKA Y, CHARPIAT G, et al. Can semantic labeling methods generalize to any city? the inria aerial image labeling benchmark[C]//Proceedings of 2017 IEEE International Geoscience and Remote Sensing Symposium (IGARSS). Fort Worth, TX, USA:IEEE, 2017:3226-3229.