MAFNet: building extraction method from remote sensing images based on multi-scale atrous fusion network

doi:10.11947/j.AGCS.2025.20240439

Abstract

Abstract:

Building extraction from remote sensing images is of great significance to disaster management, urban planning, and change monitoring. Due to the different sizes of urban buildings, there are buildings of multiple spatial scales in a remote sensing image, which makes the accuracy of building extraction in the image insufficient. In order to improve the extraction accuracy of buildings of different scales in the image, this paper proposes a remote sensing image building extraction method using a multi-scale atrous fusion network. Based on the U-Net network, the residual structure is first fused in the encoder and decoder parts to better propagate the gradient during the training process. At the same time, a multi-scale atrous fusion (MAF) module is proposed in the bridge part of the encoder-decoder. This module uses multiple atrous convolutions to capture global context features, and further enhances feature expression through channel and spatial attention mechanisms, effectively improving the extraction accuracy of buildings of different scales in the image. Finally, a hybrid loss function is designed to improve the overall boundary extraction effect. This paper conducts experiments based on the WHU building and Massachusetts building datasets, and compares the proposed method with the current mainstream semantic segmentation network. Experimental results show that the proposed method can significantly improve the accuracy of building extraction in images, can adapt to the extraction of buildings of various sizes, and can extract building boundaries more completely and smoothly.

Key words: remote sensing images, building extraction, U-Net, multi-scale atrous fusion, hybrid loss function

CLC Number:

P237

Zibo DONG, Jingxue WANG, Lijing BU, Lin FANG, Zhenghui XU. MAFNet: building extraction method from remote sensing images based on multi-scale atrous fusion network[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(6): 1094-1106.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Tab. 1

Fig. 5

Fig. 6

Tab. 2

Tab. 3

Fig. 7

Tab. 4

Fig. 8

Tab. 5

References 36

[1]	LUO Lin, LI Pengpeng, YAN Xuesong. Deep learning-based building extraction from remote sensing images: a comprehensive review[J]. Energies, 2021, 14(23): 7982.
[2]	范荣双, 陈洋, 徐启恒, 等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1): 34-41. DOI: . doi: 10.11947/j.AGCS.2019.20170638
	FAN Rongshuang, CHEN Yang, XU Qiheng, et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1): 34-41. DOI: . doi: 10.11947/j.AGCS.2019.20170638
[3]	吕少云, 李佳田, 阿晓荟, 等. Res_ASPP_UNet++：结合分离卷积与空洞金字塔的遥感影像建筑物提取网络[J]. 遥感学报, 2023, 27(2): 502-519.
	LÜ Shaoyun, LI Jiatian, A Xiaohui, et al. Res_ASPP_UNet++: building an extraction network from remote sensing imagery combining depthwise separable convolution with atrous spatial pyramid pooling[J]. National Remote Sensing Bulletin, 2023, 27(2): 502-519.
[4]	PESARESI M, GERHARDINGER A, KAYITAKIRE F. A robust built-up area presence index by anisotropic rotation-invariant textural measure[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2008, 1(3): 180-192.
[5]	谭衢霖. 高分辨率多光谱影像城区建筑物提取研究[J]. 测绘学报, 2010, 39(6): 618-623.
	TAN Qulin. Urban building extraction from VHR multi-spectral images using object-based classification[J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(6): 618-623.
[6]	杜守基, 邹峥嵘, 张云生, 等. 融合LiDAR点云与正射影像的建筑物图割优化提取方法[J]. 测绘学报, 2018, 47(4): 519-527. DOI: . doi: 10.11947/j.AGCS.2018.20160534
	DU Shouji, ZOU Zhengrong, ZHANG Yunsheng, et al. A building extraction method via graph cuts algorithm by fusion of LiDAR point cloud and orthoimage[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(4): 519-527. DOI: . doi: 10.11947/j.AGCS.2018.20160534
[7]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[8]	HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-778.
[9]	SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2024-05-03]. https://arxiv.org/abs/1409.1556.
[10]	SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[11]	RONNEBERGER O, FISCHER P, BROX T. U-Net: convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 Medical Image Computing and Computer-Assisted Intervention. Cham: Springer International Publishing, 2015: 234-241.
[12]	ZHOU Zongwei, RAHMAN SIDDIQUEE M M, TAJBAKHSH N, et al. UNet++: a nested U-Net architecture for medical image segmentation[C]//Proceedings of 2018 Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision Support. Cham: Springer International Publishing, 2018: 3-11.
[13]	ZHANG Renhe, ZHANG Qian, ZHANG Guixu. SDSC-UNet: dual skip connection ViT-based U-shaped model for building extraction[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 3270303.
[14]	LI Yuxuan, HONG Danfeng, LI Chenyu, et al. HD-Net: high-resolution decoupled network for building footprint extraction via deeply supervised body and boundary decomposition[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2024, 209: 51-65.
[15]	QIU Weiyan, GU Lingjia, GAO Fang, et al. Building extraction from very high-resolution remote sensing images using refine-UNet[J]. IEEE Geoscience and Remote Sensing Letters, 2023, 20: 3243609.
[16]	徐孝彬, 张好杰, 白建波, 等. 基于改进Unet的分布式光伏建筑物高精度分割方法[J]. 太阳能学报, 2023, 44(11): 82-90.
	XU Xiaobin, ZHANG Haojie, BAI Jianbo, et al. High-precision segmentation method of distributed photovoltaic buildings based on improved UNet[J]. Acta Energiae Solaris Sinica, 2023, 44(11): 82-90.
[17]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[18]	DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: Transformers for image recognition at scale[EB/OL]. [2024-05-03]. https://arxiv.org/abs/2010.11929.
[19]	WANG Libo, FANG Shenghui, MENG Xiaoliang, et al. Building extraction with vision transformer[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-11.
[20]	胡明洪, 李佳田, 姚彦吉, 等. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817. DOI: . doi: 10.11947/j.AGCS.2023.20210691
	HU Minghong, LI Jiatian, YAO Yanji, et al. SER-UNet algorithm for building extraction from high-resolution remote sensing image combined with multipath[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(5): 808-817. DOI: . doi: 10.11947/j.AGCS.2023.20210691
[21]	刘卓涛, 龚循强, 夏元平, 等. KU-Net：改进U-Net的高分辨率遥感影像建筑物提取方法[J]. 遥感信息, 2024, 39(5): 121-131.
	LIU Zhuotao, GONG Xunqiang, XIA Yuanping, et al. KU-Net: an improved U-Net method for building extraction from high resolution remote sensing imagery[J]. Remote Sensing Information, 2024, 39(5): 121-131.
[22]	WANG Xiaolei, HU Zirong, SHI Shouhai, et al. A deep learning method for optimizing semantic segmentation accuracy of remote sensing images based on improved UNet[J]. Scientific Reports, 2023, 13(1): 7600.
[23]	WANG Libo, LI Rui, ZHANG Ce, et al. UNetFormer: a UNet-like Transformer for efficient semantic segmentation of remote sensing urban scene imagery[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 190: 196-214.
[24]	YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL]. [2024-05-03]. https://arxiv.org/abs/1511.07122.
[25]	CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1-10
[26]	HU Jie, SHEN Li, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[27]	JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[C]//Proceedings of 2015 Advances in Neural Information Processing Systems. Montreal: MIT Press, 2015: 2017-2025.
[28]	WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of 2018 European Conference on Computer Vision. Cham: Springer International Publishing, 2018: 3-19.
[29]	MAO Anqi, MOHRI M, ZHONG Yutao. Cross-entropy loss functions: theoretical analysis and applications[EB/OL]. [2024-05-03]. https://arxiv.org/abs/2304.07288v2.
[30]	LI Xiaoya, SUN Xiaofei, MENG Yuxian, et al. Dice loss for data-imbalanced NLP tasks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 465-476.
[31]	BERMAN M, TRIKI A R, BLASCHKO M B. The lovász-Softmax loss: a tractable surrogate for the optimization of the intersection-over-union measure in neural networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 4413-4421.
[32]	JI Shunping, WEI Shiqing, LU Meng. Fully convolutional networks for multisource building extraction from an open aerial and satellite imagery data set[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(1): 574-586.
[33]	MNIH V. Machine learning for aerial image labeling[D]. Toronto: University of Toronto, 2013.
[34]	BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[35]	CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017: 1800-1807.
[36]	WANG Qilong, WU Banggu, ZHU Pengfei, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Seattle: IEEE, 2020: 11531-11539.

模型	IoU	Accuracy	Precision	Recall	F₁值
FCN-8s	79.71	95.43	88.68	89.76	88.60
U-Net	77.47	95.36	88.36	87.46	87.90
DeepLabV3+	86.07	96.48	92.28	90.05	91.15
SegNet	82.70	96.95	91.39	89.74	90.55
BuildFormer	85.96	97.03	89.51	89.45	89.47
HD-Net	84.59	96.41	89.03	88.89	88.96
SDSC-UNet	86.04	97.24	91.97	90.88	91.42
MAFNet	88.01	97.55	92.38	93.22	92.79

模型	IoU	Accuracy	Precision	Recall	F₁值
FCN-8s	71.82	93.96	81.44	80.94	81.18
U-Net	73.43	94.12	82.35	83.66	82.99
DeepLabV3+	76.67	94.74	85.87	86.67	86.26
SegNet	74.29	94.21	84.76	83.11	83.93
BuildFormer	77.31	95.01	87.26	86.32	86.79
HD-Net	76.44	94.49	85.20	85.66	85.43
SDSC-UNet	79.38	94.87	87.22	86.61	86.91
MAFNet	82.21	95.15	88.37	87.38	87.87

模型	训练过程处理速度/（batch/s）	训练总时长/h	测试过程处理速度/（batch/s）	测试总时长/min	IoU/（%）	F₁值/（%）
FCN-8s	3.0	30.1	6.2	3.2	79.71	88.60
U-Net	3.2	28.6	6.8	3.0	77.47	87.90
DeepLabV3+	2.2	39.6	5.0	4.0	86.07	91.15
SegNet	2.8	32.2	6.0	3.4	82.70	90.55
BuildFormer	2.5	36.3	5.6	3.6	85.96	89.47
HD-Net	2.7	32.4	5.8	3.5	84.59	88.96
SDSC-UNet	2.4	37.8	5.1	3.9	86.04	91.42
MAFNet	2.7	32.4	5.9	3.4	88.01	92.79

模型	IoU	Accuracy	Precision	Recall	F₁值
U-Net	77.47	95.36	88.36	87.46	87.90
Res_UNet	82.24	96.27	90.50	91.41	90.95
Res_CBAM_UNet	79.37	96.05	90.69	89.15	89.91
Res_ECA_UNet	85.09	97.15	92.47	91.34	92.07
Res_SE_UNet	83.41	95.71	91.40	90.51	90.94
MAFNet	88.01	97.55	92.38	93.22	92.79

损失函数	IoU	Accuracy	Precision	Recall	F₁值
L_B	86.58	97.05	92.14	89.67	91.03
L_D	85.41	95.84	91.84	90.76	91.29
L_L	86.18	96.27	92.01	93.26	92.63
0.8L_B+0.1L_D+0.1L_L	85.24	96.17	90.34	90.67	90.51
0.6L_B+0.1L_D+0.3L_L	86.96	97.34	90.91	93.31	92.09
0.6L_B+0.2L_D+0.2L_L	88.01	97.55	92.38	93.22	92.79
0.6L_B+0.3L_D+0.1L_L	85.94	96.21	90.66	92.29	91.47
0.4L_B+0.3L_D+0.3L_L	86.38	96.54	91.06	91.43	91.24
0.33L_B+0.33L_D+0.33L_L	85.99	96.42	91.13	90.95	91.03
0.2L_B+0.6L_D+0.2L_L	85.61	96.23	90.45	91.30	90.88
0.2L_B+0.4L_D+0.4L_L	85.72	96.67	90.87	91.56	91.21
0.2L_B+0.2L_D+0.6L_L	86.43	96.59	90.98	91.27	91.12