改进U-Net的遥感图像语义分割方法

doi:10.11947/j.AGCS.2023.20210684

测绘学报 ›› 2023, Vol. 52 ›› Issue (6): 980-989.doi: 10.11947/j.AGCS.2023.20210684

改进U-Net的遥感图像语义分割方法

胡功明¹, 杨春成^1,2,3, 徐立¹, 尚海滨¹, 王泽凡², 秦志龙¹

1. 中国地质大学(武汉)国家地理信息系统工程技术研究中心, 湖北武汉 430078;
2. 中国地质大学(武汉)地质探测与评估教育部重点实验室, 湖北武汉 430074;
3. 中国地质大学(武汉)地理与信息工程学院, 湖北武汉 430078

收稿日期:2021-12-10 修回日期:2022-05-23 发布日期:2023-07-08
通讯作者: 杨春成 E-mail:yangcc@cug.edu.cn
作者简介:胡功明(1997-),男,硕士,主要研究方向为遥感语义分割。E-mail:gongminghu@cug.edu.cn
基金资助:
国家自然科学基金(42171438)

Improved U-Net remote sensing image semantic segmentation method

HU Gongming¹, YANG Chuncheng^1,2,3, XU Li¹, SHANG Haibin¹, WANG Zefan², QIN Zhilong¹

1. National Engineering Research Center of Geographic Information System, China University of Geosciences, Wuhan 430078, China;
2. Key Laboratory of Geological Survey and Evaluation of Ministry of Education, China University of Geosciences, Wuhan 430074, China;
3. School of Geography and Information Engineering, China University of Geosciences, Wuhan 430078, China

Received:2021-12-10 Revised:2022-05-23 Published:2023-07-08
Supported by:
The National Natural Science Foundation of China (No. 42171438)

摘要/Abstract

摘要： 利用深度神经网络进行遥感影像语义分割是遥感智能解译的一个重要内容,在城市规划、灾害评估及农业生产等领域具有十分重要的作用。高分辨率遥感影像具有背景复杂、尺度多样及形状不规则等特点,使用自然场景语义分割方法处理遥感图像往往存在分割精度低的问题。针对上述情况,本文在U-Net模型基础上,提出了一种多尺度跳跃连接方法来融合不同层次的语义特征,获取准确的分割边界与位置信息;引入注意力机制和金字塔池化解决复杂背景下的精细分割问题。为了验证本文方法的有效性,在WHDLD和LandCover.ai数据集上进行试验,并与主流语义分割方法进行对比。试验结果表明,本文方法的mIoU分别达到74.28%和82.04%,F₁均值达到84.47%和89.76%,均优于其他对比方法;相比于U-Net的分割结果,IoU在建筑物、道路等占比较少的类别上提升明显,且优于其他对比方法。

关键词: 遥感语义分割, U-Net, 注意力机制, 多尺度跳跃连接, 金字塔池化

Abstract: Semantic segmentation of remote sensing images by deep neural network is an important content of remote sensing intelligent interpretation, which plays a very important role in urban planning, disaster assessment, agricultural production and other fields. High resolution remote sensing images are characterized by complex background, diverse scales and irregular shape, etc. Therefore, using natural scene semantic segmentation methods to process remote sensing images often has the problem of low segmentation accuracy. Based on the U-Net model, a multi-scale skip connection method is proposed to integrate semantic features of different levels and obtain accurate segmentation boundary and location information. Attention mechanism and pyramid pooling are introduced to solve the problem of fine segmentation in complex background. In order to verify the effectiveness of our proposed method, experiments were carried out on the WHDLD and LandCover.ai dataset and compared with the mainstream semantic segmentation methods. The experimental results show that the proposed method outperforms other comparison methods, with mIoU reaching 74.28% and 82.04% respectively, and with average of F₁ score reaching 84.47% and 89.76% respectively; compared with the segmentation results of U-Net, the value of IoU improves significantly in buildings, roads and other categories with a relatively small proportion, and is superior to other comparison methods.

Key words: remote sensing semantic segmentation, U-Net, attention mechanism, multi-scale skip connetion, pyramid pooling

中图分类号:

P237

胡功明, 杨春成, 徐立, 尚海滨, 王泽凡, 秦志龙. 改进U-Net的遥感图像语义分割方法[J]. 测绘学报, 2023, 52(6): 980-989.

HU Gongming, YANG Chuncheng, XU Li, SHANG Haibin, WANG Zefan, QIN Zhilong. Improved U-Net remote sensing image semantic segmentation method[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(6): 980-989.

参考文献

[1] 李道纪, 郭海涛, 卢俊, 等. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020,49(8):1051-1064. DOI: 10.11947/j.AGCS.2020.20190407. LI Daoji, GUO Haitao, LU Jun,et al. A remote sensing image classfication procedure based on multilevel attention fusion U-Net[J].Acta Geodaetica et Cartographica Sinica, 2020,49(8):1051-1064. DOI: 10.11947/j.AGCS.2020.20190407.
[2] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[3] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[4] KAMPFFMEYER M, SALBERG A B, JENSSEN R. Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Las Vegas:IEEE,2016: 680-688.
[5] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[C]//Proceedings of 2015 International Conference on Medical Image Computing and Computer-assisted Intervention.[S.l.]: Springer,2015.
[6] PAN Xuran, YANG Fan, GAO Lianru, et al. Building extraction from high-resolution aerial imagery using a generative adversarial network with spatial and channel attention mechanisms[J]. Remote Sensing, 2019,11(8):917.
[7] 张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报,2022,51(1):135-144. DOI:10.11947/j.AGCS.2021.20200508. ZAHNG Yuxin,YAN Qingsong,DENG Fei.Multi-path RSU network method for high-resolution remote sensing image building extraction[J]. Acta Geodaetica et Cartographica Sinica,2022,51(1):135-144.DOI:10.11947/j.AGCS.2021.20200508.
[8] HE Hao, WANG Wang, WANG Wang, et al.A road extraction method for remote sensing image based on encoder-de-coder network[J]. Journal of Geodesy and Geoinformation Science,2020,3(2): 415-426.
[9] ZHOU Z, SIDDIQUEE M M R, TAJBAKHSH N, et al. UNet: redesigning skip connections to exploit multiscale features in image segmentation[J]. IEEE Transactions on Medical Imaging, 2020, 39(6): 1856-1867.
[10] MNIH V, HEESS N, GRAVES A. Recurrent models of visual attention: advances in neural information processing systems[EB/OL]. [2022-11-20].https://arxiv.org/abs/1406.6247.
[11] HU Jie, SHEN Li, SUN Gang. Squeeze-and-excitation networks[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018: 7132-7141.
[12] ROY A G, NAVAB N, WACHINGER C. Concurrent spatial and channel 'squeeze & excitation’ in fully convolutional networks[C]//Proceedings of 2018 Medical Image Computing and Computer Assisted Intervention. Cham: Springer, 2018: 421-429.
[13] OKTAY O, SCHLEMPER J, FOLGOC L L, et al. Attention U-net: learning where to look for the pancreas[EB/OL]. [2022-11-24]. https://arxiv.org/abs/1804.03999.
[14] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[15] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu:IEEE, 2017: 6230-6239.
[16] YU F, KOLTUN V. Multi-scale context aggregation by dilated convolutions[EB/OL].[2022-12-04].https://arxiv.org/abs/1511.07122.
[17] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York: ACM, 2015: 448-456.
[18] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. Semantic image segmentation with deep convolutional nets and fully connected CRFs[EB/OL].[2023-01-04]. https://arxiv.org/abs/1412.7062.
[19] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(4): 834-848.
[20] CHEN L C, PAPANDREOU G, SCHROFF F, et al. Rethinking atrous convolution for semantic image segmentation[EB/OL].[2023-03-24]. https://arxiv.org/abs/1706.05587.
[21] CHEN L C, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of 15th European Conference of Computer Vision.Munich:ACM, 2018: 833-851.
[22] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J].Computer Science, 2014,10(1409):1556.
[23] ZHOU B, KHOSLA A, LAPEDRIZA A, et al. Object detectors emerge in deep scene CNNs[C]//Proceedings of 2015 International Conference on Learning Representations. San Diego:ICLR,2015.
[24] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV).Venice:IEEE, 2017: 2999-3007.
[25] SHAO Zhenfeng, YANG Ke, ZHOU Weixun. Performance evaluation of single-label and multi-label remote sensing image retrieval using a dense labeling dataset[J]. Remote Sensing, 2018, 10(6): 964.
[26] SHAO Zhenfeng, ZHOU Weixun, DENG Xueqing, et al. Multilabel remote sensing image retrieval based on fully convolutional network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 318-328.
[27] BOGUSZEWSKI A, BATORSKI D, ZIEMBA-JANKOWSKA N, et al. LandCover.ai: dataset for automatic mapping of buildings, woodlands and water from aerial imagery[J].Journal of the Indian Society of Remote Sensing,2018, 46:2057-2068.
[28] CHOLLET F. Xception: deep learning with depthwise separable convolutions[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu:IEEE, 2017: 1800-1807.

改进U-Net的遥感图像语义分割方法

Improved U-Net remote sensing image semantic segmentation method

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

[1]	顾小虎, 李正军, 缪健豪, 李星华, 沈焕锋. 高分遥感影像双通道并行混合卷积分类方法[J]. 测绘学报, 2023, 52(5): 798-807.
[2]	胡明洪, 李佳田, 姚彦吉, 阿晓荟, 陆美, 李文. 结合多路径的高分辨率遥感影像建筑物提取SER-UNet算法[J]. 测绘学报, 2023, 52(5): 808-817.
[3]	余东行, 徐青, 赵传, 郭海涛, 卢俊, 林雨准, 刘相云. 注意力引导特征融合与联合学习的遥感影像场景分类[J]. 测绘学报, 2023, 52(4): 624-637.
[4]	方莉娜, 王爽, 赵志远, 付化胜, 陈崇成. 车载激光点云中交通标线自动分类与矢量化[J]. 测绘学报, 2021, 50(9): 1251-1265.
[5]	方莉娜, 沈贵熙, 游志龙, 郭迎亚, 付化胜, 赵志远, 陈崇成. 融合点云和多视图的车载激光点云路侧多目标识别[J]. 测绘学报, 2021, 50(11): 1558-1573.
[6]	李道纪, 郭海涛, 卢俊, 赵传, 林雨准, 余东行. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8): 1051-1064.
[7]	李雪, 张力, 王庆栋, 艾海滨. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报, 2020, 49(11): 1473-1484.