高分辨率遥感图像双解耦语义分割网络模型

doi:10.11947/j.AGCS.2023.20210455

测绘学报 ›› 2023, Vol. 52 ›› Issue (4): 638-647.doi: 10.11947/j.AGCS.2023.20210455

高分辨率遥感图像双解耦语义分割网络模型

刘帅^1,2, 李笑迎¹, 于梦¹, 邢光龙^1,2

1. 燕山大学信息科学与工程学院, 河北秦皇岛 066004;
2. 河北省信息传输与信号处理重点实验室, 河北秦皇岛 066004

收稿日期:2021-08-11 修回日期:2022-05-08 发布日期:2023-05-05
通讯作者: 邢光龙 E-mail:xinggl@ysu.edu.cn
作者简介:刘帅(1982-),男,博士,副教授,研究方向为遥感信息处理、分析与应用。E-mail:liushuai@ysu.edu.cn
基金资助:
国家自然科学基金(61671401);河北省自然科学基金(F2020203099)

Dual decoupling semantic segmentation model for high-resolution remote sensing images

LIU Shuai^1,2, LI Xiaoying¹, YU Meng¹, XING Guanglong^1,2

1. School of Information Science and Engineering, Yanshan University, Qinhuangdao 066004, China;
2. Hebei Key Laboratory of Information Transmission and Signal Processing, Qinhuangdao 066004, China

Received:2021-08-11 Revised:2022-05-08 Published:2023-05-05
Supported by:
The National Natural Science Foundation of China (No. 61671401);The Natural Science Foundation of Hebei Province (No. F2020203099)

摘要/Abstract

摘要： 语义分割是高空间分辨率遥感图像分析和理解的核心内容之一。现有基于深度学习的语义分割网络会导致遥感图像高频信息损失,边界分割不准确。针对此问题,本文提出一种双解耦语义分割网络模型,将提取的两级特征图解耦为具有高频特性的边界特征和具有低频特性的主体特征,并将解耦后的边界和主体特征图进行融合,从而改善高分辨率遥感图像语义分割性能。进一步提出了一种顾及边界和主体的损失函数,对地物要素及其边界和主体部分进行优化学习。在ISPRS Vaihingen和Potsdam 2D高空间分辨率遥感图像数据集上进行试验,与已有的遥感图像语义分割网络模型结果比较,双解耦语义分割网络模型能有效提高地物要素分割精度。

关键词: 高分辨率遥感图像, 语义分割, 双解耦网络, 深度学习, 特征融合

Abstract: Semantic segmentation is one of the core contents of high spatial resolution remote sensing images analysis and understanding. The existing semantic segmentation network based on deep learning will lead to the loss of high-frequency information and inaccurate edge segmentation of remote sensing images. Aiming at this problem,this study designs a dual decoupling semantic segmentation network model to improve the semantic segmentation performance of high-resolution remote sensing images. The extracted two-level feature maps are decoupled into edge features with high-frequency characteristics and body features with low-frequency characteristics,and the decoupled edge and body feature maps are fused. Furthermore,a loss function considering edge and body is proposed to optimize the ground feature elements.Experiments on ISPRS Vaihingen and ISPRS Potsdam 2D high spatial resolution remote sensing image datasets. Compared with the results of the existing remote sensing images semantic segmentation network model,the dual decoupling semantic segmentation network model can effectively improve the segmentation accuracy of ground feature elements.

Key words: high-resolution remote sensing image, semantic segmentation, dual decoupling network, deep learning, feature fusion

中图分类号:

P237

刘帅, 李笑迎, 于梦, 邢光龙. 高分辨率遥感图像双解耦语义分割网络模型[J]. 测绘学报, 2023, 52(4): 638-647.

LIU Shuai, LI Xiaoying, YU Meng, XING Guanglong. Dual decoupling semantic segmentation model for high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(4): 638-647.

参考文献

[1] 刘东升, 廖通逵, 孙焕英, 等. 中国遥感软件研制进展与发展方向: 以像素专家PIE为例[J]. 中国图象图形学报, 2021, 26(5): 1169-1178. LIU Dongsheng, LIAO Tongkui, SUN Huanying, et al. Research progress and development direction of Chinese remote sensing software: taking PIE as an example[J]. Journal of Image and Graphics, 2021, 26(5): 1169-1178.
[2] HUANG D, WANG C. Optimal multi-level thresholding using a two-stage Otsu optimization approach[J]. Pattern Recognition Letters, 2009, 30(3): 275-284.
[3] AL-AMRI M, KALYANKAR N V, KHAMITKAR S D . Image segmentation by using edge detection[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2010, 2(3):804-807.
[4] NAZIF A M, LEVINE M D. Low level image segmentation: an expert system[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1984, 6(5): 555-577.
[5] SHI J, MALIK J. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 888-905.
[6] SATHYA P, MALATHI L. Classification and segmentation in satellite imagery using back propagation algorithm of ANN and K-means algorithm[J]. International Journal of Machine Learning and Computing, 2011, 1(4): 422-426.
[7] LIZARAZO I. SVM-based segmentation and classification of remotely sensed data[J]. International Journal of Remote Sensing, 2008, 29(24): 7277-7283.
[8] BELGIU M, DRAGUT L. Random forest in remote sensing: a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114: 24-31.
[9] ZHU X, TUIA D, MOU L, et al. Deep learning in remote sensing: a comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4): 8-36.
[10] LI Shutao, SONG Weiwei, FANG Leyuan, et al. Deep learning for hyperspectral image classification: an overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6690-6709.
[11] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[12] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[13] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[14] CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of 2018 European Conference on Computer Vision.Munich, Germany: Springer, 2018:801-818.
[15] CHEN L C, PAPANDREOU G, KOKKINOS I, et al.DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018, 40(4):834-848.
[16] JIAO Licheng, LIANG Miaomiao, CHEN Huan, et al. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5585-5599.
[17] 李道纪, 郭海涛, 卢俊, 等. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8): 1051-1064. DOI: 10.11947/j.AGCS.2020.20190407. LI Daoji, GUO Haitao, LU Jun, et al. A remote sensing image classification procedure based on multilevel attention fusion U-Net[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8): 1051-1064.DOI: 10.11947/j.AGCS.2020.20190407.
[18] 左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6): 718-726.DOI: 10.11947/j.AGCS.2019.20170740. ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 718-726.DOI: 10.11947/j.AGCS.2019.20170740.
[19] NIU Zijia, LIU Wen, ZHAO Jingyi, et al. DeepLab-based spatial feature extraction for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(2): 251-255.
[20] DU Shouji, DU Shihong, LIU Bo, et al. Incorporating DeepLabv3+ and object-based image analysis for semantic segmentation of very high resolution remote sensing images[J]. International Journal of Digital Earth, 2021, 14(3): 357-378.
[21] 王忠武, 王志盼, 尤淑撑, 等. Landsat影像冰川提取的上下文感知语义分割网络法[J]. 测绘学报, 2020, 49(12): 1575-1582.DOI: 10.11947/j.AGCS.2020.20190313. WANG Zhongwu, WANG Zhipan, YOU Shucheng, et al. Landsat image glacier extraction based on context semantic segmentation network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(12): 1575-1582.DOI: 10.11947/j.AGCS.2020.20190313.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016: 770-778.
[23] LI Xiangtai,Li Xia, Zhang Li,et al. Improving semantic segmentation via decoupled body and edge supervision[C]//Proceedings of 2020 European Conference on Computer Vision. Glasgow, UK: Springer, 2020:435-452.
[24] ZHU Xizhou, XIONG Yuwen, DAI Jifeng, et al. Deep feature flow for video recognition[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017: 4141-4150.
[25] DOSOVITSKIY A, FISCHER P, ILG E, et al. FlowNet: learning optical flow with convolutional networks[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2016: 2758-2766.
[26] JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks[C]//Proceedings of 2015 IEEE Conference on Neural Information. Montreal, Quebec, Canada: Morgan Kaufmann, 2015: 2017-2025.
[27] POHLEN T, HERMANS A, MATHIAS M, et al. Full-resolution residual networks for semantic segmentation in street scenes[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu, HI, USA:IEEE, 2017: 3309-3318.
[28] BOKHOVKIN A, BURNAEV E. Boundary loss for remote sensing imagery semantic segmentation[C]//Proceedings of 2019 International Symposium on Neural Networks.Moscow, Russia: Springer, 2019: 388-401.

高分辨率遥感图像双解耦语义分割网络模型

Dual decoupling semantic segmentation model for high-resolution remote sensing images

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	余东行, 徐青, 赵传, 郭海涛, 卢俊, 林雨准, 刘相云. 注意力引导特征融合与联合学习的遥感影像场景分类[J]. 测绘学报, 2023, 52(4): 624-637.
[2]	张永显, 马国锐, 崔志祥, 张志军. 面向大视角差的无人机影像序列学习型特征匹配[J]. 测绘学报, 2023, 52(2): 230-243.
[3]	卜丽静, 吴文玉, 张正鹏, 杨银. 先验引导的多降质特征“珞珈一号”夜光遥感影像质量提升方法[J]. 测绘学报, 2023, 52(2): 244-259.
[4]	孙根云, 王鑫, 安娜, 张爱竹. 基于多源遥感的大尺度高分辨率不透水面深度学习提取方法[J]. 测绘学报, 2023, 52(2): 272-282.
[5]	禹文豪, 魏铖, 陈佳鑫. 面向不平衡POI类别分布的电子地图多层次标签预测模型[J]. 测绘学报, 2023, 52(2): 318-328.
[6]	杨敏, 陈果, 李连营, 黄浩然, 苗静, 晏雄锋. 序列卷积神经网络支持下线状地图目标的分段方法[J]. 测绘学报, 2023, 52(1): 108-116.
[7]	眭海刚, 刘畅, 干哲, 江政杰, 徐川. 多模态遥感图像匹配方法综述[J]. 测绘学报, 2022, 51(9): 1848-1861.
[8]	张良培, 何江, 杨倩倩, 肖屹, 袁强强. 数据驱动的多源遥感信息融合研究进展[J]. 测绘学报, 2022, 51(7): 1317-1337.
[9]	杨必胜, 陈驰, 董震. 面向智能化测绘的城市地物三维提取[J]. 测绘学报, 2022, 51(7): 1476-1484.
[10]	程涛, 张洋, James Haworth. 基于网络和图的时空智能——概念、方法和应用[J]. 测绘学报, 2022, 51(7): 1629-1639.
[11]	龚健雅, 宦麟茜, 郑先伟. 影像解译中的深度学习可解释性分析方法[J]. 测绘学报, 2022, 51(6): 873-884.
[12]	张勤, 赵超英, 陈雪蓉. 多源遥感地质灾害早期识别技术进展与发展趋势[J]. 测绘学报, 2022, 51(6): 885-896.
[13]	龚健雅, 张觅, 胡翔云, 张展, 李彦胜, 姜良存. 智能遥感深度学习框架与模型设计[J]. 测绘学报, 2022, 51(4): 475-487.
[14]	毛文婧, 王卫林, 焦利民, 刘安宝. 基于深度学习的中国连续空间覆盖PM_2.5浓度预报[J]. 测绘学报, 2022, 51(3): 361-372.
[15]	金飞, 官恺, 刘智, 韩佳容, 芮杰, 李庆高. 无监督密集匹配特征提取网络性能分析[J]. 测绘学报, 2022, 51(3): 426-436.