Dual decoupling semantic segmentation model for high-resolution remote sensing images

doi:10.11947/j.AGCS.2023.20210455

Abstract

Abstract: Semantic segmentation is one of the core contents of high spatial resolution remote sensing images analysis and understanding. The existing semantic segmentation network based on deep learning will lead to the loss of high-frequency information and inaccurate edge segmentation of remote sensing images. Aiming at this problem,this study designs a dual decoupling semantic segmentation network model to improve the semantic segmentation performance of high-resolution remote sensing images. The extracted two-level feature maps are decoupled into edge features with high-frequency characteristics and body features with low-frequency characteristics,and the decoupled edge and body feature maps are fused. Furthermore,a loss function considering edge and body is proposed to optimize the ground feature elements.Experiments on ISPRS Vaihingen and ISPRS Potsdam 2D high spatial resolution remote sensing image datasets. Compared with the results of the existing remote sensing images semantic segmentation network model,the dual decoupling semantic segmentation network model can effectively improve the segmentation accuracy of ground feature elements.

Key words: high-resolution remote sensing image, semantic segmentation, dual decoupling network, deep learning, feature fusion

CLC Number:

P237

LIU Shuai, LI Xiaoying, YU Meng, XING Guanglong. Dual decoupling semantic segmentation model for high-resolution remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(4): 638-647.

References

[1] 刘东升, 廖通逵, 孙焕英, 等. 中国遥感软件研制进展与发展方向: 以像素专家PIE为例[J]. 中国图象图形学报, 2021, 26(5): 1169-1178. LIU Dongsheng, LIAO Tongkui, SUN Huanying, et al. Research progress and development direction of Chinese remote sensing software: taking PIE as an example[J]. Journal of Image and Graphics, 2021, 26(5): 1169-1178.
[2] HUANG D, WANG C. Optimal multi-level thresholding using a two-stage Otsu optimization approach[J]. Pattern Recognition Letters, 2009, 30(3): 275-284.
[3] AL-AMRI M, KALYANKAR N V, KHAMITKAR S D . Image segmentation by using edge detection[J]. International Journal of Advanced Trends in Computer Science and Engineering, 2010, 2(3):804-807.
[4] NAZIF A M, LEVINE M D. Low level image segmentation: an expert system[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1984, 6(5): 555-577.
[5] SHI J, MALIK J. Normalized cuts and image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(8): 888-905.
[6] SATHYA P, MALATHI L. Classification and segmentation in satellite imagery using back propagation algorithm of ANN and K-means algorithm[J]. International Journal of Machine Learning and Computing, 2011, 1(4): 422-426.
[7] LIZARAZO I. SVM-based segmentation and classification of remotely sensed data[J]. International Journal of Remote Sensing, 2008, 29(24): 7277-7283.
[8] BELGIU M, DRAGUT L. Random forest in remote sensing: a review of applications and future directions[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2016, 114: 24-31.
[9] ZHU X, TUIA D, MOU L, et al. Deep learning in remote sensing: a comprehensive review and list of resources[J]. IEEE Geoscience and Remote Sensing Magazine, 2017, 5(4): 8-36.
[10] LI Shutao, SONG Weiwei, FANG Leyuan, et al. Deep learning for hyperspectral image classification: an overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(9): 6690-6709.
[11] SHELHAMER E, LONG J, DARRELL T. Fully convolutional networks for semantic segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(4): 640-651.
[12] RONNEBERGER O, FISCHER P, BROX T. U-net: convolutional networks for biomedical image segmentation[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2015: 234-241.
[13] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[14] CHEN L, ZHU Y, PAPANDREOU G, et al. Encoder-decoder with atrous separable convolution for semantic image segmentation[C]//Proceedings of 2018 European Conference on Computer Vision.Munich, Germany: Springer, 2018:801-818.
[15] CHEN L C, PAPANDREOU G, KOKKINOS I, et al.DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence. 2018, 40(4):834-848.
[16] JIAO Licheng, LIANG Miaomiao, CHEN Huan, et al. Deep fully convolutional network-based spatial distribution prediction for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(10): 5585-5599.
[17] 李道纪, 郭海涛, 卢俊, 等. 遥感影像地物分类多注意力融和U型网络法[J]. 测绘学报, 2020, 49(8): 1051-1064. DOI: 10.11947/j.AGCS.2020.20190407. LI Daoji, GUO Haitao, LU Jun, et al. A remote sensing image classification procedure based on multilevel attention fusion U-Net[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8): 1051-1064.DOI: 10.11947/j.AGCS.2020.20190407.
[18] 左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6): 718-726.DOI: 10.11947/j.AGCS.2019.20170740. ZUO Zongcheng, ZHANG Wen, ZHANG Dongying. A remote sensing image semantic segmentation method by combining deformable convolution with conditional random fields[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(6): 718-726.DOI: 10.11947/j.AGCS.2019.20170740.
[19] NIU Zijia, LIU Wen, ZHAO Jingyi, et al. DeepLab-based spatial feature extraction for hyperspectral image classification[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(2): 251-255.
[20] DU Shouji, DU Shihong, LIU Bo, et al. Incorporating DeepLabv3+ and object-based image analysis for semantic segmentation of very high resolution remote sensing images[J]. International Journal of Digital Earth, 2021, 14(3): 357-378.
[21] 王忠武, 王志盼, 尤淑撑, 等. Landsat影像冰川提取的上下文感知语义分割网络法[J]. 测绘学报, 2020, 49(12): 1575-1582.DOI: 10.11947/j.AGCS.2020.20190313. WANG Zhongwu, WANG Zhipan, YOU Shucheng, et al. Landsat image glacier extraction based on context semantic segmentation network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(12): 1575-1582.DOI: 10.11947/j.AGCS.2020.20190313.
[22] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016: 770-778.
[23] LI Xiangtai,Li Xia, Zhang Li,et al. Improving semantic segmentation via decoupled body and edge supervision[C]//Proceedings of 2020 European Conference on Computer Vision. Glasgow, UK: Springer, 2020:435-452.
[24] ZHU Xizhou, XIONG Yuwen, DAI Jifeng, et al. Deep feature flow for video recognition[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA: IEEE, 2017: 4141-4150.
[25] DOSOVITSKIY A, FISCHER P, ILG E, et al. FlowNet: learning optical flow with convolutional networks[C]//Proceedings of 2015 IEEE International Conference on Computer Vision (ICCV). Santiago, Chile: IEEE, 2016: 2758-2766.
[26] JADERBERG M, SIMONYAN K, ZISSERMAN A. Spatial transformer networks[C]//Proceedings of 2015 IEEE Conference on Neural Information. Montreal, Quebec, Canada: Morgan Kaufmann, 2015: 2017-2025.
[27] POHLEN T, HERMANS A, MATHIAS M, et al. Full-resolution residual networks for semantic segmentation in street scenes[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu, HI, USA:IEEE, 2017: 3309-3318.
[28] BOKHOVKIN A, BURNAEV E. Boundary loss for remote sensing imagery semantic segmentation[C]//Proceedings of 2019 International Symposium on Neural Networks.Moscow, Russia: Springer, 2019: 388-401.