High-resolution remote sensing image semantic segmentation based on semi-supervised full convolution network method

doi:10.11947/j.AGCS.2020.20190044

Abstract

Abstract: In the field of remote sensing, the method of realizing image semantic segmentation by using a large amount of label image data to supervise training full convolution network will result in expensive label drawing cost, while the use of a small amount of label data would lead to network performance degradation. To solve this problem, this paper proposes a semi-supervised full convolution network based semantic segmentation method for high resolution remote sensing images. Specifically, we explore an ensemble prediction technique to train the end-to-end semantic segmentation network by simultaneously optimizing a standard supervised classification loss on labeled samples along with an additional unsupervised consistence loss term imposed on labeled and unlabeled data. In the experiments, the image data set of Vaihingen in Germany provided by ISPRS and satellite GF-1 data were used, and the experimental results show that the proposed method can effectively improve the network performance degradation caused by using only a small amount of label data.

Key words: remote sensing image, semantic segmentation, semi-supervised, full convolution network

CLC Number:

P231

GENG Yanlei, TAO Chao, SHEN Jing, ZOU Zhengrong. High-resolution remote sensing image semantic segmentation based on semi-supervised full convolution network method[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4): 499-508.

References

[1] 何小飞, 邹峥嵘, 陶超, 等. 联合显著性和多层卷积神经网络的高分影像场景分类[J]. 测绘学报, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612. HE Xiaofei, ZOU Zhengrong, TAO Chao, et al. Combined saliency with multi-convolutional neural network for high resolution remote sensing scene classification[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(9):1073-1080. DOI:10.11947/j.AGCS.2016.20150612.
[2] ZUO Tongchun, FENG Juntao, CHEN Xuejin. HF-FCN:hierarchically fused fully convolutional network for robust building extraction[C]//Proceedings of the 13th Asian Conference on Computer Vision. Taipei, Taiwan, China:Springer, 2016.
[3] NORONHA S, NEVATIA R. Detection and modeling of buildings from multiple aerial images[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2001, 23(5):501-518.
[4] COTE M, SAEEDI P. Automatic rooftop extraction in nadir aerial imagery of suburban regions using corners and variational level set evolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(1):313-328.
[5] LI Er, FEMIANI J, XU Shibiao, et al. Robust rooftop extraction from visible band images using higher order CRF[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(8):4483-4495.
[6] 胡翔云, 巩晓雅, 张觅. 变分法遥感影像人工地物自动检测[J]. 测绘学报, 2018, 47(6):780-789. DOI:10.11947/j.AGCS.2018.20170642. HU Xiangyun, GONG Xiaoya, ZHANG Mi. A variational approach for automatic man-made object detection from remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6):780-789. DOI:10.11947/j.AGCS.2018.20170642.
[7] 林祥国, 张继贤. 面向对象的形态学建筑物指数及其高分辨率遥感影像建筑物提取应用[J]. 测绘学报, 2017, 46(6):724-733. DOI:10.11947/j.AGCS.2017.20170068. LIN Xiangguo, ZHANG Jixian. Object-based morphological building index for building extraction from high resolution remote sensing imagery[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(6):724-733. DOI:10.11947/j.AGCS.2017.20170068.
[8] WANG Weixing, YANG Nan, ZHANG Yi, et al. A review of road extraction from remote sensing images[J]. Journal of Traffic and Transportation Engineering, 2016, 3(3):271-282.
[9] LECUN Y, BOTTOU L, BENGIO Y, et al. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11):2278-2324.
[10] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Lake Tahoe, Nevada:Curran Associates Inc., 2012.
[11] SAITO S, YAMASHITA T, AOKI Y. Multiple object extraction from aerial imagery with convolutional neural networks[J]. Journal of Imaging Science and Technology, 2016, 60(1):010402.
[12] ALSHEHHI R, MARPU P R, WOON W L, et al. Simultaneous extraction of roads and buildings in remote sensing imagery with convolutional neural networks[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2017(130):139-149.
[13] SAITO S, AOKI Y. Building and road detection from large aerial imagery[C]//Proceedings of SPIE 9405, Image Processing:Machine Vision Applications VIII. San Francisco, CA:SPIE, 2015:94050K.
[14] 范荣双, 陈洋, 徐启恒, 等. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1):34-41. DOI:10.11947/j.AGCS.2019.20170638. FAN Rongshuang, CHEN Yang, XU Qiheng, et al. A high-resolution remote sensing image building extraction method based on deep learning[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(1):34-41. DOI:10.11947/j.AGCS.2019.20170638.
[15] LONG J, SHELHAMER E, DARRELL T. Fully convolutional networks for semantic segmentation[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA:IEEE, 2015:3431-3440.
[16] LAINE S, AILA T. Temporal Ensembling for semi-supervised learning[C]//Proceedings of 2017 International Conference on Learning Representations. Toulon, France:ICLR, 2017.
[17] TORRALBA A, FERGUS R, FREEMAN W T. 80 million tiny images:a large data set for nonparametric object and scene recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(11):1958-1970.
[18] RONNEBERGER O, FISCHER P, BROX T. U-Net:convolutional networks for biomedical image segmentation[C]//Proceedings of the 18th Medical Image Computing and Computer-Assisted Intervention. Munich, Germany:Springer, 2015:234-241.
[19] CHEN L C, ZHU Yukun, PAPANDREOU G, et al. Encoder-decoder with Atrous separable convolution for semantic image segmentation[C]//Proceedings of the 15th European Conference on Computer Vision. Munich, Germany:Springer, 2018:833-851.
[20] GATYS L A, ECKER A S, BETHGE M. Image style transfer using convolutional neural networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016.
[21] IGLOVIKOV V, SHVETS A A. TernausNet:U-Net with VGG11 encoder pre-trained on ImageNet for image segmentation. Computer Vision and Pattern Recognition, 2018.
[22] DENG Jia, DONG Wei, SOCHER R, et al. ImageNet:a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, FL:IEEE, 2009.
[23] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal, Canada:MIT Press, 2014.