DCLS-GAN: cloud removal method for plateau area of TH-1 satellite image

doi:10.11947/j.AGCS.2021.20200020

Abstract

Abstract: It has been a research hotspot to apply deep learning to remove cloud on satellite images. In this paper, we propose a cloud removal method based on DCLS-GAN for the plateau image of TH-1 satellite. The generator is constructed with the structure of encoder-decoder, and two types of fixed and removable cloud masks are used in training. The least squarereconstruction loss and cross-entropy adversarial loss are used to generate the prediction image of cloud coverage area, whilel east square loss is also used in the discriminator to identify the authenticity of the generated image. Joint optimization of generator and discriminator is achieved by continuous iteration, after which, bilinear interpolation is used to improve the restoration accuracy of cloud coverage area, and Poisson editing is used to smooth the prediction boundary and reduce the influence of artifacts. The experimental results on the testing dataset show that the cloud removal effect of proposed method exceeds classical methods and the original Context Encoder in peak signal-to-noise ratio and structure similarity, and experiments on images with real cloud area also show that proposed method has lower indicators under blind image quality assessment. Finally,the speed is faster than classical methods and equals Context Encoder, thus it has a better practical application prospect.

Key words: plateau area, satellite image, cloud removal, TH-1, DC-GAN, the least square

CLC Number:

P237

ZHENG Kai, LI Jiansheng, WANG Junqiang, OUYANG Wen, GU Youyi, ZHANG Xun. DCLS-GAN: cloud removal method for plateau area of TH-1 satellite image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 248-259.

References

[1] 葛均波, 徐永健. 内科学[M]. 8版. 北京:人民卫生出版社, 2013:923-927. GE Junbo, XU Yongjian. Internal medicine[M]. 8th ed. Beijing:People's Health Publishing House, 2013:923-927.
[2] ZHU Zhe, WOODCOCK C E. Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data:an algorithm designed specifically for monitoring land cover change[J]. Remote Sensing of Environment, 2014, 152:217-234.
[3] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9):1200-1212.
[4] CHEN Bin, HUANG Bo, CHEN Lifan, et al. Spatially and temporally weighted regression:a novel method to produce continuous cloud-free Landsat imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(1):27-37.
[5] ZHANG Qiang, YUAN Qiangqiang, ZENG Chao, et al. Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8):4274-4288.
[6] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal:NIPS, 2014:2672-2680.
[7] 林懿伦, 戴星原, 李力, 等. 人工智能研究的新前线:生成式对抗网络[J]. 自动化学报, 2018, 44(5):775-792. LIN Yilun, DAI Xingyuan, LI Li, et al. The new frontier of AI research:generative adversarial networks[J]. Acta Automatica Sinica, 2018, 44(5):775-792.
[8] 赵树阳, 李建武. 基于生成对抗网络的低秩图像生成方法[J]. 自动化学报, 2018, 44(5):829-839. ZHAO Shuyang, LI Jianwu. Generative adversarial network for generating low-rank images[J]. Acta Automatica Sinica, 2018, 44(5):829-839.
[9] 李雪, 张力, 王庆栋, 等. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报,2020,49(11):1473-1484. DOI:10.11947/j.AGCS.2020.20190439. LI Xue, ZHANG Li, WANG Qingdong, et al. Multi-temporal remote sensing imagery semantic segmentation color consistency adversarial network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(11):1473-1484. DOI:10.11947/j.AGCS.2020.20190439.
[10] 陈玮, 李正旺, 尹钟. 基于生成对抗网络的图像去雾算法[J]. 信息与控制, 2019, 48(6):707-714, 722. CHEN Wei, LI Zhengwang, YIN Zhong. Image deblurring algorithm based on generative adversarial network[J]. Information and Control, 2019, 48(6):707-714, 722.
[11] PATHAK D, KRÄHENBÜHL P, DONAHUE J, et al. Context encoders:feature learning by inpainting[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV:IEEE, 2016:2536-2544.
[12] 张思雨, 李从利. 基于改进Criminisi算法的航拍图像厚云修复[J]. 激光与光电子学进展, 2018, 55(12):275-281. DOI:10. 3788/LOP55. 121012 ZHANG Siyu, LI Congli. Thick cloud restoration of aerial images based on improved criminisi algorithm[J]. Laser & Optoelectronics Progress, 2018, 55(12):275-281 DOI:10. 3788/LOP55. 121012.
[13] 李从利, 张思雨, 韦哲, 等. 基于深度卷积生成对抗网络的航拍图像去厚云方法[J]. 兵工学报, 2019, 40(7):1434-1442. LI Congli, ZHANG Siyu, WEI Zhe, et al. Thick cloud removal for aerial images based on deep convolutional generative adversarial networks[J]. Acta Armamentarii, 2019, 40(7):1434-1442.
[14] 李松明, 李岩, 李劲东. "天绘一号"传输型摄影测量与遥感卫星[J]. 遥感学报, 2012, 16(S1):10-16. LI Songming, LI Yan, LI Jindong. Mapping Satellite-1 transmission type photogrammetric and remote sensing satellite[J]. Journal of Remote Sensing, 2012, 16(S1):10-16.
[15] 林竹翀, 柳丽, 王翔, 等. 天绘一号卫星覆盖性能分析与优化方法研究[J]. 遥感学报, 2019, 23(3):547-554. LIN Zhuchong, LIU Li, WANG Xiang, et al. Analysis and optimization of the coverage performance for the TH01 satellite[J]. Journal of Remote Sensing, 2019, 23(3):547-554.
[16] DARABI S, SHECHTMAN E, BARNES C, et al. Image melding:combining inconsistent images using patch-based synthesis[J]. ACM Transactions on Graphics, 2012, 31(4):82.
[17] HUANG Jiabin, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4):129.
[18] MAO Xudong, LI Qing, XIE Haoran, et al. Least squares generative adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice:IEEE, 2017:2813-2821.
[19] PÉREZ P, GANGNET M, BLAKE A. Poisson image editing[C]//ACM SIGGRAPH 2003 Papers. California:ACM, 2003:313-318. DOI:10.1145/1201775.882269.
[20] GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Ft. Lauderdale:[s.n.], 2011:315-323.
[21] ZHANG Xiaohu, ZOU Yuexian, SHI Wei. Dilated convolution neural network with LeakyReLU for environmental sound classification[C]//Proceedings of the 22nd International Conference on Digital Signal Processing (DSP). London:IEEE, 2017:1-5.
[22] MARTIN Arjovsky S C, BOTTOU L. Wasserstein generative adversarial networks[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia. 2017.
[23] Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics letters, 2008, 44(13):800-801.
[24] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[25] MITTAL A, MURALIDHAR G S, GHOSH J, et al. Blind image quality assessment without human training using latent quality factors[J]. IEEE Signal Processing Letters, 2012, 19(2):75-78.