DCLS-GAN:利用生成对抗网络的天绘一号卫星高原地区影像去云方法

doi:10.11947/j.AGCS.2021.20200020

摘要/Abstract

摘要： 利用深度学习开展高原地区卫星影像去云是一个研究热点。本文提出了基于DCLS-GAN的天绘一号卫星高原地区影像的去云方法，采用对抗学习的思想构建深度卷积对抗生成网络，自主学习影像中云覆盖部分的典型地表特征，从而恢复云覆盖下垫面形貌。基于Encoder-Decoder结构生成网络，构建固定与可移动2种云区掩膜，在矩形固定中心掩模预训练之后进行随机位置云掩模迁移训练，使用最小二乘重建损失与交叉熵对抗损失的联合损失函数，用于精确修复云覆盖区域地表；基于CNN鉴别网络，判别生成影像的真实性。采用双线性插值提高云覆盖区域的修复精度，后处理使用泊松编辑处理平滑预测边界，减少伪迹的影响。在测试数据集上的试验结果表明，本文方法的总体去云效果在峰值信噪比、结构相似性与自然影像无参考质量评价算法指标上优于经典方法与原始Context Encoder，速度上较经典图像重建方法优势较大，具有较好的实际应用前景。

关键词: 高原地区, 卫星影像, 去云, 天绘一号, 深度卷积生成对抗网络, 最小二乘

Abstract: It has been a research hotspot to apply deep learning to remove cloud on satellite images. In this paper, we propose a cloud removal method based on DCLS-GAN for the plateau image of TH-1 satellite. The generator is constructed with the structure of encoder-decoder, and two types of fixed and removable cloud masks are used in training. The least squarereconstruction loss and cross-entropy adversarial loss are used to generate the prediction image of cloud coverage area, whilel east square loss is also used in the discriminator to identify the authenticity of the generated image. Joint optimization of generator and discriminator is achieved by continuous iteration, after which, bilinear interpolation is used to improve the restoration accuracy of cloud coverage area, and Poisson editing is used to smooth the prediction boundary and reduce the influence of artifacts. The experimental results on the testing dataset show that the cloud removal effect of proposed method exceeds classical methods and the original Context Encoder in peak signal-to-noise ratio and structure similarity, and experiments on images with real cloud area also show that proposed method has lower indicators under blind image quality assessment. Finally,the speed is faster than classical methods and equals Context Encoder, thus it has a better practical application prospect.

Key words: plateau area, satellite image, cloud removal, TH-1, DC-GAN, the least square

中图分类号:

P237

郑凯, 李建胜, 王俊强, 欧阳文, 谷友艺, 张迅. DCLS-GAN:利用生成对抗网络的天绘一号卫星高原地区影像去云方法[J]. 测绘学报, 2021, 50(2): 248-259.

ZHENG Kai, LI Jiansheng, WANG Junqiang, OUYANG Wen, GU Youyi, ZHANG Xun. DCLS-GAN: cloud removal method for plateau area of TH-1 satellite image[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(2): 248-259.

参考文献

[1] 葛均波, 徐永健. 内科学[M]. 8版. 北京:人民卫生出版社, 2013:923-927. GE Junbo, XU Yongjian. Internal medicine[M]. 8th ed. Beijing:People's Health Publishing House, 2013:923-927.
[2] ZHU Zhe, WOODCOCK C E. Automated cloud, cloud shadow, and snow detection in multitemporal Landsat data:an algorithm designed specifically for monitoring land cover change[J]. Remote Sensing of Environment, 2014, 152:217-234.
[3] CRIMINISI A, PÉREZ P, TOYAMA K. Region filling and object removal by exemplar-based image inpainting[J]. IEEE Transactions on Image Processing, 2004, 13(9):1200-1212.
[4] CHEN Bin, HUANG Bo, CHEN Lifan, et al. Spatially and temporally weighted regression:a novel method to produce continuous cloud-free Landsat imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(1):27-37.
[5] ZHANG Qiang, YUAN Qiangqiang, ZENG Chao, et al. Missing data reconstruction in remote sensing image with a unified spatial-temporal-spectral deep convolutional neural network[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018, 56(8):4274-4288.
[6] GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems. Montreal:NIPS, 2014:2672-2680.
[7] 林懿伦, 戴星原, 李力, 等. 人工智能研究的新前线:生成式对抗网络[J]. 自动化学报, 2018, 44(5):775-792. LIN Yilun, DAI Xingyuan, LI Li, et al. The new frontier of AI research:generative adversarial networks[J]. Acta Automatica Sinica, 2018, 44(5):775-792.
[8] 赵树阳, 李建武. 基于生成对抗网络的低秩图像生成方法[J]. 自动化学报, 2018, 44(5):829-839. ZHAO Shuyang, LI Jianwu. Generative adversarial network for generating low-rank images[J]. Acta Automatica Sinica, 2018, 44(5):829-839.
[9] 李雪, 张力, 王庆栋, 等. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报,2020,49(11):1473-1484. DOI:10.11947/j.AGCS.2020.20190439. LI Xue, ZHANG Li, WANG Qingdong, et al. Multi-temporal remote sensing imagery semantic segmentation color consistency adversarial network[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(11):1473-1484. DOI:10.11947/j.AGCS.2020.20190439.
[10] 陈玮, 李正旺, 尹钟. 基于生成对抗网络的图像去雾算法[J]. 信息与控制, 2019, 48(6):707-714, 722. CHEN Wei, LI Zhengwang, YIN Zhong. Image deblurring algorithm based on generative adversarial network[J]. Information and Control, 2019, 48(6):707-714, 722.
[11] PATHAK D, KRÄHENBÜHL P, DONAHUE J, et al. Context encoders:feature learning by inpainting[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV:IEEE, 2016:2536-2544.
[12] 张思雨, 李从利. 基于改进Criminisi算法的航拍图像厚云修复[J]. 激光与光电子学进展, 2018, 55(12):275-281. DOI:10. 3788/LOP55. 121012 ZHANG Siyu, LI Congli. Thick cloud restoration of aerial images based on improved criminisi algorithm[J]. Laser & Optoelectronics Progress, 2018, 55(12):275-281 DOI:10. 3788/LOP55. 121012.
[13] 李从利, 张思雨, 韦哲, 等. 基于深度卷积生成对抗网络的航拍图像去厚云方法[J]. 兵工学报, 2019, 40(7):1434-1442. LI Congli, ZHANG Siyu, WEI Zhe, et al. Thick cloud removal for aerial images based on deep convolutional generative adversarial networks[J]. Acta Armamentarii, 2019, 40(7):1434-1442.
[14] 李松明, 李岩, 李劲东. "天绘一号"传输型摄影测量与遥感卫星[J]. 遥感学报, 2012, 16(S1):10-16. LI Songming, LI Yan, LI Jindong. Mapping Satellite-1 transmission type photogrammetric and remote sensing satellite[J]. Journal of Remote Sensing, 2012, 16(S1):10-16.
[15] 林竹翀, 柳丽, 王翔, 等. 天绘一号卫星覆盖性能分析与优化方法研究[J]. 遥感学报, 2019, 23(3):547-554. LIN Zhuchong, LIU Li, WANG Xiang, et al. Analysis and optimization of the coverage performance for the TH01 satellite[J]. Journal of Remote Sensing, 2019, 23(3):547-554.
[16] DARABI S, SHECHTMAN E, BARNES C, et al. Image melding:combining inconsistent images using patch-based synthesis[J]. ACM Transactions on Graphics, 2012, 31(4):82.
[17] HUANG Jiabin, KANG S B, AHUJA N, et al. Image completion using planar structure guidance[J]. ACM Transactions on Graphics, 2014, 33(4):129.
[18] MAO Xudong, LI Qing, XIE Haoran, et al. Least squares generative adversarial networks[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice:IEEE, 2017:2813-2821.
[19] PÉREZ P, GANGNET M, BLAKE A. Poisson image editing[C]//ACM SIGGRAPH 2003 Papers. California:ACM, 2003:313-318. DOI:10.1145/1201775.882269.
[20] GLOROT X, BORDES A, BENGIO Y. Deep sparse rectifier neural networks[C]//Proceedings of the 14th International Conference on Artificial Intelligence and Statistics. Ft. Lauderdale:[s.n.], 2011:315-323.
[21] ZHANG Xiaohu, ZOU Yuexian, SHI Wei. Dilated convolution neural network with LeakyReLU for environmental sound classification[C]//Proceedings of the 22nd International Conference on Digital Signal Processing (DSP). London:IEEE, 2017:1-5.
[22] MARTIN Arjovsky S C, BOTTOU L. Wasserstein generative adversarial networks[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Australia. 2017.
[23] Huynh-Thu Q, Ghanbari M. Scope of validity of PSNR in image/video quality assessment[J]. Electronics letters, 2008, 44(13):800-801.
[24] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[25] MITTAL A, MURALIDHAR G S, GHOSH J, et al. Blind image quality assessment without human training using latent quality factors[J]. IEEE Signal Processing Letters, 2012, 19(2):75-78.