顾及全局特征和纹理特征的遥感影像超分辨率重建方法

doi:10.11947/j.AGCS.2023.20210571

测绘学报 ›› 2023, Vol. 52 ›› Issue (4): 648-659.doi: 10.11947/j.AGCS.2023.20210571

顾及全局特征和纹理特征的遥感影像超分辨率重建方法

胡安娜¹, 刘睿², 吴亮³, 张进⁴, 徐永洋³, 陈思琼²

1. 中国地质大学(武汉)国家地理与信息系统工程技术研究中心, 武汉 430078;
2. 中国地质大学(武汉)地理与信息工程学院, 武汉 430078;
3. 中国地质大学(武汉)计算机学院, 武汉 430078;
4. 武汉科技大学计算机科学与技术学院, 武汉 430081

收稿日期:2021-10-20 修回日期:2022-05-10 发布日期:2023-05-05
通讯作者: 徐永洋 E-mail:yongyangxu@cug.edu.cn
作者简介:胡安娜(1994-),女,博士生,研究方向为遥感影像处理。E-mail:huanna@cug.edu.cn
基金资助:
国家自然科学基金(41871311;42001340)

Super-resolution reconstruction method for remote sensing images considering global features and texture features

HU Anna¹, LIU Rui², WU Liang³, ZHANG Jin⁴, XU Yongyang³, CHEN Siqiong²

1. National Engineering Research Center of Geographic Information System, China University of Geosciences, Wuhan 430078, China;
2. School of Geography and Information Engineering, China University of Geosciences, Wuhan 430078, China;
3. School of Computer Science, China University of Geosciences, Wuhan 430078, China;
4. School of Computer Science and Technology, Wuhan University of Science and Technology, Wuhan 430081, China

Received:2021-10-20 Revised:2022-05-10 Published:2023-05-05
Supported by:
The National Natural Science Foundation of China (Nos. 41871311;42001340)

摘要/Abstract

摘要： 由于遥感设备的性能限制,使得采集的遥感影像质量受到影响,低分辨率的遥感影像限制了遥感解译应用的精度。当前针对遥感影像的超分辨率重建研究仍然存在重建后的遥感影像地物全局信息和纹理细节不足的问题。因此,本文提出顾及全局特征和纹理特征的遥感影像超分辨率重建方法,该方法利用生成对抗网络的特征学习能力,并对模型全局和纹理进行增强。一方面,地物全局特征增强部分用于解决当前研究中超分辨率重建模型对低分辨率遥感影像中全局遥感地物信息没有重视和利用的问题。在生成网络中引入自注意力模块,以获取全局地物注意力图的方式将遥感影像中相距较远的地物信息作为重建过程的参考。另一方面,遥感影像纹理增强部分用于解决超分辨率重建模型中超分辨率影像纹理信息不足的问题。本文方法引入纹理损失以优化生成网络参数并增强超分辨率重建后影像中的纹理信息。另外,为避免重建结果中的“伪影”现象,研究采用权值归一化代替批量归一化方法。试验结果表明,本文方法在遥感影像超分辨率重建过程中能增强遥感地物特征,同时可以实现地物的纹理细节精细化恢复,而且超分辨率重建结果的图像质量评价指标SSIM、FSIM和PSNR值分别达到了0.756、0.595和26.005。

关键词: 高分辨率遥感影像, 超分辨率重建, 生成对抗网络, 纹理增强, 全局特征

Abstract: Due to the performance limitation of remote sensing equipment, the quality of the remote sensing image is affected, and the low-resolution remote sensing image limits the accuracy of remote sensing interpretation applications. Insufficient global information and texture details of reconstructed remote sensing images are still in super-resolution reconstruction research. Therefore, this study proposes a super-resolution reconstruction method for remote sensing images considering global features and texture features. The method utilizes the feature learning ability of the generative adversarial network to optimize the model in two aspects: global information enhancement and texture information enhancement. On the one hand, the global feature enhancement part is used to solve the problem that the current super-resolution reconstruction model does not pay attention to the global remote sensing information of low-resolution remote sensing images. The self-attention module is introduced to the generation network, which is used to obtain the global object attention map, and the remote object information in remote sensing image is used as a reference in the reconstruction process. On the other hand, the texture enhancement part is used to solve the problem of insufficient texture information of the reconstructed remote sensing image. Texture loss is introduced to the optimized generated network that the texture information of ground objects can be improved. In addition, weight normalization is adopted to replace batch normalization to avoid false shadows in the reconstruction result. The experimental results show that the proposed super-resolution algorithm can not only enhance the features of the ground object, but also recovery the texture details for ground objects, and the SSIM, FSIM, and PSNR values of the reconstructed super-resolution image quality evaluation index are 0.756, 0.595 and 26.005, respectively.

Key words: super-resolution reconstruction, high-resolution remote sensing image, generative adversarial networks, texture enhancement, global feature

中图分类号:

P227

胡安娜, 刘睿, 吴亮, 张进, 徐永洋, 陈思琼. 顾及全局特征和纹理特征的遥感影像超分辨率重建方法[J]. 测绘学报, 2023, 52(4): 648-659.

HU Anna, LIU Rui, WU Liang, ZHANG Jin, XU Yongyang, CHEN Siqiong. Super-resolution reconstruction method for remote sensing images considering global features and texture features[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(4): 648-659.

参考文献

[1] WANG Kexian, ZHENG Shunyi, LI Rui, et al. A deep double-channel dense network for hyperspectral image classification[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(4): 46-62.
[2] WANG Yanjun, LI Shaochun, WANG Mengjie, et al. A simple deep learning network for classification of 3D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3): 49-59.
[3] YAO Jiaqi, LI Guoyuan, CHEN Jiyi, et al. Cloud detection and centroid extraction of laser footprint image of GF-7 satellite laser altimetry[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3): 1-12.
[4] 吴俊斌, 吴晟, 吴兴蛟. 矢量填充和插值算法的图像放大[J]. 计算机与数字工程, 2016, 44(6): 1146-1150, 1166. WU Junbin, WU Sheng, WU Xingjiao. Vector filling and interpolation algorithms of image magnification[J]. Computer & Digital Engineering, 2016, 44(6): 1146-1150, 1166.
[5] 龙四春, 周威, 文佳胜, 等. 雷达地形测绘DEM空洞插补方法研究[J]. 遥感信息, 2015, 30(4): 20-24, 36. LONG Sichun, ZHOU Wei, WEN Jiasheng, et al. SRTM DEM voids interpolation method based on Matlab[J]. Remote Sensing Information, 2015, 30(4): 20-24, 36.
[6] 边刚, 金绍华, 夏伟, 等. 线性插值的海洋磁力测量测线布设评价方法[J]. 测绘学报, 2014, 43(7): 675-680, 697. BIAN Gang, JIN Shaohua, XIA Wei, et al. Evaluating method of the survey line layout based on the linear interpolation in marine magnetic survey[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(7): 675-680, 697.
[7] 秦彦平, 张军, 多化琼, 等. 木材节子图像增强的小波变换与双三次插值融合方法[J]. 西北林学院学报, 2021, 36(5): 183-189. QIN Yanping, ZHANG Jun, DUO Huaqiong, et al. Fusion method of wavelet transform and bicubic interpolation for wood knot image enhancement[J]. Journal of Northwest Forestry University, 2021, 36(5): 183-189.
[8] 汪智易. 基于流形学习的图像超分辨率重建研究[D]. 西安: 西安电子科技大学, 2014. WANG Zhiyi. Research on image super-resolution reconstruction based on manifold learning[D]. Xi'an: Xidian University, 2014.
[9] 陶志强, 李海林, 张红兵. 基于新边缘指导插值的迭代反投影超分辨率重建算法[J]. 计算机工程, 2016, 42(6): 255-260. TAO Zhiqiang, LI Hailin, ZHANG Hongbing. Iterative back projection super resolution reconstruction algorithm based on new edge directed interpolation[J]. Computer Engineering, 2016, 42(6): 255-260.
[10] 苏锦程, 胡勇, 巩彩兰. 一种混合红外云图超分辨率重建算法[J]. 红外, 2018, 39(8): 34-39. SU Jincheng, HU Yong, GONG Cailan. A super-resolution reconstruction algorithm for hybrid infrared cloud images[J]. Infrared, 2018, 39(8): 34-39.
[11] 郭桐宇. 基于导向滤波与迭代反向投影的遥感影像超分辨率重建[J]. 测绘与空间地理信息, 2019, 42(1): 195-197, 205. GUO Tongyu. Super resolution reconstruction of remote sensing image based on improved iterative back projection algorithm[J]. Geomatics & Spatial Information Technology, 2019, 42(1): 195-197, 205.
[12] 杨琇卿. 基于Lanczos插值的迭代反投影图像超分辨率算法[D]. 济南: 山东大学, 2018. YANG Xiuqing. An iterative back-projection image super-resolution algorithm based on Lanczos interpolation[D]. Jinan: Shandong University, 2018.
[13] 许丽娜, 何鲁晓. 基于凸集投影的高分四号卫星影像超分辨率重建[J]. 测绘学报, 2017, 46(8): 1026-1033. DOI: 10.11947/j.AGCS.2017.20170070. XU Lina, HE Luxiao. GF-4 images super resolution reconstruction based on POCS[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(8): 1026-1033. DOI: 10.11947/j.AGCS.2017.20170070.
[14] 陈波. 自适应光学图像复原理论与算法研究[D]. 郑州:信息工程大学, 2008. CHEN Bo. The theory and algorithms of adaptive optics image restoration[D]. Zhengzhou:Information Engineering University, 2008.
[15] 武兴睿. 基于PSF重构和改进的最大后验估计的自适应光学图像复原算法[J]. 液晶与显示, 2019, 34(9): 921-927. WU Xingrui. Adaptive optical image restoration method based on PSF reconstruction and improved maximum posteriori estimation[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(9): 921-927.
[16] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
[17] CAO Hangpu, MI Sicheng. Weighted SRGAN and reconstruction loss analysis for accurate image super resolution[C]//Proceedings of 2021 Modelling and Intelligent Computing Conference. Guilin, China: CAMMIC, 2021.
[18] WANG X, YU K, WU S, et al. Esrgan: enhanced super-resolution generative adversarial networks[C]//Proceedings of 2018 European Conference on Computer Vision (ECCV) Workshops. Munich, Germany: Springer: 2018.
[19] ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[C]//Proceedings of 2019 International Conference on Machine Learning. [S.l.]:PMLR, 2019: 7354-7363.
[20] SALIMANS T, KINGMA D P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: ACM Press, 2016: 901-909.
[21] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York, NY, USA: ACM Press, 2015, 37: 448-456.
[22] JOLICOEUR-MARTINEAU A. The relativistic discriminator: a key element missing from standard GAN[EB/OL]. [2022-01-01]. https://arxiv.org/abs/1807.00734.
[23] 余志凡, 李昊, 李登实, 等. 基于Gram矩阵和卷积神经网络的风格迁移算法[J]. 江汉大学学报(自然科学版), 2020, 48(3): 62-68. YU Zhifan, LI Hao, LI Dengshi, et al. Style transfer algorithm based on gram matrix and convolutional[J]. Journal of Jianghan University (Natural Science Edition), 2020, 48(3): 62-68.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-11-20]. https://arxiv.org/abs/1409.1556.
[25] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA:IEEE, 2009: 248-255.
[26] KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2021-11-20]. https://arxiv.org/abs/1412.6980.
[27] SHI W, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 1874-1883.
[28] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Honolulu, USA:IEEE, 2017: 136-144.
[29] LEI Sen, SHI Zhenwei. Hybrid-scale self-similarity exploitation for remote sensing image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-10.

顾及全局特征和纹理特征的遥感影像超分辨率重建方法

Super-resolution reconstruction method for remote sensing images considering global features and texture features

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 15

编辑推荐

Metrics

本文评价

[1]	卜丽静, 吴文玉, 张正鹏, 杨银. 先验引导的多降质特征“珞珈一号”夜光遥感影像质量提升方法[J]. 测绘学报, 2023, 52(2): 244-259.
[2]	张广斌, 高贤君, 冉树浩, 杨元维, 李丽珊, 张妍. 高分遥感影像云雪共存区轻量云高精度检测方法[J]. 测绘学报, 2023, 52(1): 93-107.
[3]	何直蒙, 丁海勇, 安炳琪. 高分辨率遥感影像建筑物提取的空洞卷积E-Unet算法[J]. 测绘学报, 2022, 51(3): 457-467.
[4]	葛小三, 陈曦, 赵文智, 李瑞祥. 基于生成对抗网络的建筑物损毁检测[J]. 测绘学报, 2022, 51(2): 238-247.
[5]	张玉鑫, 颜青松, 邓非. 高分辨率遥感影像建筑物提取多路径RSU网络法[J]. 测绘学报, 2022, 51(1): 135-144.
[6]	郑凯, 李建胜, 王俊强, 欧阳文, 谷友艺, 张迅. DCLS-GAN:利用生成对抗网络的天绘一号卫星高原地区影像去云方法[J]. 测绘学报, 2021, 50(2): 248-259.
[7]	张涛, 丁乐乐, 史芙蓉. 高分辨率遥感影像城中村提取的景观语义指数方法[J]. 测绘学报, 2021, 50(1): 97-104.
[8]	王舒洋, 慕晓冬, 贺浩, 杨东方, 马晨晖. 航拍图像跨数据域特征迁移道路提取方法[J]. 测绘学报, 2020, 49(5): 611-621.
[9]	卜丽静, 郑新杰, 张正鹏. 顾及运动估计误差的“凝视”卫星视频运动场景超分辨率重建[J]. 测绘学报, 2020, 49(2): 214-224.
[10]	李雪, 张力, 王庆栋, 艾海滨. 多时相遥感影像语义分割色彩一致性对抗网络[J]. 测绘学报, 2020, 49(11): 1473-1484.
[11]	张瑞辰, 边少锋, 刘雁春, 李厚朴. 大面积水深异常检测的条件变化自编码算法[J]. 测绘学报, 2019, 48(9): 1182-1189.
[12]	左宗成, 张文, 张东映. 融合可变形卷积与条件随机场的遥感影像语义分割方法[J]. 测绘学报, 2019, 48(6): 718-726.
[13]	董志鹏, 王密, 李德仁, 王艳丽, 张致齐. 遥感影像目标的尺度特征卷积神经网络识别法[J]. 测绘学报, 2019, 48(10): 1285-1295.
[14]	范荣双, 陈洋, 徐启恒, 王竞雪. 基于深度学习的高分辨率遥感影像建筑物提取方法[J]. 测绘学报, 2019, 48(1): 34-41.
[15]	董志鹏, 王密, 李德仁, 王艳丽, 张致齐. 利用对象光谱与纹理实现高分辨率遥感影像云检测方法[J]. 测绘学报, 2018, 47(7): 996-1006.