Super-resolution reconstruction method for remote sensing images considering global features and texture features

doi:10.11947/j.AGCS.2023.20210571

Abstract

Abstract: Due to the performance limitation of remote sensing equipment, the quality of the remote sensing image is affected, and the low-resolution remote sensing image limits the accuracy of remote sensing interpretation applications. Insufficient global information and texture details of reconstructed remote sensing images are still in super-resolution reconstruction research. Therefore, this study proposes a super-resolution reconstruction method for remote sensing images considering global features and texture features. The method utilizes the feature learning ability of the generative adversarial network to optimize the model in two aspects: global information enhancement and texture information enhancement. On the one hand, the global feature enhancement part is used to solve the problem that the current super-resolution reconstruction model does not pay attention to the global remote sensing information of low-resolution remote sensing images. The self-attention module is introduced to the generation network, which is used to obtain the global object attention map, and the remote object information in remote sensing image is used as a reference in the reconstruction process. On the other hand, the texture enhancement part is used to solve the problem of insufficient texture information of the reconstructed remote sensing image. Texture loss is introduced to the optimized generated network that the texture information of ground objects can be improved. In addition, weight normalization is adopted to replace batch normalization to avoid false shadows in the reconstruction result. The experimental results show that the proposed super-resolution algorithm can not only enhance the features of the ground object, but also recovery the texture details for ground objects, and the SSIM, FSIM, and PSNR values of the reconstructed super-resolution image quality evaluation index are 0.756, 0.595 and 26.005, respectively.

Key words: super-resolution reconstruction, high-resolution remote sensing image, generative adversarial networks, texture enhancement, global feature

CLC Number:

P227

HU Anna, LIU Rui, WU Liang, ZHANG Jin, XU Yongyang, CHEN Siqiong. Super-resolution reconstruction method for remote sensing images considering global features and texture features[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(4): 648-659.

References

[1] WANG Kexian, ZHENG Shunyi, LI Rui, et al. A deep double-channel dense network for hyperspectral image classification[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(4): 46-62.
[2] WANG Yanjun, LI Shaochun, WANG Mengjie, et al. A simple deep learning network for classification of 3D mobile LiDAR point clouds[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3): 49-59.
[3] YAO Jiaqi, LI Guoyuan, CHEN Jiyi, et al. Cloud detection and centroid extraction of laser footprint image of GF-7 satellite laser altimetry[J]. Journal of Geodesy and Geoinformation Science, 2021, 4(3): 1-12.
[4] 吴俊斌, 吴晟, 吴兴蛟. 矢量填充和插值算法的图像放大[J]. 计算机与数字工程, 2016, 44(6): 1146-1150, 1166. WU Junbin, WU Sheng, WU Xingjiao. Vector filling and interpolation algorithms of image magnification[J]. Computer & Digital Engineering, 2016, 44(6): 1146-1150, 1166.
[5] 龙四春, 周威, 文佳胜, 等. 雷达地形测绘DEM空洞插补方法研究[J]. 遥感信息, 2015, 30(4): 20-24, 36. LONG Sichun, ZHOU Wei, WEN Jiasheng, et al. SRTM DEM voids interpolation method based on Matlab[J]. Remote Sensing Information, 2015, 30(4): 20-24, 36.
[6] 边刚, 金绍华, 夏伟, 等. 线性插值的海洋磁力测量测线布设评价方法[J]. 测绘学报, 2014, 43(7): 675-680, 697. BIAN Gang, JIN Shaohua, XIA Wei, et al. Evaluating method of the survey line layout based on the linear interpolation in marine magnetic survey[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(7): 675-680, 697.
[7] 秦彦平, 张军, 多化琼, 等. 木材节子图像增强的小波变换与双三次插值融合方法[J]. 西北林学院学报, 2021, 36(5): 183-189. QIN Yanping, ZHANG Jun, DUO Huaqiong, et al. Fusion method of wavelet transform and bicubic interpolation for wood knot image enhancement[J]. Journal of Northwest Forestry University, 2021, 36(5): 183-189.
[8] 汪智易. 基于流形学习的图像超分辨率重建研究[D]. 西安: 西安电子科技大学, 2014. WANG Zhiyi. Research on image super-resolution reconstruction based on manifold learning[D]. Xi'an: Xidian University, 2014.
[9] 陶志强, 李海林, 张红兵. 基于新边缘指导插值的迭代反投影超分辨率重建算法[J]. 计算机工程, 2016, 42(6): 255-260. TAO Zhiqiang, LI Hailin, ZHANG Hongbing. Iterative back projection super resolution reconstruction algorithm based on new edge directed interpolation[J]. Computer Engineering, 2016, 42(6): 255-260.
[10] 苏锦程, 胡勇, 巩彩兰. 一种混合红外云图超分辨率重建算法[J]. 红外, 2018, 39(8): 34-39. SU Jincheng, HU Yong, GONG Cailan. A super-resolution reconstruction algorithm for hybrid infrared cloud images[J]. Infrared, 2018, 39(8): 34-39.
[11] 郭桐宇. 基于导向滤波与迭代反向投影的遥感影像超分辨率重建[J]. 测绘与空间地理信息, 2019, 42(1): 195-197, 205. GUO Tongyu. Super resolution reconstruction of remote sensing image based on improved iterative back projection algorithm[J]. Geomatics & Spatial Information Technology, 2019, 42(1): 195-197, 205.
[12] 杨琇卿. 基于Lanczos插值的迭代反投影图像超分辨率算法[D]. 济南: 山东大学, 2018. YANG Xiuqing. An iterative back-projection image super-resolution algorithm based on Lanczos interpolation[D]. Jinan: Shandong University, 2018.
[13] 许丽娜, 何鲁晓. 基于凸集投影的高分四号卫星影像超分辨率重建[J]. 测绘学报, 2017, 46(8): 1026-1033. DOI: 10.11947/j.AGCS.2017.20170070. XU Lina, HE Luxiao. GF-4 images super resolution reconstruction based on POCS[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(8): 1026-1033. DOI: 10.11947/j.AGCS.2017.20170070.
[14] 陈波. 自适应光学图像复原理论与算法研究[D]. 郑州:信息工程大学, 2008. CHEN Bo. The theory and algorithms of adaptive optics image restoration[D]. Zhengzhou:Information Engineering University, 2008.
[15] 武兴睿. 基于PSF重构和改进的最大后验估计的自适应光学图像复原算法[J]. 液晶与显示, 2019, 34(9): 921-927. WU Xingrui. Adaptive optical image restoration method based on PSF reconstruction and improved maximum posteriori estimation[J]. Chinese Journal of Liquid Crystals and Displays, 2019, 34(9): 921-927.
[16] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
[17] CAO Hangpu, MI Sicheng. Weighted SRGAN and reconstruction loss analysis for accurate image super resolution[C]//Proceedings of 2021 Modelling and Intelligent Computing Conference. Guilin, China: CAMMIC, 2021.
[18] WANG X, YU K, WU S, et al. Esrgan: enhanced super-resolution generative adversarial networks[C]//Proceedings of 2018 European Conference on Computer Vision (ECCV) Workshops. Munich, Germany: Springer: 2018.
[19] ZHANG H, GOODFELLOW I, METAXAS D, et al. Self-attention generative adversarial networks[C]//Proceedings of 2019 International Conference on Machine Learning. [S.l.]:PMLR, 2019: 7354-7363.
[20] SALIMANS T, KINGMA D P. Weight normalization: a simple reparameterization to accelerate training of deep neural networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems. Barcelona, Spain: ACM Press, 2016: 901-909.
[21] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on International Conference on Machine Learning. New York, NY, USA: ACM Press, 2015, 37: 448-456.
[22] JOLICOEUR-MARTINEAU A. The relativistic discriminator: a key element missing from standard GAN[EB/OL]. [2022-01-01]. https://arxiv.org/abs/1807.00734.
[23] 余志凡, 李昊, 李登实, 等. 基于Gram矩阵和卷积神经网络的风格迁移算法[J]. 江汉大学学报(自然科学版), 2020, 48(3): 62-68. YU Zhifan, LI Hao, LI Dengshi, et al. Style transfer algorithm based on gram matrix and convolutional[J]. Journal of Jianghan University (Natural Science Edition), 2020, 48(3): 62-68.
[24] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[EB/OL]. [2021-11-20]. https://arxiv.org/abs/1409.1556.
[25] DENG J, DONG W, SOCHER R, et al. ImageNet: a large-scale hierarchical image database[C]//Proceedings of 2009 IEEE Conference on Computer Vision and Pattern Recognition. Miami, USA:IEEE, 2009: 248-255.
[26] KINGMA D P, BA J. Adam: a method for stochastic optimization[EB/OL]. [2021-11-20]. https://arxiv.org/abs/1412.6980.
[27] SHI W, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, USA: IEEE, 2016: 1874-1883.
[28] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Honolulu, USA:IEEE, 2017: 136-144.
[29] LEI Sen, SHI Zhenwei. Hybrid-scale self-similarity exploitation for remote sensing image super-resolution[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-10.