A priori guided method for improving the quality of Luojia01-1 NTL image with multiple degradation features

doi:10.11947/j.AGCS.2023.20210679

Abstract

Abstract: There are many complex degradation phenomena in nighttime light remote sensing image of Luojia01-1, such as cloud, glow, reduced resolution, and so on. The existing deep learning-based networks for image quality improvement often only aim at a certain type of degradation problems,and do not make full use of the priori information of the image, the interpretability of the training and learning process is poor, and the type of degradation removed is single. Therefore, aiming at the problem of image quality improvement with multiple complex degradation features, an interpretable a priori guided nighttime light remote sensing image quality improvement method with multiple degradation features is proposed. Firstly, the degradation process and performance are analyzed, and a comprehensive degradation model of glow, cloud noise, and spatial resolution degradation is derived. Taking the model as a priori guide, three kinds of data sets of cloud glow, and resolution are constructed. In terms of network structure, for three kinds of degradation, a residual dense convolution neural network including channel attention module and pixel attention module is designed, and the ratio sparse constraint loss function is used to further improve the clarity of the image. The experiment is carried out with the nighttime light remote sensing image of Luojia01-1. The results show that the method proposed in this paper can effectively remove the influence of cloud and fog and glow. After processed, the image light edge information is clearer, the spatial resolution is improved, and the image quality is improved obviously.

Key words: nighttime light remote sensing, image quality improvement, deep learning, remove the clouds, degradation model, super-resolution reconstruction

CLC Number:

P237

BU Lijing, WU Wenyu, ZHANG Zhengpeng, YANG Yin. A priori guided method for improving the quality of Luojia01-1 NTL image with multiple degradation features[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(2): 244-259.

References

[1] 李德仁, 李熙. 论夜光遥感数据挖掘[J]. 测绘学报, 2015, 44(6):591-601.DOI:10.11947/j.AGCS.2015.20150149. LI Deren, LI Xi. An overview on data mining of nighttime light remote sensing[J]. Acta Geodaetica et Cartographica Sinica, 2015, 44(6):591-601.DOI:10.11947/j.AGCS.2015.20150149.
[2] LI Xi, LI Deren, XU Huimin, et al. Intercalibration between DMSP/OLS and VIIRS night-time light images to evaluate city light dynamics of Syria's major human settlement during Syrian Civil War[J]. International Journal of Remote Sensing, 2017, 38(21):5934-5951.
[3] BENNETT M M, SMITH L C. Advances in using multitemporal night-time lights satellite imagery to detect, estimate, and monitor socioeconomic dynamics[J]. Remote Sensing of Environment, 2017, 192:176-197.
[4] JIANG Wei, HE Guojin, LONG Tengfei, et al. Potentiality of using Luojia 1-01 nighttime light imagery to investigate artificial light pollution[J]. Sensors (Basel, Switzerland), 2018, 18(9):2900.
[5] 张洋. 微纳卫星低照度成像像质提升研究[D]. 长春:中国科学院长春光学精密机械与物理研究所, 2017. ZHANG Yang. The improvement of micro-satellite's low-light remote sensing image quality[D]. Changchun:Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, 2017.
[6] 董慧颖, 王欣威, 沈凤龙, 等. 基于多散射模型的光源退化图像复原方法及应用[J]. 沈阳理工大学学报, 2010, 29(3):9-12, 34. DONG Huiying, WANG Xinwei, SHEN Fenglong, et al. A method of restoring light source degradation image based on multiple scattering model and its application[J]. Journal of Shenyang Ligong University, 2010, 29(3):9-12, 34.
[7] ABRAHAMS A, ORAM C, LOZANO-GRACIA N. Deblurring DMSP nighttime lights:a new method using Gaussian filters and frequencies of illumination[J]. Remote Sensing of Environment, 2018, 210:242-258.
[8] KARAM G S. Blurred image restoration with unknown point spread function[J]. Al-Mustansiriyah Journal of Science, 2018, 29(1):189-194.
[9] CHAVEZ P. Radiometric calibration of Landsat thematic and mapper multispectral images[J].Photogrammetric Engineering and Remote Sensing,1989,55:1285-1294.
[10] HE K, SUN J, TANG X. Single image haze removal using dark channel prior[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(12):2341-2353.
[11] PARK S C, PARK M K, KANG M G. Super-resolution image reconstruction:a technical overview[J]. IEEE Signal Processing Magazine, 2003, 20(3):21-36.
[12] AL ISMAEIL K, AOUADA D, OTTERSTEN B, et al. Multi-frame super-resolution by enhanced shift and add[C]//Proceedings of the 8th International Symposium on Image and Signal Processing and Analysis (ISPA). Trieste, Italy:IEEE, 2014:171-176.
[13] NASROLLAHI K, MOESLUND T B. Super-resolution:a comprehensive survey[J]. Machine Vision and Applications, 2014, 25(6):1423-1468.
[14] FARSIU S, ROBINSON M D, ELAD M, et al. Fast and robust multiframe super resolution[J]. IEEE Transactions on Image Processing:a Publication of the IEEE Signal Processing Society, 2004, 13(10):1327-1344.
[15] 苏衡, 周杰, 张志浩. 超分辨率图像重建方法综述[J]. 自动化学报, 2013, 39(8):1202-1213. SU Heng, ZHOU Jie, ZHANG Zhihao. Survey of super-resolution image reconstruction methods[J]. Acta Automatica Sinica, 2013, 39(8):1202-1213.
[16] FREEMAN W T, JONES T R, PASZTOR E C. Example-based super-resolution[J]. IEEE Computer Graphics and Applications, 2002, 22(2):56-65.
[17] CHANG Hong, YEUNG D Y, XIONG Yimin. Super-resolution through neighbor embedding[C]//Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.Washington, DC, USA:IEEE, 2004:I.
[18] YANG Jianchao, WRIGHT J, HUANG T S, et al. Image super-resolution via sparse representation[J]. IEEE Transactions on Image Processing, 2010, 19(11):2861-2873.
[19] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553):436-444.
[20] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018(1):1-15.
[21] CAI Bolun, XU Xiangmin, JIA Kui, et al. DehazeNet:an end-to-end system for single image haze removal[J]. IEEE Transactions on Image Processing, 2016, 25(11):5187-5198.
[22] ZHANG He, PATEL V M. Densely connected pyramid dehazing network[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT, USA:IEEE, 2018:3194-3203.
[23] LI Boyi, PENG Xiulian, WANG Zhangyang, et al. AOD-net:all-in-one dehazing network[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV).Venice, Italy:IEEE, 2017:4780-4788.
[24] REN Wenqi, MA Lin, ZHANG Jiawei, et al. Gated fusion network for single image dehazing[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT, USA:IEEE, 2018:3253-3261.
[25] BHARATH RAJ N, VENKETESWARAN N. Single image haze removal using a generative adversarial network[C]//Proceedings of 2020 International Conference on Wireless Communications Signal Processing and Networking (WiSPNET). Chennai, India:IEEE, 2020:37-42.
[26] CHEN Dongdong, HE Mingming, FAN Qingnan, et al. Gated context aggregation network for image dehazing and deraining[C]//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision (WACV). Waikoloa, HI, USA:IEEE, 2019:1375-1383.
[27] LIAO Yinghong, SU Zhuo, LIANG Xiangguo, et al. HDP-net:haze density prediction network for nighttime dehazing[C]//Proceedings of 2018 PCM Advances in Multimedia Information. Cham, Germany:Springer International Publishing, 2018:469-480.
[28] DONG Chao, LOY C C, HE Kaiming, et al. Learning a deep convolutional network for image super-resolution[C]//Proceedings of 2014 Computer Vision. Cham, Germany:Springer International Publishing, 2014:184-199.
[29] DONG Chao, LOY C C, TANG Xiaoou. Accelerating the super-resolution convolutional neural network[C]//Proceedings of Computer Vision-ECCV 2016. Cham, Germany:Springer International Publishing, 2016:391-407.
[30] SHI Wenzhe, CABALLERO J, HUSZÁR F, et al. Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Las Vegas, NV, USA:IEEE, 2016:1874-1883.
[31] MAO Xiaojiao, SHEN Chunhua, YANG Yubin. Image restoration using convolutional autoencoders with symmetric skip connections[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).[S.l.]:IEEE,2016.
[32] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV, USA:IEEE, 2016:1646-1654.
[33] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).Honolulu, HI, USA:IEEE, 2017:1132-1140.
[34] ZHANG Yulun, TIAN Yapeng, KONG Yu, et al. Residual dense network for image super-resolution[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT, USA:IEEE, 2018:2472-2481.
[35] TAI Ying, YANG Jian, LIU Xiaoming. Image super-resolution via deep recursive residual network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:2790-2798.
[36] ZHANG Yulun, TIAN Yapeng, KONG Yu, et al. Residual dense network for image super-resolution[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Salt Lake City, UT, USA. IEEE, 2018:2472-2481.
[37] GU Jinjin, LU Hannan, ZUO Wangmeng, et al. Blind super-resolution with iterative kernel correction[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Long Beach, CA, USA:IEEE, 2020:1604-1613.
[38] LEDIG C, THEIS L, HUSZáR F, et al. Photo-realistic single image super-resolution using a generative adversarial network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:105-114.
[39] WANG Xintao, YU Ke, WU Shixiang, et al. ESRGAN:enhanced super-resolution generative adversarial networks[M]//Lecture Notes in Computer Science. Cham:Springer International Publishing, 2019:63-79.
[40] 冈萨雷斯,理查德·伍兹.数字图像处理(MATLAB版)[M].北京:电子工业出版社,2005:103-104. RAFAEL C. GONZALEZ, RICHARD E. Woods. Digial image processing using MATLAB[M].Beijing:Publishing House of Electronics Industry, 2005:103-104.
[41] AUBÉ M. Physical behaviour of anthropogenic light propagation into the nocturnal environment[J]. Philosophical Transactions of the Royal Society of London Series B, Biological Sciences, 2015, 370(1667):194-199.
[42] KYBA C C, RUHTZ T, FISCHER J, et al. Cloud coverage acts as an amplifier for ecological light pollution in urban ecosystems[J]. PLoS One, 2011, 6(3):e17307.
[43] SANCHEZ DE MIGUEL A, KYBA C C M, ZAMORANO J, et al. The nature of the diffuse light near cities detected in nighttime satellite imagery[J]. Scientific Reports, 2020, 10(1):7829.
[44] LI Xuecao, ZHOU Yuyu. Urban mapping using DMSP/OLS stable night-time light:a review[J]. International Journal of Remote Sensing, 2017, 38(21):6030-6046.
[45] ZHANG J, REID J S, MILLER S D, et al. Strategy for studying nocturnal aerosol optical depth using artificial lights[J]. International Journal of Remote Sensing, 2008, 29(16):4599-4613.
[46] MILLER S D, TURNER R E. A dynamic lunar spectral irradiance data set for NPOESS/VIIRS day/night band nighttime environmental applications[J]. IEEE Transactions on Geoscience and Remote Sensing, 2009, 47(7):2316-2329.
[47] HE Kaiming, SUN Jian, TANG Xiaoou. Single image haze removal using dark channel prior[C]//Proceedings of 2010 IEEE Transactions on Pattern Analysis and Machine Intelligence.[S.l.]:IEEE, 2010:2341-2353.
[48] FATTAL R. Single image dehazing[J]. ACM Transactions on Graphics, 2008, 27(3):1-9.
[49] YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks?[C]//Proceedings of the 27th International Conference on Neural Information Processing Systems-Volume 2. Montreal, Canada. ACM Press, 2014:3320-3328.
[50] ERHAN D, BENGIO Y, COURVILLE A, et al. Why does unsupervised pre-training help deep learning[J]. Journal of Machine Learning Research, 2010, 11(3):625-660.
[51] ABRAHAMS A, ORAM C, LOZANO-GRACIA N. Deblurring DMSP nighttime lights:a new method using Gaussian filters and frequencies of illumination[J]. Remote Sensing of Environment, 2018, 210:242-258.
[52] 李欣. 基于深度学习的单幅遥感图像超分辨重建[D]. 北京:中国科学院大学, 2018. LI Xin. Super-resolution reconstruction of single remote sensing image based on deep learning[D].Beijing:Chinese Academy of Sciences,2018.
[53] LIU Shuying, DENG Weihong. Very deep convolutional neural network based image classification using small training sample size[C]//Proceedings of the 3rd IAPR Asian Conference on Pattern Recognition (ACPR).Kuala Lumpur, Malaysia:IEEE, 2016:730-734.
[54] BUŠTA M, NEUMANN L, MATAS J. Deep TextSpotter:an end-to-end trainable scene text localization and recognition framework[C]//Proceedings of 2017 IEEE International Conference on Computer Vision (ICCV).Venice, Italy:IEEE, 2017:2223-2231.
[55] QIN Xu, WANG Zhilin, BAI Yuanchao, et al. FFA-net:feature fusion attention network for single image dehazing[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2020, 34(7):11908-11915.
[56] 卜欣彤. 基于PSF优化估计的图像盲复原方法研究[D]. 阜新:辽宁工程技术大学, 2017. BU Xintong. Research on image blind restoration based on PSF optimization estimation[D]. Fuxin:Liaoning Technical University, 2017.
[57] 王正明, 朱炬波,谢美华,等. SAR图像提高分辨率技术[M]. 北京:科学出版社, 2006:23-24. WANG Zhengming, ZHU Jubo,XIE Meihua,等. SAR image resolution improvement technology[M]. Beijing:Science Press, 2006:23-24.