高分七号影像大场景DSM生成深度学习方法

doi:10.11947/j.AGCS.2023.20220567

摘要/Abstract

摘要： 立体匹配是利用卫星影像生成DSM的重要步骤,近年来基于深度学习的立体匹配方法有较好的性能,然而,由于模型预测的视差范围固定有限及缺少训练数据,深度学习很少直接用于大场景卫星影像的立体匹配。本文提出了一种分层动态匹配策略,根据上一层匹配结果来动态确定本层影像分块的区域,使左右核线影像块的视差相对较小,利于深度学习模型进行预测;本文提出了一套卫星影像立体匹配样本制作方案,利用人工编辑的DSM或LiDAR点云获取视差真值,构建了一个高分七号立体匹配数据集。使用该数据集和现有数据集训练Stereo-Net和DSM-Net并基于分层匹配策略,实现了结合深度学习技术的高分七号影像高质量DSM生成。3个城市的影像试验表明,本文方法匹配的视差图的平均视差绝对误差为1像素左右,错误视差像素比例不超过3.8%,生成的DSM质量优于传统方法。

关键词: 卫星影像, 立体匹配, DSM, 深度学习, 高分七号

Abstract: Stereo matching is an important step to generate DSM from satellite imageries. Recently, studies have shown that deep learning-based methods have better performance. However, due to the fixed and limited disparity range predicted by models and the lack of training data, deep learning is rarely directly applied to the stereo matching of satellite images in large scenes. In this paper, a hierarchical dynamic matching strategy is proposed to dynamically determine the region of image blocks of the current level according to the matching results of the previous level, so that the disparity between the left and right epipolar image blocks is relatively small, which is conducive to the prediction of deep learning models. Besides, a scheme for the production of samples is introduced, and a GF-7 dataset is constructed by using manually edited DSM or LiDAR point clouds to obtain ground truth disparity values. In the experiment, this dataset, together with an existing dataset, is used to train Stereo-Net and DSM-Net, and based on the hierarchical matching strategy, the generation of high-quality DSM from Gaofen-7 imagery combined with deep learning technology is achieved for the first time. Experiments in imageries from three cities show that the average endpoint error is about 1 pixel, and the fraction of erroneous pixels is less than 3.8%. The quality of the generated DSMs is better than that of the traditional method.

Key words: satellite imagery, stereo matching, DSM, deep learning, GF-7

中图分类号:

P236

何升, 张建凯, 陈凤, 李绅弘, 江万寿. 高分七号影像大场景DSM生成深度学习方法[J]. 测绘学报, 2023, 52(12): 2103-2114.

HE Sheng, ZHANG Jiankai, CHEN Feng, LI Shenhong, JIANG Wanshou. Deep learning method for large scene DSM generation of GF-7 imagery[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(12): 2103-2114.

参考文献

[1] 季顺平, 罗冲, 刘瑾. 基于深度学习的立体影像密集匹配方法综述[J]. 武汉大学学报(信息科学版), 2021, 46(2): 193-202. JI Shunping, LUO Chong, LIU Jin. A review of dense stereo image matching methods based on deep learning[J]. Geomatics and Information Science of Wuhan University, 2021, 46(2): 193-202.
[2] SCHARSTEIN D, SZELISKI R, ZABIH R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[J]. International Journal of Computer Vision, 2002, 47(1): 7-42.
[3] 岳庆兴, 高小明, 唐新明. 基于半全局优化的资源三号卫星影像DSM提取方法[J]. 武汉大学学报(信息科学版), 2016, 41(10): 1279-1285. YUE Qingxing, GAO Xiaoming, TANG Xinming. ZY-3 DSM generation method based on semi-global optimization[J]. Geomatics and Information Science of Wuhan University, 2016, 41(10): 1279-1285.
[4] HIRSCHMVLLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2): 328-341.
[5] GEHRKE S, MORIN K, DOWNEY M, et al. Semi-global matching: an alternative to LiDAR for DSM generation? [EB/OL].[2022-10-04]. https://www.isprs.org/proceedings/XXXVIII/part1/11/11_01_Paper_121.pdf.
[6] D'ANGELO P, REINARTZ P. Semiglobal matching results on the ISPRS stereo matching benchmark[EB/OL]. [2022-10-04]. https://pdfs.semanticscholar.org/63a1/166a69bb236bf0c07c4d0c421aa8ecad3e81.pdf.
[7] 张力, 张继贤. 基于多基线影像匹配的高分辨率遥感影像DEM的自动生成[J]. 武汉大学学报(信息科学版), 2008, 33(9): 943-946. ZHANG Li, ZHANG Jixian. Automatic DEM generation from high-resolution satellite imagery based on multiple-baseline image matching[J]. Geomatics and Information Science of Wuhan University, 2008, 33(9): 943-946.
[8] 刘瑾, 季顺平. 基于深度学习的航空遥感影像密集匹配[J]. 测绘学报, 2019, 48(9): 1141-1150. DOI: 10.11947/j.AGCS.2019.20180247. LIU Jin, JI Shunping. Deep learning based dense matching for aerial remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9): 1141-1150. DOI: 10.11947/j.AGCS.2019.20180247.
[9] POGGI M, TOSI F, BATSOS K, et al. On the synergies between machine learning and binocular stereo for depth estimation from images: a survey[EB/OL].[2022-10-04]. https://arxiv.org/abs/2004.08566.
[10] ZBONTAR J, LECUN Y. Computing the stereo matching cost with a convolutional neural network[EB/OL].[2022-10-04]. https://arxiv.org/pdf/1409.4326.pdf.
[11] SEKI A, POLLEFEYS M. SGM-nets: semi-global matching with neural networks[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017:6640-6649.
[12] SHAKED A, WOLF L. Improved stereo matching with constant highway networks and reflective confidence learning[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu: IEEE, 2017:6901-6910.
[13] MAYER N, ILG E, HAUSSER P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016:4040-4048.
[14] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. San Diego:IEEE, 2005: 539-546.
[15] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice: IEEE, 2017: 66-75.
[16] CHANG Jiaren, CHEN Yongsheng. Pyramid stereo matching network[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City: IEEE, 2018: 5410-5418.
[17] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1904-1916.
[18] GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence: IEEE, 2012: 3354-3361.
[19] MENZE M, GEIGER A. Object scene flow for autonomous vehicles[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston: IEEE, 2015: 3061-3070.
[20] SCHARSTEIN D, HIRSCHMVLLER H, KITAJIMA Y, et al. High-resolution stereo datasets with subpixel-accurate ground truth[M]//Lecture Notes in Computer Science. Cham: Springer International Publishing, 2014: 31-42.
[21] MAYER N, ILG E, FISCHER P, et al. What makes good synthetic training data for learning disparity and optical flow estimation?[J]. International Journal of Computer Vision, 2018, 126(9): 942-960.
[22] TAO Rongshu, XIANG Yuming, YOU Hongjian. An edge-sense bidirectional pyramid network for stereo matching of VHR remote sensing images[J]. Remote Sensing, 2020, 12(24): 4025.
[23] HE Sheng, LI Shenhong, JIANG San, et al. HMSM-Net: hierarchical multi-scale matching network for disparity estimation of high-resolution satellite stereo images[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2022, 188: 314-330.
[24] RAO Zhibo, HE Mingyi, ZHU Zhidong, et al. Bidirectional guided attention network for 3D semantic detection of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 59(7): 6138-6153.
[25] KHAMIS S, FANELLO S, RHEMANN C, et al. StereoNet: guided hierarchical refinement for real-time edge-aware depth prediction[C]//Proceedings of 2018 ECCV. New York: ACM Press, 2018: 596-613.
[26] HE Sheng, ZHOU Ruqin, LI Shenhong, et al. Disparity estimation of high-resolution remote sensing images with dual-scale matching network[J]. Remote Sensing, 2021, 13(24): 5050.
[27] HE Kaiming, ZHANG Xiangyu, REN Shaoqing, et al. Deep residual learning for image recognition[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas: IEEE, 2016: 770-780.
[28] WU T, VALLET B, PIERROT-DESEILLIGNY M, et al. A new stereo dense matching benchmark dataset for deep learning[J]. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences, 2021, XLⅢ-B2-2021: 405-412.
[29] BOSCH M, FOSTER K, CHRISTIE G, et al. Semantic stereo for incidental satellite images[C]//Proceedings of 2019 IEEE Winter Conference on Applications of Computer Vision.Waikoloa Village: IEEE, 2019: 1524-1532.
[30] PATIL S, COMANDUR B, PRAKASH T, et al. A new stereo benchmarking dataset for satellite images[EB/OL]. [2022-10-04]. https://arxiv.org/abs/1907.04404.pdf.
[31] LI Shenhong, HE Sheng, JIANG San, et al. WHU-stereo: a challenging benchmark for stereo matching of high-resolution satellite images[EB/OL]. [2022-10-04]. https://arxiv.org/abs/2206.02342.pdf
[32] 杨幸彬, 吕京国, 江珊, 等. 高分辨率遥感影像DSM的改进半全局匹配生成方法[J]. 测绘学报, 2018, 47(10): 1372-1384. DOI: 10.11947/j.AGCS.2018.20180091. YANG Xingbin, LÜ Jingguo, JIANG Shan, et al. Digital surface model generation for high resolution satellite stereo image based on modified semi-global matching[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(10): 1372-1384. DOI: 10.11947/j.AGCS.2018.20180091.
[33] 袁修孝, 曹金山. 高分辨率卫星遥感精确对地目标定位理论与方法[M]. 北京: 科学出版社, 2012: 182-183. YUAN Xiuxiao, CAO Jinshan. Theory and method of precise object positioning of high resolution satellite imagery[M]. Beijing: Science Press, 2012: 182-183.