A performing analysis of unsupervised dense matching feature extraction networks

doi:10.11947/j.AGCS.2022.20200503

Abstract

Abstract: With the development of artificial intelligence, supervised dense matching method based on deep learning has achieved good performance in virtual, indoor and driving data sets. In view of the difficulty in obtaining aerial image dense matching tag data, we use unsupervised dense matching framework for reference, and test the matching accuracy on aerial image data set and referential close range data set respectively, and realize the qualitative analysis of the relationship between network structure module and precision, which provides a further exploration of the practical application of deep learning in the field of surveying and mapping, and has important reference value. Under the same loss function condition, DispNetS, DispNetC, iResNet, GCNet, PSMNetB and PSMNetS network structures are used to test. Through analysis, the following conclusions are obtained:① Among the tested network structures, PSMNetS has the highest accuracy in aerial image data set and close range data set, and has the potential of practical application; ② The network with better performance in the supervised method may not have better performance in the unsupervised method. Its accuracy depends not only on the matching ability of the network itself, but also on the compatibility between the network and the loss function; ③ The twin network module, related information fusion module, pyramid pooling module and stacked hourglass module have good compatibility with unsupervised loss function, which can improve the network accuracy, while iResNet's image reconstruction iterative refinement module and reconstruction loss function repeat constraints, which will produce a "negative optimization" effect.

Key words: dense matching, deep learning, unsupervised learning, feature extraction, aerial imaging

CLC Number:

P237

JIN Fei, GUAN Kai, LIU Zhi, HAN Jiarong, RUI Jie, LI Qinggao. A performing analysis of unsupervised dense matching feature extraction networks[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(3): 426-436.

References

[1] JANAI J, GVNEY F, BEHL A, et al. Computer vision for autonomous vehicles:problems, datasets and state of the art[J]. Foundations and Trends^® in Computer Graphics and Vision, 2020, 12(1-3):1-308.
[2] 纪艳华. 顾及边缘信息的航摄影像半全局密集匹配算法研究[D]. 武汉:武汉大学, 2020. JI Yanhua. Research on semi-global dense matching algorithm for aerial images using edge information[D]. Wuhan:Wuhan University, 2020.
[3] GONG Jianya, JI Shunping. Photogrammetry and deep learning[J]. Journal of Geodesy and Geoinformation Science, 2018(1):1-15.
[4] SCHARSTEIN D, SZELISKI R, ZABIH R. A taxonomy and evaluation of dense two-frame stereo correspondence algorithms[C]//Proceedings of IEEE Workshop on Stereo and Multi-Baseline Vision. Kauai, HI, USA:IEEE, 2001:131-140.
[5] ZABIH R, WOODFILL J. Non-parametric local transforms for computing visual correspondence[J]. Lecture Notes in Computer Science, 1994, 801(1):151-158.
[6] BIRCHFIELD S, TOMASI C. Depth discontinuities by pixel-to-pixel stereo[C]//Proceedings of 6th International Conference on Computer Vision. Bombay, India:IEEE, 1998:1073-1080.
[7] BOYKOV Y, VEKSLER O, ZABIH R. Fast approximate energy minimization via graph cuts[C]//Proceedings of the 7th IEEE International Conference on Computer Vision. Kerkyra, Greece:IEEE, 1999:377-384.
[8] HIRSCHMVLLER H. Stereo processing by semiglobal matching and mutual information[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(2):328-341.
[9] ŽBONTAR J, LECUN Y. Computing the stereo matching cost with a convolutional neural network[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition.Boston, MA, USA:IEEE, 2015:1592-1599.
[10] MAYER N, ILG E, HÄUSSER P, et al. A large dataset to train convolutional networks for disparity, optical flow, and scene flow estimation[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV, USA:IEEE, 2016:4040-4048.
[11] DOSOVITSKIY A, FISCHER P, ILG E, et al. FlowNet:learning optical flow with convolutional networks[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015:2758-2766.
[12] LIANG Z, FENG Y, GUO Y, et al. Learning deep correspondence through prior and posterior feature constancy[C]//Proceedings of IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, USA:IEEE,2017:2403-2411.
[13] KENDALL A, MARTIROSYAN H, DASGUPTA S, et al. End-to-end learning of geometry and context for deep stereo regression[C]//Proceedings of the IEEE International Conference on Computer Vision. Venice, Italy:IEEE, 2017:66-75.
[14] NIBALI A, HE Z, MORGAN S, et al. Numerical coordinate regression with convolutional neural networks[EB/OL].[2020-08-25].https://arxiv.org/abs/1801.07372.
[15] CHANG Jiaren, CHEN Yongsheng. Pyramid stereo matching network[C]//Proceedings of 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Salt Lake City, UT, USA:IEEE, 2018:5410-5418.
[16] ZHAO Hengshuang, SHI Jianping, QI Xiaojuan, et al. Pyramid scene parsing network[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Honolulu, HI, USA:IEEE, 2017:6230-6239.
[17] NEWELL A, YANG Kaiyu, DENG Jia. Stacked hourglass networks for human pose estimation[C]//Proceedings of 2016 Computer Vision. Amsterdam,The Netherlands:ECCV, 2016:483-499.
[18] XIE Junyuan, GIRSHICK R, FARHADI A. Deep3D:fully automatic 2D-to-3D video conversion with deep convolutional neural networks[C]//Proceedings of 2016 Computer Vision.Amsterdam,The Netherlands:ECCV, 2016:842-857.
[19] GARG R, VIJAY KUMAR B G, CARNEIRO G, et al. Unsupervised CNN for single view depth estimation:geometry to the rescue[C]//Proceedings of 2016 Computer Vision.Amsterdam,The Netherlands:ECCV 2016:740-756.
[20] GODARD C, AODHA O M, BROSTOW G J. Unsupervised monocular depth estimation with left-right consistency[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition. Honolulu, HI, USA:IEEE, 2017:6602-6611.
[21] JADERBERG M, SIMONYAN K, ZISSERMAN A, et al. Spatial transformer networks[EB/OL] [2020-08-25]. http://cn.arxiv.org/abs/1506.02025.
[22] ZHONG Y, DAI Y, LI H. Self-supervised learning for stereo matching with self-improving ability[EB/OL] [2020-08-25]. http://cn.arxiv.org/abs/1709.00930.
[23] 王玉锋, 王宏伟, 吴晨, 等. 基于共同视域的自监督立体匹配算法[J]. 光学学报, 2019, 39(2):287-296. WANG Yufeng, WANG Hongwei, WU Chen, et al. Self-supervised stereo matching algorithm based on common view[J]. Acta Optica Sinica, 2019, 39(2):287-296.
[24] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Diego, CA, USA:IEEE, 2005:539-546.
[25] WANG Zhou, BOVIK A C, SHEIKH H R, et al. Image quality assessment:from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4):600-612.
[26] 刘瑾, 季顺平. 基于深度学习的航空遥感影像密集匹配[J]. 测绘学报, 2019, 48(9):1141-1150.DOI:10.11947/j.AGCS.2019.20180247. LIU Jin, JI Shunping. Deep learning based dense matching for aerial remote sensing images[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(9):1141-1150.DOI:10.11947/j.AGCS.2019.20180247.
[27] LIU Jin, JI Shunping. A novel recurrent encoder-decoder structure for large-scale multi-view stereo reconstruction from an open aerial dataset[C]//Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Seattle, WA, USA. IEEE, 2020:6049-6058.
[28] GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? The KITTI vision benchmark suite[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI, USA:IEEE, 2012:3354-3361.
[29] MENZE M, GEIGER A. Object scene flow for autonomous vehicles[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition. Boston, MA, USA:IEEE, 2015:3061-3070.
[30] PASZKE A, GROSS S, CHINTALA S, et al. Automatic differentiation in pytorch[EB/OL].[2020-08-25]. https://openreview.net/pdf?id=BJJsrmfCZ.
[31] KINGMA D, BA J. Adam:a method for stochastic optimization[EB/OL].[2020-08-25]. http://arxiv.org/abs/1412.6980.