图像匹配的物体空间关系推理表达

doi:10.11947/j.AGCS.2021.20190497

摘要/Abstract

摘要： 物体空间关系指的是物体在欧氏空间中的邻近关系，根据图像中包含物体的邻近关系解决图像匹配的问题。本文首先基于对比机制训练物体块特征提取网络，构建物体块深度特征，该特征可以有效匹配不同图像中的相同物体块；其次，基于已有的先验图像数据推理表达图像中物体的空间邻近关系，构建场景物体空间邻近图；进而基于该空间邻近图计算场景图像对的空间邻近度，完成图像空间关系匹配。试验表明不匹配图像间的空间邻近度一般为0，而匹配图像间的空间邻近度一般大于0，本文空间关系匹配涉及多个物体间的相互关系，具有更强的稳健性，其匹配效果明显优于对比试验中的其他方法，可以高效稳定地完成图像匹配任务。

关键词: 深度特征, 空间邻近图, 空间关系匹配, 物体块所属节点

Abstract: The spatial relationship in the paper means the spatial adjacency relation between the objects in word space, and the paper proposes a novel image matching method based on analyzing the spatial adjacency relation between the object contained in the image pair. In the method, the feature extraction network is first trained based on comparison mechanism, and the well-trained model could produce the deep features for the objects, which could effectively match the consistent objects across images. Then the priori images are employed to deduce the spatial adjacency information between different objects, which is further represented by the spatial adjacency graph. Finally, the spatial relation matching between the image pair is conducted by calculating the spatial adjacency score based on the spatial adjacency graph. The experimental results demonstrate that the spatial adjacency scores of the non-matched image pairs generally equal to 0, and those of the matched pairs are generally greater than 0. As several objects are involved, the proposed spatial relation matching method could achieve high robustness, it outperforms other methods in the comparison experiments, and it could effectively complete the image matching task with high efficiency.

Key words: deep feature, spatial adjacency graph, image matching, corresponding node of object

中图分类号:

P208

李钦, 游雄, 李科, 王玮琦. 图像匹配的物体空间关系推理表达[J]. 测绘学报, 2021, 50(1): 117-131.

LI Qin, YOU Xiong, LI Ke, WANG Weiqi. Spatial relation reasoning and representation for image matching[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(1): 117-131.

参考文献

[1] CUMMINS M, NEWMAN P. FAB-MAP:appearance-based place recognition and mapping using a learned visual vocabulary model[C]//Proceedings of the 27th International Conference on Machine Learning (ICML-10).Haifa, Israel:Omnipress, 2010.
[2] QIU Kaichang, WAN Wenhui, ZHAO Hongying, et al. Progress and applications of visual SLAM[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(2):38-49.
[3] 李向阳, 庄越挺, 潘云鹤. 基于内容的图像检索技术与系统[J]. 计算机研究与发展, 2001, 38(3):344-354. LI Xiangyang, ZHUANG Yueting, PAN Yunhe. The technique and systems of content-based image retrieval[J]. Journal of Computer Research and Development, 2001, 38(3):344-354.
[4] 杨珂, 李从敏, 周维勋, 等. 卷积神经网络多层特征联合的遥感图像检索[J]. 测绘科学, 2019, 44(7):9-15, 34. YANG Ke, LI Congmin, ZHOU Weixun, et al. Remote sensing image retrieval based on multi-layer feature integration of convolution neural networks[J]. Science of Surveying and Mapping, 2019, 44(7):9-15, 34.
[5] 袁一, 程亮, 宗雯雯, 等. 互联网众源照片的三维重建定位技术[J]. 测绘学报,2018, 47(5):631-643. DOI:10.11947/j.AGCS.2018.20170365. YUAN Yi, CHENG Liang, ZONG Wenwen, et al. Crowd-sourced pictures geo-localization method based on 3D reconstruction[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(5):631-643. DOI:10.11947/j.AGCS.2018.20170365.
[6] BVRKI M, CADENA C, GILITSCHENSKI I, et al. Appearance-based landmark selection for visual localization[J]. Journal of Field Robotics, 2019, 36(6):1041-1073.
[7] LI Qin, LI Ke, YOU Xiong, et al. Place recognition based on deep feature and adaptive weighting of similarity matrix[J]. Neurocomputing, 2016, 199:114-127.
[8] LOWE D G. Object recognition from local scale-invariant features[C]//Proceedings of the 7th IEEE International Conference on Computer Vision.Kerkyra, Greece:IEEE,1999.
[9] BAY H, TUYTELAARS T, VAN GOOL L. SURF:speeded up robust features[C]//Proceedings of the 9th European Conference on Computer Vision. Graz, Austria:Springer,2006.
[10] BAY H, ESS A, TUYTELAARS T, et al. Speeded-up robust features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359.
[11] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB:An efficient alternative to SIFT or SURF[C]//Proceedings of International Conference on Computer Vision. Barcelona:IEEE, 2012.
[12] SIVIC J, ZISSERMAN A. Video Google:A text retrieval approach to object matching in videos[C]//Proceedings of the 9th IEEE International Conference on Computer Vision. Nice, France:IEEE, 2003.
[13] CAO Song, SNAVELY N. Learning to match images in large-scale collections[C]//Proceedings of 2012 European Conference on Computer Vision. Florence, Italy:Springer, 2012.
[14] JÉGOU H, DOUZE M, SCHMID C, et al. Aggregating local descriptors into a compact image representation[C]//Proceedings of 2010 IEEE Conference on Computer Vision and Pattern Recognition. San Francisco, CA:IEEE, 2010.
[15] JAAKKOLA T S, HAUSSLER D. Exploiting generative models in discriminative classifiers[C]//Proceedings of the 11th International Conference on Neural Information Processing Systems. Cambridge, MA:MIT Press, 1998:487-493.
[16] PERRONNIN F, DANCE C. Fisher kernels on visual vocabularies for image categorization[C]//Proceedings of 2007 IEEE Conference on Computer Vision and Pattern Recognition. Minneapolis, MN:IEEE, 2007:1-8.
[17] ARANDJELOVIĆ R, GRONAT P, TORII A, et al. NetVLAD:CNN architecture for weakly supervised place recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(6):1437-1451.
[18] 李钦, 游雄, 李科, 等. 图像深度层次特征提取算法[J]. 模式识别与人工智能, 2017, 30(2):127-136. LI Qin, YOU Xiong, LI Ke, et al. Deep hierarchical feature extraction algorithm[J]. Pattern Recognition and Artificial Intelligence, 2017, 30(2):127-136.
[19] KRIZHEVSKYA, SUTSKEVERI, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. London:MIT Press,2012:1097-1105.
[20] SVNDERHAUF N, SHIRAZI S, DAYOUB F, et al. On the performance of convNet features for place recognition[C]//Proceedings of 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). Hamburg, Germany:IEEE, 2015:4297-4304.
[21] MELEKHOV I, KANNALA J, RAHTU E. Siamese network features for image matching[C]//Proceedings of the 23rd International Conference on Pattern Recognition (ICPR). Cancun, Mexico:IEEE, 2016:378-383.
[22] CHOPRA S, HADSELL R, LECUN Y. Learning a similarity metric discriminatively, with application to face verification[C]//Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05). San Diego, CA:IEEE, 2005:539-546.
[23] FINMAN R, PAULL L, LEONARD J. Toward object-based place recognition in dense RGB-D maps[C]//Proceedings of ICRA Workshop Visual Place Recognition in Changing Environments. Seattle, WA:[s.n.], 2015.
[24] PEPPERELL E, CORKE P I, MILFORD M. Routed roads:Probabilistic vision-based place recognition for changing conditions, split streets and varied viewpoints[J]. The International Journal of Robotics Research, 2016, 35(9):1057-1179.
[25] CASCIANELLI S, COSTANTE G, BELLOCCHIO E, et al. Robust visual semi-semantic loop closure detection by a covisibility graph and CNN features[J]. Robotics and Autonomous Systems, 2017, 92:53-65.
[26] ZITNICK C L, DOLLÁR P. Edge boxes:locating object proposals from edges[C]//Proceedings of the 13th European Conference on Computer Vision. Zurich, Switzerland:Springer, 2014.
[27] DOLLÁR P, ZITNICK C L. Structured forests for fast edge detection[C]//Proceedings of 2013 IEEE International Conference on Computer Vision. Sydney, NSW, Australia:IEEE, 2014.
[28] BROWN M, HUA Gang, WINDER S. Discriminative learning of local image descriptors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(1):43-57.
[29] GEIGER A, LENZ P, URTASUN R. Are we ready for autonomous driving? thekitti vision benchmark suite[C]//Proceedings of 2012 IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2012:3354-3361.
[30] HUANG Xinyu, WANG Peng, CHENG Xinjing, et al. The apolloscape open dataset for autonomous driving and its application[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(10):2702-2719.
[31] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas, NV:IEEE, 2016:2818-282.
[32] CHRISTIAN S, LIU Wei, JIA Yangqing, et al. Going deeper with convolutions[C]//Proceedings of 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA:IEEE, 2015:1-9.