基于深度卷积特征的影像关系表创建方法

doi:10.11947/j.AGCS.2018.20180040

测绘学报 ›› 2018, Vol. 47 ›› Issue (6): 882-891.doi: 10.11947/j.AGCS.2018.20180040

• 数字摄影测量与深度学习方法 • 上一篇下一篇

基于深度卷积特征的影像关系表创建方法

万杰^1,2, Alper YILMAZ²

1. 北京大学地球与空间科学学院遥感与地理信息系统研究所, 空间信息集成与3S工程应用北京市重点实验室, 北京 100871;
2. 俄亥俄州立大学土木与环境工程系, 美国俄亥俄 43210

收稿日期:2018-01-23 修回日期:2018-04-17 出版日期:2018-06-20 发布日期:2018-06-21
通讯作者: Alper YILMAZ E-mail:alpery74@gmail.com
作者简介:万杰(1990-),男,博士生,研究方向为摄影测量学和计算机视觉。
基金资助:
国家自然科学基金（41571432）

Machine Vision Special Issue: Building Match Graph Using Deep Convolution Feature for Structure from Motion

WAN Jie^1,2, Alper YILMAZ²

1. The Institute of Remote Sensing and Geographic Information System, the School of Earth and Space Sciences, Peking University, Beijing Key Laboratory of Spatial Information Integration and 3S Engineering Application, Beijing 100871, China;
2. Department of Civil, Environment and Geodetic Engineering, Ohio State University, Ohio 43210, USA

Received:2018-01-23 Revised:2018-04-17 Online:2018-06-20 Published:2018-06-21
Supported by:
The National Natural Science Foundation of China (No.41571432)

摘要/Abstract

摘要： 在从运动恢复结构（structure from motion，SfM）的过程中，无序影像间的匹配非常耗时，一方面受制于特征匹配本身，另一方面受制于大量的图像间匹配，其计算复杂度为O（n²）。为减少匹配次数，本文提出基于深度卷积特征（deep convolution feature，DCF）的影像关系表创建方法。首先利用在ImageNet上训练好的VGG-16卷积神经网络提取影像的卷积层特征图，然后对特征图进行和池化操作，最后将该向量归一化，作为图像的特征。通过向量点乘，计算数据集中的每张影像和其余所有影像的相似度，选取相似度最大的10张影像作为影像的潜在匹配像对，并由此构建影像关系表。结果表明，本文提出的DCF能够有效的创建影像关系表，找出潜在匹配像对。在Urban和South Building数据集上，基于DCF创建的关系表匹配的SfM重建的结果和穷举匹配的重建结果基本一致，但匹配次数分别减少97.4%和92.1%。同时基于DCF创建的关系表优于主流ORB-SLAM2系统中的DBoW3创建的关系表。

关键词: 深度卷积特征, 影像关系表, 从运动恢复结构, 迁移学习

Abstract: Image matching in an unordered image dataset is quite time-consuming for structure from motion (SfM) due to image matching by comparing features and large number of matches between all image pairs. To reduce matching times, deep convolution feature (DCF) is proposed to create image match graph in this paper. Firstly, the convolutional feature map of an image is extracted using the VGG-16 convolutional neural network trained on ImageNet. Then, the sum pooling is used to process the feature map. Finally, the vector is normalized and used to represent the image. The similarities between an image and all other images is calculated by calculating the distances between these feature vectors. Thus, the match graph is constructed by selecting the top 10 images with highest similarities. The experiment results showed that the proposed DCF can create the match graph effectively, find the potential image pairs. On the Urban and South Building datasets, the results of the SfM reconstruction based on the match graph created by the proposed DCF are almost the same as those of the exhaustive matching, but the number of matches are reduced by 97.4% and 92.1%, respectively. At the same time, the match graph created by the proposed DCF is obviously better than the match graph crated by the DBoW3 in the most advanced SLAM system.

Key words: deep convolution feature, match graph, structure from motion, transfer learning

中图分类号:

P237

万杰, Alper YILMAZ. 基于深度卷积特征的影像关系表创建方法[J]. 测绘学报, 2018, 47(6): 882-891.

WAN Jie, Alper YILMAZ. Machine Vision Special Issue: Building Match Graph Using Deep Convolution Feature for Structure from Motion[J]. Acta Geodaetica et Cartographica Sinica, 2018, 47(6): 882-891.

参考文献

[1] LOWE D G. Object Recognition from Local Scale-invariant Features[C]//Proceedings of the Seventh IEEE International Conference on Computer Vision. Kerkyra, Greece:IEEE, 1999.
[2] BAY H, ESS A, TUYTELAARS T, et al, Speeded-up Robust Features (SURF)[J]. Computer Vision and Image Understanding, 2008, 110(3):346-359.
[3] RUBLEE E, RABAUD V, KONOLIGE K, et al. ORB:An Efficient Alternative to SIFT or SURF[C]//IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011.
[4] LEUTENEGGER S, CHLI M, SIEGWART R Y. BRISK:Binary Robust Invariant Scalable Keypoints[C]//IEEE International Conference on Computer Vision. Barcelona, Spain:IEEE, 2011.
[5] CRANDALL D, OWENS A, SNAVELY N, et al. Discrete-continuous Optimization for Large-scale Structure from Motion[C]//IEEE Conference on Computer Vision and Pattern Recognition. Providence, RI:IEEE, 2011.
[6] WU Changchang. SiftGPU:A GPU Implementation of Scale Invariant Feature Transform (SIFT).(2007). URL http://cs.unc.edu/~ccwu/siftgpu.
[7] SNAVELY N, SEITZ S M, SZELISKI R. Modeling the World from Internet Photo Collections[J]. International Journal of Computer Vision, 2008, 80(2):189-210.
[8] Wu Changchang. VisualSFM:A Visual Structure from Motion System[EB/OL].[2017-12-12]. http://www.cs.washington.edu/homes/ccwu/vsfm.
[9] FUHRMANN S, LANGGUTH F, MOEHRLE N, et al. MVE:An Image-based Reconstruction Environment[J]. Computers & Graphics, 2015, 53:44-53.
[10] LECUN Y, BENGIO Y, HINTON G. Deep Learning[J]. Nature, 2015, 521(7553):436-444.
[11] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Imagenet Classification with Deep Convolutional Neural Networks[J]. Communications of the ACM, 2017, 60(6):84-90.
[12] BENGIO Y. Deep Learning of Representations for Unsupervised and Transfer Learning[C]//Proceedings of 2011 International Conference on Unsupervised and Transfer Learning workshop. Washington, USA:JMLR, 2012.
[13] AGARWAL S, FURUKAWA Y, SNAVELY N, et al. Building Rome in a Day[J]. Communications of the ACM, 2011, 54(10):105-112.
[14] HAVLENA M, SCHINDLER K. VocMatch:Efficient Multiview Correspondence for Structure from Motion[M]//FLEET D, PAJDLA T, SCHIELE B, et al. Computer Vision-ECCV 2014. Cham:Springer, 2014.
[15] ZHAN Zongqian, WANG Xin, WEI Minglu. Fast Method of Constructing Image Correlations to Build a Free Network Based on Image Multivocabulary Trees[J]. Journal of Electronic Imaging, 2015, 24(3):033029.
[16] SCHÖNBERGER J L, FRAHM J M. Structure-from-Motion Revisited[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas, NV:IEEE, 2016.
[17] SCHÖNBERGER J L, PRICE T, SATTLER T, et al. A Vote-and-verify Strategy for Fast Spatial Verification in Image Retrieval[M]//LAI S H, LEPETIT V, NISHINO K, et al. Computer Vision-ACCV 2016. Cham:Springer, 2016.
[18] ANGELI A, FILLIAT D, DONCIEUX S, et al. Fast and Incremental Method for Loop-closure Detection Using Bags of Visual Words[J]. IEEE Transactions on Robotics, 2008, 24(5):1027-1037.
[19] CUMMINS M,NEWMAN P. Appearance-only SLAM at Large Scale with FAB-MAP 2.0[J]. The International Journal of Robotics Research, 2011, 30(9):1100-1123.
[20] GALVEZ-LÓPE, D, TARDOS J D. Bags of Binary Words for Fast Place Recognition in Image Sequences[J]. IEEE Transactions on Robotics, 2012, 28(5):1188-1197.
[21] MUR-ARTAL R, TARDÓS J D. ORB-SLAM2:An Open-source SLAM System for Monocular, Stereo, and RGB-D Cameras[J]. IEEE Transactions on Robotics, 2017, 33(5):1255-1262.
[22] GAO Xiang, ZHANG Tao. Unsupervised Learning to Detect Loops Using Deep Neural Networks for Visual Slam System[J]. Autonomous Robots, 2017, 41(1):1-18.
[23] ZHANG Xiwu, SU Yan, ZHU Xinhua. Loop Closure Detection for Visual SLAM Systems Using Convolutional Neural Network[C]//The 23rd International Conference on Automation and Computing. Huddersfield, UK:IEEE, 2017.
[24] BABENKO A, SLESAREV A, CHIGORIN A, et al. Neural Codes for Image Retrieval[M]//FLEET D, PAJDLA T, SCHIELE B, et al. Computer Vision-ECCV 2014. Cham:Springer, 2014.
[25] RAZAVIAN A S, AZIZPOUR H, SULLIVAN J, et al. CNN Features Off-the-shelf:An Astounding Baseline for Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops. Columbus, OH:IEEE, 2014.
[26] YANDE A B, LEMPITSKY V. Aggregating Local Deep Features for Image Retrieval[C]//Proceedings of the IEEE International Conference on Computer Vision. Santiago, Chile:IEEE, 2015.
[27] DESEILLIGNY M P, CLÉRY I. Apero, An Open Source Bundle Adjusment Software for Automatic Calibration and Orientation of Set of Images[C]//Proceedings of the ISPRS Symposium.[S.l.]:ISPRS, 2011:269-276.

基于深度卷积特征的影像关系表创建方法

Machine Vision Special Issue: Building Match Graph Using Deep Convolution Feature for Structure from Motion

PDF

可视化

摘要/Abstract

引用本文

使用本文

参考文献

相关文章 7

编辑推荐

Metrics

本文评价

[1]	施慧慧, 徐雁南, 滕文秀, 王妮. 高分辨率遥感影像深度迁移可变形卷积的场景分类法[J]. 测绘学报, 2021, 50(5): 652-663.
[2]	汤寓麟, 金绍华, 边刚, 张永厚, 李凡. 侧扫声呐识别沉船影像的迁移学习卷积神经网络法[J]. 测绘学报, 2021, 50(2): 260-269.
[3]	郑鑫, 潘斌, 张健. 可变形网络与迁移学习相结合的电力塔遥感影像目标检测法[J]. 测绘学报, 2020, 49(8): 1042-1050.
[4]	王舒洋, 慕晓冬, 贺浩, 杨东方, 马晨晖. 航拍图像跨数据域特征迁移道路提取方法[J]. 测绘学报, 2020, 49(5): 611-621.
[5]	赵传, 郭海涛, 卢俊, 余东行, 张保明. 基于深度残差网络的机载LiDAR点云分类[J]. 测绘学报, 2020, 49(2): 202-213.
[6]	胡根生, 周文利, 梁栋, 鲍文霞. 融合引导滤波和迁移学习的薄云图像中地物信息恢复算法[J]. 测绘学报, 2018, 47(3): 348-358.
[7]	吴田军骆剑承夏列钢杨海平沈占锋胡晓东. 迁移学习支持下的遥感影像对象级分类样本自动选择方法[J]. 测绘学报, 2014, 43(9): 908-916.