Spatio-temporal information extraction method for dynamic targets in multi-perspective surveillance video

doi:10.11947/j.AGCS.2022.20200507

Abstract

Abstract: As an important research direction in the field of computer vision, the precise positioning and tracking of dynamic targets in surveillance video has become a research hotspot in the surveillance field in recent years.Traditional video dynamic target detection relies only on image feature data, ignoring accurate matching with geographic coordinate systems.Especially for the area covered by multiple cameras, the spatial resolution of the image formed after projection is different when the shooting angle is different.Therefore, it is difficult to satisfy intelligent monitoring in complex geographical scenes with all-round spatio-temporal information perception.In this paper, we propose a collaborative video surveillance image and geospatial data inter-mapping model construction method to obtain spatio-temporal information such as the outline and geographic location of dynamic targets.Firstly, the mutual mapping relationship between surveillance image information and geospatial data is established, and the surveillance images with different observation angles, scales and spatial resolutions are placed under the same coordinate system.And based on this, the edge information of the target is detected by fusing Canny operator and background subtraction, and then the actual position of the target in the scene is restored by using the center-of-mass offset algorithm.This enables continuous tracking of targets under multiple angles, improves the spatio-temporal understanding and analysis of geographic scenes, and enhances the precise positioning and tracking of dynamic targets.

Key words: multi-view, dynamic target detection, feature information extraction, geospatial mapping

CLC Number:

P208
TP391

LI Jingwen, WEI Jingshan, JIANG Jianwu, LU Yanling, LIU Lei, TANG Yifei, LI Xu. Spatio-temporal information extraction method for dynamic targets in multi-perspective surveillance video[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(3): 388-400.

References

[1] 张旭, 郝向阳, 李建胜, 等. 监控视频中动态目标与地理空间信息的融合与可视化方法[J]. 测绘学报, 2019, 48(11):1415-1423. DOI:10.11947/j.AGCS.2019.20180572. ZHANG Xu, HAO Xiangyang, LI Jiansheng, et al. Fusion and visualization method of dynamic targets in surveillance video with geospatial information[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(11):1415-1423. DOI:10.11947/j.AGCS.2019.20180572.
[2] 管皓, 薛向阳, 安志勇. 深度学习在视频目标跟踪中的应用进展与展望[J]. 自动化学报, 2016, 42(6):834-847. GUAN Hao, XUE Xiangyang, AN Zhiyong. Advances on application of deep learning for video object tracking[J]. Acta Automatica Sinica, 2016, 42(6):834-847.
[3] 李志欣, 施智平, 张灿龙, 等. 混合生成式和判别式模型的图像自动标注[J]. 中国图象图形学报, 2015, 20(5):687-699. LI Zhixin, SHI Zhiping, ZHANG Canlong, et al. Hybrid generative/discriminative model for automatic image annotation[J]. Journal of Image and Graphics, 2015, 20(5):687-699.
[4] 朱文青, 刘艳, 卞乐, 等. 基于生成式模型的目标跟踪方法综述[J]. 微处理机, 2017, 38(1):41-47. ZHU Wenqing, LIU Yan, BIAN Le, et al. Survey on object tracking method base on generative model[J]. Microprocessors, 2017, 38(1):41-47.
[5] 陈旭, 孟朝晖. 基于深度学习的目标视频跟踪算法综述[J]. 计算机系统应用, 2019, 28(1):1-9. CHEN Xu, MENG Zhaohui. Survey on video object tracking algorithms based on deep learning[J]. Computer Systems & Applications, 2019, 28(1):1-9.
[6] HORN B K P, SCHUNCK B G. Determining optical flow[J]. Artificial Intelligence, 1981, 17(1-3):185-203.
[7] MEI Xue, LING Haibin. Robust visual tracking and vehicle classification via sparse representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(11):2259-2272.
[8] CRUZ-MOTA J, BOGDANOVA I, PAQUIER B, et al. Scale invariant feature transform on the sphere:theory and applications[J]. International Journal of Computer Vision, 2012, 98(2):217-241.
[9] 李玺, 查宇飞, 张天柱, 等. 深度学习的目标跟踪算法综述[J]. 中国图象图形学报, 2019, 24(12):2057-2080. LI Xi, ZHA Yufei, ZHANG Tianzhu, et al. Survey of visual object tracking algorithms based on deep learning[J]. Journal of Image and Graphics, 2019, 24(12):2057-2080.
[10] 欧阳谷, 钟必能, 白冰,等. 深度神经网络在目标跟踪算法中的应用与最新研究进展[J]. 小型微型计算机系统, 2018, 39(2):315-323. OUYANG Gu, ZHONG Bineng, BAI Bing, et al. Recent research advances and application of object tacking algorithm based on deep neural network[J]. Journal of Chinese Computer Systems, 2018, 39(2):315-323.
[11] HARE S, GOLODETZ S, SAFFARI A, et al. Struck:structured output tracking with kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2016, 38(10):2096-2109.
[12] MA Chao, HUANG Jiabin, YANG Xiaokang, et al. Hierarchical convolutional features for visual tracking[C]//Proceedings of 2015 IEEE International Conference on Computer Vision. Santiago. Chile:IEEE, 2015:3074-3082.
[13] 王志军, 马凯. 基于成像角度的特征点质心提取精度研究[J]. 激光杂志, 2018, 39(4):90-93. WANG Zhijun, MA Kai. Study on the precision of feature point centroid extraction based on imaging angle[J]. Laser Journal, 2018, 39(4):90-93.
[14] 王敏, 赵金宇, 陈涛. 基于各向异性高斯曲面拟合的星点质心提取算法[J]. 光学学报, 2017, 37(5):226-235. WANG Min, ZHAO Jinyu, CHEN Tao. Center extraction method for star-map targets based on anisotropic Gaussian surface fitting[J]. Acta Optica Sinica, 2017, 37(5):226-235.
[15] 王宇岚, 孙韶媛, 刘致驿, 等. 基于多视角融合的夜间无人车三维目标检测[J]. 应用光学, 2020, 41(2):296-301. WANG Yulan, SUN Shaoyuan, LIU Zhiyi, et al. Nighttime three-dimensional target detection of driverless vehicles based on multi-view channel fusion network[J]. Journal of Applied Optics, 2020, 41(2):296-301.
[16] 胡玉兰, 石心蕊. 基于多摄像头协同的目标融合[J]. 电子世界, 2019(13):58-59. HU Yulan, SHI Xinrui. Object fusion based on multi-camera collaboration[J]. Electronics World, 2019(13):58-59.
[17] 张兴国. 地理场景协同的多摄像机目标跟踪研究[D]. 南京:南京师范大学, 2014. ZHANG Xingguo. Object tracking methods based on geographic scene and multi-camera[D]. Nanjing:Nanjing Normal University, 2014.
[18] 刘垒. 基于地理约束场景的动态目标检测及行为识别方法研究[D]. 桂林:桂林理工大学, 2020. LIU Lei. Research of dynamic target detection and behavior recognition method based on geographic constraint scene[D]. Guilin:Guilin University of Technology, 2020.
[19] ZHANG Z. A flexible new technique for camera calibration[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2000, 22(11):1330-1334.
[20] 李莉. OpenCV耦合改进张正友算法的相机标定算法[J]. 轻工机械, 2015, 33(4):60-63, 68. LI Li. Camera calibration algorithm based on OpenCV and improved Zhang zhengyou algorithm[J]. Light Industry Machinery, 2015, 33(4):60-63, 68.
[21] 刘艳, 李腾飞. 对张正友相机标定法的改进研究[J]. 光学技术, 2014, 40(6):565-570. LIU Yan, LI Tengfei. Reaserch of the improvement of Zhang zhengyou camera calibration method[J]. Optical Technique, 2014, 40(6):565-570.
[22] 李云功. 基于轮廓特征描述的目标识别算法研究[D]. 沈阳:沈阳理工大学, 2019. LI Yungong. Research on object recognition algorithm based on contour feature description[D]. Shenyang:Shenyang Ligong University, 2019.
[23] WANG Shengchun, WANG Hao, ZHOU Yunlai, et al. Automatic laser profile recognition and fast tracking for structured light measurement using deep learning and template matching[J]. Measurement, 2021, 169:108362.
[24] 华媛蕾, 刘万军. 改进混合高斯模型的运动目标检测算法[J]. 计算机应用, 2014, 34(2):580-584. HUA Yuanlei, LIU Wanjun. Moving object detection algorithm of improved Gaussian mixture model[J]. Journal of Computer Applications, 2014, 34(2):580-584.
[25] 徐亮, 魏锐. 基于Canny算子的图像边缘检测优化算法[J]. 科技通报, 2013, 29(7):127-131, 150. XU Liang, WEI Rui. An optimal algorithm of image edge detection based on canny[J]. Bulletin of Science and Technology, 2013, 29(7):127-131, 150.