Motion state monitoring and estimation of ship target based on optical satellite video

doi:10.11947/j.AGCS.2023.20220257

Abstract

Abstract: Optical satellite video can continuously monitor moving targets, which has become a current research hotspot. However, most of the current researches only focus on image-side object detection, tracking and trajectory extraction, and do not make full use of the imaging geometric parameters for high-precision quantitative estimation of the target motion state. To solve this problem, this paper proposes a method for quantitative estimation of ship target motion state based on optical satellite video. Firstly, the first frame of satellite video is accurately orientated. Then, the video is stabilized through inter-frame registration. Aiming at the problem of low positioning accuracy of the rectangular anchor tracking method, a robust tracking method for surface ships considering rotation and scale estimation is proposed for target tracking of satellite video. Finally, the geometric model is used to estimate the ground motion trajectory, speed and direction of the target through object mapping. The effectiveness of the method in this paper is verified by carrying out simultaneous observation experiments between satellites and ships on the sea based on Jilin-1 optical video satellite. Under the condition of sparse control points, the experimental results show that the geometric positioning accuracy of the ship target is 1.60 meters, the speed accuracy is 0.18 m/s, and the direction accuracy is 4.05°, which is significantly improved compared with similar methods.

Key words: optical satellite video, ground target positioning, motion state estimation, ship target, precision

CLC Number:

P237

WANG Taoyang, HONG Jianzhi, ZHANG Guo, JIANG Yonghua, LI Xin, DONG Tiancheng, HAN Yuqi, WANG Jingyin, YANG Yapeng, XU Congan. Motion state monitoring and estimation of ship target based on optical satellite video[J]. Acta Geodaetica et Cartographica Sinica, 2023, 52(10): 1679-1692.

References

[1] 王彦情,马雷,田原.光学遥感图像舰船目标检测与识别综述[J].自动化学报, 2011, 37(9):1029-1039. WANG Yanqing, MA Lei, TIAN Yuan. State-of-the-art of ship detection and recognition in optical remotely sensed imagery[J]. Acta Automatica Sinica, 2011, 37(9):1029-1039.
[2] HAN Yuqi, LIU Huaping, WANG Yufeng, et al. A comprehensive review for typical applications based upon unmanned aerial vehicle platform[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15:9654-9666.
[3] 吴佳奇,张过,汪韬阳,等.结合运动平滑约束与灰度特征的卫星视频点目标跟踪[J].测绘学报, 2017, 46(9):1135-1146. DOI:10.11947/j.AGCS.2017. 20160599. WU Jiaqi, ZHANG Guo, WANG Taoyang, et al. Satellite video point-target tracking in combination with motion smoothness constraint and grayscale feature[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(9):1135-1146. DOI:10.11947/j.AGCS.2017. 20160599.
[4] ZHANG Junpeng, JIA Xiuping, HU Jiankun. Error bounded foreground and background modeling for moving object detection in satellite videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 58(4):2659-2669.
[5] ZHANG Junpeng, JIA Xiuping, HU Jiankun, et al. Online structured sparsity-based moving-object detection from satellite videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(9):6420-6433.
[6] WANG Yunming, WANG Taoyang, ZHANG Guo, et al. Small target tracking in satellite videos using background compensation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(10):7010-7021.
[7] SHAO Jia, DU Bo, WU Chen, et al. HRSiam:high-resolution siamese network, towards space-borne satellite video tracking[J]. IEEE Transactions on Image Processing, 2021, 30:3056-3068.
[8] FENG Jie, ZENG Dening, JIA Xiuping, et al. Cross-frame keypoint-based and spatial motion information-guided networks for moving vehicle detection and tracking in satellite videos[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2021, 177:116-130.
[9] 卜丽静,郑新杰,张正鹏.顾及运动估计误差的"凝视"卫星视频运动场景超分辨率重建[J].测绘学报, 2020, 49(2):214-224. DOI:10.11947/j.AGCS.2020. 20180544. BU Lijing, ZHENG Xinjie, ZHANG Zhengpeng. Super-resolution reconstruction of "straring" satellite video motion scene considering motion estimation error[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(2):214-224. DOI:10.11947/j.AGCS.2020. 20180544.
[10] 姚烨.高分辨率视频卫星影像超分辨率重建技术研究[D].长春:中国科学院大学(中国科学院长春光学精密机械与物理研究所), 2018. YAO Ye. Research on super-resolution reconstruction of high resolution video satellite images[D]. Changchun:University of Chinese Academy of Sciences (Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences), 2018.
[11] XIE Qifang, YAO Guoqing, LIU Pin. Super-resolution reconstruction of satellite video images based on interpolation method[J]. Procedia Computer Science, 2017, 107:454-459.
[12] 范江.基于视频卫星影像的DSM生成技术研究[D].武汉:华中科技大学, 2019. FAN Jiang. Research on DSM generation technology based on video satellite images[D]. Wuhan:Huazhong University of Science and Technology, 2019.
[13] CHERNETSKIY M, TAO Y, MULLER J P. 3 D stereo reconstruction:high resolution satellite video[C]//Proceedings of 2019 SPIE International Society for Optical Engineering. Strasbourg:SPIE, 2019:582-593.
[14] 王少威.卫星视频影像建筑高度提取方法研究[D].武汉:华中科技大学, 2018. WANG Shaowei. Research on building height extraction method of satellite video images[D]. Wuhan:Huazhong University of Science and Technology, 2018.
[15] 罗亦乐,梁艳平,王妍.基于光流法的卫星视频交通流参数提取研究[J].计算机工程与应用, 2018, 54(10):204-207, 255. LUO Yile, LIANG Yanping, WANG Yan. Traffic flow parameter estimation from satellite video data based on optical flow[J]. Computer Engineering and Applications, 2018, 54(10):204-207, 255.
[16] 赵旭辉. SLAM技术在视频卫星处理中的应用研究[D].武汉:武汉大学, 2019. ZHAO Xuhui. Application research of SLAM in video satellite processing[D]. Wuhan:Wuhan University, 2019.
[17] 李建国,杨保河,石胜斌.遥感视频卫星多目标检测、跟踪和定位算法[J].火力与指挥控制, 2021, 46(2):114-119. LI Jianguo, YANG Baohe, SHI Shengbin. Multi-object detection locationing and locating algorithm for remote sensing video satellites[J]. Fire Control&Command Control, 2021, 46(2):114-119.
[18] 朱泽尧,胡庆武,王玥,等.基于吉林一号视频卫星影像的实时车辆跟踪方法研究[J].武汉理工大学学报(交通科学与工程版), 2021, 45(4):655-660. ZHU Zeyao, HU Qingwu, WANG Yue, et al. Research on real-time vehicle tracking method based on Jilin-1 video satellite imagery[J]. Journal of Wuhan University of Technology (Transportation Science&Engineering), 2021, 45(4):655-660.
[19] 张过.卫星视频处理与应用进展[J].应用科学学报, 2016, 34(4):361-370. ZHANG Guo. Satellite video processing and applications[J]. Journal of Applied Sciences, 2016, 34(4):361-370.
[20] LIU Zikun, HU Jingao, WENG Lubin, et al. Rotated region based CNN for ship detection[C]//Proceedings of 2017 IEEE International Conference on Image Processing. Beijing:IEEE, 2017:900-904.
[21] LIU Lei, PAN Zongxu, LEI Bin. Learning a rotation invariant detector with rotatable bounding box[EB/OL].[2023-09-15]. https://arxiv.org/abs/1711.09405.
[22] CUI Yanyu, HOU Biao, WU Qian, et al. Remote sensing object tracking with deep reinforcement learning under occlusion[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-13.
[23] HU Zhongyi, ZOU Mianzu, CHEN Changzu, et al. Tracking via context-aware regression correlation filter with a spatial-temporal regularization[J]. Journal of Electronic Imaging, 2020, 29(2):023029.
[24] XUAN Shiyu, LI Shengyang, HAN Mingfei, et al. Object tracking in satellite videos by improved correlation filters with motion estimations[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 58(2):1074-1086.
[25] ZHANG Wenhua, JIAO Licheng, LIU Fang, et al. MBLT:learning motion and background for vehicle tracking in satellite videos[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-15.
[26] SHAO Jia, DU Bo, WU Chen, et al. Tracking objects from satellite videos:a velocity feature based correlation filter[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(10):7860-7871.
[27] HAN Yuqi, WANG Hongshuo, ZHANG Zengshuo, et al. Boundary-aware vehicle tracking upon UAV[J]. Electronics Letters, 2020, 56(17):873-876.
[28] HAN Yuqi, DENG Chenwei, ZHAO Baojun, et al. State-aware anti-drift object tracking[J]. IEEE Transactions on Image Processing, 2019, 28(8):4075-4086.
[29] DENG Chenwei, HE Shuangcheng, HAN Yuqi, et al. Learning dynamic spatial-temporal regularization for UAV object tracking[J]. IEEE Signal Processing Letters, 2021, 28:1230-1234.
[30] SHAO Jia, DU Bo, WU Chen, et al. Can we track targets from space?A hybrid kernel correlation filter tracker for satellite video[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019, 57(11):8719-8731.
[31] WU Jialian, SU Xin, YUAN Qiangqiang, et al. Multivehicle object tracking in satellite video enhanced by slow features and motion features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2021, 60:1-26.
[32] WANG Qiang, ZHANG Li, BERTINETTO L, et al. Fast online object tracking and segmentation:a unifying approach[C]//Proceedings of 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Long Beach:IEEE, 2019:1328-1338.
[33] YANG Xue, YAN Junchi. Arbitrary-oriented object detection with circular smooth label[C]//Proceedings of 2020 European Conference on Computer Vision. Berlin:Springer, 2020:677-694.
[34] 岳琪琪.基于深度学习的尺度与角度自适应海面目标跟踪算法[D].上海:上海大学, 2021. YUE Qiqi. Adaptive scale and angle for sea surface object tracking based on deep learning[D]. Shanghai:Shanghai University, 2021.
[35] LI Yang, ZHU Jianke, HOI S C H, et al. Robust estimation of similarity transformation for visual object tracking[C]//Proceedings of 2019 AAAI Conference on Artificial Intelligence. Menlo Park:AAAI Press, 2019:8666-8673.
[36] 李忠新,茅耀斌,王执铨.基于对数极坐标映射的图像拼接方法[J].中国图象图形学报, 2005, 10(1):59-63. LI Zhongxin, MAO Yaobin, WANG Zhiquan. A method of image mosaicing using log polar coordinate mapping[J]. Journal of Image and Graphics, 2005, 10(1):59-63.
[37] DANELLJAN M, BHAT G, KHAN F S, et al. ECO:efficient convolution operators for tracking[C]//Proceedings of 2017 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Honolulu:IEEE, 2017:6931-6939.
[38] 汪韬阳.光学卫星视频数据处理与应用[M].北京:科学出版社, 2020. WANG Taoyang. Processing and application of optical satellite video data[M]. Beijing:Science Press, 2020.
[39] 张过,蒋永华,汪韬阳,等.高分辨率视频卫星标准产品分级体系[M].北京:科学出版社, 2017. ZHANG Guo, JIANG Yonghua, WANG Taoyang, et al. Standard product grading system for high resolution video satellites[M]. Beijing:Science Press, 2017.
[40] 张过.缺少控制点的高分辨率卫星遥感影像几何纠正[D].武汉:武汉大学, 2005. ZHANG Guo. Rectification for high resolution remote sensing image under lack of ground control points[D]. Wuhan:Wuhan University, 2005.
[41] 王霞,张过,沈欣,等.顾及像面畸变的卫星视频稳像[J].测绘学报, 2016, 45(2):194-198. DOI:10.11947/j.AGCS.2016. 20150238. WANG Xia, ZHANG Guo, SHEN Xin, et al. Satellite video stabilization with geometric distortion[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(2):194-198. DOI:10.11947/j.AGCS.2016. 20150238.
[42] FISCHLER M A, BOLLES R C. Random sample consensus:a paradigm for model fitting with applications to image analysis and automated cartography[J]. Communications of the ACM, 1981, 24(6):381-395.
[43] DANELLJAN M, ROBINSON A, KHAN F S, et al. Beyond correlation filters:learning continuous convolution operators for visual tracking[C]//Proceedings of 2016 European Conference on Computer Vision. Berlin:Springer, 2016:472-488.
[44] DANELLJAN M, KHAN F S, FELSBERG M, et al. Adaptive color attributes for real-time visual tracking[C]//Proceedings of 2014 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Columbus:IEEE, 2014:1090-1097.
[45] WANG Yanbo, WANG Fasheng, WANG Chang, et al. Learning saliency-aware correlation filters for visual tracking[J]. The Computer Journal, 2022, 65(7):1846-1859.
[46] HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(3):583-596.
[47] HENRIQUES J F, CASEIRO R, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]//Proceedings of 2012 European Conference on Computer Vision. Berlin:Springer, 2012:702-715.
[48] LI Feng, TIAN Cheng, ZUO Wangmeng, et al. Learning spatial-temporal regularized correlation filters for visual tracking[C]//Proceedings of 2018 IEEE Conference on Computer Vision and Pattern Recognition. Salt Lake City:IEEE, 2018:4904-4913.
[49] GALOOGAHI H K, FAGG A, LUCEY S. Learning background-aware correlation filters for visual tracking[C]//Proceedings of 2017 IEEE International Conference on Computer Vision. Venice:IEEE, 2017:1144-1152.
[50] XU Tianyang, FENG Zhenhua, WU Xiaojun, et al. Learning adaptive discriminative correlation filters via temporal consistency preserving spatial feature selection for robust visual object tracking[J]. IEEE Transactions on Image Processing, 2019, 28(11):5596-5609.
[51] BERTINETTO L, VALMADRE J, GOLODETZ S, et al. Staple:complementary learners for real-time tracking[C]//Proceedings of 2016 IEEE Conference on Computer Vision and Pattern Recognition. Las Vegas:IEEE, 2016:1401-1409.
[52] ZHAO Manqi, LI Shengyang, XUAN Shiyu, et al. SatSOT:a benchmark dataset for satellite video single object tracking[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60:1-11.
[53] WU Yi, LIM J, YANG M H. Online object tracking:a benchmark[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland:IEEE, 2013:2411-2418.
[54] WU Yi, LIM J, YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9):1834-1848