Neural network-based GNSS stochastic model generation method by fisheye images

doi:10.11947/j.AGCS.2025.20250213

Abstract

Abstract:

Global navigation satellite system (GNSS) can provide high-precision positioning services. However, in complex urban environments, the presence of multipath effects and non-line-of-sight (NLOS) signals leads to a mismatch between GNSS observation quality and prior stochastic models, significantly degrading positioning performance. Methods based on fisheye cameras can utilize sky-view information to mitigate the impact of NLOS observations, but existing solutions are mostly limited to semantic segmentation applications and fail to fully exploit the high-dimensional environmental features in images. To address this issue, this paper proposes a neural network-based GNSS stochastic model generation method using fisheye images. The proposed method employs neural networks to extract high-dimensional environmental features from images that reflect GNSS observation conditions and tightly integrates GNSS and image features in a cross-attention layer to predict the stochastic model of satellite observations. Experimental results demonstrate that the proposed method can effectively capture the correlation between fisheye images and GNSS observation environments, accurately inflating the variance of abnormal observations. Moreover, in scenarios where fisheye images are affected by errors, the method can leverage GNSS feature information to reduce the impact of image errors on prediction results. When further applied to a RTK/IMU integrated navigation system, the proposed method improves positioning accuracy by 32.9%, verifying that the proposed method can significantly reduce the influence of abnormal observations and enhance system performance in complex urban environments.

Key words: GNSS stochastic model, fisheye camera, neural networks, attention mechanism, complex urban scenes

CLC Number:

P228

Yupeng GU, Wanke LIU, Xiaohong ZHANG, Jie HU, Shujie HU, Weihao LEI, Kai ZHENG. Neural network-based GNSS stochastic model generation method by fisheye images[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(12): 2206-2218.

Figures/Tables 21

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Tab. 1

Fig. 5

Fig. 6

Tab. 2

Tab. 3

Tab. 4

Fig. 7

Fig. 8

Tab. 5

Fig. 9

Tab. 6

Tab. 7

Fig. 10

Fig. 11

Tab. 8

Fig. 12

Fig. 13

References 28

[1]	耿江辉, 常华, 郭将, 等. 面向城市复杂环境的3种多频多系统GNSS单点高精度定位方法及性能分析[J]. 测绘学报, 2020, 49(1): 1-13. DOI: . doi: 10.11947/j.AGCS.2020.20190106
	GENG Jianghui, CHANG Hua, GUO Jiang, et al. Three multi-frequency and multi-system GNSS high-precision point positioning methods and their performance in complex urban environment[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(1): 1-13. DOI: . doi: 10.11947/j.AGCS.2020.20190106
[2]	刘慧娟, 党亚民, 王潜心. 多星座实时导航中一种快速次优的选星方法[J]. 测绘科学, 2013, 38(1): 20-22.
	LIU Huijuan, DANG Yamin, WANG Qianxin. A fast sub-optimal satellite selection method in multi-constellation real-time navigation[J]. Science of Surveying and Mapping, 2013, 38(1): 20-22.
[3]	黄观文, 雷哲哲, 王进, 等. 亚太区域不同遮挡情形下多系统组合PPP定位效能分析[J]. 大地测量与地球动力学, 2018, 38(6): 562-567.
	HUANG Guanwen, LEI Zhezhe, WANG Jin, et al. Efficiency analysis of multi-GNSS combined PPP in complex conditions of Asia-Pacific region[J]. Journal of Geodesy and Geodynamics, 2018, 38(6): 562-567.
[4]	ZHENG Fu, LI Qingcheng, WANG Jiale, et al. GNSS NLOS detection method based on stacking ensemble learning and applications in smartphones[J]. GPS Solutions, 2024, 28(3): 129.
[5]	SUN Rui, WANG Guanyu, ZHANG Wenyu, et al. A gradient boosting decision tree based GPS signal reception classification algorithm[J]. Applied Soft Computing, 2020, 86: 105942.
[6]	LI Xin, XU Qi, LI Xingxing, et al. Improving PPP-RTK-based vehicle navigation in urban environments via multilayer perceptron-based NLOS signal detection[J]. GPS Solutions, 2023, 28(1): 29.
[7]	YOZEVITCH R, BEN MOSHE B, WEISSMAN A. A robust GNSS LOS/NLOS signal classifier[J]. Navigation, 2016, 63(4): 429-442.
[8]	朱彬, 杨诚, 刘岩. GNSS多径信号3种非监督学习法分析与比较[J]. 测绘学报, 2021, 50(12): 1762-1771. DOI: . doi: 10.11947/j.AGCS.2021.20210233
	ZHU Bin, YANG Cheng, LIU Yan. Analysis and comparison of three unsupervised learning clustering methods for GNSS multipath signals[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(12): 1762-1771. DOI: . doi: 10.11947/j.AGCS.2021.20210233
[9]	ZHANG Xiaohong, WANG Xinyu, LIU Wanke, et al. A reliable NLOS error identification method based on LightGBM driven by multiple features of GNSS signals[J]. Satellite Navigation, 2024, 5(1): 31.
[10]	ZENG Kungan, LI Zhenni, ZHAO Haoli, et al. A spatiotemporal information-driven cross-attention model with sparse representation for GNSS NLOS signal classification[J]. IEEE Internet of Things Journal, 2024, 11(19): 31892-31908.
[11]	WANG Lei, GROVES P D, ZIEBART M K. Smartphone shadow matching for better cross-street GNSS positioning in urban environments[J]. Journal of Navigation, 2015, 68(3): 411-433.
[12]	NG H F, ZHANG Guohao, HSU L T. Robust GNSS shadow matching for smartphones in urban canyons[J]. IEEE Sensors Journal, 2021, 21(16): 18307-18317.
[13]	XIN Shaoming, GENG Jianghui, ZHANG Guohao, et al. 3D-mapping-aided PPP-RTK aiming at deep urban canyons[J]. Journal of Geodesy, 2022, 96: 78.
[14]	NG H F, ZHANG G, HSU L T. Range-based 3D mapping aided GNSS with NLOS correction based on skyplot with building boundaries[C]//Proceedings of 2019 ION Pacific PNT Meeting, Honolulu: ION, 2019: 737-751.
[15]	ZHANG Liyuan, NG H F, ZHANG Guohao, et al. Ray-tracing correction for GNSS velocity estimation using Doppler frequency: a feasibility analysis[J]. IEEE Transactions on Instrumentation and Measurement, 2024, 73: 1-10.
[16]	WEN Weisong, BAI Xiwei, KAN Y C, et al. Tightly coupled GNSS/INS integration via factor graph and aided by fish-eye camera[J]. IEEE Transactions on Vehicular Technology, 2019, 68(11): 10651-10662.
[17]	SUZUKI T, KUBO N. N-LOS GNSS signal detection using fisheye camera for vehicle navigation in urban environments[C]//Proceedings of 2014 International Technical Meeting of the Satellite Division of the Institute of Navigation. Tampa: ION, 2014: 1897-1906.
[18]	袁蕾, 张守建, 王景荣, 等. 城市遮挡环境下天空图辅助的GNSS/INS组合导航[J]. 测绘科学, 2023, 48(9): 1-8.
	YUAN Lei, ZHANG Shoujian, WANG Jingrong, et al. GNSS/INS integrated navigation aided by sky images in urban occlusion environment[J]. Science of Surveying and Mapping, 2023, 48(9): 1-8.
[19]	田哲铭, 李旭, 胡悦, 等. 城市峡谷下视觉辅助的GNSS/INS多阶段定位方法[J]. 仪器仪表学报, 2024, 45(4): 217-225.
	TIAN Zheming, LI Xu, HU Yue, et al. Multi-stage localization method based on camera-aided GNSS/INS integration in urban canyon areas[J]. Chinese Journal of Scientific Instrument, 2024, 45(4): 217-225.
[20]	MARAIS J, KAZIM S A, COCHERIL Y, et al. Multipath and NLOS detection based on the combination of CN0 values and a fisheye camera[C]//Proceedings of 2020 European Navigation Conference. Dresden: IEEE, 2020: 1-13.
[21]	HU Runzhi, WEN Weisong, HSU L. Fisheye camera aided GNSS NLOS detection and learning-based pseudorange bias correction for intelligent vehicles in urban canyons[C]//Proceedings of 2023 International Conference on Intelligent Transportation Systems. Bilbao: IEEE, 2023: 6088-6095.
[22]	XIE Enze, WANG Wenhai, YU Zhiding, et al. SegFormer: simple and efficient design for semantic segmentation with Transformers[C]//Proceedings of the 35th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2021.
[23]	SCARAMUZZA D. Omnidirectional camera[M]//IKEUCHI K. Computer vision: a reference guide. Cham: Springer International Publishing, 2021: 900-909.
[24]	OTH L, FURGALE P, KNEIP L, et al. Rolling shutter camera calibration[C]//Proceedings of 2013 IEEE Conference on Computer Vision and Pattern Recognition. Portland: IEEE, 2013: 1360-1367.
[25]	蔡亲青, 朱锋, 陈曦, 等. 高度角与信噪比混合的GNSS随机模型精化及其对RTK定位性能的影响[J]. 全球定位系统, 2023, 48(1): 24-31.
	CAI Qinqing, ZHU Feng, CHEN Xi, et al. Refinement of GNSS stochastic model combining elevation angle and SNR and its effect on RTK positioning performance[J]. GNSS World of China, 2023, 48(1): 24-31.
[26]	张小红, 陶贤露, 王颖喆, 等. 城市场景智能手机GNSS/MEMS融合车载高精度定位[J]. 武汉大学学报(信息科学版), 2022, 47(10): 1740-1749.
	ZHANG Xiaohong, TAO Xianlu, WANG Yingzhe, et al. MEMS-enhanced smartphone GNSS high-precision positioning for vehicular navigation in urban conditions[J]. Geomatics and Information Science of Wuhan University, 2022, 47(10): 1740-1749.
[27]	RAMACHANDRAN S, SISTU G, MCDONALD J, et al. Woodscape fisheye semantic segmentation for autonomous driving—CVPR 2021 OmniCV Workshop Challenge[EB/OL]. [2025-04-05]. https://arxiv.org/pdf/2107.08246.
[28]	KIRILLOV A, MINTUN E, RAVI N, et al. Segment anything[C]//Proceedings of 2023 IEEE/CVF International Conference on Computer Vision. Paris: IEEE, 2023: 3992-4003.

传感器类型	传感器说明	传感器参数
POMS-7683	高精度组合导航设备	陀螺零偏不稳定性：0.1°/h
POMS-7683	高精度组合导航设备	数据频率：200 Hz
ICM-20602	低成本IMU	陀螺零偏不稳定性：10°/h
ICM-20602	低成本IMU	数据频率：200 Hz
Ublox F9P	低成本GNSS接收机	卫星系统：G/R/E/C/J
Ublox F9P	低成本GNSS接收机	数据频率：1 Hz
鱼眼相机	相机	图像分辨率：2048×2048
鱼眼相机	相机	数据频率：1 Hz

项目	配置
观测数据	GPS（L1、L2）
	BDS-2（B1I、B2I）
	BDS-3（B1I）
	Galileo（E1、E5）
	QZSS（L1、L2）
星历	广播星历
截止高度角/（°）	15
截止信噪比/dBHz	25

网络设计	GNSS随机模型RMSE/m
GI：图像特征+GNSS特征	1.79
CI：交叉注意力层+图像特征	3.20
CGI：交叉注意力层+图像特征+GNSS特征	1.59

网络结构	GNSS随机模型RMSE/m
模态A	1.62
模态B	1.59
模态C	5.52

场景	定位偏差/m			PDOP
场景	Trad	NLOS-seg	F-GNet	Trad	NLOS-seg	F-GNet
场景a	16.935	N/A	3.101	1.688	N/A	1.688
场景b	16.082	12.425	7.227	1.792	2.471	1.792
场景c	6.661	4.605	2.892	1.585	1.877	1.585
场景d	6.536	3.840	2.096	1.521	1.960	1.521