An index of multicollinearity diagnosis based on Rayleigh quotient

doi:10.11947/j.AGCS.2025.20240338

Abstract

Abstract:

Multicollinearity widely exists in measurement data processing, and its harms are very serious. The premise of effective treatment of multicollinearity problem is to effectively diagnose and measure the multicollinearity of design matrix. In view of the shortcomings of multicollinearity indicators such as the condition number and orthogonal degree of matrix, based on the nature of multicollinearity, the paper adopts the Rayleigh quotient of normal matrix as the measure of linear correlation between the array vectors of the design matrix, and proposes a multicollinearity indicator according to the properties of the Rayleigh quotient and the eigenvalues of the edge-matrix. This indicator can be used to diagnose multiple multicollinearity relationships. The effectiveness and superiority of the proposed indicator for multicollinearity diagnosis are demonstrated by experiments. When the multicollinearity relationships are strong, the rate of accuracy in diagnosing the multicollinearitycan reach 100%. When the multicollinearity relationships are weak, the diagnostic accuracy is 84.9%, and by adjusting the threshold, the diagnostic accuracy can be increased to 99.8%.

Key words: multicollinearity, Rayleigh quotient, condition number, minimum eigenvalue

CLC Number:

P207

Xinna LI, Songhui HAN, Ke CHEN, Jie GUO. An index of multicollinearity diagnosis based on Rayleigh quotient[J]. Acta Geodaetica et Cartographica Sinica, 2025, 54(3): 422-431.

Figures/Tables 7

Fig. 1

Fig. 2

Tab. 1

Tab. 2

Tab. 3

Tab. 4

Tab. 5

References 19

[1]	FARRAR D E, GLAUBER R R. Multicollinearity in regression analysis: the problem revisited[J]. The Review of Economics and Statistics, 1967, 49(1): 92.
[2]	杨元喜, 张丽萍. 中国大地测量数据处理60年重要进展　第二部分：大地测量参数估计理论与方法的主要进展[J]. 地理空间信息, 2010, 8(1): 1-6.
	YANG Yuanxi, ZHANG Liping. Progress of geodetic data processing for 60 years in China part 2: progress of parameter estimation theory and methodology[J]. Geospatial Information, 2010, 8(1): 1-6.
[3]	李德仁, 袁修孝. 误差处理与可靠性理论[M]. 2版. 武汉: 武汉大学出版社, 2012.
	LI Deren, YUAN Xiuxiao. Error processing and reliability theory[M]. 2nd ed. Wuhan: Wuhan University Press, 2012.
[4]	林东方, 姚宜斌, 郑敦勇, 等. 利用TSVD参数估值变化特性确定算法截断参数[J]. 测绘学报, 2022, 51(8): 1787-1796. DOI:. doi: 10.11947/j.AGCS.2022.20210377
	LIN Dongfang, YAO Yibin, ZHENG Dunyong, et al. Determination of truncation parameter based on the differences of TSVD parameter estimates for ill-posed problems in geodesy[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(8): 1787-1796. DOI:. doi: 10.11947/j.AGCS.2022.20210377
[5]	王振杰. 测量中不适定问题的正则化解法[M]. 北京: 科学出版社, 2006.
	WANG Zhenjie. Regular solution of ill-posed problems in measurement[M]. Beijing: Science Press, 2006.
[6]	GARCÍA C B, GARCÍA J, LÓPEZ MARTÍN M M, et al. Collinearity: revisiting the variance inflation factor in ridge regression[J]. Journal of Applied Statistics, 2015, 42(3): 648-661.
[7]	归庆明, 姚绍文, 顾勇为, 等. 诊断复共线性的条件指标-方差分解比法[J]. 测绘学报, 2006, 35(3): 210-214.
	GUI Qingming, YAO Shaowen, GU Yongwei, et al. A new method to diagnose multicollinearity based on condition index and variance decomposition proportion (CIVDP)[J]. Acta Geodaetica et Cartographica Sinica, 2006, 35(3): 210-214.
[8]	韩松辉, 杜兰, 归庆明, 等. 诊断复共线性的特征分析法及其在GEO定轨中的应用[J]. 测绘学报, 2013, 42(1): 19-26.
	HAN Songhui, DU Lan, GUI Qingming, et al. Characteristics analysis approach for multicollinearity diagnosis and its applications in orbit determination of GEO satellites[J]. Acta Geodaetica et Cartographica Sinica, 2013, 42(1): 19-26.
[9]	周萱影. 回归模型复共线性诊断方法及其在惯导误差分离中的应用[D]. 长沙: 国防科技大学, 2019.
	ZHOU Xuanying. Multicollinearity diagnosis method of regression model and its application in inertial navigation error separation[D]. Changsha: National University of Defense Technology, 2019.
[10]	顾勇为, 归庆明. 航空重力测量数据向下延拓基于信噪比的正则化方法的研究[J]. 测绘学报, 2010, 39(5): 458-464.
	GU Yongwei, GUI Qingming. Study of regularization based on signal-to-noise index in airborne gravity downward to the Earth surface[J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(5): 458-464.
[11]	郭杰, 归庆明, 郭淑妹, 等. 利用复共线性诊断确定偏差矫正项的截断型岭估计[J]. 武汉大学学报(信息科学版), 2015, 40(6): 785-789,794.
	GUO Jie, GUI Qingming, GUO Shumei, et al. Truncation ridge estimation based on the biased-corrected theory by multicollinearity diagnosis[J]. Geomatics and Information Science of Wuhan University, 2015, 40(6): 785-789,794.
[12]	CHATTERJEE S, PRICE B. Regression analysis by example[M]. New York: Wiley, 1977.
[13]	王孝青, 党亚民, 薛树强. 一种病态问题诊断的数值指标：矩阵向量正交度[J]. 武汉大学学报(信息科学版), 2009, 34(2): 244-247.
	WANG Xiaoqing, DANG Yamin, XUE Shuqiang. A numerical value index applied to diagnosis of ill-conditioning problems: orthogonal degree of matrices[J]. Geomatics and Information Science of Wuhan University, 2009, 34(2): 244-247.
[14]	张凯院, 徐仲. 矩阵论[M]. 西安: 西北工业大学出版社, 2017.
	ZHANG Kaiyuan, XU Zhong. Matrix theory[M]. Xi'an: Northwestern Polytechnical University Press, 2017.
[15]	王松桂, 吴密霞, 贾忠贞. 矩阵不等式[M]. 2版. 北京: 科学出版社, 2006.
	WANG Songgui, WU Mixia, JIA Zhongzhen. Matrix inequality[M]. 2nd ed. Beijing: Science Press, 2006.
[16]	SILVEY S D. Multicollinearity and imprecise estimation[J]. Journal of the Royal Statistical Society: Series B (Methodological), 1969, 31(3): 539-552.
[17]	陈希孺, 王松桂. 近代回归分析：原理方法及应用[M]. 合肥: 安徽教育出版社, 1987.
	CHEN Xiru, WANG Songgui. Advanced regression analysis[M]. Hefei: Anhui Education Publishing House, 1987.
[18]	嵇昆浦, 沈云中. TSVD正则化解法的单位权方差无偏估计[J]. 武汉大学学报(信息科学版), 2020, 45(4): 626-632.
	JI Kunpu, SHEN Yunzhong. Unbiased estimation of unit weight variance by TSVD regularization[J]. Geomatics and Information Science of Wuhan University, 2020, 45(4): 626-632.
[19]	张玉祥. 人造卫星测轨方法[M]. 北京: 国防工业出版社, 2007.
	ZHANG Yuxiang. Satellite orbit determination[M]. Beijing: National Defense Industry Press, 2007.

扰动	诊断结果（1000次试验）
扰动	两个强复共线性关系	一强一弱复共线性关系	两个弱复共线性关系	一个复共线性关系
	1000	0	0	0
	1000	0	0	0
	222	778	0	0
	222	778	0	0
	3	412	453	132
	3	412	453	132

参数	主对角线元素	特征值
6参数	6.5×10³、334.9、9.38、4.6×10¹¹、1.4×10¹²、4.95×10⁸	0.32、0.67、278.4、7.8×10⁷、8.3×10¹⁰、1.8×10¹²
12参数	6.5×10³、334.9、9.38、4.6×10¹¹、1.4×10¹²、4.95×10⁸、1.66×10⁶、1.63×10⁴、720、720、1.71×10²⁰、1.64×10²⁰	7.7×10^-5、0.51、2.97、4.67、59.12、160.5、570.4、7.6×10⁷、1.0×10⁹、5.6×10¹⁰、7.2×10¹⁸、3.3×10²⁰

设计阵	条件数	矩阵列向量正交度	本文提出的指标LI_i	线性相关状态
6参数	5.54×10¹²	0.000 326	0.034	良态
12参数	4.25×10²⁴	1.08×10^-13	8.22×10^-6、0.054	一个复共线性关系

列行	a₁q_1,1	a₂q_2,1	a₃q_3,1	a₄q_4,1	a₉q_9,1	a₁₀q_10,1
1	0.253	-0.247	-0.003	0.000	0.000	0.000
2	0.256	-0.216	-0.001	0.000	-0.042	0.000
3	0.255	-0.221	-0.002	0.000	0.000	-0.029
4	0.253	-0.243	-0.003	-0.004	0.000	0.000
5	0.256	-0.212	-0.001	-0.004	-0.042	0.000

列行	a₁q_1,1	a₂q_2,1	a₃q_3,1	a₄q_4,1	a₅q_5,1	a₆q_6,1	a₇q_7,1	a₈q_8,1	a₉q_9,1	a₁₀q_10,1	a₁₁q_11,1	a₁₂q_12,1
1	-0.300	0.530	-0.002	0.713	-0.726	-0.001	-6.000	0.030	-0.042	0.000	6.458	-0.659
2	-0.272	0.487	-0.003	0.675	-0.688	-0.001	-6.019	0.030	0.000	-0.029	6.450	-0.625
3	-0.140	0.287	-0.004	0.498	-0.515	-0.001	-6.146	0.033	0.000	0.000	6.453	-0.474
4	-0.300	0.534	-0.002	0.709	-0.726	-0.001	-5.988	0.030	-0.042	0.000	6.451	-0.664
5	-0.272	0.491	-0.003	0.671	-0.689	-0.001	-6.007	0.030	0.000	-0.029	6.443	-0.630