Optimization of regularization parameter based on minimum MSE

doi:10.11947/j.AGCS.2020.20190148

Abstract

Abstract: Tikhonov regularization method is widely used in geodesy for ill-posed problems. The regularization parameter is an important factor for regularization method to solve the ill-posed problem. However, it is very difficult to determine an optimal regularization parameter. L-curve method is proposed to determine the feasible regularization parameter, which is well known to be a stable and reliable method. However, the extensive application researches show that the regularization parameter determined by L-curve method often leads to oversmoothed results. As a result, the regularization method cannot effectively improve the estimation accuracy of model parameters. Concerning this issue, this paper analyzes the effectiveness of regularization parameter on MSE (mean square error) of regularized estimation. Then, an MSE calculation method is proposed by using SVD (singular value decomposition) technology. In the method, the MSE is divided into several parts that correspond to the singular values. Therefore, the iterative calculation of MSE is avoided and the reasonable regularization parameter can be determined part to part. Using the reliable parts of MSE, the most useful regularization parameter can be determined to optimize the L-curve determined regularization parameter. Finally, the regularization parameter optimization method is proposed. Numerical example and PolInSAR vegetation inversion experiment are carried out to demonstrate the effectiveness of the regularization parameter optimization method. The results show that the regularization parameter optimization method can greatly improves the model parameter estimation of regularization method.

Key words: ill-posed problem, regularization method, regularization parameter, mean square error, L-curve

CLC Number:

P207

LIN Dongfang, ZHU Jianjun, FU Haiqiang, ZHANG Bing. Optimization of regularization parameter based on minimum MSE[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(4): 443-451.

References

[1] 崔希璋, 於宗俦, 陶本藻, 等. 广义测量平差[M]. 2版. 武汉:武汉大学出版社, 2009. CUI Xizhang, YU Zongchou, TAO Benzao, et al. Generalized surveying adjustment[M]. 2nd ed. Wuhan:Wuhan University Press, 2009.
[2] 朱建军, 田玉淼, 陶肖静. 带准则参数的平差准则及其统一与解算[J]. 测绘学报, 2012, 41(1):8-13. ZHU Jianjun, TIAN Yumiao, TAO Xiaojing. United expression and solution of adjustment criteria with parameters[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(1):8-13.
[3] 欧吉坤. 测量平差中不适定问题解的统一表达与选权拟合法[J]. 测绘学报, 2004, 33(4):283-288. OU Jikun. Uniform expression of solutions of Ill- posed problems in surveying adjustment and the fitting method by selection of the parameter weights[J]. Acta Geodaetica et Cartographica Sinica, 2004, 33(4):283-288.
[4] TIKHONOV A N. Regularization of ill-posed problems[J]. Doklady Akademii Nauk SSSR, 1963, 151(1):49-52.
[5] SAVE H, BETTADPUR S, TAPLEY B D. Reducing errors in the GRACE gravity solutions using regularization[J]. Journal of Geodesy, 2012, 86(9):695-711.
[6] LI Bofeng, SHEN Yunzhong, FENG Yanming. Fast GNSS ambiguity resolution as an Ill-posed problem[J]. Journal of Geodesy, 2010, 84(11):683-698.
[7] GUI Qingming, HAN Songhui. New algorithm of GPS rapid positioning based on double-k-type ridge estimation[J]. Journal of Surveying Engineering, 2007, 133(4):173-178.
[8] 王振杰, 欧吉坤, 柳林涛. 单频GPS快速定位中病态问题的解法研究[J]. 测绘学报, 2005, 34(5):196-201. WANG Zhenjie, OU Jikun, LIU Lintao. Investigation on solutions of ill-conditioned problems in rapid positioning using single frequency GPS receivers[J]. Acta Geodaetica et Cartographica Sinica, 2005, 34(5):196-201.
[9] 徐新禹, 李建成, 王正涛, 等. Tikhonov正则化方法在GOCE重力场求解中的模拟研究[J]. 测绘学报, 2010, 39(5):465-470. XU Xinyu, LI Jiancheng, WANG Zhengtao, et al. The simulation research on the Tikhonov regularization applied in gravity field determination of GOCE satellite mission[J]. Acta Geodaetica et Cartographica Sinica, 2010, 39(5):465-470.
[10] HOERL A E, KENNARD RW. Ridge regression:biased estimation for nonorthogonal problems[J]. Technometrics, 1970, 12(1):55-67.
[11] 林东方, 朱建军, 宋迎春, 等. 正则化的奇异值分解参数构造法[J]. 测绘学报, 2016, 45(8):883-889. DOI:10.11947/j.AGCS.2016.20150134. LIN Dongfang, ZHU Jianjun, SONG Yingchun, et al. Construction method of regularization by singular value decomposition of design matrix[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(8):883-889. DOI:10.11947/j.AGCS.2016.20150134.
[12] GOLUB G H, HEATH M, WAHBA G. Generalized cross-validation as a method for choosing a good ridge parameter[J]. Technometrics, 1979, 21(2):215-223.
[13] XU Peiliang. Iterative generalized cross-validation for fusing heteroscedastic data of inverse ill-posed problems[J]. Geophysical Journal International, 2009, 179(1):182-200.
[14] FENU C, REICHEL L, RODRIGUEZ G, et al. GCV for Tikhonov regularization by partial SVD[J]. BIT Numerical Mathematics, 2017, 57(4):1019-1039.
[15] HANSEN P C. Analysis of discrete ill-posed problems by means of the L-curve[J]. SIAM Review, 1992, 34(4):561-580.
[16] KUSCHE J, KLEES R. Regularization of gravity field estimation from satellite gravity gradients[J]. Journal of Geodesy, 2002, 76(6):359-368.
[17] XU Peiliang. Determination of surface gravity anomalies using gradiometric observables[J]. Geophysical Journal International, 1992, 110(2):321-332.
[18] XU Peiliang, SHEN Yunzhong, FUKUDA Y, et al. Variance component estimation in linear inverse ill-posed models[J]. Journal of Geodesy, 2006, 80(2):69-81.
[19] SCHAFFRIN B. Minimum mean squared error (MSE) adjustment and the optimal Tykhonov-Phillips regularization parameter via reproducing best invariant quadratic uniformly unbiased estimates (repro-BIQUUE)[J]. Journal of Geodesy, 2008, 82(2):113-121.
[20] 徐天河, 杨元喜. 均方误差意义下正则化解优于最小二乘解的条件[J]. 武汉大学学报(信息科学版), 2004, 29(3):223-226. XU Tianhe, YANG Yuanxi. Condition of regularization solution superior to LS solution based on MSE principle[J]. Geomatics and Information Science of Wuhan University, 2004, 29(3):223-226.
[21] SHEN Yunzhong, XU Peiliang, LI Bofeng. Bias-corrected regularized solution to inverse Ill-posed models[J]. Journal of Geodesy, 2012, 86(8):597-608.
[22] 王兴涛, 石磐, 朱非洲. 航空重力测量数据向下延拓的正则化算法及其谱分解[J]. 测绘学报, 2004, 33(1):33-38. WANG Xingtao, SHI Pan, ZHU Feizhou. Regularization methods and spectral decomposition for the downward continuation of airborne gravity data[J]. Acta Geodaetica et Cartographica Sinica, 2004, 33(1):33-38.
[23] BAUER F, LUKAS M A. Comparing parameter choice methods for regularization of ill-posed problems[J]. Mathematics and Computers in Simulation, 2011, 81(9):1795-1841.
[24] REICHEL L, RODRIGUEZ G. Old and new parameter choice rules for discrete ill-posed problems[J]. Numerical Algorithms, 2013, 63(1):65-87.
[25] 王振杰, 欧吉坤. 用L-曲线法确定岭估计中的岭参数[J]. 武汉大学学报(信息科学版), 2004, 29(3):235-238. WANG Zhenjie, OU Jikun. Determining the ridge parameter in a ridge estimation using L-curve method[J]. Geomatics and Information Science of Wuhan University, 2004, 29(3):235-238.
[26] 付海强, 朱建军, 汪长城, 等. 极化干涉SAR植被高反演复数最小二乘平差法[J]. 测绘学报, 2014, 43(10):1061-1067. FU Haiqiang, ZHU Jianjun, WANG Changcheng, et al. Polarimetric SAR interferometry vegetation height inversion method of complex least squares adjustment[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(10):1061-1067.
[27] 朱建军, 解清华, 左廷英, 等. 复数域最小二乘平差及其在PolInSAR植被高反演中的应用[J]. 测绘学报, 2014, 43(1):45-51, 59. ZHU Jianjun, XIE Qinghua, ZUO Tingying, et al. Criterion of complex least squares adjustment and its application in tree height inversion with PolInSAR data[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(1):45-51, 59.