The method and application for solving separable nonlinear least squares based on matrix decomposition

doi:10.11947/j.AGCS.2022.20200502

Abstract

Abstract: For the special structure of separable function models, the variable projection (VP) method is used to separate the linear and the nonlinear parameters in this paper, and respectively combined with the full-rank decomposition, QR decomposition, singular value decomposition and Gram-Schmidt orthogon-alization to calculate the parameters, which shorten the calculation time of solving equations by computer, and enable the algorithm more efficient and equations with a certain ill-conditioning maintain the stability in the process of solving. The superiority-inferiority of the algorithms based on different matrix decomposition methods is analyzed by Mackey-Glass time series fitting experiment and parameters calculation for spatial rectangular coordinate transformation. The experimental results show that the improved VP algorithms based on matrix decomposition are highly efficient and stable, and suitable to solve the parameters of spatial rectangular coordinate transformation models.

Key words: separable nonlinear least squares, variable projection, matrix decomposition, Mackey-Glass time series, spatial rectangular coordinates transformation

CLC Number:

P207

WANG Luyao, LIU Guolin, WANG Fengyun, WANG Ke, HAN Yu. The method and application for solving separable nonlinear least squares based on matrix decomposition[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(11): 2317-2327.

References

[1] 姜兆英, 刘国林, 陶秋香. 短基线集形变模型反演的正则化解算方法[J]. 测绘学报, 2016, 45(5): 566-573. DOI: 10.11947/j.AGCS.2016.20150143. JIANG Zhaoying, LIU Guolin, TAO Qiuxiang. Regularization solution of small baseline subset deformation model inversion[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(5):556-573. DOI: 10.11947/j.AGCS.2016.20150143.
[2] 薛树强, 杨元喜, 党亚民. 测距定位方程非线性平差的封闭牛顿迭代公式[J]. 测绘学报, 2014, 43(8): 771-777. XUE Shuqiang, YANG Yuanxi, DANG Yamin. A closed-form of Newton iterative formula for nonlinear adjustment of distance equations[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(8): 771-777.
[3] 高飞, 童恒庆. 基于改进粒子群优化的非线性最小二乘估计[J]. 系统工程与电子技术, 2006, 28(5): 775-778. GAO Fei, TONG Hengqing. Nonlinear least squares estimation based on improved particle swarm optimization[J]. Systems Engineering and Electronics, 2006, 28(5): 775-778.
[4] FANG Xing. On non-combinatorial weighted total least squares with inequality constraints[J]. Journal of Geodesy,2014,88(8): 805-816.
[5] SHEN Yunqiu, YPAM Y J. Solving nonlinear systems of equations with only one nonlinear variable [J]. Journal of Computational & Applied Mathematics, 1990, 30(2): 235-246.
[6] HONG J H, ZACH C, FITZGIBBON A. Revisiting the variable projection method for separable nonlinear least squares problems[C]//Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR.). Los Alamitos, USA: IEEE Computer Society, 2017: 5939-5947.
[7] GAN Min, CHEN C L P, CHEN Guangyong, et al. On some separated algorithms for separable nonlinear least squares problems[J]. IEEE Transactions on Cybernetics, 2018, 48(10): 2866-2874.
[8] KAUFMAN L. A variable projection method for solving separable nonlinear least squares problems[J].Bit,1975, 15(1): 49-57.
[9] CORRADI C. A note on the solution of separable nonlinear least-squares problems with separable nonlinear equality constraints[J]. SIAM Journal on Numerical Analysis, 1981, 18(6): 1134-1138.
[10] DIANNE P O’L, BERT W R. Variable projection for nonlinear least squares problems[J]. Computational Optimization and Applications, 2013, 54(3): 579-593.
[11] RUHE A, WEDIN P A. Algorithms for separable nonlinear least squares problems[J].SIAM Review, 1980, 22(3): 318-337.
[12] CHEN Guangyong, GAN Min, DING Feng, et al. Modified Gram-Schmidt method based variable projection algorithm for separable nonlinear models[J]. IEEE Transactions on Neural Networks and Learning Systems, 2019, 30(8): 2410-2418.
[13] FU Zhengqing, GUO Lanlan. Tikhonov regularized variable projection algorithms for separable nonlinear least squares problems[EB/OL].[2019-10-12]. https://doi.org/10.1155/2019/4861708.
[14] RICKETT J. The variable projection method for waveform inversion with an unknown source function [J]. Geophysical Prospecting, 2013, 61(4): 874-881.
[15] CHUNG J, NAGY J G. An efficient iterative approach for large-scale separable nonlinear inverse problems [J]. SIAM Journal on Scientific Computing, 2010, 31(6): 4654-4674.
[16] CHEN G Y, GAN M, CHEN L P, et al. A two-stage estimation algorithm based on variable projection method for GPS positioning[J]. IEEE Transactions on Instrumentation and Measurement, 2018, 67(11): 2518-2525.
[17] FU Zhengqing, LIU Guolin, GUO Lanlan. Sequential quadratic programming method for separable nonlinear least squares estimation and its application[EB/OL]. [2019-10-10]. https://doi.org/10.1155/2019/3087949.
[18] DUAN Pengshuo, LIU Genyou, GONG Youliang, et al. The functional gradient description method of space coordinate transformation[J]. Journal of Geodesy and Geoinformation Science, 2019, 2(1): 66-71.
[19] 彭荣华,胡祥云,韩波.基于高斯牛顿法的频率域可控源电磁三维反演研究[J].地球物理学报, 2016, 59(9): 3470-3481. PENG Ronghua, HU Xiangyun, HAN Bo. 3D inversion of frequency-domain CSEM data based on Gauss-Newton optimization[J]. Chinese Journal of Geophysics, 2016, 59(9): 3470-3481.
[20] 邓兴升, 孙虹虹, 汤仲安. 高斯牛顿迭代法解算非线性Bursa-Wolf模型的精度分析[J]. 测绘科学, 2014, 39(5): 93-95. DENG Xingsheng, SUN Honghong, TANG Zhongan. Precision of Gauss-Newton iterative algorithm for solving nonlinear Bursa-Wolf model[J]. Science of Surveying and Mapping, 2014, 39(5):93-95.
[21] GOLUB G H, PEREYRA V. The differentiation of pseudo-inverses and nonlinear least squares problems whose variables separate [J]. SIAM Journal on Numerical Analysis, 1973, 10(2): 413-432.
[22] MARQUARDT D W. An algorithm for the least-squares estimation of nonlinear parameters [J]. Journal of the Society for Industrial and Applied Mathematics, 1963, 11(2): 431-441.
[23] LEVENBERG K. A method for the solution of certain nonlinear problems in least squares [J]. Quarterly Journal of Applied Mathmatics, 1944, 2(2): 164-168.
[24] 甘敏, 彭辉, 陈晓红, 等. RBF-AR模型在非线性时间序列预测中的应用[J]. 系统工程理论与实践, 2010, 30(6): 1055-1061. GAN Min, PENG Hui, CHEN Xiaohong, et al. RBF-AR model-based nonlinear time series prediction[J]. Systems Engineering-Theory & Practice, 2010, 30(6): 1055-1061.
[25] GAN Min, LI Hanxiong. An efficient variable projection formulation for separable nonlinear least squares problems[J]. IEEE Transactions on Cybernetics, 2014, 44(5): 707-710.
[26] 方兴, 曾文宪, 刘经南, 等. 三维坐标转换的通用整体最小二乘算法[J]. 测绘学报, 2014, 43(11): 1139-1143. FANG Xing, ZENG Wenxian, LIU Jingnan, et al. A general total least squares algorithm for three-dimensional coordinate transformation[J]. Acta Geodaetica et Cartographica Sinica, 2014, 43(11): 1139-1143.
[27] 陈宇, 白征东. 基于非线性最小二乘算法的空间坐标转换[J]. 大地测量与地球动力学, 2010, 30(2): 129-132. CHEN Yu, BAI Zhengdong. A nonlinear least square algorithm for spatial coordinate transformation[J]. Journal of Geodesy and Geodynamics, 2010, 30(2): 129-132.
[28] 钮群. 解非线性的最小二乘法拟合曲线的数值延拓法[J]. 河海大学学报(自然科学版), 2003, 31(5): 597-600. NIU Qun. Numerical continuation method for nonlinear curve fitting by use of least square method[J]. Journal of Hohai University (Natural Sciences), 2003, 31(5): 597-600.