A separable nonlinear least squares solution method based on Moore-Penrose generalized inverse and solid matrix

doi:10.11947/j.AGCS.2022.20200597

Abstract

Abstract: A separable nonlinear least squares algorithm based on Moore-Penrose generalized inverse and solid matrix is proposed to solve the special structure of linear combination of nonlinear functions in the field of surveying and mapping. Firstly, the variable projection algorithm is used to eliminate the linear parameters in the separable nonlinear model, and the original nonlinear optimization problem with two kinds of parameters is transformed into the least squares problem with only nonlinear parameters. Then, the first-order partial derivative of the least squares objective function is calculated based on the theory of differentiation of Moore-Penrose inverse matrix and solid matrix. Then the LM method of nonlinear optimization is used to solve the optimal estimation of nonlinear parameters. Finally, the optimal solution of linear parameters is obtained by linear least square method. The exponential model fitting experiment and airborne LiDAR full-waveform parameter solving experiment are used to compare the proposed method with the traditional optimization method without separation of parameters. The results show that the separable nonlinear least squares solution method based on the Moore-Penrose generalized inverse and the solid matrix is less dependent on the initial value of the parameter, avoids the ill-conditioned problem caused by the linear parameter in the iterative process, and the algorithm is robust. It provides a new idea for solving the separable nonlinear least squares problem in the field of surveying and mapping, and also expands the application of separable nonlinear least squares method.

Key words: separable nonlinear least squares method, nonlinear model, parameter estimation, variable projection algorithm

CLC Number:

P207

WANG Ke, LIU Guolin, FU Zhengqing, WANG Luyao. A separable nonlinear least squares solution method based on Moore-Penrose generalized inverse and solid matrix[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(3): 340-350.

References

[1] 杨元喜, 张丽萍. 中国大地测量数据处理60年重要进展第二部分:大地测量参数估计理论与方法的主要进展[J]. 地理空间信息, 2010, 8(1):1-6. YANG Yuanxi, ZHANG Liping. Progress of geodetic data processing for 60 years in China part 2:progress of parameter estimation theory and methodology[J]. Geospatial Information, 2010, 8(1):1-6.
[2] GHILANI C D. Adjustment computations:spatial data analysis[D]. New Jersey:The Pennsylvania State University, 2017.
[3] 陶本藻. 关于测量中非线性模型估计问题[J]. 测绘通报, 1998(2):3-5. TAO Benzao. On the estimation problem of nonlinear model in measurement[J]. Bulletin of Surveying and Mapping, 1998(2):3-5.
[4] 王新洲. 非线性模型参数估计的直接解法[J]. 武汉测绘科技大学学报, 1999(1):64-67. WANG Xinzhou. A direct solution method of parameter estimation of nonlinear model[J]. Journal of Wuhan Techonical University of Surveying and Mapping, 1999(1):64-67.
[5] 刘国林. 非线性最小二乘与测量平差[M]. 北京:测绘出版社, 2002. LIU Guolin. Nonlinear least squares and measurement adjustment[M]. Beijing:Surveying and Mapping Press, 2002.
[6] XU P, LIU J, SHI C. Total least squares adjustment in partial errors-in-variables models:algorithm and statistical analysis[J]. Journal of Geodesy, 2012, 86(8):661-675.
[7] GOLUB G H, PEREYRA V. The differentiation of Pseudo-inverses and nonlinear least squares problems whose variables separate[J]. SIAM Journal on Numerical Analysis, 1973, 10(2):413-432.
[8] GAN M, CHEN C L P, CHEN G Y, et al. On some separated algorithms for separable nonlinear least squares problems[J]. IEEE Transactions on Cybernetics, 2018, 48(10):2866-2874.
[9] 魏木生.广义最小二乘问题的理论和计算[M]. 北京:科学出版社, 2006. WEI Musheng. Theory and calculation of generalized least squares problem[M]. Beijing:Science Press, 2006.
[10] 王滨辉, 宋沙磊, 龚威, 等. 全波形激光雷达的波形优化分解算法[J]. 测绘学报, 2017, 46(11):1859-1867. DOI:10.11947/j.AGCS.2017.20170045. WANG Binhui, SONG Shalei, Gong Wei, et al. Optimization decomposition method of full-waveform LiDAR[J]. Acta Geodaetica et Cartographica Sinica, 2017, 46(11):1859-1867. DOI:10.11947/j.AGCS.2017.20170045.
[11] WAGNER W, ULLRICH A, DUCIC V, et al. Gaussian decomposition and calibration of a novel small-footprint full-waveform digitising airborne laser scanner[J]. ISPRS Journal of Photogrammetry and Remote Sensing, 2006, 60(2):100-112.
[12] AKYILMAZ O. Total least squares solution of coordinate transformation[J]. Survey Review, 2007, 39(303):68-80.
[13] LU Z P, WEI Z Q, LI J, et al. Grid model for high-accuracy coordinate transformation of China geodetic coordinate system 2000[J].Journal of Geodesy and Geoinformation Science,2019,2(1):17-25, 36.
[14] GAN M, LI H X, PENG H. A variable projection approach for efficient estimation of RBF-ARX model[J]. IEEE Transactions on Cybernetics, 2015, 45(3):476-485.
[15] PALANCZ B, AWANGE J L, ZALETNYIK P, et al. Linear homotopy solution of nonlinear systems of equations in geodesy[J]. Journal of Geodesy, 2010, 84(1):79-95.
[16] 张勤, 陶本藻. 基于同伦法的非线性最小二乘平差统一模型[J]. 武汉大学学报(信息科学版), 2004(8):708-710. ZHANG Qin, TAO Benzao. Uniform model of nonlinear least squares adjustment based on homotopy method[J]. Geomatics and Information Science of Wuhan University, 2004(8):708-710.
[17] HOFTON M A, MINSTER J B, BLAIR J B. Decomposition of laser altimeter waveforms[J]. IEEE Transactions on Geoscience & Remote Sensing, 2000, 38(4):1989-1996.
[18] 郭锴, 刘焱雄, 徐文学, 等. 机载激光测深波形分解中LM与EM参数优化方法比较[J]. 测绘学报, 2020, 49(1):117-131. DOI:10.11947/j.AGCS.2020.20180242. GUO Kai, LIU Yanxiong, XU Wenxue, et al. Comparison of LM and EM parameter optimization method for airborne laser bathymetric full-waveform decomposition[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(1):117-131. DOI:10.11947/j.AGCS.2020.20180242.
[19] 郭迎钢, 李宗春, 何华, 等. 三维坐标转换公共点最优权值的单纯形搜索算法[J]. 测绘学报, 2020, 49(8):1004-1013. DOI:10.11947/j.AGCS.2020.20190409. GUO Yinggang, LI Zongchun, HE Hua, et al. Asimplex search algorithm for the optimal weight of common point of 3D coordinate transformation[J]. Acta Geodaetica et Cartographica Sinica, 2020, 49(8):1004-1013. DOI:10.11947/j.AGCS.2020.20190409.
[20] 孙振, 曲国庆, 苏晓庆, 等. 测距定位方程参数估计的Frozen-Barycentre算法[J]. 武汉大学学报(信息科学版), 2020, 45(9):1478-1484. SUN Zhen, QU Guoqing, SU Xiaoqing, et al. Frozen-Barycentre algorithm for solving distance equations[J]. Geomatics and Information Science of Wuhan University, 2020, 45(9):1478-1484.
[21] XU W, ZHENG N, HAYAMI K. Jacobian-free implicit inner-iteration preconditioner for nonlinear least squares problems[J]. Journal of Scientific Computing, 2016, 68(3):1055-1081.
[22] BELLAVIA S, GRATTON S, RICCIETTI E. A Levenberg-Marquardt method for large nonlinear least-squares problems with dynamic accuracy in functions and gradients[J]. Numerische Mathematik, 2018, 140(3):791-825.
[23] WANG K, LIU G L, TAO Q X, et al. Efficient parameters estimation method for the separable nonlinear least squares problem[J]. Complexity, 2020(1):1-16.
[24] 方兴, 曾文宪, 刘经南, 等. 基于非线性高斯-赫尔默特模型的混合整体最小二乘估计[J]. 测绘学报, 2016, 45(3):291-296. DOI:10.11947/j.AGCS.2016.20150157. FANG Xing, ZENG Wenxian, LIU Jingnan, et al. Mixed LS-TLS estimation based on nonlinear Gauss-Helmert model[J]. Acta Geodaetica et Cartographica Sinica, 2016, 45(3):291-296. DOI:10.11947/j.AGCS.2016.20150157.
[25] 姚宜斌, 黄书华, 张良, 等. 求解三维坐标转换参数的整体最小二乘新方法[J]. 武汉大学学报(信息科学版), 2015, 40(7):853-857. YAO Yibin, HUANG Shuhua, ZHANG Liang, et al. A new method of TLS for solving the parameters of three-dimensional coordinate transformation[J]. Geomatics and Information Science of Wuhan University, 2015, 40(7):853-857.
[26] XUE Shugiang, YANG Yuanxi. Adjustment model and colored noise compensation of continuous observation system[J]. Journal of Geodesy and Geoinformation Science, 2018, 1(1):39-45.
[27] JIATIAN LI C W, CHENGLIN JIA,YIRU NIU, et al. A hybrid conjugate gradient algorithm for solving relative orientation of big rotation angle stereo pair[J]. Journal of Geodesy and Geoinformation Science, 2020, 3(2):62-70.
[28] 吕志鹏, 隋立芬. 基于非线性高斯-赫尔默特模型的结构总体最小二乘法[J]. 武汉大学学报(信息科学版), 2019, 44(12):1808-1815. LÜ Zhipeng, SUI Lifen. A structured total least squares method based on nonlinear Gauss-Helmert model[J]. Geomatics and Information Science of Wuhan University, 2019, 44(12):1808-1815.
[29] STOJANOVIC K S, MOSIC D. Generalization of the Moore-Penrose inverse[J]. Revista de la Real Academia de Ciencias Exactas, Físicas y Naturales. Serie A. Matemáticas, 2020, 114(4):1-16.
[30] 吕志鹏, 隋立芬. 基于变量投影的结构总体最小二乘算法[J]. 武汉大学学报(信息科学版), 2021, 46(3):388-394. LÜ Zhipeng, SUI Lifen. Structured total least squares method based on variable projection[J]. Geomatics and Information Science of Wuhan University, 2021, 46(3):388-394.
[31] 张贤达. 矩阵分析与应用[M]. 北京:清华大学出版社, 2013. ZHAN Xianda. Matrix analysis and application[M]. Beijing:Tsinghua University Press, 2013.
[32] 韦博成. 近代非线性回归分析[M]. 南京:东南大学出版社, 1989. WEI Bocheng. Modern nonlinear regression analysis[M]. Nanjing:Southeast University Press, 1989.
[33] SHI Y, XU P, PENG J, et al. Adjustment of measurements with multiplicative errors:error analysis, estimates of the variance of unit weight, and effect on volume estimation from LiDAR-type digital elevation models[J]. Sensors, 2014, 14(1):1249-1266.
[34] XU P, LIU J. Variance components in errors-in-variables models:estimability, stability and bias analysis[J]. Journal of Geodesy, 2014, 88(8):719-734.
[35] 鲁铁定, 吴光明, 周世健. 病态不确定性平差模型的岭估计算法[J]. 测绘学报, 2019, 48(4):403-411. DOI:10.11947/j.AGCS.2019.20180044. LU Tieding, WU Guangming, ZHOU Shijian. Ridge estimation algorithm to ill-posed uncertainty adjustment model[J]. Acta Geodaetica et Cartographica Sinica, 2019, 48(4):403-411. DOI:10.11947/j.AGCS.2019.20180044.
[36] 唐利民. 非线性最小二乘问题的不适定性及算法研究[J]. 测绘学报, 2012, 41(4):630. TANG Limin. Research on the ill-posed and solving methods of nonlinear least squares problem[J]. Acta Geodaetica et Cartographica Sinica, 2012, 41(4):630.