岭回归

岭回归（英语：ridge regression）是一种在自变量高度相关的情况下估计多元回归模型系数的方法，它已被应用于计量经济学、化学和工程学等许多领域^[1]，也称为吉洪诺夫正则化（英语：Tikhonov regularization）^[2]，以苏联数学家安德烈·吉洪诺夫的名字命名，是一种不适定问题的正则化方法^[a]。对于缓解线性回归中的多重共线性问题特别有用，这种问题通常出现在具有大量参数的模型中^[3]。一般来说，该方法提高了参数估计问题的效率，以换取可容忍的偏差量（参见偏差-方差权衡）^[4]。

该理论最初由Hoerl和Kennard于1970年在他们发表在《Technometrics》上的论文《RIDGE回归：非正交问题的偏差估计》（英语：RIDGE regressions: biased estimation of nonorthogonal problems）和《RIDGE回归：在非正交问题中的应用》（英语：RIDGE regressions: applications in nonorthogonal problems）中引入^[5]^[6]^[1] 。

当线性回归模型具有一些多重共线性（高度相关）自变量时^[7]，通过创建岭回归估计器（RR），岭回归被开发为解决最小二乘估计器不精确问题的可能解决方案。这提供了更精确的岭参数估计，因为其方差和均方估计量通常小于先前导出的最小二乘估计量^[8]^[2]。

注释

^ In statistics, the method is known as ridge regression, in machine learning it and its modifications are known as weight decay, and with multiple independent discoveries, it is also variously known as the Tikhonov–Miller method, the Phillips–Twomey method, the constrained linear inversion method, $L 2$ regularization, and the method of linear regularization. It is related to the Levenberg–Marquardt algorithm for non-linear least-squares problems.

参考资料

^ ^1.0 ^1.1 Hilt, Donald E.; Seegrist, Donald W. Ridge, a computer program for calculating ridge regression estimates. 1977 [2023-10-09]. doi:10.5962/bhl.title.68934. （原始内容存档于2023-02-10）. ^{[页码请求]}
^ ^2.0 ^2.1 Gruber, Marvin. Improving Efficiency by Shrinkage: The James--Stein and Ridge Regression Estimators. CRC Press. 1998: 2 [2023-10-09]. ISBN 978-0-8247-0156-7. （原始内容存档于2022-05-10）.
^ Kennedy, Peter. A Guide to Econometrics Fifth. Cambridge: The MIT Press. 2003: 205–206. ISBN 0-262-61183-X.
^ Gruber, Marvin. Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Boca Raton: CRC Press. 1998: 7–15. ISBN 0-8247-0156-9.
^ Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics. 1970, 12 (1): 55–67. JSTOR 1267351. doi:10.2307/1267351.
^ Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics. 1970, 12 (1): 69–82. JSTOR 1267352. doi:10.2307/1267352.
^ Beck, James Vere; Arnold, Kenneth J. Parameter Estimation in Engineering and Science. James Beck. 1977: 287 [2023-10-09]. ISBN 978-0-471-06118-2. （原始内容存档于2022-04-26）.
^ Jolliffe, I. T. Principal Component Analysis. Springer Science & Business Media. 2006: 178 [2023-10-09]. ISBN 978-0-387-22440-4. （原始内容存档于2022-04-18）.

[3] In statistics, the method is known as ridge regression, in machine learning it and its modifications are known as weight decay, and with multiple independent discoveries, it is also variously known as the Tikhonov–Miller method, the Phillips–Twomey method, the constrained linear inversion method, $L 2$ regularization, and the method of linear regularization. It is related to the Levenberg–Marquardt algorithm for non-linear least-squares problems.

[Hilt-1] 1.0 ^1.1 Hilt, Donald E.; Seegrist, Donald W. Ridge, a computer program for calculating ridge regression estimates. 1977 [2023-10-09]. doi:10.5962/bhl.title.68934. （原始内容存档于2023-02-10）. ^{[页码请求]}

[Gruber-2] 2.0 ^2.1 Gruber, Marvin. Improving Efficiency by Shrinkage: The James--Stein and Ridge Regression Estimators. CRC Press. 1998: 2 [2023-10-09]. ISBN 978-0-8247-0156-7. （原始内容存档于2022-05-10）.

[4] Kennedy, Peter. A Guide to Econometrics Fifth. Cambridge: The MIT Press. 2003: 205–206. ISBN 0-262-61183-X.

[5] Gruber, Marvin. Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Boca Raton: CRC Press. 1998: 7–15. ISBN 0-8247-0156-9.

[6] Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics. 1970, 12 (1): 55–67. JSTOR 1267351. doi:10.2307/1267351.

[7] Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics. 1970, 12 (1): 69–82. JSTOR 1267352. doi:10.2307/1267352.

[Beck-8] Beck, James Vere; Arnold, Kenneth J. Parameter Estimation in Engineering and Science. James Beck. 1977: 287 [2023-10-09]. ISBN 978-0-471-06118-2. （原始内容存档于2022-04-26）.

[Jolliffe-9] Jolliffe, I. T. Principal Component Analysis. Springer Science & Business Media. 2006: 178 [2023-10-09]. ISBN 978-0-387-22440-4. （原始内容存档于2022-04-18）.

[1]

[2]

[a]

[3]

[4]

[5]

[6]

[7]

[8]