嶺回歸

嶺回歸（英語：ridge regression）是一種在自變量高度相關的情況下估計多元回歸模型係數的方法，它已被應用於計量經濟學、化學和工程學等許多領域^[1]，也稱為吉洪諾夫正則化（英語：Tikhonov regularization）^[2]，以蘇聯數學家安德烈·吉洪諾夫的名字命名，是一種不適定問題的正則化方法^[a]。對於緩解線性回歸中的多重共線性問題特別有用，這種問題通常出現在具有大量參數的模型中^[3]。一般來說，該方法提高了參數估計問題的效率，以換取可容忍的偏差量（參見偏差-方差權衡）^[4]。

該理論最初由Hoerl和Kennard於1970年在他們發表在《Technometrics》上的論文《RIDGE回歸：非正交問題的偏差估計》（英語：RIDGE regressions: biased estimation of nonorthogonal problems）和《RIDGE回歸：在非正交問題中的應用》（英語：RIDGE regressions: applications in nonorthogonal problems）中引入^[5]^[6]^[1] 。

當線性回歸模型具有一些多重共線性（高度相關）自變量時^[7]，通過創建嶺回歸估計器（RR），嶺回歸被開發為解決最小二乘估計器不精確問題的可能解決方案。這提供了更精確的嶺參數估計，因為其方差和均方估計量通常小於先前導出的最小二乘估計量^[8]^[2]。

注釋

^ In statistics, the method is known as ridge regression, in machine learning it and its modifications are known as weight decay, and with multiple independent discoveries, it is also variously known as the Tikhonov–Miller method, the Phillips–Twomey method, the constrained linear inversion method, $L 2$ regularization, and the method of linear regularization. It is related to the Levenberg–Marquardt algorithm for non-linear least-squares problems.

參考資料

^ ^1.0 ^1.1 Hilt, Donald E.; Seegrist, Donald W. Ridge, a computer program for calculating ridge regression estimates. 1977 [2023-10-09]. doi:10.5962/bhl.title.68934. （原始內容存檔於2023-02-10）. ^{[頁碼請求]}
^ ^2.0 ^2.1 Gruber, Marvin. Improving Efficiency by Shrinkage: The James--Stein and Ridge Regression Estimators. CRC Press. 1998: 2 [2023-10-09]. ISBN 978-0-8247-0156-7. （原始內容存檔於2022-05-10）.
^ Kennedy, Peter. A Guide to Econometrics Fifth. Cambridge: The MIT Press. 2003: 205–206. ISBN 0-262-61183-X.
^ Gruber, Marvin. Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Boca Raton: CRC Press. 1998: 7–15. ISBN 0-8247-0156-9.
^ Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics. 1970, 12 (1): 55–67. JSTOR 1267351. doi:10.2307/1267351.
^ Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics. 1970, 12 (1): 69–82. JSTOR 1267352. doi:10.2307/1267352.
^ Beck, James Vere; Arnold, Kenneth J. Parameter Estimation in Engineering and Science. James Beck. 1977: 287 [2023-10-09]. ISBN 978-0-471-06118-2. （原始內容存檔於2022-04-26）.
^ Jolliffe, I. T. Principal Component Analysis. Springer Science & Business Media. 2006: 178 [2023-10-09]. ISBN 978-0-387-22440-4. （原始內容存檔於2022-04-18）.

[3] In statistics, the method is known as ridge regression, in machine learning it and its modifications are known as weight decay, and with multiple independent discoveries, it is also variously known as the Tikhonov–Miller method, the Phillips–Twomey method, the constrained linear inversion method, $L 2$ regularization, and the method of linear regularization. It is related to the Levenberg–Marquardt algorithm for non-linear least-squares problems.

[Hilt-1] 1.0 ^1.1 Hilt, Donald E.; Seegrist, Donald W. Ridge, a computer program for calculating ridge regression estimates. 1977 [2023-10-09]. doi:10.5962/bhl.title.68934. （原始內容存檔於2023-02-10）. ^{[頁碼請求]}

[Gruber-2] 2.0 ^2.1 Gruber, Marvin. Improving Efficiency by Shrinkage: The James--Stein and Ridge Regression Estimators. CRC Press. 1998: 2 [2023-10-09]. ISBN 978-0-8247-0156-7. （原始內容存檔於2022-05-10）.

[4] Kennedy, Peter. A Guide to Econometrics Fifth. Cambridge: The MIT Press. 2003: 205–206. ISBN 0-262-61183-X.

[5] Gruber, Marvin. Improving Efficiency by Shrinkage: The James–Stein and Ridge Regression Estimators. Boca Raton: CRC Press. 1998: 7–15. ISBN 0-8247-0156-9.

[6] Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Biased Estimation for Nonorthogonal Problems. Technometrics. 1970, 12 (1): 55–67. JSTOR 1267351. doi:10.2307/1267351.

[7] Hoerl, Arthur E.; Kennard, Robert W. Ridge Regression: Applications to Nonorthogonal Problems. Technometrics. 1970, 12 (1): 69–82. JSTOR 1267352. doi:10.2307/1267352.

[Beck-8] Beck, James Vere; Arnold, Kenneth J. Parameter Estimation in Engineering and Science. James Beck. 1977: 287 [2023-10-09]. ISBN 978-0-471-06118-2. （原始內容存檔於2022-04-26）.

[Jolliffe-9] Jolliffe, I. T. Principal Component Analysis. Springer Science & Business Media. 2006: 178 [2023-10-09]. ISBN 978-0-387-22440-4. （原始內容存檔於2022-04-18）.

[1]

[2]

[a]

[3]

[4]

[5]

[6]

[7]

[8]