Development of a Bootstrap-Stability Adaptive Ridge Regression Method for Multicollinear Data
Keywords:
Adaptive penalty, Bootstrap stability, Multicollinearity, Ridge regression.Abstract
This study develops Bootstrap-Stability Adaptive Ridge Regression (BSA-Ridge), a methodological extension of classical ridge regression for multicollinear regression data. Classical ridge regression controls coefficient variance by adding a uniform quadratic penalty, but it does not distinguish predictors whose coefficients are empirically unstable from predictors whose coefficients are relatively stable. The proposed method estimates coefficient instability through bootstrap resampling and converts the bootstrap variance into predictor-specific penalty weights. The empirical illustration uses the Longley benchmark dataset, a public dataset widely used to examine numerical instability and multicollinearity in least-squares regression. The results show that BSA-Ridge produces interpretable adaptive shrinkage and competitive predictive performance relative to ordinary least squares and classical ridge regression. The contribution of this article is a transparent, equation-based, and reproducible extension of ridge regression that can be further evaluated through simulation and high-dimensional applications.
References
[1] A. E. Hoerl and R. W. Kennard, "Ridge regression: Biased estimation for nonorthogonal problems," Technometrics, vol. 12, no. 1, pp. 55-67, 1970, doi: 10.1080/00401706.1970.10488634.
[2] A. E. Hoerl, R. W. Kennard, and K. F. Baldwin, "Ridge regression: Some simulations," Communications in Statistics, vol. 4, no. 2, pp. 105-123, 1975.
[3] J. W. Longley, "An appraisal of least squares programs for the electronic computer from the viewpoint of the user," Journal of the American Statistical Association, vol. 62, no. 319, pp. 819-841, 1967.
[4] NIST, "Statistical Reference Datasets: Longley," National Institute of Standards and Technology, Gaithersburg, MD, USA, public benchmark dataset.
[5] D. W. Marquardt and R. D. Snee, "Ridge regression in practice," The American Statistician, vol. 29, no. 1, pp. 3-20, 1975.
[6] G. H. Golub, M. Heath, and G. Wahba, "Generalized cross-validation as a method for choosing a good ridge parameter," Technometrics, vol. 21, no. 2, pp. 215-223, 1979, doi: 10.1080/00401706.1979.10489751.
[7] R. Tibshirani, "Regression shrinkage and selection via the lasso," Journal of the Royal Statistical Society: Series B, vol. 58, no. 1, pp. 267-288, 1996.
[8] H. Zou and T. Hastie, "Regularization and variable selection via the elastic net," Journal of the Royal Statistical Society: Series B, vol. 67, no. 2, pp. 301-320, 2005, doi: 10.1111/j.1467-9868.2005.00503.x.
[9] B. Efron and R. J. Tibshirani, An Introduction to the Bootstrap. New York, NY, USA: Chapman and Hall, 1993.
[10] A. Belsley, E. Kuh, and R. E. Welsch, Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. New York, NY, USA: Wiley, 1980.
[11] C. Montgomery, E. A. Peck, and G. G. Vining, Introduction to Linear Regression Analysis, 5th ed. Hoboken, NJ, USA: Wiley, 2012.
[12] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, 2nd ed. New York, NY, USA: Springer, 2009.
[13] I. E. Frank and J. H. Friedman, "A statistical view of some chemometrics regression tools," Technometrics, vol. 35, no. 2, pp. 109-135, 1993.
[14] J. Shao, "Linear model selection by cross-validation," Journal of the American Statistical Association, vol. 88, no. 422, pp. 486-494, 1993.
[15] G. C. McDonald, "Ridge regression," Wiley Interdisciplinary Reviews: Computational Statistics, vol. 1, no. 1, pp. 93-100, 2009, doi: 10.1002/wics.14.
[16] Friedman, T. Hastie, and R. Tibshirani, "Regularization paths for generalized linear models via coordinate descent," Journal of Statistical Software, vol. 33, no. 1, pp. 1-22, 2010, doi: 10.18637/jss.v033.i01.
[17] F. Pedregosa et al., "Scikit-learn: Machine learning in Python," Journal of Machine Learning Research, vol. 12, pp. 2825-2830, 2011.
[18] N. R. Draper and H. Smith, Applied Regression Analysis, 3rd ed. New York, NY, USA: Wiley, 1998.
[19] J. Fox and S. Weisberg, An R Companion to Applied Regression, 3rd ed. Thousand Oaks, CA, USA: Sage, 2019.
[20] A. C. Rencher and G. B. Schaalje, Linear Models in Statistics, 2nd ed. Hoboken, NJ, USA: Wiley, 2008.
Downloads
Published
Versions
- 2026-06-21 (2)
- 2026-05-12 (1)
