INVESTIGATING THE IMPACT OF MULTICOLLINEARITY ON LINEAR REGRESSION ESTIMATES
DOI:
https://doi.org/10.24191/mjoc.v6i1.10540Keywords:
Lasso & Elastic, Multicollinearity, RidgeAbstract
The study was to investigate the impact of multicollinearity on linear regression estimates. The study was guided by the following specific objectives, (i) to examine the asymptotic properties of estimators and (ii) to compare lasso, ridge, elastic net with Ordinary Least Squares (OLS). The study employed Monte-Carlo simulation to generate set of highly collinear and induced multicollinearity variables with sample sizes of 25, 50, 100, 150, 200, 250, 1000 as a source of data in this research work and the data was analyzed with lasso, ridge, elastic net and ordinary least squares using statistical package. The study findings revealed that absolute bias of ordinary least squares was consistent at all sample sizes as revealed by past researched on multicollinearity as well while lasso type estimators fluctuated alternately. Also revealed that, mean square error of ridge regression outperformed other estimators with minimum variance at small sample size and OLS was the best at large sample size. The study recommended that OLS was asymptotically consistent at a specified sample sizes on this research work and ridge regression was efficient at small and moderate sample size.
References
Chenlei, L., Yi, L. and Grace W. (2006). A note on the LASSO and related procedures in model selection. Published by Institute of Statistical Science, Academia Sinical. Pp.1273-1284.
Esra, P and Semra, T. (2016). The comparison of classical and Robust Biased Regression methods for determining unemployment rate in Turkey. Journal of data science: JDS, pp 739-768.
Esra, P and Suleyman, G. (2015). The comparison of partial least Squares Regression, Principal Component Regression and Ridge Regression with Multiple Linear Regression for predicting PM10 concentration level Based on Meteorological parameter. Journal of Data science, 663-692.
Gujrati, D. N. (2004). Basic econometrics 4th edition, Tata McGraw-Hill, New Delhi. McGraw Companies
Hawking, R. R. and Pendleton, O. J. (1983). The regression dilemma, Commun. Stat. Theo. Meth, 12, 497-527.
Hoerl, A. E. and Kennard, R. W. (1970). Ridge regression: Biased estimation for Nonorthogonal problems. Technometrics, 12, 55-67.
Hoerl, A. E. and Kennard, R. W. (1976). Ridge regression: Iterative estimation of biasing parameter, Commun. Stat. Theo. Meth, 5, 77-88.
Hoerl, A. E., Kennard, R. W. and Baldwin, K. F. (1975). Ridge regression: some simulations, Commun. Stat. Theo. Meth., 4, 105-123.
Zakari Y., Yau S. A. and Usman, U. (2018). Handling multicollinearity; A comparative study of the prediction performance of some methods based on some probability distribution. Annals. Computer Science Series pp 15-21.
Published
Issue
Section
License
Copyright (c) 2021 Adewoye Kunle Bayo, Ayinla Bayo Rafiu, Aminu Titilope Funmilayo, Onikola Isaac Oluyemi
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
