Multicollinearity

Multicollinearity is a statistical condition that occurs when two or more independent variables in a regression model are highly correlated with each other. This high correlation makes it difficult to determine the individual contribution of each predictor to the dependent variable, inflating the standard errors of the regression coefficients and producing unstable, unreliable estimates.

Regression analysis is the workhorse of quantitative research, and multicollinearity is one of its most common and insidious problems. When present, it does not bias the overall model predictions, but it destabilizes the individual coefficient estimates, making them sensitive to small changes in the data or model specification. This means that the estimated effect of a single predictor can swing dramatically, or even change sign, depending on which other predictors are included in the model. For researchers seeking to understand the unique contribution of each variable, or to make claims about which factors matter most, multicollinearity undermines the interpretability and reliability of their results.

Multicollinearity arises whenever two or more predictors share substantial information. At the extreme, perfect multicollinearity (one predictor is an exact linear function of another) makes the regression model mathematically unsolvable. More commonly, high but imperfect correlation among predictors produces estimates that are technically valid but practically unreliable.

The variance inflation factor (VIF) is a widely used diagnostic for multicollinearity. It is calculated for each predictor as 1 divided by (1 minus R²ⱼ), where R²ⱼ is the coefficient of determination from regressing the jth predictor on all other predictors. A VIF of 1 indicates that the predictor has no linear association with the other predictors (R²ⱼ = 0). The square root of the VIF is the factor by which that predictor's standard error is larger than it would be if the predictor were uncorrelated with the others, so a VIF of 4 doubles the standard error. Commonly cited rules of thumb treat VIF values above 5 as moderate multicollinearity and values above 10 as serious multicollinearity that warrants attention. These are conventions rather than strict cutoffs: whether a given VIF matters also depends on the sample size and on how precisely the coefficient needs to be estimated (O'Brien, 2007). The tolerance (1 divided by VIF, which equals 1 minus R²ⱼ) is the reciprocal measure; values below 0.10, the counterpart of a VIF above 10, are commonly treated as concerning.

The condition number of the predictor matrix provides a global assessment of multicollinearity across all predictors. Values above 30 are commonly taken to signal serious multicollinearity, a rule of thumb from Belsley, Kuh and Welsch rather than a strict cutoff. Examining the correlation matrix of predictors is a useful first step, but pairwise correlations alone can miss multicollinearity involving three or more variables (for example, X₃ being a near perfect linear combination of X₁ and X₂ even though neither pairwise correlation is extreme).

Remedies for multicollinearity include removing one of the highly correlated predictors, combining correlated predictors into a single composite variable (such as a principal component), centering predictors (subtracting their means, which reduces the multicollinearity between a predictor and its own interaction or polynomial terms, though it does not change the model's fit or predictions), collecting more data (which reduces standard errors but does not eliminate the correlation), and using ridge regression or other regularization methods that shrink coefficients toward zero at the cost of introducing a small amount of bias.

Check for multicollinearity whenever building a regression model with multiple predictors. It is a diagnostic procedure rather than a method, and it should be performed before interpreting individual regression coefficients. Pay particular attention when including predictors that are conceptually related (such as height and weight), when including multiple indicators of the same construct, or when including polynomial terms or interaction terms alongside their component variables.

A common error is ignoring multicollinearity and interpreting individual coefficients as if they were reliable, when in fact they are unstable. Another mistake is diagnosing multicollinearity based only on the pairwise correlation matrix, which can miss collinearity among three or more predictors. Some researchers respond to multicollinearity by dropping predictors without considering the theoretical implications, removing variables that may be important confounders. Overcorrecting for multicollinearity by combining predictors that should remain separate can also obscure meaningful distinctions. Finally, confusing multicollinearity (a problem of interpretation of individual coefficients) with model failure (the overall model can still predict well despite multicollinearity) leads to unnecessary alarm.

A health researcher models hospital readmission risk using seven predictors: age, number of comorbidities, Charlson comorbidity index, length of stay, number of medications, number of procedures, and insurance type. VIF calculations reveal that "number of comorbidities" (VIF = 12.3) and "Charlson comorbidity index" (VIF = 14.1) are highly collinear because the Charlson index is partly derived from the number of comorbidities. The researcher retains only the Charlson index (which is a more comprehensive measure) and reruns the model. After removing the redundant predictor, all VIFs are below 3, and the remaining coefficients are stable across model specifications.