Regression Analysis

Regression analysis is a family of statistical methods used to model the relationship between one or more independent (predictor) variables and a dependent (outcome) variable. It estimates the strength, direction, and form of these relationships, allowing researchers to predict outcomes, test hypotheses about associations, and control for confounding variables.

Regression analysis is arguably the most widely used analytical framework in quantitative research across virtually all disciplines. It enables researchers to quantify associations between exposures and outcomes while adjusting for confounders, estimate the independent contribution of each predictor, predict outcomes for new observations, and test theoretical models. In clinical research, regression models are used to identify risk factors, develop prediction rules, analyze trial data with covariate adjustment, and evaluate dose response relationships. Understanding regression analysis is essential for both producing and critically appraising quantitative research.

The type of regression model selected depends on the nature of the outcome variable. Linear regression is used when the outcome is continuous, modeling the expected value of the outcome as a linear function of the predictors. Logistic regression is used for binary outcomes (such as disease present versus absent), modeling the log odds of the outcome. Poisson regression models count outcomes (such as the number of hospital admissions). Cox proportional hazards regression models time to event outcomes (such as time to death or disease recurrence). Ordinal and multinomial logistic regression handle categorical outcomes with more than two levels.

In its simplest form, linear regression fits a line through the data that minimizes the sum of squared residuals (differences between observed and predicted values). The regression coefficient for each predictor represents the average change in the outcome associated with a one unit change in that predictor, holding all other predictors constant. The coefficient's sign indicates the direction of the relationship (positive or negative), and its magnitude gives the size of the association in the units of the predictor and the outcome, so coefficients for predictors measured in different units cannot be compared directly.

Model building involves selecting predictors based on theoretical relevance, prior evidence, and statistical criteria. Forward, backward, and stepwise selection procedures automate this process but can produce unstable models and should be used with caution. Theory driven selection, guided by causal diagrams (directed acyclic graphs), is generally preferred. Model diagnostics check assumptions such as linearity, independence of errors, homoscedasticity (constant variance of residuals), and normality of residuals. Violations of these assumptions may require data transformations, alternative models, or resistant estimation methods.

Multicollinearity, the condition where predictor variables are highly correlated with each other, inflates the variance of regression coefficients and makes them unstable. The variance inflation factor (VIF) is used to detect multicollinearity, with values above 5 or 10 indicating concern.

Use regression analysis whenever the research question involves understanding the relationship between predictor variables and an outcome, predicting outcomes, or adjusting for confounders. Select the type of regression based on the outcome variable type: linear for continuous, logistic for binary, Poisson for counts, Cox for time to event. Regression is appropriate for both observational and experimental data, though the causal interpretation of coefficients requires appropriate study design and analytical assumptions.

A common error is including too many predictors relative to the sample size, leading to overfitting and unstable estimates. For logistic and Cox regression, a common rule of thumb is at least 10 events per candidate predictor, but it is only a rule of thumb: Riley and colleagues (2020) show that the sample size needed to develop a prediction model depends on the number of candidate predictor parameters, how common the outcome is and the expected model fit. Another mistake is omitting important confounders from the model, leading to biased estimates (omitted variable bias). Conversely, including mediators (variables on the causal pathway between the exposure and outcome) as covariates can mask or distort the exposure effect. Researchers sometimes interpret regression coefficients as causal effects without adequate consideration of confounding and study design. Finally, reporting only p values without effect estimates and confidence intervals provides incomplete information about the clinical significance of the associations.

A cohort study of 8,500 adults examines risk factors for incident diabetes over 10 years. Multivariable Cox regression yields adjusted hazard ratios of 1.42 (95% CI: 1.18 to 1.71) per 5 kg/m² increase in BMI, 1.28 (95% CI: 1.09 to 1.50) for family history of diabetes, and 0.82 (95% CI: 0.71 to 0.95) per 30 minutes of daily moderate physical activity, after adjusting for age, sex, smoking, alcohol intake, and socioeconomic status. The model's discrimination, assessed by the c statistic, is 0.74.