Generalized Linear Model

A generalized linear model (GLM) is a flexible extension of ordinary linear regression that accommodates dependent variables following non normal distributions from the exponential family, including binomial, Poisson, gamma, and negative binomial distributions. GLMs connect the expected value of the outcome to a linear combination of predictors through a link function, allowing researchers to model binary outcomes, count data, proportions, and other types of response variables within a unified framework.

Not all research outcomes are continuous and normally distributed. Whether a patient recovers or not is binary. The number of adverse events in a hospital is a count. The time until an event occurs may follow an exponential distribution. Ordinary linear regression is inappropriate for these outcomes because it assumes normality, constant variance, and an unbounded response. GLMs provide a principled approach to modeling each of these outcome types while retaining the interpretive framework of regression analysis: coefficients relate predictors to outcomes, and statistical tests evaluate the significance of those relationships. The GLM framework unifies what were once considered separate statistical procedures (linear regression, logistic regression, Poisson regression, log linear models) under a single theoretical umbrella, making it one of the most important developments in applied statistics.

A GLM has three components. The random component specifies the probability distribution of the outcome variable. Common choices include the normal distribution (for continuous outcomes), the binomial distribution (for binary or proportional outcomes), and the Poisson distribution (for count data). The systematic component is the linear predictor: a weighted sum of explanatory variables (predictors) with their regression coefficients. The link function connects the expected value of the outcome to the linear predictor.

Each distribution has a canonical link function: the identity link for normal (the predicted value equals the linear predictor), the logit link for binomial (the log odds of the outcome equals the linear predictor), and the log link for Poisson (the log of the expected count equals the linear predictor). Ordinary linear regression is the special case of a GLM with a normal distribution and an identity link. Other link functions can also be used (for example, the probit link for binary outcomes or the complementary log log link for binary outcomes such as discrete time survival data).

Parameters are estimated using maximum likelihood estimation (or quasi likelihood for overdispersed data), typically through iteratively reweighted least squares (IRLS). Model fit is assessed through deviance (a generalization of the residual sum of squares), the Akaike Information Criterion (AIC), residual diagnostics, and comparison of nested models using likelihood ratio tests or Wald tests.

GLMs should be considered whenever the dependent variable is not continuous or does not meet the assumptions of ordinary linear regression. Logistic regression (a GLM with binomial distribution and logit link) is appropriate for binary outcomes. Poisson regression (a GLM with Poisson distribution and log link) is appropriate for count data. Gamma regression is appropriate for positively skewed continuous outcomes. GLMs are standard in epidemiology, clinical research, ecology, insurance, and any field where outcomes are non normal.

A common error is applying ordinary linear regression to binary or count outcomes, which can produce predicted values outside the valid range (negative counts, probabilities above 1 or below 0) and violate model assumptions. Another mistake is ignoring overdispersion in Poisson models; when the observed variance exceeds the mean (which is common in real data), a negative binomial model or quasi Poisson approach should be used instead. Misinterpreting coefficients is common: in logistic regression, coefficients are log odds ratios, not probabilities or risk differences, and in Poisson regression, exponentiated coefficients are rate ratios. Failing to check model diagnostics (residual plots, influence statistics, Cook's distance) can result in models that fit poorly or are driven by influential observations. Finally, confusing GLMs with generalized linear mixed models (GLMMs), which add random effects for clustered data, is a source of misunderstanding.

An epidemiologist investigates risk factors for emergency department visits among 10,000 adults in a cohort study. The outcome (number of ED visits in one year) is a count variable with many zeros and right skew. A Poisson GLM with a log link models the count as a function of age, sex, insurance status, and number of chronic conditions. The model estimates that, holding the other predictors constant, each additional chronic condition is associated with 1.32 times as many expected ED visits (rate ratio = 1.32, 95% CI: 1.28 to 1.36, from the Poisson fit). A test for overdispersion reveals significant excess variance, so the analyst refits the model as a negative binomial GLM, which produces similar point estimates but wider confidence intervals that allow for the excess variance.