Missing Data

Missing data are observations or measurements that are absent from a dataset, arising from a variety of causes including participant dropout, nonresponse to survey items, equipment malfunction, data entry errors, or administrative gaps in record systems. The presence of missing data reduces statistical power and, depending on the mechanism of missingness, can introduce bias into study results.

Missing data are present in virtually every research dataset, yet their handling is often given insufficient attention. The consequences of missing data depend on how much data are missing and why they are missing. Even small amounts of missing data can introduce substantial bias if the missing values are systematically different from the observed values. For example, in a clinical trial, patients who drop out due to adverse effects are likely experiencing worse outcomes than those who remain, and ignoring their missing data can produce an overly optimistic estimate of the treatment effect. Regulatory agencies, journal editors, and systematic reviewers expect researchers to report the amount and pattern of missing data, describe the assumed missing data mechanism, and use appropriate analytic methods.

Missing data are classified into three mechanisms based on Little and Rubin's framework. Data missing completely at random (MCAR) occur when the probability of being missing is unrelated to any observed or unobserved variable. Under MCAR, complete case analysis produces unbiased estimates but loses power. Data missing at random (MAR) occur when the probability of being missing depends on observed variables but not on the missing values themselves, after conditioning on those observed variables. Under MAR, methods such as multiple imputation and maximum likelihood estimation can produce valid estimates, provided the imputation or analysis model is appropriate, while complete case analysis can be biased. Data missing not at random (MNAR) occur when the probability of being missing depends on the unobserved values themselves (for example, patients with the worst outcomes are the most likely to drop out). MNAR is the most problematic mechanism and requires sensitivity analyses or specialized models.

Several strategies address missing data. Prevention is the first line of defense: well designed studies minimize missing data through careful planning, participant engagement, and follow up protocols. When data are missing, analytic approaches include complete case analysis (analyzing only participants with complete data, valid under MCAR), available case analysis (using all available data for each analysis), multiple imputation (generating multiple plausible datasets and combining results), maximum likelihood estimation (using all observed data directly, such as in mixed models for repeated measures), and pattern mixture or selection models (for MNAR data).

The choice of method should be guided by the assumed missing data mechanism, the amount and pattern of missing data, and the study design. Sensitivity analyses that vary the assumptions about the missing data mechanism (for example, assuming MNAR after primary analyses assume MAR) are essential for assessing the influence of missing data on conclusions.

Assess and address missing data in every quantitative study. Report the proportion of missing data for each variable and for the primary analysis. Investigate whether missingness is associated with observed characteristics: an association shows that the data are not MCAR, but observed data cannot distinguish MAR from MNAR, so MAR remains an assumption. Choose an analytic approach appropriate to the assumed mechanism and conduct sensitivity analyses to test the robustness of results. Use multiple imputation or likelihood based methods as the primary approach when data are assumed MAR, and supplement with MNAR sensitivity analyses when dropout or nonresponse may be related to the unobserved outcomes.

A common mistake is ignoring missing data entirely and analyzing only complete cases without assessing whether this introduces bias. Another error is using simple imputation methods (mean imputation, LOCF) that can produce biased estimates or underestimate standard errors. Some researchers report the amount of missing data but fail to investigate the mechanism or justify their analytic approach. Performing sensitivity analyses only after a reviewer requests them, rather than including them in the original analysis plan, suggests that missing data were an afterthought rather than a planned aspect of the analysis. Finally, leaving the outcome variable out of the imputation model when imputing missing covariates can bias their estimated associations with the outcome toward zero, so the imputation model should include the outcome and, where they predict the missing values, auxiliary variables.

A randomized trial of a new treatment in 1,500 patients with heart failure measures quality of life using the Kansas City Cardiomyopathy Questionnaire (KCCQ) at baseline, 6 months, and 12 months. At 12 months, 22% of patients have missing KCCQ data: 8% died, 9% withdrew, and 5% missed the assessment for unknown reasons. Because quality of life cannot be measured after death, the 8% who died are handled by a prespecified strategy, such as reporting death alongside quality of life, rather than treated like the other missing values. Analysis of baseline characteristics shows that patients with missing 12 month data had higher baseline NYHA class and lower baseline KCCQ scores, so the data are not MCAR; MAR given these observed variables is a plausible working assumption, although observed data cannot rule out MNAR. The primary analysis uses a mixed model for repeated measures (MMRM) that includes all patients with at least one post baseline observation. Sensitivity analysis using a pattern mixture model under an MNAR assumption (assuming that missing patients in the treatment group had KCCQ scores 10 points lower than predicted) shows that the treatment effect remains statistically significant but is attenuated, in this illustrative scenario by approximately 20%.