Quasi Experimental Design

A quasi experimental design is a research design that shares features with true experimental designs, such as the manipulation of an independent variable, but does not use random assignment to allocate participants to treatment and control conditions. Common quasi experimental designs include the pre post design, nonequivalent control group design, interrupted time series, regression discontinuity, and difference in differences analysis.

In many real world settings, randomization is impractical, unethical, or impossible. Policy evaluations, educational interventions, organizational changes, and public health programs often cannot randomly assign individuals or communities to treatment and control conditions. Quasi experimental designs fill this gap by providing methods that can approximate causal inference without randomization, albeit with important limitations. Understanding the strengths and threats to validity of different quasi experimental designs is essential for researchers who must work with nonexperimental data and for readers who must judge the credibility of the resulting evidence.

Quasi experimental designs differ from true experiments primarily in the absence of random assignment. In many evaluations, such as of a new law or policy, the intervention is introduced by a government or organization rather than by the researcher (a natural experiment), and the same designs are used to analyze it. Without randomization, the treated and comparison groups may differ in ways that confound the treatment effect, making it difficult to attribute observed differences to the intervention. Each quasi experimental design addresses this challenge differently.

The nonequivalent control group design includes a comparison group that was not randomly assigned. Researchers attempt to select a group that is similar to the treatment group and use statistical adjustments (such as analysis of covariance, propensity score methods, or difference in differences) to account for observed baseline differences. The plausibility of the causal inference depends on how comparable the groups are and whether unmeasured confounders exist.

The interrupted time series design examines a series of measurements taken before and after an intervention to assess whether the intervention changed the level or trend of the outcome. By modeling the pre intervention trend, the design can separate the intervention effect from ongoing temporal patterns. It is particularly useful for evaluating policy changes or system level interventions that affect an entire population simultaneously.

The regression discontinuity design assigns treatment based on whether an individual falls above or below a cutoff on a continuous assignment variable. Because the assignment mechanism is known (and, in a sharp design, deterministic), the design produces causal estimates that are considered among the strongest of all quasi experimental methods, though only for individuals near the cutoff.

The difference in differences design compares the change in outcomes over time between a treatment group and a control group, accounting for both time invariant group differences and common temporal trends. It is widely used in health policy and economics.

Use quasi experimental designs when randomization is not feasible but causal inference is the goal. The choice of specific design depends on the context: use interrupted time series when the intervention affects an entire population at a known time point, regression discontinuity when treatment assignment is based on a cutoff score, and nonequivalent control groups when a comparable untreated group can be identified. Quasi experimental designs are appropriate for evaluating natural experiments, policy changes, and organizational interventions. They should not be used when a randomized trial is feasible and ethical, as randomization provides stronger causal evidence.

A common error is failing to consider the specific threats to validity that each quasi experimental design is and is not able to address. For example, a pre post design without a control group cannot rule out maturation or historical confounding, while a nonequivalent control group design cannot rule out unmeasured confounders. Another mistake is treating the results of a quasi experimental study as equivalent to those of a randomized trial without discussing the limitations. Researchers sometimes fail to check the assumptions underlying their chosen design, such as the parallel trends assumption in difference in differences, which cannot be tested directly but can be probed by comparing pre intervention trends. Finally, selecting a comparison group that differs from the treatment group in important ways undermines the validity of the causal inference.

A state implements a new prescription drug monitoring program (PDMP) to reduce opioid prescribing. Researchers use a difference in differences design, comparing the change in opioid prescribing rates in the implementing state (treatment) versus three neighboring states without the program (control) over the two years before and two years after implementation. The analysis controls for state and time fixed effects and finds that the PDMP is associated with a 12% reduction in opioid prescriptions (95% CI: 8% to 16%) relative to the control states. The parallel trends assumption is supported by similar pre intervention trajectories in treatment and control states.