Effect Size

An effect size is a quantitative measure of the magnitude of a phenomenon, the strength of an association between variables, or the size of a difference between groups. Unlike p values, which indicate only whether an effect is statistically significantly different from zero, effect sizes convey how large the effect is, providing information essential for clinical interpretation and practical decision making. Common effect size measures include Cohen's d (standardized mean difference), odds ratio, risk ratio, risk difference, Pearson's r (correlation coefficient), and eta squared (proportion of variance explained).

Statistical significance alone does not indicate whether a finding is meaningful. A study with a very large sample size can detect a statistically significant difference that is trivially small and clinically irrelevant. Effect sizes provide the information needed to evaluate whether a result matters in practice. Clinical guideline developers need effect sizes to assess whether the expected benefit of an intervention is large enough to justify its costs and risks. Researchers conducting meta analyses combine effect sizes across studies to estimate pooled treatment effects. Power analysis requires an expected effect size to calculate the sample size needed for a study. Journal guidelines such as the American Psychological Association's reporting standards for quantitative articles (JARS-Quant) ask authors to report effect size estimates and confidence intervals for each inferential test where possible, and some journals require effect sizes as a condition of publication.

Effect sizes fall into three broad families. The d family measures standardized differences between means. Cohen's d divides the difference between two group means by the pooled standard deviation, producing a unit free measure that allows comparison across studies using different measurement scales. Because the standard deviation is the denominator, the same raw difference produces a larger d in a sample with little variability, such as a narrowly selected one, than in a more varied sample. Hedges' g applies a small sample correction to Cohen's d. By convention, d values of 0.2 are considered small, 0.5 medium, and 0.8 large. Cohen offered these labels with a caution, as a fallback for when no better basis exists; what counts as a meaningful effect varies across fields and should be judged in context.

The r family measures associations between variables. Pearson's r ranges from negative 1 (perfect negative correlation) to positive 1 (perfect positive correlation). R squared indicates the proportion of variance in one variable explained by another. Eta squared and partial eta squared measure the proportion of variance explained by a factor in ANOVA designs. By convention, r values of 0.1 are considered small, 0.3 medium, and 0.5 large.

The odds ratio family measures the strength of association between a binary exposure and a binary outcome. An odds ratio of 1.0 indicates no association. An odds ratio of 2.0 means the odds of the outcome are twice as high in the exposed group. Risk ratios (relative risks) compare probabilities rather than odds and are often more interpretable clinically. Risk differences (absolute risk reductions) express the difference in event rates between groups, providing information about the clinical impact in absolute terms.

The choice of effect size measure depends on the study design, the type of data, and the analytical question. Effect sizes can be converted approximately between families using standard formulas, which rest on assumptions about the underlying data (for example, that a binary outcome reflects an underlying continuous distribution), enabling comparison across studies that use different measures.

Effect sizes should be reported alongside confidence intervals in quantitative studies that estimate differences or associations. They are essential for power analysis during study planning, for clinical interpretation of results, and for meta analysis. They are particularly important when comparing results across studies that use different outcome measures, as standardized effect sizes enable comparison on a common scale. Effect sizes are not interpretable in isolation; they should be considered alongside confidence intervals (which indicate the precision of the estimate) and the clinical context (which determines what magnitude of effect is meaningful).

A common mistake is reporting p values without effect sizes, or interpreting a statistically significant p value as evidence of a clinically important effect. Another error is applying Cohen's conventional benchmarks (small, medium, large) without considering whether these benchmarks are appropriate in the specific research context. In some fields, an effect at or even below the "small" benchmark can have enormous practical significance (the effect of aspirin on heart attack prevention in the Physicians' Health Study corresponds to a correlation of only about 0.03, well below the small benchmark of 0.1 for r), while in others, a "large" effect size of 0.8 may not be clinically meaningful. Confusing odds ratios with risk ratios is another common mistake; when the event is common, odds ratios lie further from 1 than the corresponding risk ratios, exaggerating the apparent strength of association. Reporting standardized effect sizes without reporting the raw (unstandardized) difference in meaningful units deprives readers of information needed for clinical interpretation. Finally, calculating effect sizes from studies with high risk of bias produces estimates that may not reflect the true treatment effect, regardless of their magnitude.

A meta analysis of cognitive behavioral therapy (CBT) for generalized anxiety disorder pools data from 24 randomized trials. The pooled standardized mean difference for anxiety symptoms (measured by various validated scales) is d = 0.82 (95% CI: 0.65 to 0.99) compared to waitlist controls, and d = 0.38 (95% CI: 0.22 to 0.54) compared to active controls (relaxation therapy). The large effect relative to waitlist and the smaller effect relative to active treatment (between Cohen's small and medium benchmarks) help clinicians understand that CBT outperforms no treatment substantially but offers a more modest advantage over other active therapies.