Power (Statistical Power)

Statistical power is the probability that a study's statistical test will correctly detect a true effect when one exists, calculated as one minus the probability of a Type II error (beta). By convention, a power level of 80% or higher is considered the minimum acceptable threshold for most research designs. Power represents the sensitivity of a study to the effect it is trying to detect.

Studies with inadequate statistical power are unlikely to detect real effects, producing null results that may be mistakenly interpreted as evidence that no effect exists. This wastes research resources, exposes participants to risk without generating useful knowledge, and contributes to the problem of research waste. Conversely, understanding power helps researchers interpret published findings: a null result from an underpowered study does not provide strong evidence against an effect. Funding agencies and ethics review boards commonly expect a justified sample size, usually based on a power calculation, as part of study approval, and reporting guidelines such as CONSORT ask trial reports to explain how the sample size was determined. Power calculations also drive budget and timeline decisions, as larger sample sizes increase costs and recruitment timelines.

Statistical power is determined by the interplay of four factors: sample size, effect size, significance level (alpha), and variability in the outcome measure. Each of these can be adjusted to influence power, though practical and ethical constraints limit the options.

Sample size has the most direct relationship with power. As sample size increases, the standard error of the estimate decreases, making it easier to distinguish a true effect from random variation. This is why power analysis is typically used to determine the required sample size for a given study design. Effect size, which quantifies the magnitude of the difference or association being detected, also strongly influences power. Large effects are easier to detect and require smaller samples; small effects require much larger samples.

The significance level (alpha), conventionally set at 0.05, defines the threshold for rejecting the null hypothesis. Relaxing alpha (for example, to 0.10) increases power but also increases the risk of Type I errors. Variability in the outcome measure affects power by influencing the signal to noise ratio: higher variability in the data makes it harder to detect a given effect size and requires a larger sample to achieve the same power. Researchers can reduce variability through more precise measurement instruments, more homogeneous study populations, or designs that control for known sources of variation such as stratification or repeated measures.

Power can be computed prospectively (before the study) to plan sample size, or retrospectively (after the study) to understand what the study was capable of detecting. However, retrospective or post hoc power calculations are widely considered uninformative and misleading, because observed power is a direct function of the observed p value and adds no new information. The appropriate use of power analysis is prospective, during study planning.

Compute power during study design, before data collection begins, as part of the protocol development process. Power analysis is essential for any study that aims to detect a specific effect, including randomized trials, cohort studies, and survey research. It is not directly applicable to purely exploratory or qualitative studies. When reviewing published research, consider the study's power when interpreting null results: a nonsignificant finding from a study with 30% power tells you much less than a nonsignificant finding from a study with 95% power.

A common error is conducting a retrospective power analysis after a study finds a nonsignificant result and using the observed effect size to compute "observed power." This practice is circular and uninformative, as the observed power is determined entirely by the p value. A second frequent mistake is using an unrealistically large effect size in the power calculation to justify a small, inexpensive study. When the true effect is smaller than assumed, the study ends up underpowered and produces inconclusive results. Researchers also sometimes forget to account for attrition, noncompliance, or clustering effects, all of which reduce effective sample size and therefore power.

A research team plans a parallel group randomized trial comparing a new cognitive behavioral therapy protocol against standard care for insomnia. Based on prior literature, they expect a moderate effect size (Cohen's d = 0.50) and set alpha at 0.05 with 80% power. The power calculation indicates 64 participants per group. Anticipating 20% attrition, they plan to enroll 80 per group, for a total of 160 participants. If they could only recruit 40 per group, the study's power would drop to approximately 60%, so a null result would be weak evidence against an effect of that size: a trial of that size would miss a true effect of d = 0.50 about 4 times in 10.