The Importance of Beta, the Type II Error, and Sample Size in the Design and Interpretation of the Randomized Controlled Trial

Abstract

Seventy-one "negative" randomized controlled trials were reexamined to determine whether the investigators had studied large enough samples to give a high probability (>0.90) of detecting a 25 percent and a 50 percent therapeutic improvement in the response. Sixty-seven of the trials had a greater than 10 percent risk of missing a true 25 percent therapeutic improvement, and with the same risk, 50 of the trials could have missed a 50 percent improvement. Estimates of 90 percent confidence intervals for the true improvement in each trial showed that in 57 of these "negative" trials, a potential 25 percent improvement was possible, and 34 of the trials showed a potential 50 percent improvement. Many of the therapies labeled as "no different from control" in trials using inadequate samples have not received a fair test. Concern for the probability of missing an important therapeutic improvement because of small sample sizes deserves more attention in the planning of clinical trials. A follow-up of this study 10 years later, this time including 65 "negative" randomized controlled trials from 1988, revealed no essential improvement in recognition of the importance of beta.