Efficacy of Statistical Outlier Analysis for Monitoring Quality of Care

Monte Carlo methods are used to evaluate the performance of the outlier technique as parameters of the true mortality process are varied and indicate that the screening ability of the technique may be very sensitive to how widespread quality-related mortality is among hospitals but insensitive to other factors generally thought to be important.

Abstract

Researchers have proposed that hospitals with excessive statistically unexplained mortality rates are more likely to have quality-of-care problems. The U.S. Health Care Financing Administration currently uses this statistical “outlier” approach to screen for poor quality in hospitals. Little is known, however, about the validity of this technique, since direct measures of quality are difficult to obtain. We use Monte Carlo methods to evaluate the performance of the outlier technique as parameters of the true mortality process are varied. Results indicate that the screening ability of the technique may be very sensitive to how widespread quality-related mortality is among hospitals but insensitive to other factors generally thought to be important.