Bayesian Variance Estimation for Meta-Analysis

It is argued that residual variance estimates are often based on an unrealistic model of the sampling distribution of residual variance, and alternative Bayesian techniques for estimation are reviewed that avoid these problems and provide simulation results demonstrating the superiority of the Bayesian approach.

Abstract

A primary goal in meta-analysis is determining the variance across a set of correlations after taking into account statistical and psychometric artifacts. If the residual variance is large, substantive moderators of the relationship likely exist; if there is little residual variance, the meta-analytic estimate of the effect size is expected to generalize across multiple settings. Surprisingly little attention has been directed toward some critical shortcomings of traditional methods for estimating residual variance. In this article, the authors argue that residual variance estimates are often based on an unrealistic model of the sampling distribution of residual variance. The authors review alternative Bayesian techniques for estimation that avoid these problems and provide simulation results demonstrating the superiority of the Bayesian approach.