On the robustness, bias, and stability of statistics from meta-analysis of correlation coefficients: Some initial Monte Carlo findings.

Abstract

Each of several Monte Carlo simulations generated 100 sets of observed study correlations based on normal, heteroscedastic, or slightly nonlinear bivariate distributions, with one population correlation coefficient and true variance of 0. A version of J. E. Hunter and F. L. Schmidt's (1990b) meta-analysis was applied to each study set. Within simulations, ρ was accurate on average. σ ρ 2 was biased; one would correctly conclude more than half the time that no moderator effects existed. However, cases of variation in ρ and especially in σ ρ 2 indicated that results from individual meta-analyses could deviate substantially from what was found on average. Findings for these no-moderator cases offer applied psychologists some guidelines and cautions when drawing conclusions about true population correlations and true moderator effects (e.g., situational specificity, validity generalization) from meta-analytic results.