Egger Test

The Egger test is a statistical method for detecting small study effects and potential publication bias in meta analysis. It uses linear regression of the standardized effect estimates against their precision (the inverse of the standard error), which is equivalent to a weighted regression of the effect estimates on their standard errors, to test whether the regression intercept deviates significantly from zero. A significant Egger test suggests funnel plot asymmetry, which may indicate that smaller, less precise studies tend to report larger effects, a pattern consistent with publication bias or other sources of small study effects.

Publication bias, the tendency for studies with statistically significant or favorable results to be published more often than those with null or unfavorable results, is a major threat to the validity of meta analyses. If published studies represent a biased sample of all conducted studies, the pooled effect estimate in a meta analysis will tend to overestimate the true effect. The Egger test provides a formal statistical assessment of funnel plot asymmetry, complementing the visual inspection of funnel plots with a quantitative test. It is one of the most widely used and cited methods for assessing publication bias in systematic reviews. The Cochrane Handbook describes it as the first and most well known test for funnel plot asymmetry, and PRISMA 2020 asks reviews to report the methods they used to assess the risk of bias due to missing results, though neither requires the Egger test as such.

The test regresses the standardized effect size (the effect estimate divided by its standard error) against the precision (1 divided by the standard error) across all studies in the meta analysis. In the absence of small study effects, the regression line should pass through the origin (intercept of zero), because the standardized effect should then be proportional to precision, with the slope estimating the underlying effect. If smaller, less precise studies systematically report larger effects, the intercept moves away from zero, and an intercept significantly different from zero indicates asymmetry.

The test produces a regression coefficient (the intercept), its standard error, and a p value for the hypothesis that the intercept equals zero. A p value below 0.10 (a threshold commonly used because the test has limited statistical power) is typically taken as evidence of asymmetry. Variations of the test have been developed for different types of data. The original Egger test suits effect measures such as mean differences, but the Cochrane Handbook does not recommend it for odds ratios or standardized mean differences, because these estimates are naturally correlated with their standard errors, which can produce spurious asymmetry. For odds ratios, the Peters test and the Harbord test are modifications designed to overcome this problem.

It is critical to understand that the Egger test detects funnel plot asymmetry, which may be caused by publication bias but also by other factors. True heterogeneity (real differences in treatment effects across studies of different sizes), differences in methodological quality (smaller studies may be less rigorous), and chance (especially when the number of studies is small) can all produce asymmetry. Therefore, a significant Egger test should prompt investigation of possible explanations, not an automatic conclusion of publication bias.

The Egger test should be used as part of a comprehensive assessment of publication bias in meta analyses of continuous outcomes measured as mean differences. It is most informative when the meta analysis includes at least 10 studies, as the test has low statistical power with fewer studies. It should be used alongside visual inspection of funnel plots and, when applicable, other methods such as trim and fill, selection models, or the p curve approach. It is not appropriate for meta analyses with very few studies (fewer than 10), for meta analyses whose studies are all of similar size (similar standard errors), for odds ratios without modification (where the Peters or Harbord tests should be used instead), for standardized mean differences, or as the sole basis for concluding that publication bias exists.

A common error is interpreting a significant Egger test as proof of publication bias without considering alternative explanations such as true heterogeneity or methodological differences between small and large studies. Conversely, a nonsignificant Egger test does not rule out publication bias, particularly when the number of studies is small and the test lacks power. Applying the original Egger test to odds ratios or standardized mean differences, whose estimates are naturally correlated with their standard errors, produces inflated false positive rates; for odds ratios, the Peters or Harbord modifications avoid this. Some researchers report the Egger test but do not report or discuss the funnel plot, depriving readers of the visual context needed to interpret the result. Running the Egger test on meta analyses with fewer than 10 studies produces unreliable results, and authors should acknowledge this limitation rather than reporting the test as definitive. Finally, some researchers use the Egger test on meta analyses where the included studies are known to have other systematic differences (such as study quality), where asymmetry is expected for reasons unrelated to publication bias.

A meta analysis of 22 randomized trials evaluating a dietary supplement for cognitive function in older adults produces a pooled mean difference of 0.35 points (95% CI: 0.22 to 0.48) on a cognitive test scale that every trial used. The funnel plot appears asymmetric, with several small studies reporting large positive effects and no small studies reporting null or negative effects. The Egger test yields an intercept of 2.14 (SE = 0.72, p = 0.007), consistent with small study effects. The review team investigates further, finding that the small studies with the largest effects also had the highest risk of bias, suggesting that methodological quality rather than (or in addition to) selective publication may explain the asymmetry. They conduct sensitivity analyses excluding high risk of bias studies, which reduces the pooled estimate to 0.18 points (95% CI: 0.03 to 0.33).