Testing and adjusting for publication bias
Trends in Ecology & EvolutionPublished 1 October 2001Open access
Anders Pape Møller, Michael D. Jennions
Citations422
SJR quartileQ1
SJR score4.88
SNIP4.51
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
Abstract
How can the scientific literature provide unbiased conclusions if it represents a biased sample of available studies? Publication bias refers to phenomena arising from bias in submitting, reviewing, accepting and publishing scientific results. Direct and
Keywords
Decision SciencesHealth Professions
BMJBias in meta-analysis detected by a simple, graphical test
56,268 Citations1997Matthias Egger, George Davey Smith +2 more
Funnel plots, plots of the trials' effect estimates against sample size, are skewed and asymmetrical in the presence of publication bias and other biases Funnel plot asymmetry, measured by regression analysis, predicts discordance of results when meta-analyses are compared with single large trials.
BiometricsOperating Characteristics of a Rank Correlation Test for Publication Bias
17,035 Citations1994Colin B. Begg, Madhuchhanda Mazumdar
An adjusted rank correlation test is proposed as a technique for identifying publication bias in a meta-analysis, and its operating characteristics are evaluated via simulations, and the test statistic is a direct statistical analogue of the popular "funnel-graph".
BiometricsTrim and Fill: A Simple Funnel‐Plot–Based Method of Testing and Adjusting for Publication Bias in Meta‐Analysis
13,841 Citations2000Sue Duval, Richard L. Tweedie
These are simple rank-based data augmentation techniques, which formalize the use of funnel plots, which provide effective and relatively powerful tests for evaluating the existence of publication bias.
Journal of Educational StatisticsStatistical Methods for Meta-Analysis
10,479 Citations1988Stanley Wasserman, Larry V. Hedges +1 more
Psychological BulletinThe file drawer problem and tolerance for null results.
8,595 Citations1979Robert Rosenthal
Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed.
Journal of the Royal Statistical Society Series D (The Statistician)Meta-Analytic Procedures for Social Research.
4,283 Citations1994Andrew Metcalfe, R. Rosenthal
Journal of Educational StatisticsSumming up: The Science of Reviewing Research
1,490 Citations1987Ian P. Wilkinson, Richard J. Light +1 more
JAMAThe Existence of Publication Bias and Risk Factors for Its Occurrence
1,345 Citations1990Kay Dickersin
Cognitive Therapy and ResearchPublication prejudices: An experimental study of confirmatory bias in the peer review system
1,030 Citations1977M. J. Mahoney
The LancetLanguage bias in randomised controlled trials published in English and German
1,026 Citations1997Matthias Egger, Tanja Zellweger-Zähner +4 more
English language bias may be introduced in reviews and meta-analyses if they include only trials reported in English if authors are more likely to report trials with statistically significant results in English than in German.
EcologySTATISTICAL ISSUES IN ECOLOGICAL META-ANALYSES
1,020 Citations1999Jessica Gurevitch, Larry V. Hedges
A number of considerations related to choosing methods for the meta-analysis of ecological data, including the choice of parametric vs. resampling methods, reasons for conducting weighted analyses where possible, and comparisons fixed vs. mixed models in categorical and regression-type analyses are outlined.
BMJPublication bias: evidence of delayed publication in a cohort study of clinical research projects
829 Citations1997Jerome M Stern, R. J. Simes
Controlled Clinical TrialsPublication bias and clinical trials
721 Citations1987Kay Dickersin, Simon Chan +3 more
It appears that nonpublication was primarily a result of failure to write up and submit the trial results rather than rejection of submitted manuscripts, implying the existence of a publication bias of importance both to meta-analysis and the interpretation of statistically significant positive trials.
Trends in Ecology & EvolutionMeta-analysis: synthesizing research findings in ecology and evolution
571 Citations1995Göran Arnqvist, David E. Wooster
The growing number of empirical studies performed in ecology and evolution creates a need for quantitative summaries of research domains to generate higher-order conclusions about general trends and patterns.
JAMAUnderreporting Research Is Scientific Misconduct
556 Citations1990lain Chalmers
An extended use of prospective registration of trials at inception, as well as benefiting clinical research in other ways, could help people to play their respective roles in reducing underreporting of clinical trials.
Journal of Clinical EpidemiologySelecting the language of the publications included in a meta-analysis: Is there a tower of babel bias?
534 Citations1995Geneviève Grégoire, François Derderian +1 more
This study demonstrates that, in at least one out of 36 consecutive meta-analyses the exclusion of papers for linguistic reasons produced results different from those which would have been obtained if this exclusion criteria had not been used.
Health Technology AssessmentThe effectiveness and cost-effectiveness of prophylactic removal of wisdom teeth.
480 Citations2000Ping Song, O'Meara +3 more
The results of this trial suggested that the removal of third molars to prevent late incisor crowding cannot be justified, and preliminary results indicate that watchful waiting may be a promising strategy.
Annals of the New York Academy of SciencesPublication Bias: The Problem That Won't Go Away
429 Citations1993Kay Dickersin, Yuan‐I Min
Meta-analyses performed using data from four prospective investigations in which a total of 997 initiated studies were followed to learn of study results, publication status, and reasons for nonpublication indicate that there is a positive association between "significant" study results and publication.
Health Technology AssessmentPublication and related biases.
409 Citations2000Song, Eastwood +3 more
This review aims to identify systematically and appraise studies that have examined methodological issues and provide empirical evidence about publication bias and other dissemination-related biases, including biases due to the time, type and language of publication, multiple publication, selective citation of references, database index bias, and biased media attention.
Annual Review of Ecology and SystematicsQuasi-Replication and the Contract of Error: Lessons from Sex Ratios, Heritabilities and Fluctuating Asymmetry
397 Citations2000A. Richard Palmer
Funnel graphs of published results suggest selective reporting is widespread and raise doubts about the true magnitude of these phenomena, including sex-ratio variation in birds, field estimates of heritabilities, and relations between fluctuating asymmetry and individual attractiveness or fitness.
The Quarterly Review of BiologySexually Selected Traits and Adult Survival: A Meta-Analysis
395 Citations2001Michael D. Jennions, A. P. Møller +1 more
In general, males with larger ornaments or weapons, greater body size, or higher rates of courtship showed greater survivorship or longevity, suggesting that male investment in sexually selected traits is not fixed but varies in relation to the ability to pay the underlying costs of expressing these characters.
JAMAEffect on the Quality of Peer Review of Blinding Reviewers and Asking Them to Sign Their Reports
336 Citations1998Fiona Godlee, Catharine R. Galé +1 more
Neither blinding reviewers to the authors and origin of the paper nor requiring them to sign their reports had any effect on rate of detection of errors, and such measures are unlikely to improve the quality of peer review reports.
JAMAUS and Non-US Submissions
321 Citations1998Ann Marie Link
To assess whether US reviewers or non-US reviewers evaluate manuscripts differently, depending on whether the manuscripts are submitted from outside the United United States or from the United States, a retrospective analysis of all original submissions received by Gastroenterology in 1995 and 1996 is analyzed.
BMJCholesterol lowering trials in coronary heart disease: frequency of citation and outcome.
313 Citations1992Uffe Ravnskov
Lowering serum cholesterol concentrations does not reduce mortality and is unlikely to prevent coronary heart disease and claims of the opposite are based on preferential citation of supportive trials.
JAMAEffect of Blinding and Unmasking on the Quality of Peer Review
286 Citations1998Susan van Rooyen, Fiona Godlee +3 more
Blinding and unmasking made no editorially significant difference to review quality, reviewers' recommendations, or time taken to review.
The British Journal of PsychiatryOpen peer review: A randomised controlled trial
268 Citations2000Elizabeth Walsh, Maeve Rooney +2 more
This study supports the feasibility of an open peer review system and identifies such a system's potential drawbacks.
Academy of Management JournalManuscript Characteristics Which Influence Acceptance for Management and Social Science Journals.
239 Citations1977S. J. Kerr, Joseph Thomas Tolliver +1 more
JAMAThe existence of publication bias and risk factors for its occurrence
209 Citations1990Kay Dickersin
OikosThe "File Drawer Problem" of Non-Significant Results: Does It Apply to Biological Research?
204 Citations1996Ryan D. Csada, P. Coxon James +1 more
There appears to be a publication bias against non-significant results in the biological literature, and reasons why non- significant results are not published are suggested.
Proceedings of the Royal Society B Biological SciencesManipulation of host behaviour by parasites: a weakening paradigm?
181 Citations2000Robert Poulin
The relationship between the magnitude of published quantitative estimates of parasite–induced changes in host behaviour and year of publication from the time the adaptive host manipulation hypothesis was first proposed is examined.
Social Studies of ScienceAuthor Judgements about Works They Cite: Three Studies from Psychology Journals
164 Citations1995William R. Shadish, Donna Tolliver +2 more
JAMAThe Effects of Blinding on Acceptance of Research Papers by Peer Review
163 Citations1994Martin Fisher
The results suggest that blinded reviewers may provide more unbiased reviews and that nonblinded reviewers may be affected by various types of bias.
JAMAMultiple Blinded Reviews of the Same Two Manuscripts
131 Citations1994Magne Nylenna
Experienced and young referees gave a stricter assessment of the manuscripts than their less experienced and older colleagues, and an English version seemed to be accepted more easily than a national-language version of the same manuscript.
JAMABias Against Negative Studies in Newspaper Reports of Medical Research
129 Citations1991Gideon Koren
The number, length, and quality of newspaper reporting on the positive study were greater than news reports on the negative study, which suggests a bias against news reports of studies showing no effects or no adverse effects.
Statistics in MedicineMODELS FOR ESTIMATING THE NUMBER OF UNPUBLISHED STUDIES
114 Citations1996Leon Jay Gleser, Ingram Olkin
Two general models are proposed that describe how the preponderance of published studies could report significant p-values even when testing a null hypothesis that is, in fact, true.
JAMAA Citation Analysis of the Impact of Blinded Peer Review
112 Citations1994David N. Laband
Journals using nonblinded peer review published a larger fraction of papers that should not have been published than do journals using blinded peer review, when reviewers know the identity of the author(s) of an article, and substitute particularistic criteria for universalistic criteria in their evaluation process.
Proceedings of the Royal Society B Biological SciencesFluctuating paradigm
102 Citations1999Leigh W. Simmons, Joseph L. Tomkins +2 more
A surveyed the literature on fluctuating asymmetry and sexual selection and found a marked chronological decline in the proportion of studies supporting the paradigm through the 1990s, which conform with the notion of a scientific revolution.
JAMAEffect of Institutional Prestige on Reviewers' Recommendations and Editorial Decisions
96 Citations1994Joseph M. Garfunkel
Major manuscripts from institutions with greater prestige were no more likely to be recommended or accepted for publication than those from Institutions with lesser prestige, and the likelihood of recommendation for acceptance and of selection for publication of brief reports appeared to correlate with the prestige of the institution.
JAMAWhat Can and Should Be Done to Reduce Publication Bias?
82 Citations1990D. W. A. Sharp
Opportunities for bias exist, but more objective evaluation is required before journals need to alter their peer review practices, and the price of some proposed correctives is high.
PubMedReviewing the reviews. How strong is the evidence? How clear are the conclusions?
72 Citations2001Jeanette Ezzo, Barker Bausell +3 more
Behavioral EcologyThe strength of sexual selection: a meta-analysis of bird studies
69 Citations1999M.-C. Gontard-Danek
A meta-analysis of visual sexual signals in birds found some evidence consistent with publication bias, whereas a decreasing variance in effect size with increasing sample size and the most common effect size being close to the average effect size did not suggest publication bias.
PubMedReviewer bias: a blinded experimental study.
59 Citations1994E. Ernst, Karl‐Ludwig Resch
Evaluating interrater consistency in reviewing a single manuscript and determining whether a referee's likely predisposition influences his or her attitude toward this manuscript suggest significant impact of reviewer bias on referee's judgment and imply that the peer review system in its present form has room for improvements in fairness and consistency.
PubMedPublication bias in obesity treatment trials?
49 Citations1996David B. Allison, Faith Ms +1 more
There was evidence for publication bias within two of the four published meta-analyses, such that reviews of published studies were likely to overestimate clinical efficacy and suggestions are offered for managing publication bias once identified or reducing its likelihood in the first place.
JAMAMultiple blinded reviews of the same two manuscripts. Effects of referee characteristics and publication language
48 Citations1994M. Nylenna
JAMAReviewing the Reviews
43 Citations1998John W. Joyce, Sophia Rabe‐Hesketh +1 more
Test the hypothesis that the selection of literature in review articles is unsystematic and is influenced by the authors' discipline and country of residence by searching reviews in English published between 1980 and March 1996.
PubMedTesticular size and shape of 47,XYY and 47,XXY men in a double-blind, double-matched population survey.
26 Citations1979E Boisen
The testes of the XXY men were significantly smaller than those of their XY controls, and there was also a difference in shape, but the mean size in this sample of XXYs was larger than in previous reports on Klinefelter syndrome patients, indicating that previous reports of abnormal testes in XYYs suffered from selection bias.
Medical Informatics and the Internet in MedicineThe role of electronic journals in reducing publication bias
21 Citations1999Simon Gilbody
To maintain the integrity of medical publication, editorial policy needs to be changed to accept clinical trials for publication, based only on the methodological criteria and not on the impact of their findings.
PubMedNeonatal outcome and obstetric complications in women with gestational diabetes: effects of maternal body mass index.
19 Citations1996Graziano Di Cianni, Luca Benzi +7 more
The degree of overweight is associated with an earlier diagnosis of GDM; prepregnancy BMI is more predictive of macrosomia than weight gain, both in control and GDM women; GDM seems to play the most important role in increasing the possibility of the occurrence of macrosOMia.
PubMed CentralTesticular size and shape of 47,XYY and 47,XXY men in a double-blind, double-matched population survey.
10 Citations1979Boisen, E
PubMedWill publication bias vanish in the age of online journals?
8 Citations1992J A Berlin
The Online Journal of Current Clinical Trials will judge papers on the quality of the work and the importance of the findings, not on the level of statistical significance achieved in any comparisons made.