Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study
BMJPublished 3 March 2008Open access
Lesley Wood, Matthias Egger, Lise Lotte Gluud, Kenneth F. Schulz, Peter Jüni, Douglas G. Altman
Citations2,512
SJR quartileQ1
SJR score2.98
SNIP9.45
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
The average bias associated with defects in the conduct of randomised trials varies with the type of outcome, andSystematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of biases.
Abstract
The average bias associated with defects in the conduct of randomised trials varies with the type of outcome. Systematic reviewers should routinely assess the risk of bias in the results of trials, and should report meta-analyses restricted to trials at low risk of bias either as the primary analysis or in conjunction with less restrictive analyses.
Keywords
Decision SciencesMathematicsEconomics, Econometrics and Finance
Cochrane Handbook for Systematic Reviews of Interventions
24,936 Citations2008
JAMAEmpirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials
5,580 Citations1995Klaus‐Peter Schulz
Empirical evidence is provided that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias.
The LancetThe CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomised trials
5,372 Citations2001David Moher, Kenneth F. Schulz +1 more
The revised CONSORT statement presented here incorporates new evidence and addresses some criticisms of the original statement to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
Annals of Internal MedicineThe Revised CONSORT Statement for Reporting Randomized Trials: Explanation and Elaboration
3,805 Citations2001D G Altman, Kenneth F. Schulz +7 more
The CONSORT statement (or simply CONSORT) comprises a checklist of essential items that should be included in reports of RCTs and a diagram for documenting the flow of participants through a trial, aimed at first reports of two-group parallel designs.
The LancetDoes quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
3,591 Citations1998David Moher, Ba’ Pham +6 more
Study of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.
Assessing Risk of Bias in Included Studies
3,156 Citations2008Julian P. T. Higgins, Douglas G. Altman
Annals of Internal MedicineReported Methodologic Quality and Discrepancies between Large and Small Randomized Trials in Meta-Analyses
2,326 Citations2001Lise Lotte Kjaergard, John Villumsen +1 more
The potential association between reported methodologic quality and intervention effects was studied to assess whether methodologicquality may explain discrepancies between the results of large and small randomized trials in meta-analyses.
JAMAEmpirical Evidence for Selective Reporting of Outcomes in Randomized Trials
1,806 Citations2004An‐Wen Chan, Asbjørn Hróbjartsson +3 more
The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols and Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention.
Health Technology AssessmentHow important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study
1,169 Citations2003Matthias Egger, Peter Jüni +3 more
The LancetDoes the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses?
808 Citations2000Laura McAuley, Ba’ Pham +2 more
Whether exclusion of grey literature, compared with its inclusion in meta-analysis, provides different estimates of the effectiveness of interventions assessed in randomised trials is examined.
American Journal of EpidemiologyBias in Clinical Intervention Research
668 Citations2006Lise Lotte Gluud
A number of methodological studies suggest that lack of adequate randomization in published trial reports may be associated with more positive estimates of intervention effects and that bias is difficult to detect and appraise.
JAMACorrelation of Quality Measures With Estimates of Treatment Effect in Meta-analyses of Randomized Controlled Trials
475 Citations2002Ethan M. Balk
Individual quality measures are not reliably associated with the strength of treatment effect across studies and medical areas, and findings of associations with treatment effect cannot be generalized to all clinical areas or meta-analyses.
Canadian Medical Association JournalOutcome reporting bias in randomized trials funded by the Canadian Institutes of Health Research
446 Citations2004A.-W. Chan
Whether outcome reporting bias would be present in a cohort of government-funded trials subjected to rigorous peer review and whether primary outcomes specified in trial protocols were compared with those reported in publications is compared.
JAMAQuality of Reporting of Randomized Trials as a Measure of Methodologic Quality
400 Citations2002Karin Huwiler-Müntener
Similar quality of reporting may hide important differences in methodologic quality, and well-conducted trials may be reported badly, so a clear distinction should be made between these 2 dimensions of the quality of RCTs.
Statistics in MedicineStatistical methods for assessing the influence of study characteristics on treatment effects in ‘meta‐epidemiological’ research
390 Citations2002Jonathan A C Sterne, Peter Jüni +4 more
It is shown that both within- and between meta-analysis heterogeneity may be of importance in the analysis of meta-epidemiological studies, and that confounding exists between the effects of publication status and trial quality.
BMJBad reporting does not mean bad methods for randomised trials: observational study of randomised controlled trials performed by the Radiation Therapy Oncology Group
315 Citations2004Heloisa P. Soares, Stephanie Daniels +5 more
The reporting of methodological aspects of randomised controlled trials does not necessarily reflect the conduct of the trial and reviewing research protocols and contacting trialists for more information may improve quality assessment.
JAMAPhysician Interpretations and Textbook Definitions of Blinding Terminology in Randomized Controlled Trials
258 Citations2001P.J. Devereaux
It is suggested that both physicians and textbooks vary greatly in their interpretations and definitions of single, double, and triple blinding.
BMJAssessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study
237 Citations2005Lorenzo Moja, Elena Telaro +4 more
Cochrane reviews fared better than systematic reviews published in paper based journals in terms of assessment of methodological quality of primary studies, although they both largely failed to take it into account in the interpretation of results.
International Journal of EpidemiologyInvestigating patient exclusion bias in meta-analysis
213 Citations2004Jayne F. Tierney
Trial, systematic reviews, and meta-analyses may be prone to bias associated with post-randomization exclusion of patients, and wherever possible, the level of such exclusions should be taken into account when assessing the potential for bias.
BMJMeta-analysis and the meta-epidemiology of clinical research
210 Citations1997C. David Naylor
This week's BMJ contains a pot-pourri of materials that deal with the research methodology of meta-analysis, with many studies have documented publication bias favouring clinical trials that show a significant treatment effect.
BMJComparison of descriptions of allocation concealment in trial protocols and the published reports: cohort study
208 Citations2005Julie Pildal, An‐Wen Chan +4 more
Most randomised clinical trials have unclear allocation concealment on the basis of the trial publication alone, and most of these trials also have unclear allocations concealment according to their protocol.
Evidence-Based MedicineThe GATE frame: critical appraisal with pictures
201 Citations2006Rod Jackson
The GATE approach to critical appraisal is outlined, illustrated throughout using the Heart and Estrogen/progestin Replacement Study (HERS), a randomised, double blind, placebo controlled trial of the effect of daily oestrogen plus progestin on coronary heart disease (CHD) death in postmenopausal women.
Journal of the Royal Statistical Society Series A (Statistics in Society)Models for Potentially Biased Evidence in Meta-Analysis using Empirically Based Priors
151 Citations2008Nicky J. Welton, A. E. Ades +3 more
Results from an illustrative example show that the bias-adjusted treatment effect estimates are sensitive to the way in which the meta-epidemiological data are modelled, but that using point estimates for bias parameters provides an adequate approximation to using a full joint prior distribution.
Oxford University Research Archive (ORA) (University of Oxford)Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
127 Citations2004Chan, AW, Hróbjartsson, A +3 more
Statistics in MedicineMultivariable modelling for meta‐epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials
94 Citations2006Volkert Siersma, Bodil Als‐Nielsen +4 more
Two general statistical models are presented for analysis of a study of 523 randomized trials from 48 meta‐analyses in a random sample of Cochrane reviews and stratification by reviews and by broad trial types, the latter of which gives larger subgroups of the data, circumventing instabilities.
Evidence-Based NursingThe GATE frame: critical appraisal with pictures
42 Citations2006Rod Jackson, Shanthi Ameratunga +7 more
The GATE approach to critical appraisal is illustrated throughout using the Heart and Estrogen/progestin Replacement Study (HERS), a randomised, double blind, placebo controlled trial of the effect of daily oestrogen plus progestin on coronary heart disease (CHD) death in postmenopausal women.