The art of quality assessment of RCTs included in systematic reviews
Journal of Clinical EpidemiologyPublished 1 July 2001
Arianne P. Verhagen, Henrica C. W. de Vet, Rob A. de Bie, Maarten Boers, Piet A. van den Brandt
Citations246
SJR quartileQ1
SJR score3.15
SNIP2.66
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This work elaborates on questions such as: what is the concept of quality of individual studies (RCTs), can quality be measured validly and reliably and plans for future research on this issue are proposed.
Abstract
The best evidence on the efficacy of medical interventions is provided by high-quality trials summarized in high-quality systematic reviews or meta-analyses. The methodological quality of studies included in a systematic review can have a substantial impact on the estimates of the treatment effect and therefore on the conclusions of such a review. But what is the empirical evidence to support quality assessment of randomized clinical trials (RCTs)? We elaborate on questions such as: what is the concept of quality of individual studies (RCTs), can quality be measured validly and reliably? Plans for future research on this issue are proposed.
Keywords
Decision SciencesMedicineEconomics, Econometrics and Finance
Controlled Clinical TrialsAssessing the quality of reports of randomized clinical trials: Is blinding necessary?
17,565 Citations1996Alejandro R. Jadad, R Andrew Moore +5 more
An instrument to assess the quality of reports of randomized clinical trials (RCTs) in pain research is described and its use to determine the effect of rater blinding on the assessments of quality is described.
JAMAEmpirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials
5,580 Citations1995Klaus‐Peter Schulz
Empirical evidence is provided that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias.
The LancetDoes quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?
3,591 Citations1998David Moher, Ba’ Pham +6 more
Study of low methodological quality in which the estimate of quality is incorporated into the meta-analyses can alter the interpretation of the benefit of intervention, whether a scale or component approach is used in the assessment of trial quality.
JAMAImproving the quality of reporting of randomized controlled trials. The CONSORT statement
3,343 Citations1996Colin B. Begg
Journal of Clinical EpidemiologyThe Delphi List
2,352 Citations1998Arianne P. Verhagen, Henrica C. W. de Vet +5 more
The aim of this study was to obtain consensus among experts about a set of generic core items for quality assessment of randomized clinical trials (RCTs) and the final criteria list (the Delphi list) was satisfactory to all participants.
JAMAThe Hazards of Scoring the Quality of Clinical Trials for Meta-analysis
1,983 Citations1999Peter Jüni
The data indicate that the use of summary scores to identify trials of high quality is problematic, and relevant methodological aspects should be assessed individually and their influence on effect sizes explored.
Controlled Clinical TrialsA method for assessing the quality of a randomized control trial
1,370 Citations1981Thomas C. Chalmers, Harry Smith +5 more
A reasonable standard design and conduct of trials will facilitate the interpretation of those with conflicting results and help in making valid combinations of undersized trials.
Controlled Clinical TrialsAssessing the quality of randomized controlled trials: An annotated bibliography of scales and checklists
1,304 Citations1995David Moher, Alejandro R. Jadad +4 more
An annotated bibliography of scales and checklists developed to assess quality is presented, giving readers a quantitative index of the likelihood that the reported methodology and results are free of bias.
New England Journal of MedicineMeta-Analyses of Randomized Controlled Trials
1,268 Citations1987Henry S. Sacks, Jayne Berrier +3 more
Journal of Clinical EpidemiologyValidation of an index of the quality of review articles
789 Citations1991Andrew D Oxman, Gordon Guyatt
The OQAQ is a valid measure of the quality of research overviews and the primary shortcoming noted was the need for judgement in applying the index.
BMJThe unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials
773 Citations1998Regina Kunz, Andrew D Oxman
On average, failure to use randomisation or adequate concealment of allocation resulted in larger estimates of effect due to a poorer prognosis in non-randomlyselected control groups compared with randomly selected control groups.
Controlled Clinical TrialsAssessing the Quality of Randomized Trials
652 Citations1999Heather D. Clark, George A. Wells +5 more
The reliability of the Jadad scale and the effect of blinding on interrater agreement in another group of primary studies was determined and a significant effect of blindness on the quality scores was unable to be demonstrated.
Journal of Clinical EpidemiologyIncorporating variations in the quality of individual randomized trials into meta-analysis
646 Citations1992Allan S. Detsky, C. David Naylor +3 more
It is concluded that while it is possible to incorporate the effect of variation of quality of individual trials into overviews, this issue requires more study.
BMJClinical trials of homoeopathy.
610 Citations1991Jos Kleijnen, P. Knipschild +1 more
The evidence of clinical trials is positive but not sufficient to draw definitive conclusions because most trials are of low methodological quality and because of the unknown role of publication bias, which indicates that there is a legitimate case for further evaluation of homoeopathy, but only by means of well performed trials.
Statistics in MedicineHow study design affects outcomes in comparisons of therapy. I: Medical
527 Citations1989Graham A. Colditz, James N. Miller +1 more
Any evaluation of an innovation may include both bias and the true efficacy of the new therapy, therefore any evaluation of a new therapy may consider making adjustments for the average bias associated with a study design.
American Journal of EpidemiologyInvited Commentary: A Critical Look at Some Popular Meta-Analytic Methods
511 Citations1994Sander Greenland
Quality scoring adds the analyst's subjective bias to the results, wastes information, and can prevent the recognition of key sources of heterogeneity; it should be completely replaced by meta-regression on quality items (the score components).
Annals of Internal MedicineSpinal Manipulation for Low-Back Pain
501 Citations1992Paul G Shekelle, Alan H. Adams +3 more
Spinal manipulation is of short-term benefit in some patients, particularly those with uncomplicated, acute low- back pain, and data are insufficient concerning the efficacy of spinal manipulation for chronic low-back pain.
The LancetRandomisation and baseline comparisons in clinical trials
496 Citations1990Douglas G. Altman, Caroline J Doré
80 reports of randomised clinical trials in four leading general medical journals were reviewed and the reporting of the methodology of randomisation was inadequate, and the handling of comparisons of baseline characteristics was inadequate.
International Journal of Technology Assessment in Health CareAssessing the Quality of Randomized Controlled Trials:<i>Current Issues and Future Directions</i>
441 Citations1996David Moher, Alejandro R. Jadad +1 more
Assessing the quality of randomized controlled trials is a relatively new and important development, and several methods for including quality scores in systematic reviews have been proposed, but since little empirical evidence supports any given method, results must be interpreted cautiously.
Journal of Clinical EpidemiologyAcupuncture and chronic pain: A criteria-based meta-analysis
314 Citations1990Gerben ter Riet, J. Kleunen +1 more
The efficacy of acupuncture in the treatment of chronic pain remains doubtful because of the quality of even the better studies, which proved to be mediocre.
JAMAA Proposal for Structured Reporting of Randomized Controlled Trials
310 Citations1994Erik Andrew
A RANDOMIZED controlled trial (RCT) is the most reliable method of assessing the efficacy of health care interventions and reports of RCTs should provide readers with adequate information about what went on in the design, execution, analysis, and interpretation of the trial.
SpineSpinal Manipulation for Low Back Pain
279 Citations1996Bart W. Koes, Willem JJ Assendelft +2 more
The efficacy of spinal manipulation for patients with acute or chronic low back pain has not been demonstrated with sound randomized clinical trials, but there certainly are indications that manipulation might be effective in some subgroups of patients withLow back pain.
Controlled Clinical TrialsAn empirical study of the possible relation of treatment differences to quality scores in controlled randomized clinical trials
246 Citations1990John D. Emerson, Elisabeth Burdick +3 more
In an empirical study of the relation of quality scores to treatment differences in published meta-analyses of 7 groups of controlled randomized clinical trials comprising 107 primary studies, it is found no relation between treatment difference and overall quality score.
Journal of Clinical EpidemiologyImpact of Study Quality on Outcome in Placebo-Controlled Trials of Homeopathy
210 Citations1999Klaus Linde, Michael Scholz +4 more
There was clear evidence that studies with better methodological quality tended to yield less positive results, and meta-regression methods simultaneously investigating the influence of single study features seem the best method for investigating the impact of study quality on outcome.
BMJBlinding and exclusions after allocation in randomised controlled trials: survey of published parallel group trials in obstetrics and gynaecology: Table 1
161 Citations1996K. F Schulz, David A. Grimes +2 more
Journal of Clinical EpidemiologyThe efficacy of back schools: A review of randomized clinical trials
154 Citations1994Bart W. Koes, Maurits W. van Tulder +2 more
The best studies indicated that back schools may be effective in occupational settings in acute, recurrent or chronic conditions and the most promising type of interventions were (modifications of) the "Swedish back school".
Journal of Clinical EpidemiologyMeta-analysis in medicine Where we are and where we want to go
144 Citations1989Milos Jenicek
The greatest challenge of meta-analysis in medicine lies in the integration of the qualitative and quantitative assessment of given information (scoring of quality, weighing of the effect size by quality score, etc.).
Archives of Pediatrics and Adolescent MedicineGuides for Reading and Interpreting Systematic Reviews
135 Citations1998Alejandro R. Jadad, David Moher +1 more
The characteristics of the ideal search, the limitations and decisions that most reviewers face when deciding how to search the literature, and the aspects of a report that readers should evaluate to assess the comprehensiveness and appropriateness of the search strategy are described.
Journal of Clinical EpidemiologyBalneotherapy and Quality Assessment: Interobserver Reliability of the Maastricht Criteria List and the Need for Blinded Quality Assessment
101 Citations1998Arianne P. Verhagen, Henrica C. W. de Vet +4 more
It is concluded that the Maastricht criteria list is a reliable instrument in quality assessment of clinical trials and no evidence that blinding is necessary to prevent review bias is found.
Archives of SurgeryTrials on Trial
98 Citations1984Mary Evans
There is room for improvement in the conduct of clinical trials, and the ability of a small-scale random-control clinical trial comprising less than 500 patients to disclose clinically important differences between treatment groups depends on the event rate in the control group.
Journal of Clinical EpidemiologyResolving conflicting clinical trials: Guidelines for meta-analysis
91 Citations1988Zachary B. Gerbarg, Ralph I. Horwitz
An approach to meta-analysis is proposed that requires methodologic criteria to identify scientifically valid studies, and pooling criteria to combine data from each of the studies, which will have enhanced scientific validity and clinical applicability.
JAMAA proposal for structured reporting of randomized controlled trials. The Standards of Reporting Trials Group
91 Citations1994Erik Andrew, Aslam H. Anis +27 more
BMJTime to register randomised trials
90 Citations1999R. Horton, Richard Smith
A systematic review of trials using ondansetron to treat postoperative nausea and vomiting and those conducting systematic reviews need to be able to identify all trials begun on a subject to avoid the problem of publication bias.
PubMedThe efficacy of chiropractic manipulation for back pain: blinded review of relevant randomized clinical trials.
70 Citations1992Willem JJ Assendelft, Bart W. Koes +2 more
Although the small number of chiropractor RCTs and the poor general methodological quality precludes the drawing of strong conclusions, chiropractic seems to be an effective treatment of back pain.
Physical Therapy ReviewsMethodology of systematic reviews: an introduction
32 Citations1996Robert A. de Bie
The background and considerations of the method of systematic reviewing are given and specific attention is given to evaluation of the quality of studies incorporated in a review and to the description and interpretation of the results of a systematic review.
Canadian Journal of Anesthesia/Journal canadien d anesthésieQuality and retrieval of obstetrical anaesthesia randomized controlled trials
26 Citations1997Jennifer S. Bender, Stephen H. Halpern +3 more
Care must be taken when reviewing obstetrical anaesthesia research and strategies in addition to a MEDLINE search must be used to identify RCTs since more than one third were missed using MEDLINE alone.
American Journal of Obstetrics and GynecologyMethodology citations and the quality of randomized controlled trials in obstetrics and gynecology
24 Citations1996David A. Grimes, Kenneth F. Schulz
Investigators who conduct randomized controlled trials should be thoroughly familiar with this type of research or should get expert help.