Discussion Between Reviewers Does Not Improve Reliability of Peer Review of Hospital Quality
Medical CarePublished 1 February 2000
Timothy P. Hofer, Steven J. Bernstein, Sonya DeMonner, Rodney A. Hayward
Citations84
SJR quartileQ1
SJR score1.57
SNIP1.38
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
When 2 physicians discuss a record that they are reviewing, it substantially improves the agreement between those 2 physicians, but this improvement is illusory, as discussion does not improve the overall reliability as assessed by examining the reliability between physicians who were part of different discussions.
Abstract
When 2 physicians discuss a record that they are reviewing, it substantially improves the agreement between those 2 physicians. However, this improvement is illusory, as discussion does not improve the overall reliability as assessed by examining the reliability between physicians who were part of different discussions. This finding may also have implications with regard to how disagreements are resolved on consensus panels, guideline committees, and reviews of literature quality for meta-analyses.
Keywords
Health Professions
Wiley series in probability and statisticsStatistical Methods for Rates and Proportions
18,048 Citations2003Joseph L. Fleiss, Bruce Levin +1 more
The Encyclopedia of Political ScienceReliability and Validity Assessment
5,079 Citations2011Edward G. Carmines
New England Journal of MedicineIncidence of Adverse Events and Negligence in Hospitalized Patients
4,666 Citations1991Troyen A. Brennan, Lucian L. Leape +7 more
The incidence of adverse events, defined as injuries caused by medical management, and of the subgroup of such injuries that resulted from negligent or substandard care, in New York State in 1984 were estimated.
Measuring Health: A Guide to Rating Scales and Questionnaires
4,234 Citations1987Ian McDowell, Claire Newell
The theory and the leading methods of measurement, all of which rely on subjective judgments in questionnaires and rating scales are described, showing readers how to select the most suitable one, apply it, and score the results.
BiometricsAn Application of Hierarchical Kappa-type Statistics in the Assessment of Majority Agreement among Multiple Observers
3,808 Citations1977J. Richard Landis, Gary G. Koch
A subset of 'observers who demonstrate a high level of interobserver agreement can be identified by using pairwise agreement statistics betweeni each observer and the internal majority standard opinion on each subject.
Measuring Health
2,097 Citations2006Ian McDowell
The following section gives a brief introduction to the methods that are mentioned in the reviews of the topic to be measured.
Journal of Personality and Social PsychologyThe common knowledge effect: Information sharing and group judgment.
545 Citations1993Daniel Gigone, Reid Hastie
JAMATotal Parenteral Nutrition in the Critically Ill Patient
532 Citations1998Daren K. Heyland, Shaun MacDonald +2 more
Total parenteral nutrition does not influence the overall mortality rate of surgical or critically ill patients and studies including only malnourished patients were associated with lower complication rates but no difference in mortality when compared with studies of nonmalnouredished patients.
JAMAEffect of Outcome on Physician Judgments of Appropriateness of Care
366 Citations1991Robert A. Caplan
It is concluded that knowledge of the severity of outcome can influence a reviewer's judgment of the appropriateness of care.
New England Journal of MedicineThe Reproducibility of a Method to Identify the Overuse and Underuse of Medical Procedures
360 Citations1998Paul G Shekelle, James P. Kahan +4 more
A parallel, three-way replication of the RAND-University of California at Los Angeles appropriateness method as applied to two medical procedures, coronary revascularization and hysterectomy, found that the appropri ateness method is far from perfect.
BMJBenzodiazepine use in pregnancy and major malformations or oral cleft: meta-analysis of cohort and case-control studies
354 Citations1998Lisa R Dolovich, Antonio Addis +4 more
Pooled data from cohort studies showed no association between fetal exposure to benzodiazepines and the risk of major malformations or oral cleft, but on the basis of pooled data from case-control studies, however, there was a significant increased risk for major mal Formations or Oral cleft alone.
Journal of Clinical EpidemiologyEstimating the reliability of continuous measures with cronbach's alpha or the intraclass correlation coefficient: Toward the integration of two traditions
316 Citations1991Gina Bravo, Louise Potvin
The epidemiological literature has been ambiguous concerning reliability estimates for continuous variables, but based on demonstrations available in the literature this paper recalls that both estimates are equivalent and highlights the conditions under which they are equivalent.
Annals of Internal MedicinePreventable Deaths: Who, How Often, and Why?
281 Citations1988Robert W. Dubois, Robert H. Brook
A new method based on a new method found that a significant number of hospital deaths might have been prevented, and patients whose deaths were probably preventable were younger, less often demented, and less severely ill than patients whose died.
JAMAThe Reliability of Peer Assessments of Quality of Care
206 Citations1992Ronald L. Goldman
There is a need for a global reexamination of the peer review process including more objective assessment procedures, multiple reviewers, higher standards for reviewers, elimination of systematic reviewer bias, use of outcome judgements, and adoption of practice guidelines.
JAMAChanges in Quality of Care for Five Diseases Measured by Implicit Review, 1981 to 1986
160 Citations1990Lisa V. Rubenstein
Except for discharge planning processes, the quality of hospital care has continued to improve for Medicare patients despite, or because of, the introduction of the prospective payment system with its accompanying professional review organization review.
BMJCarotid endarterectomy for asymptomatic carotid stenosis: a meta-analysis
154 Citations1998Oscar Benavente, David Moher +1 more
Carotid endarterectomy in patients with asymptomatic carotid stenosis unequivocally reduces the incidence of ipsilateral stroke, though the absolute benefit is relatively small.
Medical CareReliability and Validity of Judgments Concerning Adverse Events Suffered by Hospitalized Patients
148 Citations1989Troyen A. Brennan, R J Localio +1 more
It was found that a two-step review process of medical records can produce judgments about adverse events that are both reliable and valid, and cases classified as difficult to judge from causation were judged less reliable.
Journal of the American Geriatrics SocietyAppropriateness of Medication Prescribing in Ambulatory Elderly Patients
141 Citations1994Kenneth E. Schmader, Joseph T. Hanlon +7 more
This study aims to assess the quality of medication prescribing in ambulatory elderly patients on multiple medications using the Medication Appropriateness Index (MAI).
JAMAThe Appropriateness of Hysterectomy
141 Citations1993Steven J. Bernstein
The rates of inappropriate use of hysterectomies are similar to those for other procedures and vary to a small degree among health plans and may be useful to purchasers when they consider which health plans to offer their employees.
JAMAWatching the doctor-watchers. How well do peer review organization methods detect hospital care quality problems?
124 Citations1992Haya R. Rubin
Archives of Internal MedicineIatrogenic Complications in High-Risk, Elderly Patients
123 Citations1992F. Lefèvre
Iatrogenic complications are likely to be an extremely common experience for elderly medicine service patients with long lengths of stay and a significant portion of these complications may be potentially preventable with closer attention to initial assessment and documentation of patients' functional status.
Annals of Internal MedicineEvaluating the Care of General Medicine Inpatients: How Good Is Implicit Review?
121 Citations1993Rodney A. Hayward, Laurence F. McMahon +1 more
Implicit review was evaluated for measuring quality of care and appropriateness of resource use on general medicine wards and how often poor care was identified, the types of reported quality problems, and the level of agreement between different physician reviewers (inter-rater reliability).
The Journal of Trauma: Injury, Infection, and Critical CareINTER-RATER RELIABILITY OF PREVENTABLE DEATH JUDGMENTS
111 Citations1992Ellen J. MacKenzie, Donald M. Steinwachs +3 more
JAMAThe appropriateness of hysterectomy. A comparison of care in seven health plans. Health Maintenance Organization Quality of Care Consortium
107 Citations1993Steven J. Bernstein
The rates of inappropriate use of hysterectomies are similar to those for other procedures and vary to a small degree among health plans and may be useful to purchasers when they consider which health plans to offer their employees.
The Joint Commission Journal on Quality ImprovementValidating Quality Indicators for Hospital Care
44 Citations1997Timothy P. Hofer, Steven J. Bernstein +2 more
Four corresponding steps comprise an efficient validation method to produce indicators that detect deficiencies in an important process-outcome continuum, help produce the tools to find the deficiencies, document the efficiency of using the indicator to search for process problems, and define the appropriate use of the indicator.
Archives of Internal MedicineThe Appropriateness of Use of Cardiovascular Procedures in Women and Men
33 Citations1994Steven J. Bernstein
Based on two indicators, the RAND appropriateness score and the Duke prognostic exercise treadmill score, there was unable to find any evidence of a difference in the clinical Appropriateness of use of these three cardiovascular procedures between women and men.
JAMAChanges in quality of care for five diseases measured by implicit review, 1981 to 1986
32 Citations1990Lisa V. Rubenstein
Journal of Educational and Behavioral StatisticsReliability of Essay Rating and Score Adjustment
32 Citations1994Nicholas T. Longford
Medical CareDetermining the Generalizability of Rating Scales in Clinical Settings
29 Citations1981William J. Evans, C. Gene Cayten +1 more
The FLS was found to be capable of differentiating between patients and the effect of time on patient status, regardless of the physician rater(s), illustrating how one multi-faceted design can answer questions that formerly required several separate data sets.
Archives of Internal MedicineIatrogenic complications in high-risk, elderly patients
26 Citations1992F. LeFEVRE
The Joint Commission Journal on Quality ImprovementLong-Term Follow-up in the Peer Assessment Program for Nonspecialist Physicians in Ontario, Canada
18 Citations1998Peter Norton, Earl V. Dunn +2 more
The assessment, interview, and educational interventions undertaken by the licensing authority produced an improvement in practice in the short term in the bottom 10%-15% of all physicians reviewed, which was sustained for more than six years.
AnesthesiologyThe Effect of Group Discussion on Interrater Reliability of Structured Peer Review
18 Citations1998Rhoda D. Levine, M. L. Sugarman +4 more
Journal of the American Medical AssociationINTERNAL AUDIT IN THE DEPARTMENT OF MEDICINE OF A COMMUNITY HOSPITAL
18 Citations1958John J. Butler
The medical audit here described was part of a program intended to evaluate and improve the quality of patient care in a hospital and has been effective in improving the medical records of the hospital.
Annals of Emergency MedicineA comparison of implicit and explicit methods of process quality assurance for blunt trauma patients
16 Citations1990David L. Schriger, Larry J. Baraff +1 more
It is suggested that process-of-care assessments of the quality of care must be interpreted with caution as they are method dependent and may not correlate with patient outcomes.