Consensus development methods, and their use in clinical guideline development.
Health Technology AssessmentPublished 1 January 1998Open access
Murphy Murphy, Jed Black, Lamping, Martin McKee, Sanderson, Askham
Citations1,795
SJR quartileQ1
SJR score1.36
SNIP1.69
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
This assessment aims to identify the factors that affect the decisions that emerge from consensus development methods and to assess the implications of the findings for the development of clinical guidelines.
Abstract
T he overall aim of the NHS R&D Health Technology Assessment (HTA) programme is to ensure that high-quality research information on the costs, effectiveness and broader impact of health technologies is produced in the most efficient way for those who use, manage
Keywords
Social SciencesMedicineEconomics, Econometrics and Finance
ScienceJudgment under Uncertainty: Heuristics and Biases
28,003 Citations1974Amos Tversky, Daniel Kahneman
Three heuristics that are employed in making judgements under uncertainty are described: representativeness, availability of instances or scenarios, which is often employed when people are asked to assess the frequency of a class or the plausibility of a particular development.
ScienceThe Framing of Decisions and the Psychology of Choice
17,311 Citations1981Amos Tversky, Daniel Kahneman
The psychological principles that govern the perception of decision problems and the evaluation of probabilities and outcomes produce predictable shifts of preference when the same problem is framed in different ways.
American Sociological ReviewThe Psychology of Interpersonal Relations.
11,956 Citations1958Theodore M. Newcomb, Fritz Heider
Psychological ReviewTelling more than we can know: Verbal reports on mental processes.
11,285 Citations1977Richard E. Nisbett, Timothy D. Wilson
Management ScienceAn Experimental Application of the DELPHI Method to the Use of Experts
6,017 Citations1963Norman C. Dalkey, Olaf Helmer
Journal of Abnormal & Social PsychologyA study of normative and informational social influences upon individual judgment.
4,659 Citations1955Morton Deutsch, Harold B. Gerard
Several modifications of the Asch experiment in which the S judges the length of lines in the company of a group of “stooges” who carry out the experimenter's instructions are described.
Journal of Applied PsychologyEstimating within-group interrater reliability with and without response bias.
4,592 Citations1984Lawrence R. James, Robert G. Demaree +1 more
Journal of Personality and Social PsychologyBiased assimilation and attitude polarization: The effects of prior theories on subsequently considered evidence.
4,369 Citations1979Charles G. Lord, Lee Ross +1 more
Administrative Science QuarterlyA Multimethod Examination of the Benefits and Detriments of Intragroup Conflict
3,943 Citations1995Karen A. Jehn
Medical Entomology and ZoologyAttitudes and Persuasion: Classic and Contemporary Approaches
3,406 Citations1981Richard E. Petty, John T. Cacioppo
The LancetEffect of clinical guidelines on medical practice: a systematic review of rigorous evaluations
2,811 Citations1993Jeremy Grimshaw, Ian Russell
It is concluded that explicit guidelines do improve clinical practice, when introduced in the context of rigorous evaluations, however, the size of the improvements in performance varied considerably.
Academy of Management JournalTop Management Team Demography and Corporate Strategic Change
2,177 Citations1992Margarethe F. Wiersema, Karen A. Bantel
American Journal of Public HealthConsensus methods: characteristics and guidelines for use.
2,131 Citations1984Alexander Fink, Jacqueline Kosecoff +2 more
The characteristics of several major methods (Delphi, Nominal Group, and models developed by the National Institutes of Health and Glaser) are surveyed and guidelines for those who want to use the techniques are provided.
Elsevier eBooksJudgment under Uncertainty: Heuristics and Biases
2,074 Citations1978Amos Tversky, Daniel Kahneman
Journal of Personality and Social PsychologyProductivity loss in brainstorming groups: Toward the solution of a riddle.
1,876 Citations1987Michael Diehl, Wolfgang Stroebe
SociometryEye-Contact, Distance and Affiliation
1,756 Citations1965Michael Argyle, Janet B. Dean
It is postulated that eye-contact is linked to affiliative motivation, and that approach and avoidance forces produce an equilibrium level of physical proximity, eyecontact and other aspects of intimacy.
New England Journal of MedicineAn Assessment of Clinically Useful Measures of the Consequences of Treatment
1,676 Citations1988Andreas Laupacis, David L. Sackett +1 more
The goal is to provide a clear picture of the individual components of the immune system and provide a strategy for individualized treatment of these components according to their Kesslerian importance.
New England Journal of MedicineOn the Elicitation of Preferences for Alternative Therapies
1,659 Citations1982Barbara J. McNeil, Stephen G. Pauker +2 more
It is suggested that an awareness of variations in the way information is presented to patients influence their choices between alternative therapies could help reduce bias and improve the quality of medical decision making.
Arthritis & RheumatismThe American college of rheumatology preliminary core set of disease activity measures for rheumatoid arthritis clinical trials
1,647 Citations1993David T. Felson, Jennifer J. Anderson +14 more
A core set of disease activity measures for use in rheumatoid arthritis clinical trials is proposed, which sample the broad range of improvement in RA, and all are at least moderately sensitive to change (have discriminant validity).
Annual Review of PsychologyTEAMS IN ORGANIZATIONS: Recent Research on Performance and Effectiveness
1,624 Citations1996Richard A. Guzzo, Marcus W. Dickson
This review examines recent research on groups and teams, giving special emphasis to research investigating factors that influence the effectiveness of teams at work in organizations, including group composition, cohesiveness, and motivation.
The resolution of conflict;: Constructive and destructive processes
1,458 Citations1973Morton Deutsch
British Journal of Educational PsychologyREMEMBERING: A STUDY IN EXPERIMENTAL AND SOCIAL PSYCHOLOGY
1,190 Citations1933F. C. Bartlett, Cyril Burt
Psychological ReviewSupport theory: A nonextensional representation of subjective probability.
1,169 Citations1994Amos Tversky, Derek J. Koehler
The Journal of Applied Behavioral ScienceA Group Process Model for Problem Identification and Program Planning
1,147 Citations1971André L. Delbecq, Andrew H. Van de Ven
A group process approach useful for practicing administrators charged with a program development task is set forth, suggested for involving the following critical reference groups in successive phases of program development.
Statistical ScienceCombining Probability Distributions: A Critique and an Annotated Bibliography
1,000 Citations1986Christian Genest, James V. Zidek
A taxonomy of solutions is presented which serves as the framework for a survey of recent theoretical developments in the area and a number of current research directions are mentioned and an extensive, current annotated bibliography is included.
Journal of Personality and Social PsychologyGroup polarization: A critical review and meta-analysis.
999 Citations1986Daniel J. Isenberg
Social CognitionSchematic Bases of Social Information Processing
923 Citations2022Shelley E. Taylor, Jennifer Crocker
Technological Forecasting and Social ChangeDelphi: A reevaluation of research and theory
728 Citations1991Gene Rowe, George Wright +1 more
Journal of Experimental Psychology Human Perception & PerformanceFault trees: Sensitivity of estimated failure probabilities to problem representation.
717 Citations1978Baruch Fischhoff, Paul Slovic +1 more
Major results were: people were quite insensitive to what had been left out of a fault tree, increasing the amount of detail for the tree as a whole or just for some of its branches produced small effects on perceptions, and the perceived importance of a particular branch was increased by presenting it in pieces.
Academy of Management JournalThe Effectiveness of Nominal, Delphi, and Interacting Group Decision Making Processes.
694 Citations1974Aplicación De, A. L. Delbecq
Psychological ReviewGroup decision and social interaction: A theory of social decision schemes.
690 Citations1973James H. Davis
Strategic Management JournalTop management group heterogeneity and firm performance
643 Citations1989Alan Murray
Testing a number of hypotheses relating top management group composition to firm performance found partial support for these hypotheses was found; however, the pattern of results also highlights the numerous difficulties in untangling and identifying the determinants of firm performance.
Annals of Internal MedicineReminders to Physicians from an Introspective Computer Medical Record
641 Citations1984Clement J. McDonald, Siu L. Hui +5 more
A computer-stored medical record system containing a limited set of the total clinical data base--primarily diagnostic studies and treatments responds to its own content according to physician-authored reminder rules according to 1490 rules on physician behavior.
AnesthesiologyA Clinical Grading Scale to Predict Malignant Hyperthermia Susceptibility
578 Citations1994Marilyn Green Larach, A. Russell Localio +11 more
The malignant hyperthermia clinical grading scale is recommended for use as an aid to the objective definition of this disease and may improve malignanthyperthermia research by allowing comparisons among well-defined groups of patients.
Social CognitionThe Influence of Positive Affect on Decision-Making Strategy
578 Citations1983Alice M. Isen, Barbara Means
Socio-Economic Planning SciencesThe Delphi method: Substance, context, a critique and an annotated bibliography
573 Citations1971Juri Pill
The methodology and its historical development are described, it is placed in context vis-a-vis other methods of subjective scaling, and a critique is provided.
European Review of Social PsychologySocial Identification, Self-Categorization and Social Influence
535 Citations1990Dominic Abrams, Michael A. Hogg
Journal of Applied PsychologyA disagreement about within-group agreement: Disentangling issues of consistency versus consensus.
512 Citations1992Steve W. J. Kozlowski, Keith Hattrup
Human RelationsUnderstanding Our Differences: Performance in Decision-Making Groups with Diverse Members
490 Citations1994Martha L. Maznevski
Academy of Management JournalEXPERIENTIAL EFFECTS OF DIALECTICAL INQUIRY, DEVIL'S ADVOCACY AND CONSENSUS APPROACHES TO STRATEGIC DECISION MAKING.
479 Citations1989David M. Schweiger, William Sandberg +1 more
Journal of Consumer ResearchEffects of Information Presentation Format on Consumer Information Acquisition Strategies
438 Citations1977James R. Bettman, Pradeep Kakkar
JAMAEffects of the National Institutes of Health Consensus Development Program on Physician Practice
410 Citations1987Jacqueline Kosecoff
It was concluded that the consensus development conference is an important educational tool whose effects might be enhanced by focusing on areas of practice that need improvement and by encouraging follow-up programs at the state and local level.
Journal of Experimental Psychology Human Perception & PerformancePresentation and content: The use of base rates as a continuous variable.
401 Citations1988Gerd Gigerenzer, Wolfgang Hell +1 more
American Journal of Public HealthPhysician ratings of appropriate indications for six medical and surgical procedures.
399 Citations1986Rohnert Park, Arlène Fink +6 more
There was generally better agreement on the final ratings than on the initial ratings, and based on reasonable criteria for agreement and disagreement, the panelists agreed on ratings for 42 to 56 per cent of the indications, and disagreed on 11 to 29 per cent.
JAMAMedical Decision Making in Situations That Offer Multiple Alternatives
396 Citations1995Donald A. Redelmeier
In one scenario involving a patient with osteoarthritis, family physicians were less likely to prescribe a medication when deciding between two medications than when deciding about only one medication and the difficulty in deciding between the two medications led some physicians to recommend not starting either.
Organizational Behavior and Human PerformanceA comparative study of differences in subjective likelihood estimates made by individuals, interacting groups, Delphi groups, and nominal groups
385 Citations1973David H. Gustafson, Ramesh K. Shukla +2 more
Acta PsychologicaHow much information does an expert use? Is it relevant?
360 Citations1992James Shanteau
The results from five studies show that experts often have the same (or fewer) number of significant cues as novices, but that the information used is more relevant, which has implications for measurement of expertise, analysis of expert tasks, and generalizability of conclusions about experts.
International Journal of Nursing StudiesThe Delphi technique: a comparison of results obtained using two expert panels
338 Citations1993Christine Duffield
A study in which two panels of experts were asked to identify the competencies expected of first-line nurse managers using the Delphi technique and the level at which consensus was achieved was defined as the point at which 10% or less of the 168 competencies provided to the panel moved above or below this point of inclusion.
New England Journal of MedicinePerformance of Four Computer-Based Diagnostic Systems
326 Citations1994Eta S. Berner, George D. Webster +12 more
A profile of the strengths and limitations of the diagnostic capabilities of four internal medicine diagnostic systems: Dxplain, Iliad, Meditel, and QMR is provided.
Psychological BulletinField studies of French and Raven's bases of power: Critique, reanalysis, and suggestions for future research.
323 Citations1985Philip M. Podsakoff, Chester A. Schriescheim
Organizational Behavior and Human PerformanceA note on aggregating opinions
321 Citations1978Robin M. Hogarth
European Journal of Operational ResearchDecision making - descriptive, normative, and prescriptive interactions
317 Citations1989
Annual Review of Public HealthWords Without Action? The Production, Dissemination, and Impact of Consensus Recommendations
305 Citations1991Jonathan Lomas
The review describes the decision points in the production process for consensus recommendations as a start on the development of a set of recognized standards, and offers a critical appraisal of the various methodological choices available at each decision point.
Journal of Experimental Social PsychologyThe effects of time scarcity and time abundance on group performance quality and interaction process
299 Citations1992Steven J. Karau, Janice R. Kelly
Cambridge University Press eBooksRESPONSE MODE, FRAMING, AND INFORMATION-PROCESSING EFFECTS IN RISK ASSESSMENT
258 Citations1988Paul Slovic, Baruch Fischhoff +1 more
5 Demand Characteristics and the Concept of Quasi-Controls
251 Citations2009Martin T. Orne
Organizational Behavior and Human Decision ProcessesSome compelling intuitions about group consensus decisions, theoretical and empirical research, and interpersonal aggregation phenomena: Selected examples 1950–1990
234 Citations1992James H. Davis
Management ScienceAn Experiment Assessing Group Support System and Facilitator Effects on Meeting Outcomes
229 Citations1995Robert Anson, Robert P. Bostrom +1 more
Journal of Personality and Social PsychologyThe decision processes of 6- and 12-person mock juries assigned unanimous and two-thirds majority rules.
211 Citations1975James H. Davis, Norbert L. Kerr +3 more
BMJInfluence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration
196 Citations1994Heiner C. Bucher, M. Weinbacher +1 more
Physicians' views of the effectiveness of lipid lowering drugs and the decision to prescribe such drugs is affected by the predominant use of reduction of relative risk in trial reports and advertisements.
Journal of Experimental Psychology Human Learning & MemoryJudged frequency of lethal events.
196 Citations1978Sarah Lichtenstein, et al.
Group & Organization StudiesProcess and Outcome Expectations for the Dialectical Inquiry, Devil's Advocacy, and Consensus Techniques of Strategic Decision Making
188 Citations1991Richard L. Priem, Kenneth H. Price
Small Group ResearchThe Role of the Facilitator in Computer-Supported Meetings
186 Citations1993Victoria K. Clawson, Robert P. Bostrom +1 more
A rigorous assessment, using the critical incidents research methodology, of the role of the facilitator in computer-supported environments is presented and the overall purpose of the study was to empirically develop the critical dimensions of thefacilitation's role.
Organizational Behavior and Human PerformanceImproving the quality of group judgment: Social judgment analysis and the Delphi technique
163 Citations1979John Rohrbaugh
JAMAClarification and Standardization of Substance Abuse Terminology
155 Citations1988Robert C. Rinaldi
A four-stage Delphi survey of substance abuse experts was conducted to help achieve greater clarity and uniformity in terminology associated with alcohol and other drug-related problems.
Medical CareVariations by Specialty in Physician Ratings of the Appropriateness and Necessity of Indications for Procedures
154 Citations1996James P. Kahan, Rolla Edward Park +7 more
Divergence in appropriateness ratings and mutual influence of panelists from different specialties rating a comprehensive set of indications for six surgical procedures means that fewer actual procedures are deemed performed for appropriate or necessary indications.
Journal of Experimental Social PsychologyWhat a person thinks upon learning he has chosen differently from others: Nice evidence for the persuasive-arguments explanation of choice shifts
153 Citations1975Eugene Burnstein, Amiram D. Vinokur
Journal of Behavioral Decision MakingGraphic displays in decision making — the visual salience effect
148 Citations1990Sirkka L. Järvenpää
The study investigates the visual salience of information in making differential predictions for alpha-numeric versus graphic displays in consumer decision making and partially supports predictions that with alpha- numeric displays, information is acquired in correspondence with the importance weights of the attributes.
Journal of ManagementStructured conflict and consensus outcomes in group decision making
140 Citations1995Richard L. Priem
RAND Corporation eBooksImpact of varying panel membership on ratings of appropriateness in consensus panels: A comparison of a multi- and single disciplinary panel
136 Citations1997Ian D. Coulter, Alexandra Adams +1 more
The composition of a panel clearly influences the ratings and those who use a given procedure in practice, in this case manipulation, are more likely to rate it as appropriate than those who do not use the procedure.
Organizational Behavior and Human PerformanceImproving the quality of group judgment: Social judgment analysis and the nominal group technique
128 Citations1981John Rohrbaugh
Journal of Experimental Social PsychologyBelief accessibility and context effects in attitude measurement
128 Citations1989Roger Tourangeau, Kenneth A. Rasinski +2 more
European Journal of Social PsychologyNovel argumentation and attitude change: The case of polarization following group discussion
126 Citations1978Amiram D. Vinokur, Eugene Burnstein
American Journal of Public HealthDerivation of clinical indications for carotid endarterectomy by an expert panel.
124 Citations1987Nancy J. Merrick, Arlène Fink +5 more
It is concluded that consensus methods that do not force agreement can be used with panels of physicians to produce detailed, reliable, and valid indications and can identify medically controversial reasons for using a procedure that can serve as a starting point for a research agenda.
PubMedGroup judgments of appropriateness: the effect of panel composition.
122 Citations1992L L Leape, R E Park +2 more
Management ScienceGroup Confidence Pressures in Iterative Decisions
120 Citations1982David M. Bøje, J. Keith Murnighan
Annals of Otology Rhinology & LaryngologyFinnish Approach to the Treatment of Acute Otitis Media Report of the Finnish Consensus Conference
119 Citations1987P. Karma, K Kouvalainen +6 more
It is concluded that penicillin V, in large doses, is still the drug of first choice in this disease and uncomplicated acute otitis media should be treated by physicians taking care of children at the primary health care level.
Psychological ReportsStrategic Clarity in Banking: Role of Top Management-Team Demography
115 Citations1993Karen A. Bantel
Journal of ForecastingAn examination of factors contributing to delphi accuracy
109 Citations1984Frederick Parente, Janet K. Anderson +2 more
Three experiments examined the accuracy in the Delphi method and found that predictions derived from the group were more accurate than those of 95 per cent of the individual panelists, but did not exceed in accuracy the best panelists.
Journal of ManagementStructured Conflict and Consensus Outcomes in Group Decision Making
103 Citations1995Richard L. Priem, David A. Harrison +1 more
Cambridge University Press eBooksON THE FRAMING OF MEDICAL DECISIONS
89 Citations1988Barbara J. McNeil, Stephen G. Pauker +1 more
If patients are to play an active role in medical decision making – beyond passive informed consent – the authors must find methods for presenting patients with the relevant data and devise procedures for eliciting their preferences among the available treatments.
Medical CarePeer Review of Medical Care
85 Citations1972Fred MacD. Richardson
It was shown that individual judges consistently differed in the degree of harshness or permissiveness of their respective judgments, and that the number of independent judges required to reach a stable judgment of care quality exceeded the number logistically available to meet the probable future demands of third-party payors.
American Journal of Public HealthPhysician ratings of appropriate indications for three procedures: theoretical indications vs indications used in practice.
84 Citations1989R E Park, Arlène Fink +6 more
It is found that there is also substantial disagreement about the appropriateness of indications used in actual cases if disagreement is defined by first discarding the two extreme of nine ratings, then looking for at least one rating near the bottom and one near the top of the 9-point scale.
Combining Individual Judgments
84 Citations1985William R. Ferrell
Journal of Health Politics Policy and LawDo Consensus Conferences Work? A Process Evaluation of the NIH Consensus Development Program
82 Citations1988Paul M. Wortman, Amiram D. Vinokur +1 more
It is concluded that controversial issues cannot be properly addressed within the present conference format, although that was one of its major purposes, and recommendations for improving the consensus process are presented.
Organizational Behavior and Human PerformanceWhen oracles fail—A comparison of four procedures for aggregating subjective probability forecasts
80 Citations1981Gregory W. Fischer
JAMAMedical decision making in situations that offer multiple alternatives
79 Citations1995Donald A. Redelmeier
Group Decision and NegotiationAn experimental investigation of facilitation in an EMS decision room
79 Citations1992Joey F. George, Alan R. Dennis +1 more
If the number of alternatives generated is treated as a covariate, facilitated groups made better decisions, but nonfacilitated groups were more likely to reach consensus.
PubMedPrimary health care needs of persons with physical disabilities: what are the research and service priorities?
76 Citations1990Burns Tj, Batavia Ai +2 more
There was a high degree of correlation between the rankings of physicians and nonphysicians, especially in the areas of hospitalization, aging, and access, and directions in which future research and service efforts should be focused.
GutAppropriateness of cholecystectomy in the United Kingdom--a consensus panel approach.
72 Citations1991Elizabeth Scott, Nicholas Black
A consensus development approach was used to assess the extent to which doctors in the UK agreed about the appropriate indications for cholecystectomy, with a consensus achieved for 61% (surgical panel) and 67% of indications considered.
Annual Review of PsychologyBehavioral Decision Research: A Constructive Processing Perspective
71 Citations1992John W Payne
Human Communication ResearchDECISION-MAKING PROCEDURE AND DECISION QUALITY
70 Citations1984Brant R. Burleson, Barbara J. Levine +1 more
Medical CarePatient Versus Clinician Item Generation for Quality-of-Life Measures
69 Citations1987Jonathan Lomas, Laura Pickard +1 more
The study demonstrates that language-disabled adults can generate items for a quality-of-life measure (hence, this should be possible in almost any other disease group), and that clinician-generated items are not fully representative of patient values.
Group & Organization StudiesA Comparison of Techniques for Judgmental Forecasting by Groups with Common Information
68 Citations1990Janet A. Sniezek
Results show a low accuracy of group forecasts compared to Actual Best Member forecasts in difficult tasks, an inability of groups to identify their best members, and an under-confidence in unbiased easy tasks and overconfidence in biased difficult tasks.
Administrative Science QuarterlyThe Effects of Group Decision-Making Process and Problem Situation Complexity on Implementation Attempts
68 Citations1980Sam E. White, John E. Dittrich +1 more
There was a significant complexity-by-process interaction effect among the decision-making processes, which supports the conclusion that the type of group decision- making process and the problem-situation complexity should be considered in order to optimize the number of implementation attempts.
Journal of Marketing ResearchManagerial Judgment in Marketing: The Concept of Expertise
67 Citations1983Jean-Claude Larréché, Reza Moinpour
International Journal for Quality in Health CareHOW REPRESENTATIVE ARE MEMBERS OF EXPERT PANELS?
61 Citations1991Martin McKee, Patricia Priest +2 more
Consultants working in district general hospitals seem to be more likely to be willing to participate than those employed in teaching hospitals, although this difference may be accounted for by errors in the list of teaching hospital consultants.
…