Strategies for Improving Fidelity in the National Evidence-Based Practices Project
Research on Social Work PracticePublished 27 May 2009
Gary R. Bond, Robert E. Drake, Gregory J. McHugo, Charles A. Rapp, Rob Whitley
Citations133
SJR quartileQ1
SJR score0.72
SNIP1.09
Generate an AI Snapshot to get a quick, structured summary of this paper.
Study Snapshot
ObjectiveStudy objective
MethodsResearch methodology
PopulationPopulation studied
Sample sizeSample sizes
OutcomesStudy outcomes here
ResultsStudy results comes here
LimitationsResearch study limitations comes here
A concise AI-generated summary of the paper will appear here once you click Generate AI Snapshot.
TL;DR
A multipronged implementation strategy was effective in achieving high fidelity in over half of the sites seeking to implement a new EBP.
Abstract
Background: The National Evidence-Based Practices (EBPs) Project developed and tested a model for facilitating the implementation of five psychosocial EBPs for adults with severe mental illness in the United States. Methods: The implementation model was tested in 53 sites in 8 states. In each site, one of the five EBPs was adopted for implementation and then studied for a 2-year period using a combination of qualitative and quantitative methods. Findings: At baseline, none of the sites had programs attaining high fidelity. Four factors were identified as influencing fidelity: (a) EBP-specific factors, (b) governmental factors, (c) leadership factors, and (d) fidelity review factors. Conclusion: A multipronged implementation strategy was effective in achieving high fidelity in over half of the sites seeking to implement a new EBP.
Keywords
Health Professions
BMJCrossing the Quality Chasm: A New Health System for the 21st Century
10,744 Citations2001Alexi Baker
Analyzing health care organizations as complex systems, Crossing the Quality Chasm also documents the causes of the quality gap, identifies current practices that impede quality care, and explores how systems approaches can be used to implement change.
Psychological MethodsForming inferences about some intraclass correlation coefficients.
6,685 Citations1996Kenneth O. McGraw, S. P. Wong
Implementation research: a synthesis of the literature.
3,784 Citations2005Fixsen Dl, Naoom Sf +2 more
Archives of General PsychiatryAlternative to Mental Hospital Treatment
1,889 Citations1980Leonard I. Stein
Use of the community program for 14 months greatly reduced the need to hospitalize patients and enhanced the community tenure and adjustment of the experimental patients, and the results suggest that community programming should be comprehensive and ongoing.
Medical CareSuccesses and Failures in the Implementation of Evidence-Based Guidelines for Clinical Practice
1,233 Citations2001Richard Grol
Experiences with more than 10 years of development and dissemination of clinical guidelines for family medicine in the Netherlands show a multifaceted approach with written and personal approaches seems to be effective in the dissemination of guidelines.
Schizophrenia BulletinTranslating Research Into Practice: The Schizophrenia Patient Outcomes Research Team (PORT) Treatment Recommendations
923 Citations1998Anthony F. Lehman, D. M. Steinwachs +1 more
The PORT Treatment Recommendations provide a basis for moving toward "evidence-based" practice for schizophrenia and identify both the strengths and limitations in the current knowledge base.
American Journal of EvaluationFidelity Criteria: Development, Measurement, and Validation
869 Citations2003Carol T. Mowbray, Mark C. Holter +2 more
Questions are raised about the dynamic nature of fidelity criteria, appropriate validation and statistical analysis methods, the inclusion of structure and process criteria in fidelity assessment, and the role of program theory in deciding on the balance between adaptation versus exact replication of model programs.
International Journal of Technology Assessment in Health CareEffectiveness and efficiency of guideline dissemination and implementation strategies
867 Citations2005Jeremy Grimshaw, Ruth Thomas +11 more
Schizophrenia BulletinThe Schizophrenia Patient Outcomes Research Team (PORT): Updated Treatment Recommendations 2003
765 Citations2004Anthony F. Lehman, Julie Kreyenbuhl +10 more
Evidence points to the value of treatment approaches combining medications with psychosocial treatments, including psychological interventions, family interventions, supported employment, assertive community treatment, and skills training.
Psychiatric ServicesImplementing Evidence-Based Practices in Routine Mental Health Service Settings
723 Citations2001Robert E. Drake, Howard H. Goldman +5 more
The authors discuss common concerns about the use of evidence-based practices, such as whether ethical values have a role in shaping such practices and how to deal with clinical situations for which no scientific evidence exists.
Psychiatric ServicesImplementing Dual Diagnosis Services for Clients With Severe Mental Illness
696 Citations2001Robert E. Drake, Susan M. Essock +8 more
The authors describe the critical components of effective programs, which include a comprehensive, long-term, staged approach to recovery; assertive outreach; motivational interventions; provision of help to clients in acquiring skills and supports to manage both illnesses and to pursue functional goals; and cultural sensitivity and competence.
Psychiatric ServicesImplementing Supported Employmentas an Evidence-Based Practice
691 Citations2001Gary R. Bond, Deborah R. Becker +6 more
Psychiatric ServicesIllness Management and Recovery: A Review of the Research
688 Citations2002Kim T. Mueser, Patrick W. Corrigan +9 more
Health AffairsEvidence-Based Quality Improvement: The State Of The Science
680 Citations2005Kaveh G Shojania, Jeremy Grimshaw
This work reviews problems with current approaches to QI research, outlines the steps required to make QI efforts based as much on evidence as the practices they seek to implement, and reviews the current state of quality improvement research.
Schizophrenia BulletinPatterns of Usual Care for Schizophrenia: Initial Results From the Schizophrenia Patient Outcomes Research Team (PORT) Client Survey
612 Citations1998Anthony F. Lehman, D. M. Steinwachs +1 more
Current usual treatment practices for persons with schizophrenia likely fall substantially short of what would be recommended based on the best evidence on treatment efficacy, and the need for greater efforts to ensure that treatment research results are translated into practice is highlighted.
Disease Management & Health OutcomesAssertive Community Treatment for People with Severe Mental Illness
604 Citations2001Gary R. Bond, Robert E. Drake +2 more
The critical ingredients of the assertive community treatment (ACT) model for people with severe mental illness are described and the evidence regarding its effectiveness and cost effectiveness is reviewed.
American Journal of Public HealthAdequacy of Treatment for Serious Mental Illness in the United States
589 Citations2002Philip S. Wang, Olga Demler +1 more
Inadequate treatment of serious mental illness is an enormous public health problem and public policies and cost-effective interventions are needed to improve both access to treatment and quality of treatment.
American Journal of OrthopsychiatryProgram fidelity in assertive community treatment: Development and use of a measure.
516 Citations1998Gregory B. Teague, Gary R. Bond +1 more
A measure of program fidelity to ACT and the results of its application to fifty diverse programs are presented.
Psychiatric ServicesEvidence-Based Practices for Services to Families of People With Psychiatric Disabilities
506 Citations2001Lisa B. Dixon, William R. McFarlane +8 more
Family psychoeducation dissemination efforts that have been successful to date have built consensus at all levels, including among consumers and their family members; have provided ample training, technical assistance, and supervision to clinical staff; and have maintained a long-term perspective.
Psychiatric ServicesImplementing Evidence-Based Practices for Persons With Severe Mental Illnesses
495 Citations2001William C. Torrey, Robert E. Drake +6 more
An implementation plan for evidence-based practices based on the use of toolkits to promote the consistent delivery of such practices and to address the concerns of mental health authorities (funders), administrators of provider organizations, clinicians, and consumers and their families are described.
Mental Health Services ResearchMultisystemic Treatment of Substance-Abusing and -Dependent Delinquents: Outcomes, Treatment Fidelity, and Transportability
450 Citations1999Scott W. Henggeler, Susan G. Pickrel +1 more
Examination of treatment adherence measures suggests that the modest results of MST were due, at least in part, to difficulty in transporting this complex treatment model from the direct control of its developers.
Psychiatric ServicesMoving Assertive Community Treatment Into Standard Practice
448 Citations2001Susan D. Phillips, Barbara J. Burns +6 more
The assertive community treatment model of comprehensive community-based psychiatric care for persons with severe mental illness is described and issues pertaining to implementation of the model are discussed.
Schizophrenia BulletinThe NIMH Community Support Program: Pilot Approach to a Needed Social Reform*
417 Citations1978James C. Turner, William J. TenHoor
This year, the National Institute of Mental Health launched a modest but potentially significant pilot program titled the Community Support Program (CSP), designed to improve services for one particularly vulnerable population—adult psychiatric patients whose disabilities are severe and persistent but for whom long-term skilled or semiskilled nursing care is inappropriate.
Journal of Consulting and Clinical PsychologyMeasuring the fidelity of implementation of a mental health program model.
337 Citations1994John H. McGrew, Gary R. Bond +2 more
Journal of Consulting and Clinical PsychologyThe New Hampshire study of supported employment for people with severe mental illness.
329 Citations1996Robert E. Drake, Gregory J. McHugo +3 more
Psychiatric ServicesFidelity to Assertive Community Treatment and Client Outcomes in the New Hampshire Dual Disorders Study
321 Citations1999Gregory J. McHugo, Robert E. Drake +2 more
Faithful implementation of, and adherence to, the assertive community treatment model for persons with dual disorders was associated with superior outcomes in the substance use domain, and the findings underscore the value of measures of model fidelity.
Medical Entomology and ZoologyAssertive Community Treatment of Persons With Severe Mental Illness
319 Citations1998Leonard I. Stein, Alberto B. Santos
In addition to laying out the basics of staffing and running a community based program for the mentally ill, this text offers guidance on such issues as financing the program and treating drug abusers.
Mental Health Services ResearchMeasurement of Fidelity in Psychiatric Rehabilitation
305 Citations2000Gary R. Bond, Lisa Evans +3 more
The historical roots for the development of fidelity measures are reviewed, recent applications in both research and practice are described and systematic methods for measuring their implementation are described.
Psychiatric ServicesFidelity Outcomes in the National Implementing Evidence-Based Practices Project
299 Citations2007Gregory J. McHugo, Robert E. Drake +7 more
Fidelity outcomes for five evidence-based practices that were implemented in routine public mental health settings in the National Implementing Evidence-Based Practices Project showed an increase in fidelity from baseline to 12 months, with scores leveling off between 12 and 24 months.
Behavior ModificationImplementing Evidence-Based Practices for People with Severe Mental Illness
280 Citations2003Kim T. Mueser, William C. Torrey +3 more
The authors describe the Implementing Evidence-Based Practices (EBPs) Project, designed to increase access for people with SMI to empirically supported interventions, and the six EBPs identified for initial package development.
The Joint Commission Journal on Quality ImprovementLessons from Experienced Guideline Implementers: Attend to Many Factors and Use Multiple Strategies
265 Citations2000Leif I. Solberg, Milo L. Brekke +8 more
It is believed that implementation efforts focusing on the individual physician with a single strategy are unlikely to be successful, and implementation efforts must use multiple strategies that take account of multiple characteristics of the guideline, practice organization, and external environment.
The Canadian Journal of PsychiatryEconomic Impacts of Assertive Community Treatment: A Review of the Literature
251 Citations1999Éric Latimer
The most reliable cost offset to A CT treatment costs appears to be reduced hospital use, and overall, ACT appears to result in somewhat lower costs, whatever the perspective of analysis adopted.
Milbank QuarterlyA Report Card on Continuous Quality Improvement
249 Citations1998David Blumenthal, Charles M. Kilo
Until there is a profound, organization-wide recognition of the need for change, universal commitment to CQI principles will not be achieved.
Psychiatric ServicesThe Role of Staff Turnover in the Implementation of Evidence-Based Practices in Mental Health Care
193 Citations2008E. M. Woltmann, Rob Whitley +6 more
High turnover most often had a negative impact on implementation, although some teams were able to use strategies to improve implementation through turnover, and implementation models must consider turbulent behavioral health workforce conditions.
Psychiatric ServicesFidelity of Supported Employment Programs and Employment Outcomes
187 Citations2001Deborah R. Becker, James Smith +3 more
This study attempted to identify critical components of a supported employment program that were strongly correlated with competitive employment outcomes in a state mental health system and with two program components, namely, providing services in the community as opposed to providing them in the clinic and using full-time employment specialists as oppose to staff with mixed roles.
The Journal of Behavioral Health Services & ResearchCritical ingredients of assertive community treatment: Judgments of the experts
183 Citations1995John H. McGrew, Gary R. Bond
Twenty experts on assertive community treatment rated the importance of 73 program elements, also indicating ideal model specifications (e.g., minimum time commitment for psychiatrist) when appropriate, and suggested several areas of increasing emphasis and decreasing emphasis.
New Directions for Mental Health ServicesNew Directions for Mental Health Services
154 Citations2018Logan, William S.
This work describes the evolution of mental health services research structure and organization of systems providingmental health services costs, financing, reimbursement and regulation determinants and patterns of service use outcome of serviceUse definition, classification and measurement of mental illness and services.
W.W. Norton & Company eBooksEvidence-based mental health practice: A textbook.
151 Citations2005Robert E. Drake, Matthew R. Merrens +1 more
This book discusses developing and Sustaining an Evidence-Based System of Mental Health Service Lon Herman, M.S.A.W, LICSW, and the interface of Cultural Competency and Evidence Based Practices Carole Siegel, Ph.D.
Psychiatric ServicesThe State Policy Context of Implementation Issues for Evidence-Based Practices in Mental Health
144 Citations2007Kimberley R. Isett, M. Audrey Burnam +7 more
Analyzing implementation issues related to several evidence-based practices for adults with serious mental illness that were included in a national demonstration project found that state mental health authorities should note that each of the practices requires different skill sets and involves different stakeholders.
The Joint Commission Journal on Quality ImprovementGuideline Implementation: What the Literature Doesn’t Tell Us
134 Citations2000Leif I. Solberg
Reviews of guideline implementation trials have focused on how to change the behavior of individual clinicians, with little attention to the impact of practice systems or organizational support of clinician behavior or the role of the practice environmental context within which change is being attempted.
Schizophrenia BulletinMultisite Experimental Cost Study of Intensive Psychiatric Community Care
110 Citations1995Robert A. Rosenheck, Michael C. Neale +3 more
A 2-year experimental cost study of 10 Intensive Psychiatric Community Care programs was conducted at Department of Veterans Affairs medical centers in the Northeast, finding that costs were lower for IPCC patients in two subgroups: veterans over age 45 and veterans with high levels of inpatient service use before program entry.
Community Mental Health JournalWhat Predicts Supported Employment Program Outcomes?
106 Citations2006Deborah R. Becker, Haiyi Xie +3 more
Differences in access to supported employment services and rates of competitive employment (efficiency) as well as predictors of access and efficiency are identified.
Psychiatric ServicesEvidence-Based Pharmacologic Treatment for People With Severe Mental Illness:A Focus on Guidelines and Algorithms
102 Citations2001Thomas A. Mellman, Alexander L. Miller +4 more
The authors observe that the potential of guidelines and algorithms to promote evidence-based medication treatment for persons with severe mental illness depends on refinement of tools, progress in research, and cooperation of physicians, nonphysician clinicians, administrators, and consumers and family members.
Psychiatric ServicesDissemination of assertive community treatment programs
100 Citations1995P A Deci, Alberto B. Santos +3 more
Assertive community treatment programs have disseminated quite unevenly across 33 states, with the highest concentrations of programs in midwestern and eastern states.
Psychiatric Rehabilitation JournalPractices differentiating high-performing from low-performing supported employment programs.
96 Citations2003Elizabeth L. Gowdy, Linda Carlson +1 more
The study found 13 differentiating practice factors in three areas: building consumer interest in working, getting a job, and maintaining a job that differentiate programs producing high rates of competitive employment from those with low rates.
Psychiatric Rehabilitation JournalFidelity of supported employment: Lessons learned from the National Evidence-Based Practice Project.
88 Citations2008Gary R. Bond, Gregory J. McHugo +3 more
Examination and interpreting fidelity changes over two years of program implementation suggested four factors that facilitated movement toward high fidelity: Discontinuing non-evidence-based vocational services, making rapid structural changes through administrative action, measuring key process indicators to move toward desired changes, and gradually improving integration of the employment specialists with clinical services.
Psychiatric ServicesImpact of Robustness of Program Implementation on Outcomes of Clients in Dual Diagnosis Programs
86 Citations1999Jeanette M. Jerrell, M. Susan Ridgely
Three types of treatment-behavioral skills training, a 12-step recovery model, and intensive case management-provided to 132 clients at four facilities were identified as being robustly or not robustly implemented, depending on whether core elements of these treatments were emphasized.
CNS SpectrumsImplementing Evidence-Based Psychosocial Practices: Lessons Learned from Statewide Implementation of Two Practices
81 Citations2004Lorna L. Moser, Natalie L. DeLuca +2 more
It was found that ACT was generally more successfully implemented than IDDT throughout the state, and that this difference could be traced in large part to state-level factors relating to historical preparation for the practice, establishment of standards, formation of a technical assistance center, and funding.
CNS SpectrumsPrediction of Outcome from the Dartmouth Assertive Community Treatment Fidelity Scale
79 Citations2004Gary R. Bond, Michelle P. Salyers
Even if predictive validity of the Dartmouth ACT Scale is limited, it continues to be a useful tool for program monitoring and for providing corrective feedback.
Community Mental Health JournalThe Role of State Mental Health Authorities in Promoting Improved Client Outcomes through Evidence-Based Practice
73 Citations2005Charles A. Rapp, Gary R. Bond +4 more
The role of state mental health authorities (SMHA) is critical to implementing and sustaining evidence-based practices and examples from states which have been actively pursuing evidence- based practices are provided.
The Journal of Behavioral Health Services & ResearchIs it ACT yet? real-world examples of evaluating the degree of implementation for assertive community treatment
66 Citations2003Michelle P. Salyers, Gary R. Bond +5 more
Using an exploratory, actuarial approach to defining program standards, this study applies different statistical criteria for determining whether or not a program meets ACT standards using the 28-item Dartmouth Assertive Community Treatment Scale.
Community Mental Health JournalEvidence-based Practice Implementation Strategies: Results of a Qualitative Study
63 Citations2007Charles A. Rapp, Diane Etzel-Wise +6 more
Using qualitative research methods, the study uncovered eight strategies that contributed to successful implementation in six sites of supported employment and integrated dual diagnosis treatment.
Psychiatric ServicesWhat Do We Really Know About Community Support Programs? Strategies for Better Monitoring
59 Citations1988J. S. Brekke
Developing effective community support programs requires making a commitment to evaluating programs as they are implemented, identifying program variables to monitor, devising methods for monitoring the variables chosen, and drawing from and building on existing literature about program evaluation.
Child and Adolescent Psychiatric Clinics of North AmericaPromoting the Implementation of Practices that are Supported by Research: The National Implementing Evidence-Based Practice Project
57 Citations2005William C. Torrey, David Lynde +1 more
The National Implementing Evidence-Based Practice Project is an ongoing effort to promote the implementation of effective practices for adults who have severe mental illnesses and is field-testing the approach in eight states.
Psychiatric Rehabilitation JournalConcurrent and Predictive Validity of Two Scales to Assess the Fidelity of Implementation of Supported Employment.
56 Citations2005John H. McGrew, Mélina E. Griss
The ability of the Individual Placement and Support and the Quality of Supported Employment Implementation Scale (QSEIS) fidelity scales to predict independently rated SE adherence and percentage of clients closed by the Office of Vocational rehabilitation (OVR) was tested.
Journal of Applied Behavior AnalysisAN ANALYSIS OF THE PROCEDURAL COMPONENTS OF SUPPORTED EMPLOYMENT PROGRAMS ASSOCIATED WITH EMPLOYMENT OUTCOMES
51 Citations1989John McDonnell, Diane Nofs +2 more
The results suggest that successful implementation of supported employment services led to ongoing employment of study participants in community work sites, increased wages, and ongoing opportunities for workers to interact with nondisabled peers.
New Directions for Mental Health ServicesVariations in an assertive outreach model
51 Citations1991Gary R. Bond
The author reviews recent research findings examining variations in an assertive outreach model and mental health system and client characteristics that have influenced program implementation are discussed.
Journal of Traumatic StressDissemination of evidence‐based mental health interventions: Importance to the trauma field
50 Citations2006Wayne Katon, Doug Zatzick +2 more
The authors review several examples of dissemination efforts of mental health interventions and describes the problem with prior clinical approaches, the program to be disseminated, the barriers and levers to implementation and the progress in overcoming these barriers.
Community Mental Health JournalThe State Health Authority Yardstick (SHAY)
46 Citations2009Molly Finnerty, Charles A. Rapp +4 more
The development of the State Mental Health Authority Yardstick (SHAY), a behaviorally anchored instrument designed to assess state-level facilitating conditions associated with successful implementation of EBPs in community mental health centers, is described.
Psychiatric Clinics of North AmericaStrategies for leading the implementation of evidence-based practices
46 Citations2003William C. Torrey, Molly Finnerty +2 more
The literature on health care practice change is reviewed and some strategy suggestions for administrators who are leading evidence-based practice implementation initiatives are offered.
Journal of Mental HealthDimensions of supported employment: Factor structure of the IPS fidelity scale
45 Citations2001Kathleen M. Vogler Gary R. Bond
Psychiatric Clinics of North AmericaFundamental principles of evidence-based medicine applied to mental health care
43 Citations2003Robert E. Drake, Stanley D. Rosenberg +3 more
The fundamental principles of evidence-based medicine are discussed: using the best available evidence, individualizing the evidence, attending to patient preferences, and expanding clinical expertise.
Medical CareA Fidelity Measure for Integrated Management of Depression in Primary Care
39 Citations2006Thomas E. Oxman, Herbert C. Schulberg +5 more
A fidelity measure is a promising tool for monitoring the quality of implementation of integrated care and was associated with better outcomes within this high implementation of the clinical model.
American Journal of Psychiatric RehabilitationThe Role of Technical Assistance Centers in Implementing Evidence-Based Practices: Lessons Learned
39 Citations2007Michelle P. Salyers, MICHAEL McKASSON +4 more
The ACT Center of Indiana is created in response to a competitive, broad agency announcement to create a technical assistance center for Assertive Community Treatment (ACT).
Journal of Mental HealthThe role of staffing stability in the implementation of integrated dual disorders treatment: An exploratory study
37 Citations2007Emily M. Woltmann, Rob Whitley
While challenging to overcome, agencies were often able to work through staffing problems to achieve reasonable fidelity in the face of team turnover, and training infrastructure appeared to be important in the ability of agencies to sustain implementation during times of staffing instability.
Administration and Policy in Mental Health and Mental Health Services ResearchAligning Measurement-based Quality Improvement with Implementation of Evidence-based Practices
34 Citations2006Richard C. Hermann, Jeffrey Chan +2 more
The degree to which MBQI and EBP implementation (EBPI) are aligned and whether further coordination between them could yield greater effectiveness or efficiency is explored.
Administration and Policy in Mental Health and Mental Health Services ResearchReduction of Concurrent Antipsychotic Prescribing Practices Through the Use of PSYCKES
11 Citations2006Thomas Uttaro, Molly Finnerty +3 more
The results of this evaluation suggest that the use of PSYCKES was effective in reducing such regimens for inpatients.
Research on Social Work PracticeThe Ohio Supported Employment Coordinating Center of Excellence
11 Citations2007David E. Biegel, Sarah J. Swanson +1 more
The Ohio Supported Employment Coordinating Center of Excellence is focused on helping to implement and evaluate evidence-based employment services provided to individuals with a severe mental illness.