Internal Medicine Residency Training on Topics in Ambulatory Care: A Status Report

Internal medicine residents are receiving inadequate training in chronic disease management, and training should be redesigned to improve teaching in ambulatory care.

Abstract

In the US, medical care for most consists of an ambulatory visit.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar, 2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 3Warm E.J. Schauer D.P. Diers T. et al.The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) Education Innovations Project (EIP).J Gen Intern Med. 2008; 23: 921-926Crossref PubMed Scopus (69) Google Scholar When a new medical concern arises, the general internist is typically the first contact.4Bodenheimer T. Primary care—will it survive?.N Engl J Med. 2006; 355: 861-864Crossref PubMed Scopus (325) Google Scholar Over the years, ambulatory care has grown to include postacute care from recently hospitalized patients, as well as increasingly complex chronic disease management for diabetes, hypertension, and other diseases.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar As a result, the need for general internists is growing.5The impending collapse of primary care medicine and its implications for the state of the nation's health care. American College of Physicians, Washington, DCJanuary 30, 2006http://www.acponline.org/advocacy/events/state_of_healthcare/statehc06_1.pdfGoogle Scholar Despite this projected increasing need, our current system of training is not producing more general internists. The number of medical students choosing residencies in primary care-oriented internal medicine programs is decreasing, and the number of internal medicine residents choosing careers as general internists is decreasing.6Garibaldi R.A. Popkave C. Bylsma W. Career plans for trainees in internal medicine residency programs.Acad Med. 2005; 80: 507-512Crossref PubMed Scopus (125) Google ScholarPerspectives Viewpoints•The need for general internists is growing.•We found that baseline knowledge of chronic disease management and acute/symptom-based care management among internal medicine residents is poor.•Knowledge on these topics improves only slightly during internal medicine residency training, more so at community hospital-based training programs than at university hospital-based training programs.•These findings contribute to those arguing in favor of redesign of internal medicine residency training. •The need for general internists is growing.•We found that baseline knowledge of chronic disease management and acute/symptom-based care management among internal medicine residents is poor.•Knowledge on these topics improves only slightly during internal medicine residency training, more so at community hospital-based training programs than at university hospital-based training programs.•These findings contribute to those arguing in favor of redesign of internal medicine residency training. There is a structural disconnect between internal medicine residency training and the primary care needs of the population.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar, 2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 3Warm E.J. Schauer D.P. Diers T. et al.The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) Education Innovations Project (EIP).J Gen Intern Med. 2008; 23: 921-926Crossref PubMed Scopus (69) Google Scholar, 7Larson E.B. Fihn S.D. Kirk L.M. et al.The future of general internal medicine: report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the domain of general internal medicine.J Gen Intern Med. 2004; 19: 69-77Crossref PubMed Scopus (87) Google Scholar, 8Meyers F.J. Weinberger S.E. Fitzgibbons J.P. Glassroth J. Duffy F.D. Clayton C.P. Alliance for Academic Internal Medicine Education Redesign Task ForceRedesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med. 2007; 82: 1211-1219Crossref PubMed Scopus (115) Google Scholar Internal medicine residency training is heavily inpatient-based, often at the expense of training in ambulatory care.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar, 2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 7Larson E.B. Fihn S.D. Kirk L.M. et al.The future of general internal medicine: report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the domain of general internal medicine.J Gen Intern Med. 2004; 19: 69-77Crossref PubMed Scopus (87) Google Scholar, 8Meyers F.J. Weinberger S.E. Fitzgibbons J.P. Glassroth J. Duffy F.D. Clayton C.P. Alliance for Academic Internal Medicine Education Redesign Task ForceRedesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med. 2007; 82: 1211-1219Crossref PubMed Scopus (115) Google Scholar Training in ambulatory care is valued less by residents than are the general medical wards and intensive care unit rotations, despite the fact that the majority of residents will pursue careers that include a strong ambulatory component.9Sisson S.D. Boonyasai R. Baker-Genaw K. Silverstein J. Continuity clinic satisfaction and valuation in residency training.J Gen Intern Med. 2007; 22: 1704-1710Crossref PubMed Scopus (26) Google Scholar Residents graduate from internal medicine residency training programs without the necessary skills to effectively practice ambulatory care, and report feeling unprepared to provide outpatient care.3Warm E.J. Schauer D.P. Diers T. et al.The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) Education Innovations Project (EIP).J Gen Intern Med. 2008; 23: 921-926Crossref PubMed Scopus (69) Google Scholar, 7Larson E.B. Fihn S.D. Kirk L.M. et al.The future of general internal medicine: report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the domain of general internal medicine.J Gen Intern Med. 2004; 19: 69-77Crossref PubMed Scopus (87) Google Scholar, 8Meyers F.J. Weinberger S.E. Fitzgibbons J.P. Glassroth J. Duffy F.D. Clayton C.P. Alliance for Academic Internal Medicine Education Redesign Task ForceRedesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med. 2007; 82: 1211-1219Crossref PubMed Scopus (115) Google Scholar, 10Mladenovic J. Shea J.A. Duffy F.D. Lynn L.A. Holmboe E.X. Lipner R.S. Variation in internal medicine residency clinic practices: assessing practice environments and quality of care.J Gen Intern Med. 2008; 23: 914-920Crossref PubMed Scopus (22) Google Scholar Society's need for competent ambulatory care is growing. Hospitalization rates and Medicare expenditures are both improved by access to a talented pool of general internists.4Bodenheimer T. Primary care—will it survive?.N Engl J Med. 2006; 355: 861-864Crossref PubMed Scopus (325) Google Scholar, 5The impending collapse of primary care medicine and its implications for the state of the nation's health care. American College of Physicians, Washington, DCJanuary 30, 2006http://www.acponline.org/advocacy/events/state_of_healthcare/statehc06_1.pdfGoogle Scholar The shrinking number of general internists and the shortcomings of internal medicine residency training may be contributing to suboptimal care of common chronic diseases such as diabetes and hypertension.2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 4Bodenheimer T. Primary care—will it survive?.N Engl J Med. 2006; 355: 861-864Crossref PubMed Scopus (325) Google Scholar, 11Stevens D.P. Wagner E.H. Transform residency training in chronic illness care—now.Acad Med. 2006; 81: 685-687Crossref PubMed Scopus (29) Google Scholar Internal medicine residents are receiving inadequate training in chronic disease management, and training should be redesigned to improve teaching in ambulatory care.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar, 2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 3Warm E.J. Schauer D.P. Diers T. et al.The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) Education Innovations Project (EIP).J Gen Intern Med. 2008; 23: 921-926Crossref PubMed Scopus (69) Google Scholar, 7Larson E.B. Fihn S.D. Kirk L.M. et al.The future of general internal medicine: report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the domain of general internal medicine.J Gen Intern Med. 2004; 19: 69-77Crossref PubMed Scopus (87) Google Scholar, 8Meyers F.J. Weinberger S.E. Fitzgibbons J.P. Glassroth J. Duffy F.D. Clayton C.P. Alliance for Academic Internal Medicine Education Redesign Task ForceRedesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med. 2007; 82: 1211-1219Crossref PubMed Scopus (115) Google Scholar, 12Huddle T.S. Heudebert G.R. Internal medicine training in the 21st century.Acad Med. 2008; 83: 910-915Crossref PubMed Scopus (27) Google Scholar, 13Arora V. Guardiano S. Donaldson D. Storch I. Hemstreet P. Closing the gap between internal medicine training and practice: recommendations from recent graduates.Am J Med. 2005; 118: 680-685Abstract Full Text Full Text PDF PubMed Scopus (36) Google Scholar, 14Feifer C. Mora A. White B. Barnett B.P. Challenges to improving chronic disease care and training in residencies.Acad Med. 2006; 81: 696-701Crossref PubMed Scopus (22) Google Scholar Many of the suggestions for internal medicine residency redesign have focused on structural aspects of training, including a better balance of the time committed to teaching in ambulatory care. There has been little discussion of how much of the medical knowledge needed for competent ambulatory care is actually taught during residency. We set out to evaluate data generated from a widely used curriculum in ambulatory care to determine what residents learn about topics in ambulatory care during residency training, and to compare knowledge between residents training at university-based residency training programs and residents training at community hospital-based residency training programs. The Johns Hopkins Internet Learning Center was established in 2002 and provides ambulatory care training to medical residents via 38 online training modules. Users of the curriculum include residents at internal medicine residency training programs that pay an annual fee to access the curriculum. Subscribing programs include university hospitals (UH), defined as the primary teaching hospital affiliate of a medical school, and community hospitals (CH), defined as hospitals that may or may not have a loose affiliation with a medical school and are not a military hospital.15Sisson S.D. Hughes M.T. Levine D. Brancati F.L. Effect of an Internet-based curriculum on post-graduate education: a multicenter intervention.J Gen Intern Med. 2004; 19: 503-507Crossref Scopus (42) Google Scholar Demographic data collected at registration includes postgraduate year (PGY) of training and training program. Data generated during the 2006-2007 academic year was used for analysis, at which time 41 community hospitals (representing 16.7% of a possible 245 CH programs) and 26 university hospitals (representing 21.1% of a possible 123 UH programs) were using the curriculum.16Fellowship and Residency Electronic Interactive Database (FREIDA)http://www.ama-assn.org/ama/pub/education-careers/graduate-medical-education/freida-online.shtmlGoogle Scholar During the 2006-2007 academic year, 4724 residents at these 67 residency programs registered to use the curriculum, including 2429 residents at UHs (51.4%) and 2295 residents at CHs (48.6%). All programs make completion of at least part of the curriculum mandatory for all residents, but programs differ in whether or not specific modules are assigned. Of the 4724 residents, 3032 completed at least one module (participation rate, 64.2%). Each of these residents could complete between 1 and 38 modules. A total of 38,982 modules were completed, for an average of 13 modules completed by each resident. Each completed module was treated as an independent observation; performance of any single resident was not tracked from module to module, or from year to year. Topics covered by the curriculum were based on national surveys of common visits to internal medicine clinics, and include common diseases such as diabetes and hypertension, preventive care, and symptom-based topics such as headache and back pain. For this study, modules were grouped into broad areas of chronic disease management, preventive care, or acute/symptom-based care. Each of the 38 modules consists of a pretest, the completion of which allows access to the didactics, followed by a posttest. Performance on the pretest was used to define baseline knowledge among residents, and differences in baseline knowledge between PGY1, PGY2, and PGY3 trainees was used to assess the impact of residency training on baseline knowledge. Scores on the modules range from 0% to 100% and are expressed as mean±SD. Item discrimination was calculated on each question and Cronbach alpha was calculated on each test using the method of Ferguson and Takane.17Ferguson G.A. Takane Y. Statistical Analysis in Psychology and Education.6th edn. McGraw-Hill Book Company, New York1989Google Scholar Item discrimination ranged from 0.26 to 0.64, and Cronbach alpha ranged from 0.3 to 0.8. Scores were compared by year of training and hospital type (CH or UH) using unpaired t tests. The difference in knowledge between same-year residents at UH vs CH was compared using multi-variable regression with interaction. Two sided P-value of <.05 was considered statistically significant for analyses. All statistical analyses were conducted using STATA 10 Professional (Stata Corp., College Station, Tex). Knowledge of all topics covered in the ambulatory curriculum by year of training and hospital type is shown in Figure 1. The average score on all topics for all PGY1 residents was 46.0, for PGY2 it was 50.6, and for PGY3 it was 52.7. Knowledge did not differ between UH and CH PGY1 residents (45.9% vs 46.1%, P = .68). However, CH PGY2 and PGY3 residents had higher scores than did UH residents (PGY2 51.5% vs 49.2%; PGY3 53.7% vs 51.5%; P <.001 for both). Module topics were grouped into chronic disease management, preventive care, and symptom-based acute care. Knowledge among UH and CH residents based on these groupings and year of training is shown in Figure 2. For all residents, knowledge was worst on diagnosis and management of chronic diseases, followed by diagnosis and management of acute care issues. For all residents, knowledge was best on preventive care medicine. Knowledge among UH and CH PGY3 residents on individual topics and groupings of topics is shown in the Table. On individual topics in chronic disease management, knowledge did not differ between the 2 groups. Among topics of chronic disease management, knowledge was worst on lipid management for both UH and CH PGY3 residents (31.1% for UH PGY3 residents and 35.7% for CH PGY3 residents). The average score on chronic disease management was 48% among UH PGY3 residents and 50.1% among CH PGY3 residents, a difference that was statistically significant (P <.005).TablePGY3 Baseline Knowledge by Topic and Program TypeModulesUniversity(Mean±SD)Community(Mean±SD)P-ValueChronic diseases Alcoholism37.9±26.335.9±26.2.51 Anemia33.3±21.736.5±24.3.19 Asthma50.8±22.850.4±21.1.85 Diabetes35.5±19.539.9±23.4.05 Hypertension62.2±17.865.7±21.0.09 Chronic kidney diseases51.6±18.054.8±20.9.12 Dementia56.5±21.461.4±23.6.06 Depression42.4±20.742.8±22.7.85 Gynecology63.2±23.967.7±23.4.17 Menopause/HRT47.6±22.447.1±24.1.87 Obesity43.9±20.846.9±23.1.24 Osteoporosis59.7±22.160.5±24.7.78 Lipid management31.1±21.535.7±25.0.06 Smoking53.3±22.254.4±25.0.66 Thyroid61.5±20.559.1±23.6.38 Total48.0±23.850.1±25.6<.005Preventive care Cancer screening52.9±16.451.1±17.7.31 Immunizations56.7±20.764.3±21.4<.005 Total54.6±18.557.1±20.5.1Symptom/acute care Back pain65.8±30.768.1±27.8.43 Community-acquired pneumonia50.0±21.055.5±21.5.01 Dermatitis65.9±23.363.6±25.6.41 Dizziness24.2±23.934.4±27.1<.005 GERD44.0±20.351.1±24.7<.005 Headaches53.7±24.562.4±28.7<.005 Hip knee pain52.3±22.456.3±24.9.13 Ophthalmology47.1±21.455.1±27.3.01 Pre-op evaluation62.7±21.062.6±23.0.97 Upper respiratory infections55.8±28.359.2±26.2.23 Total51.6±26.757.3±27.1<.001GERD=gastroesophageal reflux disease; HRT=hormone replacement therapy. Open table in a new tab GERD=gastroesophageal reflux disease; HRT=hormone replacement therapy. The average score on preventive care among PGY3 residents was 57.1% for CH PGY3 residents and 54.6% for UH PGY3 residents, a difference that was not statistically significant. CH PGY3 residents outperformed UH PGY3 residents on immunizations (64.3% vs 56.7%; P <.005), but when grouped with cancer screening, there was no significant difference in knowledge of preventive care between UH and CH PGY3 residents. The average score on all symptom-based/acute care topics was 57.3% for CH PGY3 residents and 51.6% for UH PGY3 residents, a difference that was statistically significant (P <.05). CH PGY3 residents outperform UH PGY3 residents on several topics in symptom-based or acute care, including community-acquired pneumonia (55.5% vs 50%; P=.01), dizziness (34.3% vs 24.2%; P <.005), gastroesophageal reflux disease (51.1% vs 44%; P <.005), headaches (62.4% vs 53.7% P <.005), and ophthalmology (55.1% vs 47%; P=.01). For all PGY3 residents, knowledge of symptom-based/acute care topics was worst on dizziness. The differences in ambulatory knowledge between same-year residents at CH and UH programs are compared in Figure 3. There was no difference in ambulatory knowledge among PGY1 residents at university and community hospitals. Among PGY2 residents, ambulatory knowledge was higher at CHs than at UHs. This same difference in knowledge also was present between PGY3 residents at community and university hospitals. We showed that baseline knowledge among residents at university and community hospitals on topics in ambulatory care is poor. PGY3 residents have greater knowledge of ambulatory topics than PGY1 residents, but this difference is modest. Baseline knowledge of ambulatory care does not differ between PGY1 residents at university hospital-based and community hospital-based training programs, but among PGY2 and PGY3 residents, residents at community programs have greater knowledge on these topics than residents at university programs. At the end of residency training, graduates from community hospitals have greater knowledge than graduates from university hospitals on chronic disease management (especially diabetes) and symptom-based/acute management, but not on preventive care. Training for all residents is particularly poor on outpatient diagnosis and management of diabetes, lipid disorders, dizziness, anemia, and alcoholism. Our results quantify some of the shortcomings in ambulatory training during internal medicine residency that has prompted so many to call for change. PGY1 residents start training with significant knowledge gaps on topics in ambulatory care regardless of where they train. PGY2 and PGY3 residents have only slightly better knowledge on ambulatory topics. Poor performance by PGY3 residents may be a result of bias towards inpatient training, hindering resident ambulatory education, which is not a priority at most institutions.2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 3Warm E.J. Schauer D.P. Diers T. et al.The ambulatory long-block: an Accreditation Council for Graduate Medical Education (ACGME) Education Innovations Project (EIP).J Gen Intern Med. 2008; 23: 921-926Crossref PubMed Scopus (69) Google Scholar, 13Arora V. Guardiano S. Donaldson D. Storch I. Hemstreet P. Closing the gap between internal medicine training and practice: recommendations from recent graduates.Am J Med. 2005; 118: 680-685Abstract Full Text Full Text PDF PubMed Scopus (36) Google Scholar Our results also suggest that university hospitals do not do as good a job as community hospitals at teaching residents about ambulatory care. The culture of a training program has a significant impact on residents.11Stevens D.P. Wagner E.H. Transform residency training in chronic illness care—now.Acad Med. 2006; 81: 685-687Crossref PubMed Scopus (29) Google Scholar Some have noted that the culture at university hospitals, where specialists often assume leadership roles in resident education, is not hospitable toward primary care, where specialized medicine is emphasized over a broad education in general internal medicine.6Garibaldi R.A. Popkave C. Bylsma W. Career plans for trainees in internal medicine residency programs.Acad Med. 2005; 80: 507-512Crossref PubMed Scopus (125) Google Scholar, 18Block S.D. Clark-Chiarelli N. Peters A.S. Singer J.D. Academia's chilly climate for primary care.JAMA. 1996; 276: 677-682Crossref PubMed Google Scholar Knowledge on topics in chronic disease management was poor among all trainees. Competent management of chronic diseases is a core skill of general internists, and as the prevalence of chronic diseases such as diabetes, hypertension, obesity, lipid disorders and others increase, more general internists will be needed.5The impending collapse of primary care medicine and its implications for the state of the nation's health care. American College of Physicians, Washington, DCJanuary 30, 2006http://www.acponline.org/advocacy/events/state_of_healthcare/statehc06_1.pdfGoogle Scholar Symptom-based and acute care knowledge among PGY3 residents was only marginally better than knowledge of chronic disease management. If the mission of internal medicine residency programs is to train physicians to meet society's health care needs, then our results suggest that they are failing. We note with irony that all surveyed programs use our ambulatory curriculum, most for several years, yet performance among all trainees, including PGY3 residents, was poor. More than just didactic modules on ambulatory care are needed to improve ambulatory training during internal medicine residency. The inpatient and specialty-based orientation of university hospital-based residency programs represents a large barrier to improving training on chronic disease management.14Feifer C. Mora A. White B. Barnett B.P. Challenges to improving chronic disease care and training in residencies.Acad Med. 2006; 81: 696-701Crossref PubMed Scopus (22) Google Scholar Our study has several limitations. We compare groups of trainees that were evaluated simultaneously; we did not track group performance over 3 years of training. Individual performance also was not assessed. Factors prompting individual residents to complete any particular module were not considered, and may have introduced selection bias. Training programs included in this study subscribed to a shared ambulatory curriculum and include only a portion of all internal medicine residency training programs, thus results may not be generalizable. We assessed only knowledge; attitudes, skills, and patient care were not assessed. We also did not do a comparative assessment with knowledge of topics on inpatient medicine. All of these limitations merit further study to better identify the strengths and weakness of internal medicine residency training in ambulatory care. The American College of Physicians, the Association of Program Directors in Internal Medicine, the Society of General Internal Medicine, and the Alliance for Academic Internal Medicine Education Redesign task force all call for redesign of internal medicine residency training programs to include greater focus on ambulatory care.1Fitzgibbons J.P. Bordley D.R. Berkowitz L.R. Miller B.W. Henderson M.C. Redesigning residency education in internal medicine: a position paper from the Association of Program Directors in Internal Medicine.Ann Intern Med. 2006; 144: 920-926Crossref PubMed Scopus (126) Google Scholar, 2Weinberger S.E. Smith L.G. Collier V.U. Redesigning training for internal medicine.Ann Intern Med. 2006; 144 (977932)Google Scholar, 7Larson E.B. Fihn S.D. Kirk L.M. et al.The future of general internal medicine: report and recommendations from the Society of General Internal Medicine (SGIM) Task Force on the domain of general internal medicine.J Gen Intern Med. 2004; 19: 69-77Crossref PubMed Scopus (87) Google Scholar, 8Meyers F.J. Weinberger S.E. Fitzgibbons J.P. Glassroth J. Duffy F.D. Clayton C.P. Alliance for Academic Internal Medicine Education Redesign Task ForceRedesigning residency training in internal medicine: the consensus report of the Alliance for Academic Internal Medicine Education Redesign Task Force.Acad Med. 2007; 82: 1211-1219Crossref PubMed Scopus (115) Google Scholar This redesign should be informed by evaluation of the strengths and weaknesses in our current training system. We have found specific areas of medical knowledge, especially of chronic disease management, for which knowledge is poor among both PGY1 and PGY3 residents. Further study is needed to define other weaknesses in training and how to address them.