Tracking Knowledge Growth across an Integrated Nutrition Curriculum

Two studies imply that, even when nutrition is not a major aspect of the medical school curriculum, firstand second-year students’ knowledge can increase and they may apply their knowledge to patient care, and increase of nutrition knowledge following exposure to the clinical curriculum is potentially of greater importance than is the nutritional content of the basic science curriculum.

Abstract

Both the academic literature and the popular press continually report the importance of nutrition for health. One example is the increasing prevalence of obesity in the United States1 and the concomitant popularity of diet books and untested remedies. A 1985 National Academy of Sciences report warned of the lack of nutrition education and the need for a required curriculum for all medical students in U.S. medical schools.2 Results from annual Association of American Medical Colleges (AAMC) Graduation Questionnaires reinforce this conclusion. In 1995, 63% of students reported that they had received inadequate nutrition in their medical school curricula.3 In 1998, nothing had changed; 64% of students still reported inadequate nutrition education.4 Following the 1985 National Academy of Sciences report, funding from the National Cancer Institute (NCI) stimulated development of nutrition curricula at a number of medical schools.5,6,7 In one study, the use of a multimedia program to teach nutritional assessment and counseling was evaluated.6 The authors found that, following exposure to the multimedia nutrition program, first-year students were more likely to use a food-frequency form while interviewing a standardized patient compared with previous students who had not received the intervention. A majority of students (51%) who completed the curriculum reported that observing a physician model nutritional assessment and counseling in the multimedia program had been helpful. In another study, the evaluation of a two-year integrated nutrition curriculum implemented during the basic sciences indicated increased knowledge for those students who had completed the curriculum.7 Although limited in scope, these studies are promising. They imply that, even when nutrition is not a major aspect of the medical school curriculum, first- and second-year students' knowledge can increase and they may apply their knowledge to patient care. The increase of nutrition knowledge following exposure to the clinical curriculum is potentially of greater importance than is the nutritional content of the basic science curriculum, since clinical exposure may be more likely to lead to application in practice. Many studies report physicians' lack of knowledge and confidence in using nutritional concepts in their practices.8,9,10,11 The paucity of physicians who model the use of nutrition concepts in their practices could have a negative effect on students' acquisition of knowledge and their application of that knowledge to patient care.12 At our institution, cognitive learning theory (i.e., actively engaging students in learning13) guided the development of a new nutrition curriculum. The curriculum's goals were to increase students' (1) learning and retention of nutritional concepts; (2) skills, such as diet-assessment methods; and (3) application of content to patients' care. To accomplish this, we planned to increase opportunities for practice with nutritional concepts throughout the four-year curriculum using active learning methods such as laboratory exercises, a dietary self-assessment, interviews with standardized patients, and discussions in small-group problem-based learning (PBL) sessions. In 1992, we started the curricular planning process by conducting a nutrition needs assessment. We received funding of an NCI R25 grant (NCI PAR 94-005) in 1994, and a Nutrition Education Committee was formed to develop and implement the new nutrition curriculum. The Committee established goals and objectives (outlined on our Web site 〈http://apps.medsch.ucla.edu/nutrition/objectives.html〉), reviewed existing courses and clerkships, and implemented changes in years one, two and three of the curriculum. New instructional and examination materials were developed to foster accomplishment of nutrition proficiencies outlined on the Web site above. The development of ongoing curricular review and evaluation processes tracked growth of nutritional knowledge in those students exposed to the revised curriculum. Modification of targeted courses to emphasize proficiency with nutritional concepts was the primary strategy of the curricular change. The nutrition curriculum is concentrated in the first-year course, Human Biochemistry and Nutrition Laboratory. A number of nutrition-related cases are also included in two first-year PBL courses. Nutrition is included in approximately ten lectures of the second-year course, Pathophysiology of Disease. New curricular material was incorporated into the required third-year family medicine clerkship and the Doctoring 3 curriculum, where students interview standardized patients. Numerous fourth-year nutrition electives are offered, but their impact is limited because very few students take these electives. In this study, we examined the effect of changes in the nutrition curriculum on students' knowledge over four years of medical school. Based on earlier findings, and further development and implementation of nutritional content in the clinical curriculum, we hypothesized that students completing an integrated four-year nutrition curriculum would demonstrate, on a Nutrition Progress Survey, a continual increase in their nutrition knowledge over time. We also hypothesized that they would demonstrate more confidence in their responses through a decrease in their use of “don't know” as a response to survey questions. Method We used a pre-/post-test intact-group design to evaluate changes in the nutrition knowledge of a cohort of medical students as they progressed from their first to fourth years (class of 1998). Test items that originally had been developed at the University of Alabama and had been demonstrated to be valid and reliable measures for assessing the nutritional knowledge of medical students formed the basis of our 90-item Nutrition Knowledge Progress Survey. In order to decrease the use of guessing, students were given an additional response option, “don't know,” for all questions. The students were informed that the test items would be scored (correct = +1, incorrect = −1, and don't know = 0). All students completed an informed consent form prior to entering the study. The nutrition survey was administered as a pre-test to the first-year class in January 1995. A 45-item subtest of the survey (30 items expected by first-year course chairs to be initially covered in the first-year curriculum and 15 randomly selected items) was administered in May 1995 (post-test 1) to the same cohort of students. Delayed post-test exams were given to third-year students in August 1996 (post-test 2) and to fourth-year students in August 1997 (post-test 3). Two forms of post-test 2 were administered to third-year students: the full 90-item and the 45-item subtests of the survey. Earlier we reported no significant difference between the scores on the 45 items in common on the two forms of the test.7 The 90-item exam was given at the start of the fourth year to a randomly selected half of the cohort (n = 76). Those students who completed all four previous surveys were asked to complete one more at the end of the fourth year (post-test 4). Each fourth-year student who participated received a $100 gift certificate as an incentive. Total scores were calculated by summing the scores for the items in each exam (correct = +1, incorrect = −1, and don't know = 0). The 45 items in common for each test were summed to form a total score for each administration of the survey. Additionally, the 30 items in the survey covered in the first-year curriculum and used in subsequent years were summed to create a total score for the first-year curriculum in order to test for learning and retention of material. In order to test whether the total numbers of correct, incorrect, and “don't know” answers changed over time, total scores for these answers were calculated by summing the number of responses for each category. A repeated-measures analysis of variance (ANOVA) was used to examine changes between the time points. The difference method was used to compare each time point with the previous one. It was possible that those students who completed the survey every time it was given differed from those students who did not (i.e., were more knowledgeable about nutrition). In order to test this, a sample was randomly selected (equaling the sample size of those who completed all four surveys) from those students who had not completed all four exams. Pre-test, post-test 1, and post-test 2 total scores were compared for these two groups. Results Approximately 90% of the cohort completed at least one of the four exams: 88% at pre-test (n = 130), 93% at post-test 1 (n = 136), 72% at post-test 2 (n = 89), and 70% at post-test 3 (53 of 76 students recruited to complete the nutrition survey). Fifty-three percent of the students (n = 78) completed the first three exams. Twenty of the 24 students (83%) who filled out the first four surveys completed post-test 4. The first set of data reported includes all students who completed the nutrition survey at the first four test administrations except for the comparison group. The second set of data reported includes only those students who completed the survey at all five administrations. There was a significant increase in knowledge over the four test administrations (see Table 1). Results of the repeated-measures ANOVA showed a significant increase in knowledge over the three-year time period using the 45-item subtest. The number of correct answers increased; the numbers of incorrect and “don't know” responses decreased. Within-subject comparisons between each time period and the previous time period were also significant (see Table 1), indicating a significant increase in knowledge from one time point to the next. In addition, knowledge relative to the content covered in the first-year curriculum (30-item subtest) increased over time (see Figure 1).TABLE 1: Students' Responses to the Nutrition Knowledge Progress Survey*Figure 1: Mean scores ± 2 standard errors on the Nutrition Knowledge Progress Survey (30 items from the first-year curriculum) for those students who completed all five test administrations (n = 20). Repeated-measures ANOVA: f = 23.3 (4, 16), p <.001. The test scored +1 for correct response, 0 for “don't know,” and −1 for an incorrect answer.Students who completed the first four nutrition surveys (n = 24) were compared with randomly selected groups of students who did not complete all four surveys on pre-test, post-test 1, and post-test 2 mean scores. There was no significant difference between the two groups on any of these measures, indicating that there was no bias in terms of nutrition knowledge as to who completed all of the four nutrition surveys. Discussion Results from this study indicate that the goals of the curriculum were met; medical students who received the longitudinal integrated nutrition curriculum did increase their knowledge over time and retained the knowledge learned in the first year through the third year (see Figure 1). In addition, students appeared to be more confident in their responses, since they decreased their use of the “don't know” response, even though they risked losing points for an incorrect answer. These results, however, do not mean that students are more able to apply their knowledge in the clinical setting. In contrast, anecdotally, we know from speaking informally with fourth-year medical students that they felt very uncomfortable being alone in an exam room with a patient who asked about diet or supplements. Results from the AAMC Graduation Questionnaire confirmed this. Even though our students clearly increased their knowledge over time, 68% of this cohort still reported inadequate education in nutrition in their curriculum, compared with 64% nationally.4 This finding might reflect the students' greater understanding of the importance of nutrition in clinical practice based on the curriculum. On the other hand, it may be that their own clinical experience, although limited, had informed them of their need to know. There are a number of limitations to this study. First, only one school was studied, so results might not be comparable in another school. There may have been a test effect from using the same survey over time, although given the time lag between administrations and the lack of grading associated with it, this seems unlikely. There may have been sample bias if those students who completed the survey all five times were more interested in nutrition. Again, this is unlikely given the comparison of those students who took all five tests with those who took only the pre-test, post-test 1, or post-test 2. Finally, it is possible that the results are purely from a maturation effect. This is not likely, however, given our earlier study results indicating no significant difference in a comparison of scores on post-test 2 of a control group (those not completing the nutrition curriculum) with scores of students who had completed the nutrition curriculum.7 Results from this study are promising, but there is still a way to go—one of the biggest hurdles remaining is incorporating nutrition into the clinical curriculum. The average number of items answered correctly by those graduating students who completed the survey was 32 of 45, indicating a marginally passing score of 71%. This denotes an increase of only 13% from their scores at the end of the first year. However, these results are similar to those of a multi-school study conducted in the late 1980s, in which fourth-year students at 11 southeastern U.S. medical schools scored an average of 69% on a similar survey. Scores were related to the amount of required nutrition curriculum the students had experienced. Although knowledge scores increased, students' attitudes with respect to the importance of nutrition for their careers deteriorated from year one to the end of the clinical curriculum.14 At our institution, nutritional content increased in the third-year curriculum, but little advancement was made into any clerkship except family medicine. Given the general lack of nutrition knowledge of clinicians,8,9,10,11 it is likely that there were few preceptor role models who demonstrated or reinforced nutritional assessment or dietary counseling of patients. Consistent with this are the results of a study comparing the nutrition knowledge of our fourth-year students with that of physicians attending a local nutrition continuing medical education course. The students significantly outscored the physicians in nutrition knowledge (68% versus 52%).15 Last, although a case with nutritional content was inserted into our senior clinical performance examination, this occurred after this cohort of students had graduated. Therefore, the only change observed, (an increase of students' knowledge of nutrition concepts) provides no evidence that students will apply this information in clinical practice—our ultimate goal. Further studies are needed to examine this potential effect of the curriculum.