[Skip to Content]
Access to paid content on this site is currently suspended due to excessive activity being detected from your IP address 54.159.197.114. Please contact the publisher to request reinstatement.
Sign In
Individual Sign In
Create an Account
Institutional Sign In
OpenAthens Shibboleth
[Skip to Content Landing]
Download PDF
Figure 1. Example Concept Map
Image description not available.
Example map of water demonstrating the basic components of hierarchical concept maps. Note the cross-link (bold arrow) between the concepts motion and states. Reproduced with permission from Cambridge University Press.14
Figure 2. Low-Scoring Concept Map
Image description not available.
Reproduction of a hand-written concept map of seizures by a first-year resident. Note the absence of cross-links and the consistent use of 2 to 3 levels of hierarchy, resulting in a low score.
Figure 3. High-Scoring Concept Map
Image description not available.
Reproduction of a hand-written concept map of seizures by a resident in the second- and third-year group. Note the complexity of the map as manifested by frequent cross-links and the consistent use of 5 to 7 levels of hierarchy, resulting in a higher map score.
Figure 4. Partial Reproduction of Preinstruction and Postinstruction Concept Maps From the Same Subject
Image description not available.
Reproduction of approximately 50% of both a preinstuction and postinstruction concept map from the same subject.
Table 1. Comparison of Total Concept Map and Subscores in Second- and Third-Year vs First-Year Residents*
Image description not available.
Table 2. Comparison of Total Concept Map and Subscores Before and After Educational Intervention*
Image description not available.
1.
Tosteson DC. New pathways in general medical education.  N Engl J Med.1990;322:234-238.
2.
Piaget J. The Language and Thought of the Child3rd ed. London, England: Routledge & Kegan Paul; 1959.
3.
Ausubel DP. The Psychology of Meaningful Verbal LearningNew York, NY: Grune & Stratton; 1963.
4.
Ausubel DP, Novak JD, Hanesian H. Educational Psychology: A Cognitive View2nd ed. New York, NY: Holt Rinehart & Winston; 1978.
5.
Markow PG, Lonning RA. Usefulness of concept maps in college chemistry laboratories: students' perceptions and effects on achievement.  J Res Sci Teach.1998;35:1015-1029.
6.
Bodner GM. Constructivism: a theory of knowledge.  J Chem Educ.1986;63:873-878.
7.
Mintzes JJ, Wandersee JH, Novak JD. Teaching Science for Understanding: A Constructivist ViewSan Diego, Calif: Academic Press; 1998.
8.
Posner GJ, Strike KA, Hewson PW, Gertzog WA. Accommodation of a scientific conception: toward a theory of conceptual change.  Sci Educ.1982;66:211-227.
9.
von Glaserfeld E. The Practice of Constructivism in Science and Mathematics EducationWashington, DC: American Association for the Advancement of Science Press; 1992.
10.
Glaser R, Bassok M. Learning theory and the study of instruction.  Annu Rev Psychol.1989;40:631-666.
11.
Royer J, Cisero CA, Carlo MS. Techniques and procedures for assessing cognitive skills.  Rev Educ Res.1993;63:201-224.
12.
Ruiz-Primo MA, Shavelson RJ. Problems and issues in the use of concept maps in science assessment.  J Res Sci Teach.1996;33:569-600.
13.
Novak JD. Audio-tutorial techiques for individualized science instruction in the elementary school. In: Triezenberg H, ed. Individualized Science: Like It Is. Washington, DC: National Science Teachers Association; 1972:14-30.
14.
Novak JD, Gowin DB. Learning How to LearnCambridge, England: Cambridge University Press; 1984:15-54.
15.
Novak JD, Musonda D. A twelve year study of science concept learning.  Am Educ Res J.1991;28:117-153.
16.
Novak JD, Gowin DB, Johansen G. The use of concept mapping and knowledge vee mapping with junior high school science students.  Sci Educ.1983;67:625-645.
17.
Markham KM, Mintzes JJ, Jones MG. The concept map as a research and evaluation tool: further evidence of validity.  J Res Sci Teach.1994;31:91-101.
18.
Beyerbach BA. Concept mapping as an approach to assessment of students' representation of structural knowledge.  Dissertation Abstracts Int.1986;46:2622A.
19.
Novak JD. Concept mapping: a strategy for organizing knowledge. In: Glynn SM, Duit R, eds. Learning Science in the Schools: Research Reforming Practice. Mahwah, NJ: Lawrence Erlbaum Associates; 1995:229-245.
20.
Novak JD. Applying learning psychology and philosophy of science to biology teaching.  Am Biol Teach.1981;43:13-20.
21.
Novak JD. Concept mapping: a useful tool for science education.  J Res Sci Teach.1990;27:937-949.
22.
Arnaudin MW, Mintzes JJ, Dunn CS, Shafer TH. Concept mapping in college science teaching.  J College Sci Teach.1984;14:117-121.
23.
McClure JR, Sonak B, Suen HK. Concept map assessment of classroom learning: reliability, validity, and logistical practicality.  J Res Sci Teach.1999;36:475-492.
24.
Roletto E, Regis A, Albertazzi PG. Concept maps in chemistry education.  J Chem Ed.1996;73:1084-1088.
25.
Lahtinen V, Lonka K, Lindbloom-Ylaenne S. Spontaneous study strategies and the quality of knowledge construction.  Br J Educ Psychol.1997;67:13-24.
26.
Coleman EB. Using explanatory knowledge during collaborative problem solving in science.  J Learn Sci.1998;7:387-427.
27.
Heinze-Fry JA, Novak JD. Concept mapping brings long-term movement toward meaningful learning.  Sci Educ.1990;74:461-472.
28.
Edmondson KM, Smith DF. Concept mapping to facilitate veterinary students' understanding of fluid and electrolyte disorders.  Teach Learn Med.1998;10:21-33.
29.
Chastonay P, Papart JP, Laporte JD.  et al.  Use of concept mapping to define learning objectives in a master of public health program.  Teach Learn Med.1999;11:21-25.
30.
Edmondson KM. Concept maps and the development of cases for problem-based learning.  Acad Med.1994;69:108-110.
31.
Bar-Lavie BZ. Enhancing meaningful learning in an environmental education program: a case study of class empowered through the use of Novak's and Gowin's principles of learning how to learn, concept mapping, interviewing, and educating.  Dissertation Abstracts Int.1988;48:2590A.
32.
Stuart HA. Should concept maps be scored numerically.  Eur J Sci Educ.1985;7:73-81.
33.
McGaghie WC, Boerger RL, McCrimmon DR, Ravitch MM. Learning pulmonary physiology: comparison of student and faculty knowledge structures.  Acad Med.1996;71(suppl):S13-S15.
Original Contribution
September 6, 2000

Critical Thinking in Graduate Medical EducationA Role for Concept Mapping Assessment?

Author Affiliations

Author Affiliations: Department of Pediatrics, School of Medicine (Drs West, Park, and Gerstenberger) and Division of Education (Drs Pomeroy and Sandoval), University of California, Davis.

JAMA. 2000;284(9):1105-1110. doi:10.1001/jama.284.9.1105
Context

Context Tools to assess the evolving conceptual framework of physicians-in-training are limited, despite their critical importance to physicians' evolving clinical expertise. Concept mapping assessment (CMA) enables teachers to view students' organization of their knowledge at various points in training.

Objective To assess whether CMA reflects expected differences and changes in the conceptual framework of resident physicians, whether concept maps can be scored reliably, and how well CMA scores relate to the results of standard in-training examination.

Design, Setting, and Participants A group of 21 resident physicians (9 first-year and 12 second- and third-year residents) from a university-based pediatric training program underwent concept map training, drew a preinstruction concept map about seizures, completed an education course on seizures, and then drew a postinstruction map. Maps were scored independently by 3 raters using a standardized method. The study was conducted in May and June 1999.

Main Outcome Measures Preinstruction map total scores and subscores in 4 categories compared with postinstruction map scores; map scores of second- and third-year residents compared with first-year residents; and interrater correlation of map scores.

Results Total CMA scores increased after instruction from a mean (SD) preinstruction map score of 429 (119) to a mean postinstruction map score of 516 (196) (P = .03). Second- and third-year residents scored significantly higher than first-year residents before instruction (mean [SD] score of 472 [116] vs 371 [102], respectively; P = .04), but not after instruction (mean [SD] scores, 561 [203] vs 456 [179], respectively; P = .16). Second- and third-year residents had greater preinstruction map complexity as measured by cross-link score (P = .01) than first-year residents. The CMA score had a weak to no correlation with the American Board of Pediatrics In-training Examination score (r = 0.10-0.54). Interrater correlation of map scoring ranged from weak to moderate for the preinstruction map (r = 0.51-0.69) and moderate to strong for the postinstruction map (r = 0.74-0.88).

Conclusions Our data provide preliminary evidence that concept mapping assessment reflects expected differences and change in the conceptual framework of resident physicians. Concept mapping assessment and standardized testing may measure different cognitive domains.

During the past few decades medical curricula have increasingly reflected the belief that students should understand concepts, as opposed to rote memorization of facts or procedural algorithms. Many medical schools have adopted alternative teaching and learning strategies that challenge students to become self-directed learners who continuously expand the limits of their knowledge and seek evidence for clinical decision making.1 The root of these ideas lies in the constructivist theory of learning, which states that meaningful learning occurs when students assimilate new knowledge into a preexisting conceptual framework.29 This evolving conceptual framework supports expertise in critical thinking and clinical problem solving.1012 Despite its critical importance in the education of future physicians, tools to evaluate the quality of students' conceptual frameworks are limited because of the difficulty in designing and administering objective measures.

In this article, we describe an application of concept mapping assessment (CMA) that provides a "window" into learners' organization of knowledge at various points in training. Concept mapping is a technique developed by Joseph Novak in the 1970s in which students create a 2-dimensional diagram outlining their understanding of the relationships between and among important concepts within a given subject area.1315 Unlike most conventional examinations, CMA may provide unique insights into how an individual organizes his or her knowledge or comes to think about a problem. While concept mapping has been used frequently as a learning tool, especially in high school and college science education, it has only rarely been used as an assessment method, primarily because there is limited information regarding its validity and reliability.12,1618

Therefore, we undertook this preliminary study to assess whether CMA can measure predicted differences and expected changes in the conceptual framework of resident physicians. In addition, we sought to determine whether concept maps could be scored in a reliable way.

METHODS
Study Subjects and Design

All pediatric residents (33 total) at our institution were eligible to participate in this study, which was approved by our institutional human subjects committee. Written informed consent was obtained. A total of 21 residents enrolled: 9 first-year residents, 8 second-year residents, and 4 third-year residents. Each participant underwent concept map training followed by drawing a preinstruction concept map about the topic seizures. They then participated in a 3-session seizure education course followed by drawing a postinstruction map about seizures.

All study subjects participated in a standardized concept map training exercise based on previously described methods.14 Within 2 to 3 weeks of completing map training, participants were given 30 minutes to independently, and without study aids, draw their concept of seizures on a blank sheet of paper (preinstruction map). Residents then participated in a 3-session (1 hour per session) seizure education course consisting of case-based instruction. One week after completing the course, residents drew a postinstruction concept map about seizures under the same conditions as before.

Concept Maps

We used a modification of the "hierarchical technique" of concept mapping in which a concept map consists of 5 components: concepts, concept links, hierarchies, cross-links, and examples (see Figure 1 for an example concept map of water).14,19

A concept is defined as a perceived regularity in events or objects designated by a label and, when depicted on a map, enclosed with a circle. Creating a concept map involves connecting related concepts using arrow lines with a statement or proposition written above the line describing how the concepts are related.

A concept linked to another concept by an arrow line with a propositional statement, called a concept link, represents the foundational unit of a concept map.3,14,20,21 In Figure 1, the concept water connected to living things by the proposition needed by represents a concept link. The expression of the hierarchy between concepts is indicated by the direction of the arrow in the concept link with the most general concepts arranged at the top and more specific concepts arranged hierarchically below. Three levels of hierarchy are represented in Figure 1. The linking of a series of related concept links in a hierarchical fashion represents a domain of knowledge. Cross-links are the method by which the map author indicates how different domains of knowledge are related to each other and are a critical measure of map complexity. In a manner similar to propositions, cross-links are labeled with a phrase that describes the relationship of the domains. In the example map, motion connected to states by the proposition determines defines a cross-link.

Examples are labeled and connected to the related concept in the most subordinate position on the map. In Figure 1, Lake Tahoe is a specific example of a lake. Theoretically, as a learner's conceptual framework changes, he or she links new propositions sequentially (concept links), general to specific (hierarchy), and across domains (cross-links), thus demonstrating integration of knowledge.22

Scoring Method

Maps were scored independently by 3 different raters, blinded to the identity of the map author, and using a modification of the structural scoring method previously described.7,14 Each rater was a board-certified pediatrician who underwent a 30-minute concept map scoring training exercise immediately prior to grading the maps. Maps were scored in 4 categories: concept links, hierarchy, cross-links, and examples. The value of each scoring category represents the depth of thinking required to form the category.7,14 Each valid concept link was given 2 points, and each level of hierarchy was given 5 points. A cross-link was given 10 points, while an example counted for 1 point. Invalid links or concepts were given 0 points. A total score and subscore for each category (concept links, cross-links, hierarchy, and examples) were generated for each map. Standard total scores and standard subscores consisting of the sum of individual rater total scores and subscores, respectively, were assigned to each map.

Data Analysis

Statistical analysis was done using STATA, version 6.0 (College Station, Tex). The correlation of each rater's total scores and subscores for each map was tested using Spearman rank correlation. All other data analyses were done using standard total scores and standard subscores. We compared preinstruction map to postinstruction map scores using the Wilcoxon signed rank test. To assess whether there was a difference in map scores based on level of training or experience, we combined the more experienced second- and third-year residents (n = 12) into 1 group and compared their scores with those of the first-year (n = 9) residents using the 2-sample Wilcoxon rank sum test. To assess the relationship of concept map scores to standard written examinations, we compared the most recent American Board of Pediatrics In-training Examination (ITE) scores of second- and third-year residents to first-year residents using the 2-sample Wilcoxon rank sum test. First- and second-year residents took the examination within 1 month of completing this study, while third-year residents took the examination 10 months prior to completing the study. We then tested the relationship of concept map scores to ITE scores using Spearman rank correlation.

RESULTS

All 21 participants completed all phases of the study including concept map training, the seizure education course, and both preinstruction and postinstruction concept maps.

Concept Map Scores Based on Level of Training

Figure 2 shows a complete low-scoring concept map from a first-year resident, while Figure 3 shows a high-scoring map from a resident in the second- and third-year group. Note the complexity, with numerous cross-links in the map from the second- and third-year group compared with the linear simplicity of the first-year resident's map. Table 1 summarizes the mean standard total and mean standard subscores of study participants by level of training. The second- and third-year residents had significantly higher total (P = .04) and cross-link scores on the preinstruction map (P = .01) compared with first-year residents. There was no significant difference between concept link, hierarchy, and example scores based on level of training. The postinstruction maps demonstrated that while the total and subscores for second- and third-year residents remained higher, these differences did not reach significance. Of particular note is that the significant difference in total score and cross-link score seen on the preinstruction maps was no longer apparent after instruction (P = .16 and .15, respectively).

Concept Map Scores After Educational Intervention

Total scores increased significantly after the educational intervention (P = .03). In addition, cross-link (P = .02) and concept link (P = .01) subscores also increased significantly. Subscores for hierarchy and examples both decreased slightly, but this change did not reach significance. Preinstruction and postinstruction maps from 1 subject are shown in Figure 4. The resident added a new conceptual domain for prevention and education as well as new cross-links to increased intracranial pressure. The vague concept cerebral irritation included as a focal point in the preinstruction map was omitted in the postinstruction map. Table 2 summarizes the mean standard score and subscores of all subjects before (preinstruction map) and after (postinstruction map) the seizure education course. Total scores increased significantly after the educational intervention (P = .03). In addition, cross-link (P = .02) and concept link (P = .01) subscores also increased significantly. Subscores for hierarchy and examples both decreased slightly, but this change did not reach significance.

Concept Map Scores and ITE

The mean (SD) ITE score of second- and third-year residents was 358 (135) and of first-year residents, 243 (94). The ITE scores of second- and third-year residents were significantly higher than those of first-year residents (P = .05). There was weak to no correlation between CMA scores (total and subscores) and ITE scores (r = 0.10-0.54). The strongest correlation was between ITE score and the example score on the preinstruction (r = 0.54; P = .01) and postinstruction map total score (r = 0.53; P = .01). The correlation between ITE scores and postinstruction map total scores, preinstruction map cross-link scores, and postinstruction map cross-link scores was r = 0.29, 0.34, and 0.20. respectively.

Interrater Correlation of Concept Map Scores

The interrater correlation was weak to moderate (r = 0.51-0.69) for the preinstruction map and moderate to strong for the postinstruction map (r = 0.74-0.88). The interrater correlation for subscores demonstrated an almost identical range to that seen in the analysis of total scores.

COMMENT

We found that concept map scores of residents increased significantly following completion of an educational program after which conceptual framework would be expected to change. This change is perhaps best illustrated by qualitative comparison of the preinstruction and postinstruction maps (eg, Figure 4), which demonstrates the incorporation of additional concepts into the postintervention map, further differentiation of existing concepts, and a change in cross-linking between concepts. Quantitatively, these changes are reflected in a statistically significant increase in total map score and concept link and cross-link subscores. Despite the small sample size, we also found that CMA scores of residents with more training and expertise were significantly higher than those with less training. This difference was the result of greater map complexity, as reflected in the significantly higher cross-link scores of second- and third-year residents. Thus, more experienced residents appeared to organize their knowledge differently, expressing links between concepts that were not observed by less experienced residents. It is important to note that this difference narrowed, as one might anticipate, after all residents participated in the same educational intervention. While not conclusive, these findings provide important preliminary evidence supporting our hypothesis that CMA is a valid measure of conceptual framework and the expected change in that framework with new learning.

We also found that while ITE scores of second- and third-year residents were significantly higher than those of first-year residents, these scores did not correlate well with CMA scores. Thus, residents who scored highly on CMA were not the same residents who scored highly on the ITE examination. This finding is consistent with other reports, which note that CMA scores of college students do not correlate with conventional measures of learning, such as final course grades or scores on standardized tests.16 While there may be other explanations, the absence of a positive correlation suggests that CMA measures a different knowledge characteristic than do multiple choice examinations.

The issue of scoring reliability is critically important to any future use of CMA. We found that interrater reliability was moderate to strong and, notably, was greater on postinstruction maps. This level of scoring reliability is similar to that reported by others and suggests that gains in scoring experience lead to increased consistency across raters.23 Therefore, rater training would be important to any future applications.

The vast majority of previous research regarding concept mapping has involved high school and college science education. In reviewing this literature, there is clear evidence that concept mapping can be an effective learning tool in a variety of classroom settings—including the education of health care professionals.1618,2430 However, evidence supporting the validity of concept mapping as an assessment tool is much more limited, probably because there is no criterion standard measure of conceptual framework to which CMA can be compared. Traditional measures of competence, such as board scores or even newer competency-based assessments (eg, Objective Structured Clinical Examinations), may not represent appropriate comparisons since they shed little light on how a learner thinks about a problem.12 Thus, other investigators have used the same strategy to build a case for the validity of CMA. For example, in a variety of secondary school and undergraduate settings, CMA has consistently been found to measure conceptual change in situations in which such change would be expected and to detect differences between students with more training in a particular field compared with those with less.14,1618,22,31,32 In medical education, we identified only a single related study in which Pathfinder networks, a substantially different form of concept mapping, was used to assess student understanding of pulmonary physiology.33 In that study, the investigators found that, after instruction, student Pathfinder networks became more similar to the networks generated by faculty experts. These findings appear consistent with our study, and while not conclusive, provide additional evidence suggesting the predictive validity of CMA.

We have provided encouraging preliminary evidence supporting the validity of CMA as a way to measure conceptual change and differences among resident physicians. Our findings also indicate that rater training is critical to the use of CMA. Future research must focus on further validating this method and addressing issues of reliability before CMA can be applied on a larger scale. Nevertheless, CMA has the potential to evaluate how students or residents organize and use knowledge in a way that traditional objective tests cannot. It also could provide a tool for educators to identify unique distortions in students' understanding of content and to identify errors of omission. Finally, CMA might provide insight into why some residents score well on objective written examinations but have difficulty applying this knowledge to clinical situations.

References
1.
Tosteson DC. New pathways in general medical education.  N Engl J Med.1990;322:234-238.
2.
Piaget J. The Language and Thought of the Child3rd ed. London, England: Routledge & Kegan Paul; 1959.
3.
Ausubel DP. The Psychology of Meaningful Verbal LearningNew York, NY: Grune & Stratton; 1963.
4.
Ausubel DP, Novak JD, Hanesian H. Educational Psychology: A Cognitive View2nd ed. New York, NY: Holt Rinehart & Winston; 1978.
5.
Markow PG, Lonning RA. Usefulness of concept maps in college chemistry laboratories: students' perceptions and effects on achievement.  J Res Sci Teach.1998;35:1015-1029.
6.
Bodner GM. Constructivism: a theory of knowledge.  J Chem Educ.1986;63:873-878.
7.
Mintzes JJ, Wandersee JH, Novak JD. Teaching Science for Understanding: A Constructivist ViewSan Diego, Calif: Academic Press; 1998.
8.
Posner GJ, Strike KA, Hewson PW, Gertzog WA. Accommodation of a scientific conception: toward a theory of conceptual change.  Sci Educ.1982;66:211-227.
9.
von Glaserfeld E. The Practice of Constructivism in Science and Mathematics EducationWashington, DC: American Association for the Advancement of Science Press; 1992.
10.
Glaser R, Bassok M. Learning theory and the study of instruction.  Annu Rev Psychol.1989;40:631-666.
11.
Royer J, Cisero CA, Carlo MS. Techniques and procedures for assessing cognitive skills.  Rev Educ Res.1993;63:201-224.
12.
Ruiz-Primo MA, Shavelson RJ. Problems and issues in the use of concept maps in science assessment.  J Res Sci Teach.1996;33:569-600.
13.
Novak JD. Audio-tutorial techiques for individualized science instruction in the elementary school. In: Triezenberg H, ed. Individualized Science: Like It Is. Washington, DC: National Science Teachers Association; 1972:14-30.
14.
Novak JD, Gowin DB. Learning How to LearnCambridge, England: Cambridge University Press; 1984:15-54.
15.
Novak JD, Musonda D. A twelve year study of science concept learning.  Am Educ Res J.1991;28:117-153.
16.
Novak JD, Gowin DB, Johansen G. The use of concept mapping and knowledge vee mapping with junior high school science students.  Sci Educ.1983;67:625-645.
17.
Markham KM, Mintzes JJ, Jones MG. The concept map as a research and evaluation tool: further evidence of validity.  J Res Sci Teach.1994;31:91-101.
18.
Beyerbach BA. Concept mapping as an approach to assessment of students' representation of structural knowledge.  Dissertation Abstracts Int.1986;46:2622A.
19.
Novak JD. Concept mapping: a strategy for organizing knowledge. In: Glynn SM, Duit R, eds. Learning Science in the Schools: Research Reforming Practice. Mahwah, NJ: Lawrence Erlbaum Associates; 1995:229-245.
20.
Novak JD. Applying learning psychology and philosophy of science to biology teaching.  Am Biol Teach.1981;43:13-20.
21.
Novak JD. Concept mapping: a useful tool for science education.  J Res Sci Teach.1990;27:937-949.
22.
Arnaudin MW, Mintzes JJ, Dunn CS, Shafer TH. Concept mapping in college science teaching.  J College Sci Teach.1984;14:117-121.
23.
McClure JR, Sonak B, Suen HK. Concept map assessment of classroom learning: reliability, validity, and logistical practicality.  J Res Sci Teach.1999;36:475-492.
24.
Roletto E, Regis A, Albertazzi PG. Concept maps in chemistry education.  J Chem Ed.1996;73:1084-1088.
25.
Lahtinen V, Lonka K, Lindbloom-Ylaenne S. Spontaneous study strategies and the quality of knowledge construction.  Br J Educ Psychol.1997;67:13-24.
26.
Coleman EB. Using explanatory knowledge during collaborative problem solving in science.  J Learn Sci.1998;7:387-427.
27.
Heinze-Fry JA, Novak JD. Concept mapping brings long-term movement toward meaningful learning.  Sci Educ.1990;74:461-472.
28.
Edmondson KM, Smith DF. Concept mapping to facilitate veterinary students' understanding of fluid and electrolyte disorders.  Teach Learn Med.1998;10:21-33.
29.
Chastonay P, Papart JP, Laporte JD.  et al.  Use of concept mapping to define learning objectives in a master of public health program.  Teach Learn Med.1999;11:21-25.
30.
Edmondson KM. Concept maps and the development of cases for problem-based learning.  Acad Med.1994;69:108-110.
31.
Bar-Lavie BZ. Enhancing meaningful learning in an environmental education program: a case study of class empowered through the use of Novak's and Gowin's principles of learning how to learn, concept mapping, interviewing, and educating.  Dissertation Abstracts Int.1988;48:2590A.
32.
Stuart HA. Should concept maps be scored numerically.  Eur J Sci Educ.1985;7:73-81.
33.
McGaghie WC, Boerger RL, McCrimmon DR, Ravitch MM. Learning pulmonary physiology: comparison of student and faculty knowledge structures.  Acad Med.1996;71(suppl):S13-S15.
×