Skip to main content Accessibility help
×
Hostname: page-component-586b7cd67f-rdxmf Total loading time: 0 Render date: 2024-11-22T16:29:18.727Z Has data issue: false hasContentIssue false
This chapter is part of a book that is no longer available to purchase from Cambridge Core

10 - Laboratory Methods for Assessing Experts' and Novices' Knowledge

from PART III - METHODS FOR STUDYING THE STRUCTURE OF EXPERTISE

Michelene T. H. Chi
Affiliation:
Learning Research and Development Center, University of Pittsburgh
K. Anders Ericsson
Affiliation:
Florida State University
Neil Charness
Affiliation:
Florida State University
Paul J. Feltovich
Affiliation:
University of West Florida
Robert R. Hoffman
Affiliation:
University of West Florida
Get access

Summary

Introduction

Expertise, by definition, refers to the manifestation of skills and understanding resulting from the accumulation of a large body of knowledge. This implies that in order to understand how experts perform and why they are more capable than non-experts, we must understand the representation of their knowledge, that is, how their knowledge is organized or structured, and how their representations might differ from those of novices. For example, if a child who is fascinated with dinosaurs and has learned a lot about them correctly infers attributes about some dinosaurs that was new to them by reasoning analogically to some known dinosaurs (e.g., the shape of teeth for carnivores versus vegetarians), we would not conclude that the “expert” child has a more powerful analogical reasoning strategy. Instead, we would conclude that such a global or domain-general reasoning strategy is available to all children, but that novice children might reason analogically to some other familiar domain, such as animals (rather than dinosaurs), as our data have shown (Chi, Hutchinson, & Robin, 1989). Thus, the analogies of domain-novice are less powerful not necessarily because they lack adequate analogical reasoning strategies, although they may, but because they lack the appropriate domain knowledge from which analogies can be drawn. Thus, in this framework, a critical locus of proficiency lies in the representation of their domain knowledge.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Akin, O. (1980). Models of architectural knowledge. London: Pion.Google Scholar
Alberdi, E., Becher, J. C., Gilhooly, K., Hunter, J., Logie, R., Lyon, A., McIntosh, N., & Reiss, J. (2001). Expertise and the interpretation of computerized physiological data: Implications for the design of computerized monitoring in neonatal intensive care. International Journal of Human-Computer Studies, 55, 191–216.CrossRefGoogle Scholar
Burton, A. M., Shadbolt, N. R., Hedgecock, A. P., & Rugg, G. (1987). A formal evaluation of knowledge elicitation techniques for expert systems: Domain 1. In Moralee, D. S. (Ed.), Research and development in expert systems, Vol. 4. (pp. 35–46). Cambridge: Cambridge University Press.Google Scholar
Burton, A. M., Shadbolt, N. R., Rugg, G., & Hedgecock, A. P. (1988). Knowledge elicitation techniques in classification domains. In Kodratoff, Y. (Ed.), ECAI-88: Proceedings of the 8th European Conference on Artificial Intelligence (pp. 85–93). London: Pittman.Google Scholar
Burton, A. M., Shadbolt, N. R., Rugg, G., & Hedgecock, A. P. (1990). The efficacy of knowledge elicitation techniques: A comparison across domains and levels of expertise. Journal of Knowledge Acquisition, 2, 167–178.CrossRefGoogle Scholar
Calderwood, R., Klein, G. A., & Crandall, B. W. (1988). Time pressure, skill, and move quality in chess. American Journal of Psychology, 101, 481–493.CrossRefGoogle Scholar
Campitelli, G., & Gobet, F. (2005). The mind's eye in blindfold chess. European Journal of Cognitive Psychology, 17, 23–45.CrossRefGoogle Scholar
Chase, W. G., & Chi, M. T. H. (1981). Cognitive skill: Implications for spatial skill in large-scale environments. In Harvey, J. (Ed.), Cognition, social behaviors, and the environment (pp. 111–136). Hillsdale, NJ: Erlbaum.Google Scholar
Chase, W. G., & Simon, H. A. (1973a). Perception in chess. Cognitive Psychology, 4, 55–81.CrossRefGoogle Scholar
Chase, W. G., & Simon, H. A. (1973b). The mind's eye in chess. In Chase, W. G. (Ed.), Visual information processing (pp. 215–281). New York: Academic Press.Google Scholar
Chi, M. T. H. (1997). Quantifying qualitative analyses of verbal data: A practical guide. Journal of the Learning Sciences, 6, 271–315.CrossRefGoogle Scholar
Chi, M. T. H. (2000). Self-explaining expository texts: The dual processes of generating inferences and repairing mental models. In Glaser, R. (Ed.), Advances in instructional psychology (pp. 161–238). Hillsdale, NJ: Erlbaum.Google Scholar
Chi, M. T. H. (2005). Common sense conceptions of emergent processes: Why some misconceptions are robust. Journal of the Learning Sciences, 14, 161–199.CrossRefGoogle Scholar
Chi, M. T. H., Feltovich, P., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5, 121–152.CrossRefGoogle Scholar
Chi, M. T. H., Glaser, R., & Rees, E. (1982). Expertise in problem solving. In Sternberg, R. J. (Ed.), Advances in the psychology of human intelligence, Vol. 1 (pp. 7–75). Hillsdale, NJ: Erlbaum.Google Scholar
Chi, M. T. H., Hutchinson, J., & Robin, A. F. (1989). How inferences about novel domain-related concepts can be constrained by structured knowledge. Merrill-Palmer Quarterly, 35, 27–62.Google Scholar
Chi, M. T. H., & Koeske, R. (1983). Network representation of a child's dinosaur knowledge. Developmental Psychology, 19, 29–39.CrossRefGoogle Scholar
Chi, M. T. H., & Ohlsson, S. (2005). Complex declarative learning. In Holyoak, K. J., & Morrison, R. G., (Eds.), The Cambridge handbook of thinking and reasoning (pp. 371–399). Cambridge: Cambridge University Press.Google Scholar
Chi, M. T. H., Siler, S. A., Jeong, H., Yamauchi, T., & Hausmann, R. G. (2001). Learning from human tutoring. Cognitive Science, 25, 471–533.CrossRefGoogle Scholar
Chiesi, H. L., Spilich, G. J., & Voss, J. F. (1979). Acquisition of domain-related information in relation to high and low domain knowledge. Journal of Verbal Learning and Verbal Behavior, 18, 257–273.CrossRefGoogle Scholar
Cooke, N. J. (1994). Varieties of knowledge elicitation techniques. International Journal of Human-Computer Studies, 41, 801–849.CrossRefGoogle Scholar
Cullen, J., & Bryman, A. (1988). The knowledge acquisition bottleneck: A time for reassessment? Expert Systems, 5, 216–225.CrossRefGoogle Scholar
De Groot, A. (1966). Perception and memory versus thought: Some old ideas and recent findings. In Kleinmuntz, B. (Ed.), Problem solving: Research, method, and theory (pp. 19–50). New York: Wiley.Google Scholar
Duncker, K. (1945). On problem-solving. (L. S. Lees, Trans.). Psychological monographs, 58 (Whole No. 270). (Original work published, 1935.)CrossRefGoogle Scholar
Egan, D. E., & Schwartz, B. J. (1979). Chunking in recall of symbolic drawings. Memory & Cognition, 7, 149–158.CrossRefGoogle ScholarPubMed
Eisenstadt, M., & Kareev, Y. (1975). Aspects of human problem solving: The use of internal representations. In Norman, D. A. & Rumelhart, D. E. (Eds.), Exploration in cognition (pp. 308–346). San Francisco: Freeman.Google Scholar
Ericsson, K. A., Delaney, P. F., Weaver, G., & Mahadevan, R. (2004). Uncovering the structure of a memorist's superior “basic” memory capacity. Cognitive Psychology, 49, 191–237.CrossRefGoogle ScholarPubMed
Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis. Cambridge, MA: MIT Press.Google Scholar
Falkenhainer, B., Forbus, K. D., & Gentner, D. (1990). The structure-mapping engine: Algorithm and examples. Artificial Intelligence, 41, 1–63.CrossRefGoogle Scholar
Gellert, E. (1962). Children's conception of the structure and function of the human body. Genetic Psychology Monographs, 65, 193–405.Google Scholar
Goel, A. K., Gomez de Silva Garza, A., Grue, N., Murdock, J. W., Recker, M. M., & Govinderaj, T. (1996). Towards designing learning environments. In Frasson, C., Gauthier, G., & Lesgold, A. (Ed.), Intelligent tutoring systems: Lecture notes in computer science (pp. 493–501). Berlin: Springer-Verlag.CrossRefGoogle Scholar
Hmelo-Silver, C. E., & Pfeffer, M. G. (2004). Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and function. Cognitive Science, 28, 127–138.CrossRefGoogle Scholar
Hoffman, R. R. (1987, Summer). The problem of extracting the knowledge of experts from the perspective of experimental psychology. The AI Magazine, 8, 53–67.Google Scholar
Hoffman, R. R., Coffey, J. W., Ford, K. M., & Novak, J. D. (in press). A method for eliciting, preserving, and sharing the knowledge of expert forecasters. Weather & Forecasting.Google Scholar
Hoffman, R. R., Shadbolt, N. R., Burton, A. M., & Klein, G. (1995). Eliciting knowledge from experts: A methodological analysis. Organizational Behavior and Human Decision Processes, 62, 129–158.CrossRefGoogle Scholar
Hoffman, R. R., Trafton, G., & Roebber, P. (2006). Minding the weather: How expert forecasters think. Cambridge, MA: MIT Press.Google Scholar
Johnson, K., & Eilers, A. T. (1998). Effects of knowledge and development on subordinate level categorization. Cognitive Development, 13, 515–545.CrossRefGoogle Scholar
, G., , R. M., , B., & , D. (2005). Problem detection. Cognition, Technology, and Work, 7, 14–28.CrossRefGoogle Scholar
Klein, G. A., & Hoffman, R. R. (1992). Seeing the invisible: Perceptual-cognitive aspects of expertise. In Rabinowitz, M. (Ed.), Cognitive science foundations of instruction (pp. 203–226). Mahwah, NJ: Erlbaum.Google Scholar
Klein, G., Wolf, S., Militello, L., & Zsambok, C. (1995). Characteristics of skilled option generation in chess. Organizational Behavior and Human Decision Processes, 62, 63–69.CrossRefGoogle Scholar
Lesgold, A., Rubinson, H., Feltovich, P., Glaser, R., Klopfer, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing X-ray pictures. In Chi, M., Glaser, R., & Farr, M. (Eds.), The nature of expertise (pp. 311–342). Hillsdale, NJ: Erlbaum.Google Scholar
Mayfield, W. A., Kardash, C. M., & Kivlighan, D. M. (1999). Differences in experienced and novice counselors' knowledge structures about clients: Implications for case conceptualization. Journal of Counseling Psychology, 46, 504–514.CrossRefGoogle Scholar
Means, M. L., & Voss, J. F. (1985). Star Wars: A developmental study of expert and novice knowledge structures. Journal of Memory and Language, 24, 746–757.CrossRefGoogle Scholar
Medin, D. L., Lynch, E. B., Coley, J. D., & Atran, S. (1997). Categorization and reasoning among tree experts: Do all roads lead to Rome? Cognitive Psychology, 32, 49–96.CrossRefGoogle ScholarPubMed
Miller, G. A. (1996). The science of words. New York: McGraw-Hill.Google Scholar
Morrow, D. G., Menard, W. E., Stine-Morrow, E. A. L., Teller, T., & Bryant, D. (2001). The influence of expertise and task factors on age differences in pilot communication. Psychology & Aging, 16, 31–46.CrossRefGoogle ScholarPubMed
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.CrossRefGoogle Scholar
Patel, V. L., & Arocha, J. F. (2001). The nature of constraints on collaborative decision-making in health care settings. In Salas, E., & Klein, G. A. (Eds.), Linking expertise and naturalistic decision making (pp. 383–405). Mahwah, NJ: Erlbaum.Google Scholar
Reitman, J. S. (1976). Skilled perception in Go: Deducing memory structures from inter-response times. Cognitive Psychology, 8, 336–356.CrossRefGoogle Scholar
Rosch, E., Mervis, C. B., Gray, W. D., Johnson, D. M., & Boyes-Braem, P. (1976). Basic objects in natural categories. Cognitive Psychology, 8, 382–439.CrossRefGoogle Scholar
Sabers, D. S., Cushing, K. S., & Berliner, D. C. (1991). Differences among teachers in a task characterized by simultaneity, multidimensionality, and immediacy. American Educational Research Journal, 28, 63–88.Google Scholar
Schvaneveldt, R. W., Durso, F. T., Goldsmith, T. E., Breen, T. J., Cooke, N. M., Tucker, R. G., & DeMaio, J. C. (1985). Measuring the structure of expertise. International Journal of Man-Machine Studies, 23, 699–728.CrossRefGoogle Scholar
Shafto, P., & Coley, J. D. (2003). Development of categorization and reasoning in natural world: Novices to experts, naïve similarity to ecological knowledge. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, 641–649.Google ScholarPubMed
Simon, D. P., & Simon, H. A. (1978). Individual differences in solving physics problems. In Siegler, R. (Ed.), Children's thinking: What develops? (pp. 325–348). Hillsdale, NJ: Erlbaum.Google Scholar
Smith, M. U., & Good, R. (1984). Problem solving and classical genetics: Successful versus unsuccessful performance. Journal of Research in Science Teaching, 21, 895–912.CrossRefGoogle Scholar
Snowden, P. T., Davies, I. R. L., & Roling, P. (2000). Perceptual learning of the detection of features in X-ray images: A functional role for improvements in adults' visual sensitivity? Journal of Experimental Psychology: Human Perception and Performance, 26, 379–390.Google Scholar
Tanaka, J. W. (2001). The entry point of face recognition: Evidence for face expertise. Journal of Experimental Psychology: General, 130, 534–543.CrossRefGoogle ScholarPubMed
Tanaka, J. W., & Taylor, M. (1991). Object categories and expertise: Is the basic level in the eye of the beholder? Cognitive Psychology, 23, 457–482.CrossRefGoogle Scholar
Vicente, K. J. (1992). Memory recall in a process control system: A measure of expertise and display effectiveness. Memory and Cognition, 20, 356–373.CrossRefGoogle Scholar
Vicente, K. J., & Wang, J. H. (1998). An ecological theory of expertise effects in memory recall. Psychological Review, 105, 33–57.CrossRefGoogle ScholarPubMed
Weiser, M., & Shertz, J. (1983). Programming problem representation in novice and expert programmers. International Journal of Man-Machine Studies, 14, 391–396.CrossRefGoogle Scholar
Wineburg, S. S. (1991). Historical problem solving: A study of the cognitive processes used in the evaluation of documentary and pictorial evidence. Journal of Educational Psychology, 83, 73–87.CrossRefGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×