Skip to main content Accessibility help
×
Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-11-03T05:36:35.110Z Has data issue: false hasContentIssue false
This chapter is part of a book that is no longer available to purchase from Cambridge Core

13 - Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts' Performance on Representative Tasks

from PART III - METHODS FOR STUDYING THE STRUCTURE OF EXPERTISE

K. Anders Ericsson
Affiliation:
Department of Psychology, Florida State University
K. Anders Ericsson
Affiliation:
Florida State University
Neil Charness
Affiliation:
Florida State University
Paul J. Feltovich
Affiliation:
University of West Florida
Robert R. Hoffman
Affiliation:
University of West Florida
Get access

Summary

The superior skills of experts, such as accomplished musicians and chess masters, can be amazing to most spectators. For example, club-level chess players are often puzzled by the chess moves of grandmasters and world champions. Similarly, many recreational athletes find it inconceivable that most other adults – regardless of the amount or type of training – have the potential ever to reach the performance levels of international competitors. Especially puzzling to philosophers and scientists has been the question of the extent to which expertise requires innate gifts versus specialized acquired skills and abilities.

One of the most widely used and simplest methods of gathering data on exceptional performance is to interview the experts themselves. But are experts always capable of describing their thoughts, their behaviors, and their strategies in a manner that would allow less-skilled individuals to understand how the experts do what they do, and perhaps also understand how they might reach expert level through appropriate training? To date, there has been considerable controversy over the extent to which experts are capable of explaining the nature and structure of their exceptional performance. Some pioneering scientists, such as Binet (1893/1966), questioned the validity of the experts' descriptions when they found that some experts gave reports inconsistent with those of other experts. To make matters worse, in those rare cases that allowed verification of the strategy by observing the performance, discrepancies were found between the reported strategies and the observations (Watson, 1913).

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2006

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abernethy, B., Neal, R. J., & Koning, P. (1994). Visual-perceptual and cognitive differences between expert, intermediate, and novice snooker players. Applied Cognitive Psychology, 18, 185–211.CrossRefGoogle Scholar
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.Google Scholar
Austin, J., & Delaney, P. F. (1998). Protocol analysis as a tool for behavior analysis. Analysis of Verbal Behavior, 15, 41–56.CrossRefGoogle ScholarPubMed
Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., & Rellinger, E. R. (1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, & Cognition, 21, 205–223.Google Scholar
Binet, A. (1893/1966). Mnemonic virtuosity: A study of chess players. (Original paper appeared in 1893 and was translated by M. L. Simmel & S. B. Barron). Genetic Psychology Monographs, 74, 127–162.Google Scholar
Binet, A. (1894). Psychologie des grands calculateurs et joueurs d'echecs [The psychology of great calculators and chess players]. Paris: Libraire Hachette.Google Scholar
Boring, E. B. (1950). A history of experimental psychology. New York: Appleton-Century Crofts.Google Scholar
Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive Science, 16, 153–184.CrossRefGoogle Scholar
Bryan, W. L., & Harter, N. (1899). Studies on the telegraphic language: The acquisition of a hierarchy of habits. Psychological Review, 6, 345–375.Google Scholar
Bühler, K. (1907). Tatsachen und Probleme zu einer Psychologie der Denkvorgaenge: I. Ueber Gedanken [Facts and problems in a psychology of thinking: I. On thoughts]. Archiv für die gesamte Psychologie, 9, 297–365.Google Scholar
Chabris, C. F., & Hearst, E. S. (2003). Visualization, pattern recognition, and forward search: Effects of playing speed and sight of the position on grandmaster chess errors. Cognitive Science, 27. 637–648.CrossRefGoogle Scholar
Chaffin, R., & Imreh, G. (1997). “Pulling teeth and torture”: Musical memory and problem solving. Thinking and Reasoning, 3, 315–336.Google Scholar
Charness, N. (1981). Search in chess: Age and skill differences. Journal of Experimental Psychology: Human Perception and Performance, 7, 467–476.Google Scholar
Chase, W. G., & Ericsson, K. A. (1981). Skilled memory. In Anderson, J. R. (Ed.), Cognitive skills and their acquisition (pp. 141–189). Hillsdale, NJ: Erlbaum.Google Scholar
Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In Bower, G. H. (Ed.), The psychology of learning and motivation, Vol. 16 (pp. 1–58). New York: Academic Press.Google Scholar
Chi, M. T. H., Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.Google Scholar
Clarkson, G. P., & Metzler, A. H. (1960). Portfolio selection: A heuristic approach. Journal of Finance, 15, 465–480.Google Scholar
Crovitz, H. F. (1970 ). Galton's walk: Methods for the analysis of thinking, intelligence, and creativity. New York: Harper & Row Publishers.Google Scholar
Crowley, R. S., Naus, G. J., Stewart, J., & Friedman, C. P. (2003). Development of visual diagnostic expertise in pathology: An information-processing study. Journal of the American Medical Informatics Association, 10, 39–51.CrossRefGoogle ScholarPubMed
Crutcher, R. J. (1994). Telling what we know: The use of verbal report methodologies in psychological research. Psychological Science, 5, 241–244.CrossRefGoogle Scholar
Groot, A. (1978). Thought and choice and chess. The Hague: Mouton. (Original work published 1946).Google Scholar
Deakin, J. M., & Cobley, S. (2003). A search for deliberate practice: An examination of the practice environments in figure skating and volleyball. In Starkes, J. & Ericsson, K. A. (Eds.), Expert performance in sport: Recent advances in research on sport expertise (pp. 115–135). Champaign, IL: Human Kinetics.Google Scholar
Djakow, I. N., Petrowski, N. W., & Rudik, P. A. (1927). Psychologie des Schachspiels [The psychology of chess]. Berlin: Walter de Gruyter.Google Scholar
Duncker, K. (1945). On problem solving. Psychological Monographs, 58(5, Whole No. 270).CrossRefGoogle Scholar
Ericsson, K. A. (1985). Memory skill. Canadian Journal of Psychology, 39, 188–231.CrossRefGoogle Scholar
Ericsson, K. A. (1988a). Concurrent verbal reports on reading and text comprehension. Text, 8, 295–325.CrossRefGoogle Scholar
Ericsson, K. A. (1988b). Analysis of memory performance in terms of memory skill. In Sternberg, R. J. (Ed.), Advances in the psychology of human intelligence, Vol. 4 (pp. 137–179). Hillsdale, NJ: Erlbaum.
Ericsson, K. A. (Ed.) (1996a). The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games. Mahwah, NJ: Erlbaum.Google Scholar
Ericsson, K. A. (1996b). The acquisition of expert performance: An introduction to some of the issues. In Ericsson, K. A. (Ed.), The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games (pp. 1–50). Mahwah, NJ: Erlbaum.Google Scholar
Ericsson, K. A. (1998) Protocol analysis. In Bechtel, W. & Graham, G. (Eds.), A companion to cognitive science (pp. 425–432). Cambridge, MA: Basil Blackwell.Google Scholar
Ericsson, K. A. (2001). Protocol analysis in psychology. In Smelser, N. & Baltes, P. (Eds.), International Encyclopedia of the Social and Behavioral Sciences (pp. 12256–12262). Oxford, UK: Elsevier. Google Scholar
Ericsson, K. A. (2002). Toward a procedure for eliciting verbal expression of nonverbal experience without reactivity: Interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis. Applied Cognitive Psychology, 16, 981–987.CrossRefGoogle Scholar
Ericsson, K. A. (2003a). Valid and non-reactive verbalization of thoughts during performance of tasks: Toward a solution to the central problems of introspection as a source of scientific data. Journal of Consciousness Studies, 10, 1–18.Google Scholar
Ericsson, K. A. (2003b). Exceptional memori-zers: Made, not born. Trends in Cognitive Sciences, 7, 233–235.CrossRefGoogle Scholar
Ericsson, K. A. (2003c). The development of elite performance and deliberate practice: An update from the perspective of the expert-performance approach. In Starkes, J. & Ericsson, K. A. (Eds.), Expert performance in sport: Recent advances in research on sport expertise (pp. 49–81). Champaign, IL: Human Kinetics.
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 10, S70–S81.Google Scholar
Ericsson, K. A., Chase, W., & Faloon, S. (1980). Acquisition of a memory skill. Science, 208, 1181–1182.CrossRefGoogle Scholar
Ericsson, K. A., & Crutcher, R. J. (1991). Introspection and verbal reports on cognitive processes – two approaches to the study of thought processes: A response to Howe. New Ideas in Psychology, 9, 57–71.CrossRefGoogle Scholar
Ericsson, K. A., Delaney, P. F., Weaver, G., & Mahadevan, R. (2004). Uncovering the structure of a memorist's superior “basic” memory capacity. Cognitive Psychology, 49, 191–237.CrossRefGoogle ScholarPubMed
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211–245.CrossRefGoogle ScholarPubMed
Ericsson, K. A., & Kintsch, W. (2000). Shortcomings of article retrieval structures with slots of the type that Gobet (1993) proposed and modeled. British Journal of Psychology, 91, 571–588.CrossRefGoogle Scholar
Ericsson, K. A., Krampe, R. Th., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406.CrossRefGoogle Scholar
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence on maximal adaptations on task constraints. Annual Review of Psychology, 47, 273–305.CrossRefGoogle ScholarPubMed
Ericsson, K. A., Patel, V. L., & Kintsch, W. (2000). How experts' adaptations to representative task demands account for the expertise effect in memory recall: Comment on Vicente and Wang (1998). Psychological Review, 107, 578–592.CrossRefGoogle Scholar
Ericsson, K. A., & Polson, P. G. (1988a). Memory for restaurant orders. In Chi, M., Glaser, R., & Farr, M. (Eds.), The nature of expertise (pp. 23–70). Hillsdale, NJ: Erlbaum.Google Scholar
Ericsson, K. A., & Polson, P. G. (1988b). An experimental analysis of a memory skill for dinner orders. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 305–316.Google Scholar
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: Bradford books/MIT Press.Google Scholar
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised edition). Cambridge, MA: Bradford books/MIT Press.Google Scholar
Ericsson, K. A., & Smith, J. (1991). Prospects and limits in the empirical study of expertise: An introduction. In Ericsson, K. A. & Smith, J. (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 1–38). Cambridge: Cambridge University Press.Google Scholar
French, K. E., Nevett, M. E., Spurgeon, J. H., Graham, K. C., Rink, J. E., & McPherson, S. L. (1996). Knowledge representation and problem solution in expert and novice youth baseball players. Research Quarterly for Exercise and Sport, 67, 386–395.CrossRefGoogle ScholarPubMed
Gagné, R. M., & Smith, E. C. (1962). A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63, 12–18.CrossRefGoogle ScholarPubMed
Galton, F. (1879). Psychometric experiments. Brain, 2, 148–162.CrossRefGoogle Scholar
Galton, F. (1883). Inquiries into human faculty and its development. New York: Dutton.CrossRefGoogle Scholar
Green, A. J. F. (1998). Using verbal protocols in language testing research: A handbook. Cambridge, UK: Cambridge University Press.Google Scholar
Henderson, R. D., Smith, M. C., Podd, J., & Varela-Alvarez, H. (1995). A comparison of the four prominent user-based methods for evaluating the usability of computer software. Ergonomics, 39, 2030–2044.CrossRefGoogle Scholar
Hoffman, R. R. (Ed.) (1992). The psychology of expertise: Cognitive research and empirical AI. New York: Springer-Verlag.CrossRefGoogle Scholar
Hughes, J., & Parkes, S. (2003). Trends in the use of verbal protocol analysis in software engineering research. Behaviour & Information Technology, 22, 127–140.CrossRefGoogle Scholar
Hunt, E., & Love, T. (1972). How good can memory be? In Melton, A. W. & Martin, E. (Eds.), Coding processes in human memory (pp. 237–260). New York: Holt.Google Scholar
Johnson, P. E., Karim, J., & Berryman, R. G. (1991). Effects of framing on auditor decisions. Organizational Behavior & Human Decision Processes, 50, 75–105CrossRefGoogle Scholar
Johnson, P. E., Grazioli, S., Jamal, K., & Berryman, R. G. (2001). Detecting deception: Adversarial problem solving in a low base-rate world. Cognitive Science, 25, 355–392.CrossRefGoogle Scholar
Karpov, A. (1995). Grandmaster musings. Chess Life, November, pp. 32–33.Google Scholar
Keller, F. S. (1958). The phantom plateau. Journal of the Experimental Analysis of Behavior, 1, 1–13.CrossRefGoogle ScholarPubMed
Koltanowski, G. (1985). In the dark. Coraopolis, PA: Chess Enterprises.Google Scholar
Luria, A. R. (1968). The mind of a mnemonist. New York: Avon.Google Scholar
McKelvie, S. J. (1995). The VVIQ and beyond: Vividness and its measurement. Journal of Mental Imagery, 19, 197–252.Google Scholar
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits of our capacity for processing information. Psychological Review, 63, 81–97.CrossRefGoogle ScholarPubMed
Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior. New York: Holt, Rinehart, and Winston.CrossRefGoogle Scholar
Neuman, Y., & Schwarz, B. (1998). Is self-explanation while solving problems helpful? The case of analogical problem-solving. British Journal of Educational Psychology, 68, 15–24.CrossRefGoogle Scholar
Neuman, Y., Leibowitz, L., & Schwarz, B. (2000). Patterns of verbal mediation during problem solving: A sequential analysis of self-explanation. Journal of Experimental Education, 68, 197–213.CrossRefGoogle Scholar
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.Google Scholar
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Nielsen, S. (1999). Regulation of learning strategies during practice: A case study of a single church organ student preparing a particular work for a concert performance. Psychology of Music, 27, 218–229.CrossRefGoogle Scholar
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.CrossRefGoogle Scholar
Norman, G. R., Trott, A. D., Brooks, L. R., & Smith, E. K. M. (1994). Cognitive differences in clinical reasoning related to postgraduate training. Teaching and Learning in Medicine, 6, 114–120.CrossRefGoogle Scholar
Patel, V. L., Arocha, J. F., & Kaufmann, D. R. (1994). Diagnostic reasoning and medical expertise. In Medin, D. (Ed.), The psychology of learning and motivation, Vol. 30 (pp. 187–251). New York: Academic Press.Google Scholar
Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A critical look. In Ericsson, K. A. & Smith, J. (Eds.), Toward a general theory of expertise (pp. 93–125). Cambridge, MA: Cambridge University Press.Google Scholar
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.Google Scholar
Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21, 1–29.CrossRefGoogle Scholar
Richardson, J. T. E. (1988). Vividness and unvividness: Reliability, consistency, and validity of subjective imagery ratings. Journal of Mental Imagery, 12, 115–122.Google Scholar
Saariluoma, P. (1991). Aspects of skilled imagery in blindfold chess. Acta Psychologica, 77, 65–89.CrossRefGoogle ScholarPubMed
Saariluoma, P. (1992). Error in chess: The apperception-restructuring view. Psychological Research/Psychologische Forschung, 54, 17–26.CrossRefGoogle ScholarPubMed
Saariluoma, P. (1995). Chess players' thinking. London: Routledge.Google Scholar
Schmidt, H. G., & Boshuizen, H. (1993). On acquiring expertise in medicine. Educational Psychology Review, 5, 205–221.CrossRefGoogle Scholar
Schraagen, J. M. (1993). How experts solve a novel problem in experimental design. Cognitive Science, 17, 285–309.CrossRefGoogle Scholar
Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Scientist, 61, 394–403.Google Scholar
Simon, H. A., & Kaplan, C. A. (1989). Foundations of cognitive science. In Posner, M. J. (Ed.), Foundations of cognitive science (pp. 1–47). Cambridge, MA: MIT Press.Google Scholar
Simpson, S. A., & Gilhooly, K. J. (1997). Diagnostic thinking processes: Evidence from a constructive interaction study of electrocardiogram (ECG) interpretation. Applied Cognitive Psychology, 11, 543–554.3.0.CO;2-C>CrossRefGoogle Scholar
Starkes, J., & Ericsson, K. A. (Eds.) (2003). Expert performance in sport: Recent advances in research on sport expertise. Champaign, IL: Human Kinetics.Google Scholar
Sudman, S., Bradburn, N. M., & Schwarz, N. (Eds.) (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.Google Scholar
Maas, H. L. J., & Wagenmakers, E. J. (2005). A psychometric analysis of chess expertise. American Journal of Psychology, 118, 29–60.Google ScholarPubMed
Verplanck, W. S. (1962). Unaware of where's awareness: Some verbal operants-notates, moments and notants. In Eriksen, C. W. (Ed.), Behavior and awareness – a symposium of research and interpretations (pp. 130–158). Durham, NC: Duke University Press.Google Scholar
Von Eckardt, B. (1998). Psychology of introspection. In Craig, E. (Ed.), Routledge encyclopedia of philosophy (pp. 842–846). London: Routledge.Google Scholar
Ward. P., Hodges, N. J., Williams, A. M., & Starkes, J. L. (2004). Deliberate practice and expert performance: Defining the path to excellence. In Williams, A. M. & Hodges, N. J. (Eds.), Skill acquisition in sport: Research, theory and practice (pp. 231–258). London, UK: Routledge.Google Scholar
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–77.CrossRefGoogle Scholar
Watson, J. B. (1920). Is thinking merely the action of language mechanisms? British Journal of Psychology, 11, 87–104.Google Scholar
Wenger, M. J., & Payne, D. G. (1995). On the acquisition of mnemonic skill: Application of skilled memory theory. Journal of Experimental Psychology: Applied, 1, 194–215.Google Scholar
Wilding, J., & Valentine, E. (1997). Superior memory. Hove, UK: Psychology Press.Google Scholar
Wundt, W. (1897). Outlines of psychology (Translated by Judd, C. H.). Leipzig: Wilhelm Engelmann.CrossRefGoogle Scholar
Wundt, W. (1907). Über Ausfrageexperimente und über die Methoden zur Psychologie des Denkens [On interrogation experiments and on the methods of the psychology of thinking]. Philosophische Studien, 3, 301–360.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×