Skip to main content Accessibility help
  • This chapter is unavailable for purchase
  • Cited by 101
  • Cited by
    This chapter has been cited by the following publications. This list is generated based on data provided by CrossRef.

    Bofferding, Laura and Wessman-Enzinger, Nicole M. 2018. Exploring the Integer Addition and Subtraction Landscape. p. 191.

    Koretsky, Milo Keeler, Jessie Ivanovitch, John and Cao, Ying 2018. The role of pedagogical tools in active learning: a case for sense-making. International Journal of STEM Education, Vol. 5, Issue. 1,

    Liang, Chaoyun Chang, Chi-Cheng and Liu, Yu-Cheng 2018. Comparison of the cerebral activities exhibited by expert and novice visual communication designers during idea incubation. International Journal of Design Creativity and Innovation, p. 1.

    Surry, Luke T. Torre, Dario Trowbridge, Robert L. and Durning, Steven J. 2018. A mixed-methods exploration of cognitive dispositions to respond and clinical reasoning errors with multiple choice questions. BMC Medical Education, Vol. 18, Issue. 1,

    Paquette, Andrew 2018. Spatial Visualization and Professional Competence. p. 183.

    Dinsmore, Daniel L. and Zoellner, Brian P. 2018. The relation between cognitive and metacognitive strategic processing during a science simulation. British Journal of Educational Psychology, Vol. 88, Issue. 1, p. 95.

    Korovin, Lev N. Farrell, Timothy M. Hsu, Chiu- Hsieh White, McKenna and Ghaderi, Iman 2018. Surgeons' expertise during critical event in laparoscopic cholecystectomy: An expert-novice comparison using protocol analysis. The American Journal of Surgery,

    Stichter, Matt 2018. The Skillfulness of Virtue.

    Paquette, Andrew 2018. Spatial Visualization and Professional Competence. p. 41.

    Laureiro‐Martínez, Daniella and Brusoni, Stefano 2018. Cognitive flexibility and adaptive decision‐making: Evidence from a laboratory study of expert decision makers. Strategic Management Journal, Vol. 39, Issue. 4, p. 1031.

    Beyer, Reinhard and Gerlach, Rebekka 2018. Sprache und Denken. p. 83.

    den Boer, Larissa van der Schaaf, Marieke F. Vincken, Koen L. Mol, Chris P. Stuijfzand, Bobby G. and van der Gijp, Anouk 2018. Volumetric image interpretation in radiology: scroll behavior and cognitive processes. Advances in Health Sciences Education, Vol. 23, Issue. 4, p. 783.

    Tewes, Christian 2018. The Phenomenology of Habits: Integrating First-Person and Neuropsychological Studies of Memory. Frontiers in Psychology, Vol. 9, Issue. ,

    Montero, Barbara Gail 2018. Chess and the conscious mind: Why Dreyfus and McDowell got it wrong. Mind & Language,

    Burns-Childers, Annie and Vidakovic, Draga 2018. Calculus students’ understanding of the vertex of the quadratic function in relation to the concept of derivative. International Journal of Mathematical Education in Science and Technology, Vol. 49, Issue. 5, p. 660.

    Roberts, Leo J. Jackson, Mervyn S. and Grundy, Ian H. 2017. Choking under pressure: Illuminating the role of distraction and self-focus. International Review of Sport and Exercise Psychology, p. 1.

    Kumcu, Asli Vermeulen, Lotte Elprama, Shirley A. Duysburgh, Pieter Platiša, Ljiljana Van Nieuwenhove, Yves Van De Winkel, Nele Jacobs, An Van Looy, Jan and Philips, Wilfried 2017. Effect of video lag on laparoscopic surgery: correlation between performance and usability at low latencies. The International Journal of Medical Robotics and Computer Assisted Surgery, Vol. 13, Issue. 2, p. e1758.

    Konrad, Klaus 2017. Handbuch Qualitative Forschung in der Psychologie. p. 1.

    Martin-Lacroux, Christelle 2017. “Without the spelling errors I would have shortlisted her…”: The impact of spelling errors on recruiters’ choice during the personnel selection process. International Journal of Selection and Assessment, Vol. 25, Issue. 3, p. 276.

    McKendy, Katherine M. Watanabe, Yusuke Bilgic, Elif Enani, Ghada Munshi, Amani Lee, Lawrence Feldman, Liane S. Fried, Gerald M. and Vassiliou, Melina C. 2017. Establishing meaningful benchmarks: the development of a formative feedback tool for advanced laparoscopic suturing. Surgical Endoscopy, Vol. 31, Issue. 12, p. 5057.

  • Print publication year: 2006
  • Online publication date: June 2012

13 - Protocol Analysis and Expert Thought: Concurrent Verbalizations of Thinking during Experts' Performance on Representative Tasks


The superior skills of experts, such as accomplished musicians and chess masters, can be amazing to most spectators. For example, club-level chess players are often puzzled by the chess moves of grandmasters and world champions. Similarly, many recreational athletes find it inconceivable that most other adults – regardless of the amount or type of training – have the potential ever to reach the performance levels of international competitors. Especially puzzling to philosophers and scientists has been the question of the extent to which expertise requires innate gifts versus specialized acquired skills and abilities.

One of the most widely used and simplest methods of gathering data on exceptional performance is to interview the experts themselves. But are experts always capable of describing their thoughts, their behaviors, and their strategies in a manner that would allow less-skilled individuals to understand how the experts do what they do, and perhaps also understand how they might reach expert level through appropriate training? To date, there has been considerable controversy over the extent to which experts are capable of explaining the nature and structure of their exceptional performance. Some pioneering scientists, such as Binet (1893/1966), questioned the validity of the experts' descriptions when they found that some experts gave reports inconsistent with those of other experts. To make matters worse, in those rare cases that allowed verification of the strategy by observing the performance, discrepancies were found between the reported strategies and the observations (Watson, 1913).

Recommend this book

Email your librarian or administrator to recommend adding this book to your organisation's collection.

The Cambridge Handbook of Expertise and Expert Performance
  • Online ISBN: 9780511816796
  • Book DOI:
Please enter your name
Please enter a valid email address
Who would you like to send this to *
Abernethy, B., Neal, R. J., & Koning, P. (1994). Visual-perceptual and cognitive differences between expert, intermediate, and novice snooker players. Applied Cognitive Psychology, 18, 185–211.
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.
Austin, J., & Delaney, P. F. (1998). Protocol analysis as a tool for behavior analysis. Analysis of Verbal Behavior, 15, 41–56.
Berardi-Coletta, B., Buyer, L. S., Dominowski, R. L., & Rellinger, E. R. (1995). Metacognition and problem solving: A process-oriented approach. Journal of Experimental Psychology: Learning, Memory, & Cognition, 21, 205–223.
Binet, A. (1893/1966). Mnemonic virtuosity: A study of chess players. (Original paper appeared in 1893 and was translated by M. L. Simmel & S. B. Barron). Genetic Psychology Monographs, 74, 127–162.
Binet, A. (1894). Psychologie des grands calculateurs et joueurs d'echecs [The psychology of great calculators and chess players]. Paris: Libraire Hachette.
Boring, E. B. (1950). A history of experimental psychology. New York: Appleton-Century Crofts.
Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive Science, 16, 153–184.
Bryan, W. L., & Harter, N. (1899). Studies on the telegraphic language: The acquisition of a hierarchy of habits. Psychological Review, 6, 345–375.
Bühler, K. (1907). Tatsachen und Probleme zu einer Psychologie der Denkvorgaenge: I. Ueber Gedanken [Facts and problems in a psychology of thinking: I. On thoughts]. Archiv für die gesamte Psychologie, 9, 297–365.
Chabris, C. F., & Hearst, E. S. (2003). Visualization, pattern recognition, and forward search: Effects of playing speed and sight of the position on grandmaster chess errors. Cognitive Science, 27. 637–648.
Chaffin, R., & Imreh, G. (1997). “Pulling teeth and torture”: Musical memory and problem solving. Thinking and Reasoning, 3, 315–336.
Charness, N. (1981). Search in chess: Age and skill differences. Journal of Experimental Psychology: Human Perception and Performance, 7, 467–476.
Chase, W. G., & Ericsson, K. A. (1981). Skilled memory. In Anderson, J. R. (Ed.), Cognitive skills and their acquisition (pp. 141–189). Hillsdale, NJ: Erlbaum.
Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In Bower, G. H. (Ed.), The psychology of learning and motivation, Vol. 16 (pp. 1–58). New York: Academic Press.
Chi, M. T. H., Leeuw, N., Chiu, M.-H., & LaVancher, C. (1994). Eliciting self-explanations improves understanding. Cognitive Science, 18, 439–477.
Clarkson, G. P., & Metzler, A. H. (1960). Portfolio selection: A heuristic approach. Journal of Finance, 15, 465–480.
Crovitz, H. F. (1970 ). Galton's walk: Methods for the analysis of thinking, intelligence, and creativity. New York: Harper & Row Publishers.
Crowley, R. S., Naus, G. J., Stewart, J., & Friedman, C. P. (2003). Development of visual diagnostic expertise in pathology: An information-processing study. Journal of the American Medical Informatics Association, 10, 39–51.
Crutcher, R. J. (1994). Telling what we know: The use of verbal report methodologies in psychological research. Psychological Science, 5, 241–244.
Groot, A. (1978). Thought and choice and chess. The Hague: Mouton. (Original work published 1946).
Deakin, J. M., & Cobley, S. (2003). A search for deliberate practice: An examination of the practice environments in figure skating and volleyball. In Starkes, J. & Ericsson, K. A. (Eds.), Expert performance in sport: Recent advances in research on sport expertise (pp. 115–135). Champaign, IL: Human Kinetics.
Djakow, I. N., Petrowski, N. W., & Rudik, P. A. (1927). Psychologie des Schachspiels [The psychology of chess]. Berlin: Walter de Gruyter.
Duncker, K. (1945). On problem solving. Psychological Monographs, 58(5, Whole No. 270).
Ericsson, K. A. (1985). Memory skill. Canadian Journal of Psychology, 39, 188–231.
Ericsson, K. A. (1988a). Concurrent verbal reports on reading and text comprehension. Text, 8, 295–325.
Ericsson, K. A. (1988b). Analysis of memory performance in terms of memory skill. In Sternberg, R. J. (Ed.), Advances in the psychology of human intelligence, Vol. 4 (pp. 137–179). Hillsdale, NJ: Erlbaum.
Ericsson, K. A. (Ed.) (1996a). The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games. Mahwah, NJ: Erlbaum.
Ericsson, K. A. (1996b). The acquisition of expert performance: An introduction to some of the issues. In Ericsson, K. A. (Ed.), The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games (pp. 1–50). Mahwah, NJ: Erlbaum.
Ericsson, K. A. (1998) Protocol analysis. In Bechtel, W. & Graham, G. (Eds.), A companion to cognitive science (pp. 425–432). Cambridge, MA: Basil Blackwell.
Ericsson, K. A. (2001). Protocol analysis in psychology. In Smelser, N. & Baltes, P. (Eds.), International Encyclopedia of the Social and Behavioral Sciences (pp. 12256–12262). Oxford, UK: Elsevier.
Ericsson, K. A. (2002). Toward a procedure for eliciting verbal expression of nonverbal experience without reactivity: Interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis. Applied Cognitive Psychology, 16, 981–987.
Ericsson, K. A. (2003a). Valid and non-reactive verbalization of thoughts during performance of tasks: Toward a solution to the central problems of introspection as a source of scientific data. Journal of Consciousness Studies, 10, 1–18.
Ericsson, K. A. (2003b). Exceptional memori-zers: Made, not born. Trends in Cognitive Sciences, 7, 233–235.
Ericsson, K. A. (2003c). The development of elite performance and deliberate practice: An update from the perspective of the expert-performance approach. In Starkes, J. & Ericsson, K. A. (Eds.), Expert performance in sport: Recent advances in research on sport expertise (pp. 49–81). Champaign, IL: Human Kinetics.
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 10, S70–S81.
Ericsson, K. A., Chase, W., & Faloon, S. (1980). Acquisition of a memory skill. Science, 208, 1181–1182.
Ericsson, K. A., & Crutcher, R. J. (1991). Introspection and verbal reports on cognitive processes – two approaches to the study of thought processes: A response to Howe. New Ideas in Psychology, 9, 57–71.
Ericsson, K. A., Delaney, P. F., Weaver, G., & Mahadevan, R. (2004). Uncovering the structure of a memorist's superior “basic” memory capacity. Cognitive Psychology, 49, 191–237.
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211–245.
Ericsson, K. A., & Kintsch, W. (2000). Shortcomings of article retrieval structures with slots of the type that Gobet (1993) proposed and modeled. British Journal of Psychology, 91, 571–588.
Ericsson, K. A., Krampe, R. Th., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363–406.
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and exceptional performance: Evidence on maximal adaptations on task constraints. Annual Review of Psychology, 47, 273–305.
Ericsson, K. A., Patel, V. L., & Kintsch, W. (2000). How experts' adaptations to representative task demands account for the expertise effect in memory recall: Comment on Vicente and Wang (1998). Psychological Review, 107, 578–592.
Ericsson, K. A., & Polson, P. G. (1988a). Memory for restaurant orders. In Chi, M., Glaser, R., & Farr, M. (Eds.), The nature of expertise (pp. 23–70). Hillsdale, NJ: Erlbaum.
Ericsson, K. A., & Polson, P. G. (1988b). An experimental analysis of a memory skill for dinner orders. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, 305–316.
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.
Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: Bradford books/MIT Press.
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised edition). Cambridge, MA: Bradford books/MIT Press.
Ericsson, K. A., & Smith, J. (1991). Prospects and limits in the empirical study of expertise: An introduction. In Ericsson, K. A. & Smith, J. (Eds.), Toward a general theory of expertise: Prospects and limits (pp. 1–38). Cambridge: Cambridge University Press.
French, K. E., Nevett, M. E., Spurgeon, J. H., Graham, K. C., Rink, J. E., & McPherson, S. L. (1996). Knowledge representation and problem solution in expert and novice youth baseball players. Research Quarterly for Exercise and Sport, 67, 386–395.
Gagné, R. M., & Smith, E. C. (1962). A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63, 12–18.
Galton, F. (1879). Psychometric experiments. Brain, 2, 148–162.
Galton, F. (1883). Inquiries into human faculty and its development. New York: Dutton.
Green, A. J. F. (1998). Using verbal protocols in language testing research: A handbook. Cambridge, UK: Cambridge University Press.
Henderson, R. D., Smith, M. C., Podd, J., & Varela-Alvarez, H. (1995). A comparison of the four prominent user-based methods for evaluating the usability of computer software. Ergonomics, 39, 2030–2044.
Hoffman, R. R. (Ed.) (1992). The psychology of expertise: Cognitive research and empirical AI. New York: Springer-Verlag.
Hughes, J., & Parkes, S. (2003). Trends in the use of verbal protocol analysis in software engineering research. Behaviour & Information Technology, 22, 127–140.
Hunt, E., & Love, T. (1972). How good can memory be? In Melton, A. W. & Martin, E. (Eds.), Coding processes in human memory (pp. 237–260). New York: Holt.
Johnson, P. E., Karim, J., & Berryman, R. G. (1991). Effects of framing on auditor decisions. Organizational Behavior & Human Decision Processes, 50, 75–105
Johnson, P. E., Grazioli, S., Jamal, K., & Berryman, R. G. (2001). Detecting deception: Adversarial problem solving in a low base-rate world. Cognitive Science, 25, 355–392.
Karpov, A. (1995). Grandmaster musings. Chess Life, November, pp. 32–33.
Keller, F. S. (1958). The phantom plateau. Journal of the Experimental Analysis of Behavior, 1, 1–13.
Koltanowski, G. (1985). In the dark. Coraopolis, PA: Chess Enterprises.
Luria, A. R. (1968). The mind of a mnemonist. New York: Avon.
McKelvie, S. J. (1995). The VVIQ and beyond: Vividness and its measurement. Journal of Mental Imagery, 19, 197–252.
Miller, G. A. (1956). The magical number seven, plus or minus two: Some limits of our capacity for processing information. Psychological Review, 63, 81–97.
Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior. New York: Holt, Rinehart, and Winston.
Neuman, Y., & Schwarz, B. (1998). Is self-explanation while solving problems helpful? The case of analogical problem-solving. British Journal of Educational Psychology, 68, 15–24.
Neuman, Y., Leibowitz, L., & Schwarz, B. (2000). Patterns of verbal mediation during problem solving: A sequential analysis of self-explanation. Journal of Experimental Education, 68, 197–213.
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.
Nielsen, S. (1999). Regulation of learning strategies during practice: A case study of a single church organ student preparing a particular work for a concert performance. Psychology of Music, 27, 218–229.
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.
Norman, G. R., Trott, A. D., Brooks, L. R., & Smith, E. K. M. (1994). Cognitive differences in clinical reasoning related to postgraduate training. Teaching and Learning in Medicine, 6, 114–120.
Patel, V. L., Arocha, J. F., & Kaufmann, D. R. (1994). Diagnostic reasoning and medical expertise. In Medin, D. (Ed.), The psychology of learning and motivation, Vol. 30 (pp. 187–251). New York: Academic Press.
Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A critical look. In Ericsson, K. A. & Smith, J. (Eds.), Toward a general theory of expertise (pp. 93–125). Cambridge, MA: Cambridge University Press.
Pressley, M., & Afflerbach, P. (1995). Verbal protocols of reading: The nature of constructively responsive reading. Hillsdale, NJ: Erlbaum.
Renkl, A. (1997). Learning from worked-out examples: A study on individual differences. Cognitive Science, 21, 1–29.
Richardson, J. T. E. (1988). Vividness and unvividness: Reliability, consistency, and validity of subjective imagery ratings. Journal of Mental Imagery, 12, 115–122.
Saariluoma, P. (1991). Aspects of skilled imagery in blindfold chess. Acta Psychologica, 77, 65–89.
Saariluoma, P. (1992). Error in chess: The apperception-restructuring view. Psychological Research/Psychologische Forschung, 54, 17–26.
Saariluoma, P. (1995). Chess players' thinking. London: Routledge.
Schmidt, H. G., & Boshuizen, H. (1993). On acquiring expertise in medicine. Educational Psychology Review, 5, 205–221.
Schraagen, J. M. (1993). How experts solve a novel problem in experimental design. Cognitive Science, 17, 285–309.
Simon, H. A., & Chase, W. G. (1973). Skill in chess. American Scientist, 61, 394–403.
Simon, H. A., & Kaplan, C. A. (1989). Foundations of cognitive science. In Posner, M. J. (Ed.), Foundations of cognitive science (pp. 1–47). Cambridge, MA: MIT Press.
Simpson, S. A., & Gilhooly, K. J. (1997). Diagnostic thinking processes: Evidence from a constructive interaction study of electrocardiogram (ECG) interpretation. Applied Cognitive Psychology, 11, 543–554.
Starkes, J., & Ericsson, K. A. (Eds.) (2003). Expert performance in sport: Recent advances in research on sport expertise. Champaign, IL: Human Kinetics.
Sudman, S., Bradburn, N. M., & Schwarz, N. (Eds.) (1996). Thinking about answers: The application of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass.
Maas, H. L. J., & Wagenmakers, E. J. (2005). A psychometric analysis of chess expertise. American Journal of Psychology, 118, 29–60.
Verplanck, W. S. (1962). Unaware of where's awareness: Some verbal operants-notates, moments and notants. In Eriksen, C. W. (Ed.), Behavior and awareness – a symposium of research and interpretations (pp. 130–158). Durham, NC: Duke University Press.
Von Eckardt, B. (1998). Psychology of introspection. In Craig, E. (Ed.), Routledge encyclopedia of philosophy (pp. 842–846). London: Routledge.
Ward. P., Hodges, N. J., Williams, A. M., & Starkes, J. L. (2004). Deliberate practice and expert performance: Defining the path to excellence. In Williams, A. M. & Hodges, N. J. (Eds.), Skill acquisition in sport: Research, theory and practice (pp. 231–258). London, UK: Routledge.
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–77.
Watson, J. B. (1920). Is thinking merely the action of language mechanisms? British Journal of Psychology, 11, 87–104.
Wenger, M. J., & Payne, D. G. (1995). On the acquisition of mnemonic skill: Application of skilled memory theory. Journal of Experimental Psychology: Applied, 1, 194–215.
Wilding, J., & Valentine, E. (1997). Superior memory. Hove, UK: Psychology Press.
Wundt, W. (1897). Outlines of psychology (Translated by Judd, C. H.). Leipzig: Wilhelm Engelmann.
Wundt, W. (1907). Über Ausfrageexperimente und über die Methoden zur Psychologie des Denkens [On interrogation experiments and on the methods of the psychology of thinking]. Philosophische Studien, 3, 301–360.