Skip to main content Accessibility help

We use cookies to distinguish you from other users and to provide you with a better experience on our websites. Close this message to accept cookies or find out how to manage your cookie settings.

Close cookie message
×
Hostname: page-component-8448b6f56d-gtxcr Total loading time: 0 Render date: 2024-04-19T14:49:37.472Z Has data issue: false hasContentIssue false

12 - Capturing Expert Thought with Protocol Analysis: Concurrent Verbalizations of Thinking during Experts’ Performance on Representative Tasks

from Part III - Methods for Studying the Structure of Expertise

Published online by Cambridge University Press:  10 May 2018

By
K. Anders Ericsson
Edited by
K. Anders Ericsson ,
Robert R. Hoffman ,
Aaron Kozbelt  and
A. Mark Williams
K. Anders Ericsson
Affiliation:
Florida State University
Robert R. Hoffman
Affiliation:
Florida Institute for Human and Machine Cognition
Aaron Kozbelt
Affiliation:
Brooklyn College, City University of New York
A. Mark Williams
Affiliation:
University of Utah
Get access

Summary

A summary is not available for this content so a preview has been provided. Please use the Get access link above for information on how to access this content.
Image of the first page of this content. For PDF version, please use the ‘Save PDF’ preceeding this image.'
Type
Chapter
Information
The Cambridge Handbook of Expertise and Expert Performance , pp. 192 - 212
Publisher: Cambridge University Press
Print publication year: 2018

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Abernethy, B., Neal, R. J., & Koning, P. (1994). Visual-perceptual and cognitive differences between expert, intermediate, and novice snooker players. Applied Cognitive Psychology, 18, 185–211.Google Scholar
Afonso, J., Garganta, J., McRobert, A., Williams, A. M., & Mesquita, I. (2014). Visual search behaviours and verbal reports during film-based and in situ representative tasks in volleyball. European Journal of Sport Science, 14, 177–184.CrossRefGoogle ScholarPubMed
Anderson, J. R. (1983). The architecture of cognition. Cambridge, MA: Harvard University Press.Google Scholar
Anzai, Y., & Simon, H. A. (1979). The theory of learning by doing. Psychological Review, 86, 124–140.CrossRefGoogle ScholarPubMed
Arsal, G., Eccles, D. W., & Ericsson, K. A. (2016). Cognitive mediation of putting: Use of a think-aloud measure and implications for studies of golf-putting in the laboratory. Psychology of Sport and Exercise, 27, 18–27.Google Scholar
Austin, J., & Delaney, P. F. (1998). Protocol analysis as a tool for behavior analysis. The Analysis of Verbal Behavior, 15, 41–56.Google Scholar
Azevedo, R., Faremo, S., & Lajoie, S. P. (2007). Expert–novice differences in mammogram interpretation. In McNamara, D. S. & Trafton, J. G. (eds.), Proceedings of the 29th Annual Cognitive Science Society (pp. 65–70). Nashville, TN: Cognitive Science Society.Google Scholar
Balslev, T., Jarodzka, H., Holmqvist, K., de Grave, W., Muijtjens, A. M. M., Eika, B., … & Scherpbier, A. J. J. A. (2011). Visual expertise in paediatric neurology. European Journal of Paediatric Neurology, 16, 161–166.CrossRefGoogle ScholarPubMed
Binet, A. (1966). Mnemonic virtuosity: A study of chess players (trans. Simmel, M. L. and Barron, S. B.). Genetic Psychology Monographs, 74, 127–162. (Original work published 1893)Google ScholarPubMed
Boring, E. B. (1950). A history of experimental psychology. New York: Appleton-Century Crofts.Google Scholar
Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the role of biomedical knowledge in clinical reasoning by experts, intermediates and novices. Cognitive Science, 16, 153–184.Google Scholar
Bryan, W. L., & Harter, N. (1899). Studies on the telegraphic language: The acquisition of a hierarchy of habits. Psychological Review, 6, 345–375.Google Scholar
Burns, B. D. (2004). The effects of speed on skilled chess performance. Psychological Science, 15, 442–447.CrossRefGoogle ScholarPubMed
Calderwood, R., Klein, G. A., & Crandall, B. W. (1988). Time pressure, skill, and move quality in chess. American Journal of Psychology, 101, 481–493.Google Scholar
Calmeiro, L., & Tenenbaum, G. (2011). Concurrent verbal protocol analysis in sport: Illustration of thought process during a golf-putting task. Journal of Clinical Sport Psychology, 5, 223–236.CrossRefGoogle Scholar
Carayon, P., Kianfar, S., Li, Y., Xie, A., Alyousef, B., & Wooldridge, A. (2015). A systematic review of mixed methods research on human factors and ergonomics in health care. Applied Ergonomics, 51, 291–321.Google Scholar
Chabris, C. F., & Hearst, E. S. (2003). Visualization, pattern recognition, and forward search: Effects of playing speed and sight of the position on grandmaster chess errors. Cognitive Science, 27, 637–648.CrossRefGoogle Scholar
Chaffin, R., Imreh, G., Lemieux, A. F., & Chen, C. (2003). “Seeing the big picture”: Piano practice as expert problem solving. Music Perception, 20, 465–490.Google Scholar
Charness, N. (1981). Search in chess: Age and skill differences. Journal of Experimental Psychology: Human Perception and Performance, 7, 467–476.Google Scholar
Chase, W. G., & Ericsson, K. A. (1981). Skilled memory. In Anderson, J. R. (ed.), Cognitive skills and their acquisition (pp. 141–189). Hillsdale, NJ: Erlbaum.Google Scholar
Chase, W. G., & Ericsson, K. A. (1982). Skill and working memory. In Bower, G. H. (ed.), The psychology of learning and motivation (Vol. 16, pp. 1–58). New York: Academic Press.Google Scholar
Collins, D., & Dunn, M. (2011). Problem-solving strategies and processes in musical composition: Observations in real time. Journal of Music, Technology and Education, 4, 47–76.Google Scholar
Cooke, L. (2010). Assessing concurrent think-aloud protocol as a usability test method: A technical communication approach. IEEE Transactions on Professional Communication, 53, 202–215.Google Scholar
Crovitz, H. F. (1970). Galton’s walk: Methods for the analysis of thinking, intelligence, and creativity. New York: Harper & Row.Google Scholar
Crowley, R. S., Naus, G. J., Stewart, J., & Friedman, C. P. (2003). Development of visual diagnostic expertise in pathology: An information-processing study. Journal of the American Medical Informatics Association, 10, 39–51.Google Scholar
Dansereau, D. F., & Gregg, L. W. (1966). An information processing analysis of mental multiplication. Psychonomic Science, 6, 71–72.Google Scholar
Darker, C. D., & French, D. P. (2009). What sense do people make of a theory of planned behaviour questionnaire? A think aloud study. Journal of Health Psychology, 14, 861–871.Google Scholar
de Groot, A. (1978). Thought and choice in chess (2nd edn.). The Hague: Mouton. (Original work published 1946)Google Scholar
Den Hartigh, R. J. R., Van Der Steen, S., De Meij, M., Van Yperen, N. W., Gernigon, C. & Van Geert, P. L. C. (2014). Characterising expert representations during real-time action: A Skill Theory application to soccer. Journal of Cognitive Psychology, 26, 754–767.Google Scholar
Duncker, K. (1945). On problem solving. Psychological Monographs, 58, 1–113.CrossRefGoogle Scholar
Ericsson, K. A. (1988). Concurrent verbal reports on reading and text comprehension. Text, 8, 295–325.Google Scholar
Ericsson, K. A. (1996). The acquisition of expert performance: An introduction to some of the issues. In Ericsson, K. A. (ed.), The road to excellence: The acquisition of expert performance in the arts and sciences, sports, and games (pp. 1–50). Mahwah, NJ: Erlbaum.Google Scholar
Ericsson, K. A. (2002). Toward a procedure for eliciting verbal expression of nonverbal experience without reactivity: Interpreting the verbal overshadowing effect within the theoretical framework for protocol analysis. Applied Cognitive Psychology, 16, 981–987.CrossRefGoogle Scholar
Ericsson, K. A. (2003). Valid and non-reactive verbalization of thoughts during performance of tasks: Toward a solution to the central problems of introspection as a source of scientific data. Journal of Consciousness Studies, 10, 1–18.Google Scholar
Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic Medicine, 79, S70–S81.Google Scholar
Ericsson, K. A. (2015). Acquisition and maintenance of medical expertise: A perspective from the expert-performance approach with deliberate practice. Academic Medicine, 90, 1471–1486.Google Scholar
Ericsson, K. A., & Charness, N. (1994). Expert performance: Its structure and acquisition. American Psychologist, 49, 725–747.Google Scholar
Ericsson, K. A., Chase, W., & Faloon, S. (1980). Acquisition of a memory skill. Science, 208, 1181–1182.Google Scholar
Ericsson, K. A., Cheng, X., Pan, Y., Ku, Y., Ge, Y., & Hu, Y. (2017). Memory skills mediating superior memory in a world-class memorist. Memory, 25, 1294–1302.Google Scholar
Ericsson, K. A., Delaney, P. F., Weaver, G., & Mahadevan, R. (2004). Uncovering the structure of a memorist’s superior “basic” memory capacity. Cognitive Psychology, 49, 191–237.Google Scholar
Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102, 211–245.CrossRefGoogle ScholarPubMed
Ericsson, K. A., Patel, V. L., & Kintsch, W. (2000). How experts’ adaptations to representative task demands account for the expertise effect in memory recall: Comment on Vicente and Wang (1998). Psychological Review, 107, 578–592.Google Scholar
Ericsson, K. A., & Polson, P. G. (1988). Memory for restaurant orders. In Chi, M., Glaser, R., & Farr, M. (eds.), The nature of expertise (pp. 23–70). Hillsdale, NJ: Erlbaum.Google Scholar
Ericsson, K. A., & Simon, H. A. (1980). Verbal reports as data. Psychological Review, 87, 215–251.CrossRefGoogle Scholar
Ericsson, K. A., & Simon, H. A. (1984). Protocol analysis: Verbal reports as data. Cambridge, MA: Bradford Books/MIT Press.Google Scholar
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (revised edn.). Cambridge, MA: Bradford Books/MIT Press.Google Scholar
Ericsson, K. A., & Smith, J. (1991). Prospects and limits in the empirical study of expertise: An introduction. In Ericsson, K. A. and Smith, J. (eds.), Toward a general theory of expertise: Prospects and limits (pp. 1–38). Cambridge University Press.Google Scholar
Ericsson, K. A., & Staszewski, J. (1989). Skilled memory and expertise: Mechanisms of exceptional performance. In Klahr, D. and Kotovsky, K. (eds.), Complex information processing: The impact of Herbert A. Simon (pp. 235–267). Hillsdale, NJ: Erlbaum.Google Scholar
Ericsson, K. A., & Ward, P. (2007). Capturing the naturally occurring superior performance of experts in the laboratory: Toward a science of expert and exceptional performance. Current Directions in Psychological Science, 16, 346–350.Google Scholar
Ericsson, K. A., & Williams, A. M. (2007). Capturing naturally-occurring superior performance in the laboratory: Translational research on expert performance. Journal of Experimental Psychology: Applied, 13, 115–123.Google Scholar
Fayena-Tawil, F., Kozbelt, A., & Sitaras, L. (2011). Think global, act local: A protocol analysis comparison of artists’ and nonartists’ cognitions, metacognitions, and evaluations while drawing. Psychology of Aesthetics, Creativity, and the Arts, 5, 135–145.Google Scholar
Fox, M. C., Ericsson, K. A., & Best, R. (2011a). Do procedures for verbal reporting of thinking have to be reactive? A meta-analysis and recommendations for best reporting methods. Psychological Bulletin, 137, 316–344.Google Scholar
Fox, M. C., Ericsson, K. A., & Best, R. (2011b). Supplemental materials. http://dx.doi.org/10.1037/a0021663.supp.Google Scholar
French, K. E., Nevett, M. E., Spurgeon, J. H., Graham, K. C., Rink, J. E., & McPherson, S. L. (1996). Knowledge representation and problem solution in expert and novice youth baseball players. Research Quarterly for Exercise and Sport, 67, 386–395.Google Scholar
Gagné, R. H., & Smith, E. C. (1962). A study of the effects of verbalization on problem solving. Journal of Experimental Psychology, 63, 12–18.CrossRefGoogle ScholarPubMed
Galton, F. (1879). Psychometric experiments. Brain, 2, 148–162.Google Scholar
Gilhooly, K. J., McGeorge, P., Hunter, J., Rawles, J. M., Kirby, I. K., Green, C., & Wynn, V. (1997). Biomedical knowledge in diagnostic thinking: The case of electrocardiogram (ECG) interpretation. European Journal of Cognitive Psychology, 9, 199–223.Google Scholar
Hu, Y., & Ericsson, K. A. (2012). Memorization and recall of very long lists accounted for within the Long-Term Working Memory framework. Cognitive Psychology, 64, 236–266.Google Scholar
Jaarsma, T., Jarodzka, H., Nap, M., Merriënboer, J., & Boshuizen, H. (2014). Expertise under the microscope: Processing histopathological slides. Medical Education, 48, 292–300.Google Scholar
Karpov, A. (1995). Grandmaster musings. Chess Life (November), 32–33.Google Scholar
Keller, F. S. (1958). The phantom plateau. Journal of the Experimental Analysis of Behavior, 1, 1–13.CrossRefGoogle ScholarPubMed
Koltanowski, G. (1985). In the dark. Coraopolis, PA: Chess Enterprises.Google Scholar
Larkin, J., McDermott, J., Simon, D. P., & Simon, H. A. (1980). Expert and novice performance in solving physics problems. Science, 208, 1335–1342.Google Scholar
Lesgold, A., Rubinson, H., Feltovitch, P., Glaser, R., Klopher, D., & Wang, Y. (1988). Expertise in a complex skill: Diagnosing X-ray pictures. In Chi, M. T. H., Glaser, R., & Farr, M. J. (eds.), The nature of expertise (pp. 311–342). Hillsdale, NJ: Erlbaum.Google Scholar
Lundgrén-Laine, H., & Salanterä, S. (2010). Think-aloud technique and protocol analysis in clinical decision making research. Qualitative Health Research, 20, 565–575.Google Scholar
MacDonald, S., Edwards, H. M., & Zhao, T. (2012). Exploring think-alouds in usability testing: An international survey. IEEE Transactions on Professional Communication, 55, 2–19.Google Scholar
Mann, D. T. Y., Williams, A. M., Ward, P., & Janelle, C. M. (2007). Perceptual-cognitive expertise in sport: A meta analysis. Journal of Sport & Exercise Psychology, 29, 457–478.Google Scholar
McLaughlin, K., Novac, K., Rikers, R., & Schmidt, H. (2010). Does applying biomedical knowledge improve diagnostic performance when solving electrolyte problems? Canadian Medical Education Journal, 1, e4–e9.Google Scholar
McPherson, S., & Kernodle, M. (2007). Mapping two new points on the tennis expertise continuum: Tactical skills of adult advanced beginners and entry-level professionals during competition. Journal of Sports Sciences, 25, 945–959.Google Scholar
McRobert, A. P., Causer, J., Vassiliadis, J., Watterson, L., Kwan, J., & Williams, M. A. (2013). Contextual information influences diagnosis accuracy and decision making in simulated emergency medicine emergencies. BMJ Quality & Safety, 22, 478–484.Google Scholar
Meissner, C. A., & Brigham, J. C. (2001). A meta-analysis of the verbal overshadowing effect in face identification. Applied Cognitive Psychology, 15, 603–616.Google Scholar
Miglioretti, D. L., Gard, C. C., Carney, P. A., Onega, T. L., Buist, D. S., Sickles, E. A., … & Elmore, J. G. (2009). When radiologists perform best: The learning curve in screening mammogram interpretation. Radiology, 253, 632–640.Google Scholar
Miller, G. A., Galanter, E., & Pribram, K. H. (1960). Plans and the structure of behavior. New York: Holt, Rinehart, & Winston.Google Scholar
Moxley, J. H., Ericsson, K. A., Charness, N., & Krampe, R. T. (2012). The role of intuition and deliberative thinking in experts’ superior tactical decision-making. Cognition, 124, 72–78.Google Scholar
Neuman, Y., & Schwarz, B. (1998). Is self-explanation while solving problems helpful? The case of analogical problem-solving. British Journal of Educational Psychology, 68, 15–24.Google Scholar
Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.Google Scholar
Newell, A., & Simon, H. A. (1972). Human problem solving. Englewood Cliffs, NJ: Prentice-Hall.Google Scholar
Nielsen, S. G. (1999) Learning strategies in instrumental music practice. British Journal of Music Education, 16, 275–291.Google Scholar
Nisbett, R. E., & Wilson, T. D. (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review, 84, 231–259.Google Scholar
Nodine, C. F., Kundel, H. L., Mello-Thoms, C., Weinstein, S. P., Orel, S. G., Sullivan, D. C., & Conant, E. F. (1999). How experience and training influence mammography expertise. Academic Radiology, 6, 575–585.CrossRefGoogle ScholarPubMed
Norman, G. R., Trott, A. D., Brooks, L. R., & Smith, E. K. M. (1994). Cognitive differences in clinical reasoning related to postgraduate training. Teaching and Learning in Medicine, 6, 114–120.Google Scholar
Patel, V. L., Arocha, J. F., & Kaufmann, D. R. (1994). Diagnostic reasoning and medical expertise. In Medin, D. (ed.), The psychology of learning and motivation (Vol. 30, pp. 187–251). New York: Academic Press.Google Scholar
Patel, V. L., & Groen, G. J. (1991). The general and specific nature of medical expertise: A critical look. In Ericsson, K. A. & Smith, J. (eds.), Toward a general theory of expertise: Prospects and limits (pp. 93–125). Cambridge University Press.Google Scholar
Roca, A., Ford, P., McRobert, A., & Williams, A. M. (2011). Identifying the processes underpinning anticipation and decision-making in a dynamic time-constrained-task. Cognitive Processing, 12, 301–310.Google Scholar
Roca, A., Williams, A. M., & Ford, P. R. (2013). Capturing and testing perceptual-cognitive expertise: A comparison of stationary and movement response methods. Behavior Research Methods, 46, 173–177.Google Scholar
Saariluoma, P. (1991). Aspects of skilled imagery in blindfold chess. Acta Psychologica, 77, 65–89.CrossRefGoogle ScholarPubMed
Saariluoma, P. (1992). Error in chess: The apperception-restructuring view. Psychological Research/Psychologische Forschung, 54, 17–26.Google Scholar
Saariluoma, P. (1995). Chess players’ thinking. London: Routledge.Google Scholar
Schmidt, H. G., & Boshuizen, H. (1993). On acquiring expertise in medicine. Educational Psychology Review, 5, 205–221.Google Scholar
Schooler, J. W., & Engstler-Schooler, T. Y. (1990). Verbal overshadowing of visual memories: Some things are better left unsaid. Cognitive Psychology, 22, 36–71.Google Scholar
Schraagen, J. M. (2006). Task analysis. In Ericsson, K. A., Charness, N., Hoffman, R. R., & Feltovich, P. J. (eds.), The Cambridge handbook of expertise and expert performance (pp. 185–201). Cambridge University Press.Google Scholar
St. Germain, J., & Tenenbaum, G. (2011) Decision-making and thought processes among poker players. High Ability Studies, 22, 3–17.Google Scholar
Thompson, C. P., Cowan, T. M., & Frieman, J. (1993). Memory search by a memorist. Hillsdale, NJ: Erlbaum.Google Scholar
Tuffiash, M., Roring, R. W., & Ericsson, K. A. (2007). Expert word play: Capturing and explaining reproducibly superior verbal task performance. Journal of Experimental Psychology: Applied, 13, 124–134.Google Scholar
van der Maas, H. L. J., & Wagenmakers, E. J. (2005). A psychometric analysis of chess expertise. American Journal of Psychology, 118, 29–60.Google Scholar
Watson, J. B. (1913). Psychology as the behaviorist views it. Psychological Review, 20, 158–177.Google Scholar
Watson, J. B. (1920). Is thinking merely the action of language mechanisms? British Journal of Psychology, 11, 87–104.Google Scholar
Whitehead, A. E., Taylor, J. A., & Polman, R. C. J. (2015). Examination of the suitability of collecting in event cognitive processes using Think Aloud protocol in golf. Frontiers in Psychology, 6, 1083.Google Scholar
Whitehead, A. E., Taylor, J. A., & Polman, R. C. (2016). Evidence for skill level differences in the thought processes of golfers during high and low pressure situations. Frontiers in Psychology, 6, 1–12.Google Scholar
Willis, G. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications.Google Scholar
Willis, G. (2015). Analysis of the cognitive interview in questionnaire design: Understanding qualitative research. Oxford University Press.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×