Cognitive Modeling for Cognitive Engineering

doi:10.1017/9781108755610.038

33 - Cognitive Modeling for Cognitive Engineering

from Part IV - Computational Modeling in Various Cognitive Fields

Published online by Cambridge University Press: 21 April 2023

Matthew L. Bolton and

Wayne D. Gray

Edited by

Ron Sun

Show author details

Ron Sun: Affiliation:
Rensselaer Polytechnic Institute, New York

Book contents

Get access

Summary

Cognitive engineering is the application of cognitive science to engineering. While the majority of the cognitive models and architectures commonly associated with cognitive engineering were created to understand human behavior, their use in engineering has been carried out with the purpose of realizing better systems. As such, cognitive engineering model fidelity varies, based on application goals. This chapter provides readers with a history of cognitive modeling in cognitive engineering and its diverse contributions by reviewing the seminal work of Card, Moran, and Newell, which laid the foundations for many developments. It then examines the use of cognitive models in complex systems engineering. The chapter concludes with a summary and a discussion of potential threats and future advances.

Keywords

cognitive engineering human factors cognitive models cognitive architecture formal methods simulation task analysis

Information

Type: Chapter
Information: The Cambridge Handbook of Computational Cognitive Sciences , pp. 1088 - 1112

DOI: https://doi.org/10.1017/9781108755610.038 [Opens in a new window]

Publisher: Cambridge University Press

Print publication year: 2023

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Book purchase

Temporarily unavailable

References

Abbate, A. J., & Bass, E. J. (2015). Using computational tree logic methods to analyze reachability in user documentation. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 59, pp. 1481–1485).CrossRef Google Scholar

Aït-Ameur, Y., & Baron, M. (2006). Formal and experimental validation approaches in HCI systems design based on a shared event B model. International Journal on Software Tools for Technology Transfer, 8 (6), 547–563.CrossRef Google Scholar

Anderson, J. R. (1993). Rules of the Mind. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar

Anderson, J. R., Bothell, D., Byrne, M. D., Douglas, S., Lebiere, C., & Quin, Y. (2004). An integrated theory of the mind. Psychological Review, 111 (4), 1036–1060.Google Scholar

Anderson, J. R., Matessa, M., & Lebiere, C. (1997). ACT-R: a theory of higher-level cognition and its relation to visual attention. Human-Computer Interaction, 12 (4), 439–462.Google Scholar

Bainbridge, L. (1983). Ironies of automation. Automatica, 19 (6), 775–780.CrossRef Google Scholar

Ball, J., Myers, C., Heiberg, A., et al. (2010). The synthetic teammate project. Computational and Mathematical Organization Theory, 16 (3), 271–299.Google Scholar

Ballard, D. H., & Sprague, N. (2007). On the role of embodiment in modeling natural behaviors. In Gray, W. D. (Ed.), Integrated Models of Cognitive Systems. New York, NY: Oxford University Press.Google Scholar

Barbosa, A., Paiva, A. C., & Campos, J. C. (2011). Test case generation from mutated task models. In Proceedings of the 3rd ACM SIGCHI Symposium on Engineering Interactive Computing Systems (pp. 175–184).CrossRef Google Scholar

Basnyat, S., Palanque, P. A., Bernhaupt, R., & Poupart, E. (2008). Formal modelling of incidents and accidents as a means for enriching training material for satellite control operations. In Proceedings of the Joint ESREL 2008 and 17th SRA-Europe Conference (CD-ROM). London: Taylor & Francis.Google Scholar

Basnyat, S., Palanque, P., Schupp, B., & Wright, P. (2007). Formal socio-technical barrier modelling for safety-critical interactive systems design. Safety Science, 45 (5), 545–565.Google Scholar

Bastide, R., & Basnyat, S. (2007). Error patterns: systematic investigation of deviations in task models. In Task Models and Diagrams for Users Interface Design 5th International Workshop (pp. 109–121). Berlin: Springer.Google Scholar

Basuki, T. A., Cerone, A., Griesmayer, A., & Schlatte, R. (2009). Model-checking user behaviour using interacting components. Formal Aspects of Computing, 1–18.Google Scholar

Bolton, M. L. (2015). Model checking human–human communication protocols using task models and miscommunication generation. Journal of Aerospace Information Systems, 12 (7), 476–489.Google Scholar

Bolton, M. L. (2017a). Novel developments in formal methods for human factors engineering. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 715–717).Google Scholar

Bolton, M. L. (2017b). A task-based taxonomy of erroneous human behavior. International Journal of Human-Computer Studies, 108, 105–121.Google Scholar

Bolton, M. L., & Bass, E. J. (2010). Formally verifying human-automation interaction as part of a system model: limitations and tradeoﬀs. Innovations in Systems and Software Engineering: A NASA Journal, 6 (3), 219–231.Google Scholar

Bolton, M. L., & Bass, E. J. (2013). Generating erroneous human behavior from strategic knowledge in task models and evaluating its impact on system safety with model checking. IEEE Transactions on Systems, Man and Cybernetics: Systems, 43 (6), 1314–1327.Google Scholar

Bolton, M. L., & Bass, E. J. (2017). Enhanced operator function model (EOFM): a task analytic modeling formalism for including human behavior in the verification of complex systems. In Weyers, B., Bowen, J., Dix, A., & Palanque, P. (Eds.), The Handbook of Formal Methods in Human-Computer Interaction. Berlin: Springer.Google Scholar

Bolton, M. L., Bass, E. J., & Siminiceanu, R. I. (2012). Generating phenotypical erroneous human behavior to evaluate human–automation interaction using model checking. International Journal of Human-Computer Studies, 70 (11), 888–906.CrossRef Google Scholar PubMed

Bolton, M. L., Bass, E. J., & Siminiceanu, R. I. (2013). Using formal verification to evaluate human-automation interaction in safety critical systems, a review. IEEE Transactions on Systems, Man and Cybernetics: Systems, 43 (3), 488–503.CrossRef Google Scholar

Bolton, M. L., Molinaro, K. A., & Houser, A. M. (2019). A formal method for assessing the impact of task-based erroneous human behavior on system safety. Reliability Engineering & System Safety, 188, 168–180.Google Scholar

Bolton, M. L., Siminiceanu, R. I., & Bass, E. J. (2011). A systematic approach to model checking human-automation interaction using task-analytic models. IEEE Transactions on Systems, Man, and Cybernetics, Part A, 41(5), 961–976.Google Scholar

Boring, R. L., & Rasmussen, M. (2016). GOMS-HRA: a method for treating subtasks in dynamic human reliability analysis. In Proceedings of the 2016 European Safety and Reliability Conference (pp. 956–963).Google Scholar

Bovair, S., Kieras, D. E., & Polson, P. G. (1990). The acquisition and performance of text-editing skill: a cognitive complexity analysis. Human-Computer Interaction, 5 (1), 1–48.CrossRef Google Scholar

Byrne, M. D. (2007). Cognitive architecture. In Sears, A. & Jacko, J. A. (Eds.), The Human-Computer Interaction Handbook (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar

Byrne, M. D., & Anderson, J. R. (1998). Perception and action. In Anderson, J. R. & Lebiѐre, C. (Eds.), The Atomic Components of Thought (pp. 167–200). Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar

Byrne, M. D., & Bovair, S. (1997). A working memory model of a common procedural error. Cognitive Science, 21 (1), 31–61.CrossRef Google Scholar

Campos, J. C., Fayollas, C., Martinie, C., Navarre, D., Palanque, P., & Pinto, M. (2016). Systematic automation of scenario-based testing of user interfaces. In Proceedings of the 8th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (pp. 138–148).Google Scholar

Card, S. K., Moran, T. P., & Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar

Clarke, E. M., Grumberg, O., & Peled, D. A. (1999). Model Checking. Cambridge, MA: MIT Press.Google Scholar

Curzon, P., & Blandford, A. (2004). Formally justifying user-centered design rules: a case study on post-completion errors. In Proceedings of the 4th International Conference on Integrated Formal Methods (pp. 461–480). Berlin: Springer.Google Scholar

Curzon, P., & Rukšėnas, R. (2017). Modelling the user. In Weyers, B., Bowen, J., Dix, A., & Palanque, P. (Eds.), The Handbook of Formal Methods in Human-Computer Interaction. Berlin: Springer.Google Scholar

Curzon, P., Rukšėnas, R., & Blandford, A. (2007). An approach to formal verification of human–computer interaction. Formal Aspects of Computing, 19 (4), 513–550.Google Scholar

Degani, A. (2004).Taming HAL: Designing Interfaces Beyond 2001. New York, NY: Macmillan.Google Scholar

Degani, A., Heymann, M., & Shafto, M. (1999). Formal aspects of procedures: the problem of sequential correctness. In Proceedings of the 43rd Annual Meeting of the Human Factors and Ergonomics Society (pp. 1113–1117). Los Angeles, CA: SAGE.Google Scholar

Demir, M., McNeese, N. J., Cooke, N. J., Ball, J. T., Myers, C., & Frieman, M. (2015). Synthetic teammate communication and coordination with humans. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 951–955). Los Angeles, CA: SAGE.Google Scholar

Demir, M., McNeese, N. J., & Cooke, N. J. (2016). Team communication behaviors of the human-automation teaming. In 2016 IEEE International Multi-Disciplinary Conference on Cognitive Methods in Situation Awareness and Decision Support (CogSIMA) (pp. 28–34). New York, NY: IEEE.CrossRef Google Scholar

Emerson, E. A. (1990). Temporal and modal logic. In Formal Models and Semantics (pp. 995–1072). Oxford: Elsevier.Google Scholar

España, S., Pederiva, I., & Panach, J. I. (2007). Integrating model-based and task-based approaches to user interface generation. In Computer-Aided Design of User Interfaces V (pp. 253–260). Amsterdam: Springer.CrossRef Google Scholar

Fahssi, R., Martinie, C., & Palanque, P. (2015). Enhanced task modelling for systematic identification and explicit representation of human errors. In Human-Computer Interaction – Interact 2015 (pp. 192–212). Cham: Springer International Publishing.Google Scholar

Fields, R. E. (2001). Analysis of erroneous actions in the design of critical systems. Unpublished doctoral dissertation, University of York, York.Google Scholar

Gluck, K. A., Ball, J. T., Gunzelmann, G., Krusmark, M., Lyon, D., & Cooke, N. (2005). A prospective look at a synthetic teammate for UAV applications. In Infotech@ Aerospace. Reston: American Institute of Aeronautics and Astronautics.Google Scholar

Gluck, K. A., Ball, J. T., & Krusmark, M. A. (2007). Cognitive control in a computational model of the predator pilot. In Gray, W. D. (Ed.), Integrated Models of Cognitive Systems (pp. 13–28). New York, NY: Oxford University Press.CrossRef Google Scholar

Gray, W. D. (2008). Cognitive modeling for cognitive engineering. In Sun, R. (Ed.), The Cambrdge Handbook of Computational Psychology (pp. 565–588). Cambridge: Cambridge University Press.Google Scholar

Gray, W. D. (Ed.). (2007). Integrated Models of Cognitive Systems. New York, NY: Oxford University Press.Google Scholar

Gray, W. D., & Boehm-Davis, D. A. (2000). Milliseconds matter: an introduction to microstrategies and to their use in describing and predicting interactive behavior. Journal of Experimental Psychology: Applied, 6 (4), 322–335.Google Scholar

Gray, W. D., John, B. E., & Atwood, M. E. (1993). Project Ernestine: validating a GOMS analysis for predicting and explaining real-world performance. Human-Computer Interaction, 8 (3), 237–309.Google Scholar

Gunning, D., & Aha, D. W. (2019). DARPA’s explainable artificial intelligence program. AI Magazine, 40 (2), 44–58.Google Scholar

Hollnagel, E. (1993). The phenotype of erroneous actions. International Journal of Man-Machine Studies, 39 (1), 1–32.Google Scholar

Jeong, H., & Liu, Y. (2017). Modeling of stimulus-response secondary tasks with different modalities while driving in a computational cognitive architecture. In Proceedings of the 9th International Driving Symposium on Human Factors in Driver Assessment, Training, and Vehicle Design (pp. 193–199). Iowa, IA: University of Iowa.Google Scholar

John, B. E. (1988). Contributions to engineering models of human-computer interaction. Unpublished doctoral dissertation, Carnegie Mellon University, Pittsburgh, PA.Google Scholar

John, B. E. (1996). TYPIST: a theory of performance in skilled typing. Human-Computer Interaction, 11 (4), 321–355.Google Scholar

Kebabjian, R. (2016). Accident statistics. planecrashinfo.com. Retrieved from www.planecrashinfo.com/cause.htm [last accessed July 30, 2022].Google Scholar

Kenny, D. J. (2015). 26th Joseph T. Nall Report: General Aviation Accidents in 2014. Technical Report. Frederick, MD: AOPA Foundation.Google Scholar

Kieras, D. E. (1997). A guide to GOMS model usability evaluation using NGOMSL. In Helander, M., Landauer, T. K., & Prabhu, P. (Eds.), Handbook of Human-Computer Interaction (2nd ed., pp. 733–766). New York, NY: Elsevier.Google Scholar

Kieras, D. E. (2007). Model-based evaluation. In Sears, A. & Jacko, J. A. (Eds.), The Human-Computer Interaction Handbook (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar

Kieras, D. E., & Bovair, S. (1986). The acquisition of procedures from text: a production-system analysis of transfer of training. Journal of Memory and Language, 25, 507–524.Google Scholar

Kieras, D. E., & Meyer, D. E. (1997). An overview of the EPIC architecture for cognition and performance with application to human-computer interaction. Human-Computer Interaction, 12 (4), 391–438.Google Scholar

Kieras, D. E., Wakefield, G. H., Thompson, E. R., Iyer, N., & Simpson, B. D. (2016). Modeling two-channel speech processing with the EPIC cognitive architecture. Topics in Cognitive Science, 8 (1), 291–304.Google Scholar

Kirwan, B., & Ainsworth, L. K. (Eds.). (1992). A Guide to Task Analysis. Washington, DC: Taylor & Francis.Google Scholar

Le Bot, P. (2004). Human reliability data, human error and accident models – illustration through the Three Mile Island accident analysis. Reliability Engineering & System Safety, 83 (2), 153–167.Google Scholar

Li, M., & Bolton, M. L. (2019). Task-based automated test case generation for human-machine interaction. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (Vol. 63, pp. 807–811).Google Scholar

Li, M., Wei, J., Zheng, X., & Bolton, M. L. (2017). A formal machine learning approach to generating human-machine interfaces from task models. IEEE Transactions of Human Machine Systems, 47 (6), 822–833.CrossRef Google Scholar

Liu, Y., Feyen, R., & Tsimhoni, O. (2006). Queueing Network-Model Human Processor (QN-MHP): a computational architecture for multitask performance in human-machine systems. ACM Transactions on Computer-Human Interaction (TOCHI), 13 (1), 37–70.Google Scholar

Lovett, M. C., & Anderson, J. R. (1996). History of success and current context in problem solving: combined influences on operator selection. Cognitive Psychology, 31, 168–217.Google Scholar

Luyten, K., Clerckx, T., Coninx, K., & Vanderdonckt, J. (2003). Derivation of a dialog model from a task model by activity chain extraction. In Proceedings of the 10th International Workshop on Interactive Systems. Design, Specification, and Verification (pp. 203–217). Berlin: Springer.Google Scholar

Manning, S. D., Rash, C. E., LeDuc, P. A., Noback, R. K., & McKeon, J. (2004). The Role of human Causal Factors in US Army Unmanned Aerial Vehicle Accidents. Technical Report No. 2004-11. Adelphi, MD: USA Army Research Laboratory.Google Scholar

Makary, M. A., & Daniel, M. (2016). Medical error – the third leading cause of death in the US. BMJ, 353, 5.Google Scholar

Mirman, J. H. (2019). A dynamical systems perspective on driver behavior. Transportation Research Part F: Traffic Psychology and Behaviour, 63, 193–203.Google Scholar

Mirman, J. H., Curry, A. E., & Mirman, D. (2019). Learning to drive: a reconceptualization. Transportation Research Part F: Traffic Psychology and Behaviour, 62, 316–326.Google Scholar

Newell, A. (1990). Unified Theories of Cognition. Cambridge, MA: Harvard University Press.Google Scholar

Newell, A., & Card, S. K. (1985). The prospects for psychological science in human-computer interaction. Human-Computer Interaction, 1 (3), 209–242.Google Scholar

NHTSA. (2008). National Motor Vehicle Crash Causation Survey: Report to Congress. Technical Report No. DOT HS 811 059. Springfield: National Highway Traﬃc Safety Administration.Google Scholar

Pan, D., & Bolton, M. L. (2018). Properties for formally assessing the performance level of human-human collaborative procedures with miscommunications and erroneous human behavior. International Journal of Industrial Ergonomics, 63, 75–88.Google Scholar

Paternò, F., & Santoro, C. (2001). Integrating model checking and HCI tools to help designers verify user interface properties. In Proceedings of the 7th International Workshop on the Design, Specification, and Verification of Interactive Systems (pp. 135–150). Berlin: Springer.Google Scholar

Paternò, F., Mancini, C., & Meniconi, S. (1997). ConcurTaskTrees: a diagrammatic notation for specifying task models. In Proceedings of the IFIP TC13 International Conference on Human-Computer Interaction (pp. 362–369). London: Chapman & Hall.Google Scholar

Pew, R. W. (2007). Some history of human performance modeling. In Gray, W. D. (Ed.), Integrated Models of Cognitive Systems. New York, NY: Oxford University Press.Google Scholar

Pritchett, A. R., Feigh, K. M., Kim, S. Y., & Kannan, S. K. (2014). Work models that compute to describe multiagent concepts of operation: part 1. Journal of Aerospace Information Systems, 11 (10), 610–622.Google Scholar

Reason, J. (1990). Human Error. New York, NY: Cambridge University Press.CrossRef Google Scholar

Rehman, U., Cao, S., & MacGregor, C. (2019). Using an integrated cognitive architecture to model the effect of environmental complexity on drivers’ situation awareness. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting (pp. 812–816).CrossRef Google Scholar

Rhie, Y. L., Lim, J. H., & Yun, M. H. (2018). Queueing network based driver model for varying levels of information processing. IEEE Transactions on Human-Machine Systems, 49 (6), 508–517.Google Scholar

Rodgers, S., Myers, C., Ball, J., & Freiman, M. (2011). The situation model in the synthetic teammate project. In Proceedings of the 20th Annual Conference on Behavior Representation in Modeling and Simulation (pp. 66–73).Google Scholar

Rukšėnas, R., Back, J., Curzon, P., & Blandford, A. (2008). Formal modelling of salience and cognitive load. In Proceedings of the 2nd International Workshop on Formal Methods for Interactive Systems (pp. 57–75). Amsterdam: Elsevier Science Publishers.Google Scholar

Rukšėnas, R., Back, J., Curzon, P., & Blandford, A. (2009). Verification-guided modelling of salience and cognitive load. Formal Aspects of Computing, 21 (6), 541–569.Google Scholar

Rukšėnas, R., Curzon, P., Back, J., & Blandford, A. (2007). Formal modelling of cognitive interpretation. In Proceedings of the 13th International Workshop on the Design, Specification, and Verification of Interactive Systems (pp. 123–136). London: Springer.CrossRef Google Scholar

Rukšėnas, R., Curzon, P., Blandford, A., & Back, J. (2014). Combining human error verification and timing analysis: a case study on an infusion pump. In Proceedings of the 13th International Workshop on the Design, Specification, and Verification of Interactive Systems (pp. 123–136). London: Springer.Google Scholar

Salvucci, D. D. (2001). Predicting the effects of in-car interface use on driver performance: an integrated model approach. International Journal of Human-Computer Studies, 55 (1), 85–107.Google Scholar

Salvucci, D. D. (2006). Modeling driver behavior in a cognitive architecture. Human Factors, 48 (2), 362–380.Google Scholar

Salvucci, D. D., & Gray, R. (2004). A two-point visual control model of steering. Perception, 33 (10), 1233–1248.Google Scholar

Salvucci, D. D., & Macuga, K. L. (2002). Predicting the effects of cellular-phone dialing on driver performance. Cognitive Systems Research, 3 (1), 95–102.CrossRef Google Scholar

Santoro, C. (2005). A Task Model-Based Approach for Design and Evaluation of Innovative User Interfaces. Belgium: Presses universitaires de Louvain.Google Scholar

Schweickert, R., Fisher, D. L., & Proctor, R. W. (2003). Steps toward building mathematical and computer models from cognitive task analyses. Human Factors, 45 (1), 77–103.Google Scholar

Shepherd, A. (1998). HTA as a framework for task analysis. Ergonomics, 41 (11), 1537–1552.Google Scholar

Shepherd, A. (2001). Hierarchical Task Analysis. New York, NY: Taylor & Francis.Google Scholar

Sheridan, T. B., & Parasuraman, R. (2005). Human-automation interaction. Reviews of Human Factors and Ergonomics, 1 (1), 89–129.Google Scholar

Simon, H. A. (1996). The Sciences of the Artificial (3rd ed.). Cambridge, MA: MIT Press.Google Scholar

Strauch, B. (2017). Ironies of automation: still unresolved after all these years. IEEE Transactions on Human-Machine Systems, 48 (5), 419–433.CrossRef Google Scholar

Thomas, M. (1994). The role of formal methods in achieving dependable software. Reliability Engineering & System Safety, 43 (2), 129–134.Google Scholar

Vieira, M., Leduc, J., Hasling, B., Subramanyan, R., & Kazmeier, J. (2006). Automation of GUI testing using a model-driven approach. In Proceedings of the 2006 International Workshop on Automation of Software Test (pp. 9–14).Google Scholar

Weyers, B., Bowen, J., Dix, A., & Palanque, P. (Eds.). (2017). The Handbook of Formal Methods in Human-Computer Interaction. Berlin: Springer.Google Scholar

Wing, J. M. (1990). A specifier’s introduction to formal methods. Computer, 23 (9), 8–22.Google Scholar

Wu, C., Rothrock, L., & Bolton, M. (2019). Editorial special issue on computational human performance modeling. IEEE Transactions on Human-Machine Systems, 49 (6), 470–473.Google Scholar

Young, R. M., Green, T. R. G., & Simon, T. (1989). Programmable user models for predictive evaluation of interface designs. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 15–19). New York: ACM.Google Scholar

Zheng, X., Bolton, M. L., Daly, C., & Biltekoﬀ, E. (2020). The development of a next-generation human reliability analysis: systems analysis for formal pharmaceutical human reliability (SAFPH℞). Reliability Engineering & System Safety, 20. https://doi.org/10.1016/j.ress.2020.106927 Google Scholar

Accessibility standard: Unknown

Why this information is here

This section outlines the accessibility features of this content - including support for screen readers, full keyboard navigation and high-contrast display options. This may not be relevant for you.

Accessibility Information

Accessibility compliance for the HTML of this chapter is currently unknown and may be updated in the future.