Hostname: page-component-89b8bd64d-z2ts4 Total loading time: 0 Render date: 2026-05-07T20:11:29.506Z Has data issue: false hasContentIssue false
  • English
  • Français

Markov Decision Process Measurement Model

Published online by Cambridge University Press:  01 January 2025

Michelle M. LaMar Open the ORCID record for Michelle M. LaMar [Opens in a new window]
Michelle M. LaMar*
Affiliation:
Educational Testing Service
*
Correspondence should be made to Michelle M. LaMar, Educational Testing Service, Princeton, NJ, USA. Email: mlamar@ets.org
Get access Rights & Permissions [Opens in a new window]

Abstract

Within-task actions can provide additional information on student competencies but are challenging to model. This paper explores the potential of using a cognitive model for decision making, the Markov decision process, to provide a mapping between within-task actions and latent traits of interest. Psychometric properties of the model are explored, and simulation studies report on parameter recovery within the context of a simple strategy game. The model is then applied to empirical data from an educational game. Estimates from the model are found to correlate more strongly with posttest results than a partial-credit IRT model based on outcome data alone.

Keywords

latent-trait modelcognitive modelperformance assessment

Information

Type
Original Paper
Information
Psychometrika , Volume 83 , Issue 1 , March 2018 , pp. 67 - 88
Copyright
Copyright © 2017 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Electronic supplementary material The online version of this article (doi:10.1007/s11336-017-9570-0) contains supplementary material, which is available to authorized users.

References

Baker, C., Saxe, R., Tenenbaum, J., (2009). Action understanding as inverse planning, Cognition, 113(3) 329–349.CrossRefGoogle ScholarPubMed
Baker, C., Saxe, R., & Tenenbaum, J., (2011). Bayesian theory of mind: Modeling joint belief-desire attribution. In Proceedings of the thirty-third annual conference of the cognitive science society (pp. 2469–2474).Google Scholar
Bock, D.R., (1972). Estimating item parameters and latent ability when responses are scored in two or more nominal categories, Psychometrika, 37(1) 29–51.CrossRefGoogle Scholar
Bradshaw, L., & Templin, J., (2014). Combining item response theory and diagnostic classification models: A psychometric model for scaling ability and diagnosing misconceptions. Psychometrika, 79(3), 403–425. doi:10.1007/s11336-013-9350-4.CrossRefGoogle ScholarPubMed
Fischer, G.H., (1973). The linear logistic test model as an instrument in educational research, Acta Psychologica, 37(6) 359–374.CrossRefGoogle Scholar
Howard, R.A., (1960). Dynamic programming and markov processes 1 Cambridge, MA: MIT Press.Google Scholar
Hulin, C.L., Lissak, R.I., Drasgow, F., (1982). Recovery of two- and three-parameter logistic item characteristic curves: A Monte Carlo study, Applied Psychological Measurement, 6(3) 249–260.CrossRefGoogle Scholar
LaMar, M.M., (2014). Models for understanding student thinking using data from complex computerized science tasks (Unpublished doctoral dissertation) Berkeley: University of California.Google Scholar
Limpert, E., Stahel, W.A., Abbt, M., (2001). Log-normal distributions across the sciences: Keys and clues, BioScience, 51(5) 341.CrossRefGoogle Scholar
Mislevy, R.J., Behrens, J.T., Dicerbo, K.E., Frezzo, D.C., West, P., (2012). Three things game designers need to know about assessment. In Ifenthaler, D., Eseryel, D., Ge, X. (Eds.), Assessment in game-based learning. New York: Springer pp.59–81.CrossRefGoogle Scholar
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A. A., Veness, J., Bellemare, M. G., & Hassabis, D., (2015, February). Human-level control through deep reinforcement learning. Nature,518(7540), 529–533..CrossRefGoogle Scholar
National Research Council. (2014). Developing assessments for the next generation science standards. Washington, DC: The National Academies Press..Google Scholar
Ng, A. Y., & Russell, S., (2000). Algorithms for inverse reinforcement learning. In Proceedings of the seventeenth international conference on machine learning (pp. 663–670) (2000)..Google Scholar
Puterman, M.L., (1994). Markov decision processes: Discrete stochastic dynamic programming New York: Wiley.CrossRefGoogle Scholar
Rafferty, A.N., LaMar, M.M., Griffiths, T.L., (2015). Inferring learners’ knowledge from their actions, Cognitive Science, 39(3) 584–618.CrossRefGoogle ScholarPubMed
Red Hill Studios. (n.d.). Lifeboat to mars. Retrieved from http://www.pbskids.org/lifeboat.Google Scholar
Reise, S.P., Yu, J., (1990). Parameter recovery in the graded response model using MULTILOG, Journal of Educational Measurement, 27(2) 133–144.CrossRefGoogle Scholar
Russell, S., Norvig, P., Artificial intelligence: A modern approach 2009 3Upper Saddle River: Pearson.Google Scholar
Rust, J. eds.Engle, R., McFadden, D., (1994). Structural estimation of Markov decision processes, Handbook of econometrics, Amsterdam: Elsevier Science 3081–3143.CrossRefGoogle Scholar
Svetina, D., Crawford, A.V., Levy, R., Green, S.B., Scott, L., Thompson, M., Kunze, K.L., (2013). Designing small-scale tests: A simulation study of parameter recovery with the 1-PL, Psychological Test and Assessment Modeling, 55(4) 335–360.Google Scholar
Thissen, D., Steinberg, L., (1986). A taxonomy of item response models, Psychometrika, 51(4) 567–577.CrossRefGoogle Scholar
Wu, M.L., Adams, R.J., Wilson, M.R., (1998). ConQuest [computer software and manual] Camberwell, VIC: Australian Council for Educational Research.Google Scholar
Supplementary material: File

LaMar supplementary material

LaMar supplementary material
Download LaMar supplementary material(File)
File 37.9 KB