Hostname: page-component-76d6cb85b7-vdhp9 Total loading time: 0 Render date: 2026-07-23T20:55:44.737Z Has data issue: false hasContentIssue false
  • English
  • Français

Modeling Learning in Doubly Multilevel Binary Longitudinal Data Using Generalized Linear Mixed Models: An Application to Measuring and Explaining Word Learning

Published online by Cambridge University Press:  01 January 2025

Sun-Joo Cho  and
Amanda P. Goodwin
Sun-Joo Cho*
Affiliation:
Vanderbilt University’s Peabody College
Amanda P. Goodwin
Affiliation:
Vanderbilt University’s Peabody College
*
Correspondence should be made to Sun-Joo Cho, Vanderbilt University’s Peabody College, Nashville, TN, USA. Email: sj.cho@vanderbilt.edu. http://www.vanderbilt.edu/psychological_sciences/bio/sun-joo-cho
Get access Rights & Permissions [Opens in a new window]

Abstract

When word learning is supported by instruction in experimental studies for adolescents, word knowledge outcomes tend to be collected from complex data structure, such as multiple aspects of word knowledge, multilevel reader data, multilevel item data, longitudinal design, and multiple groups. This study illustrates how generalized linear mixed models can be used to measure and explain word learning for data having such complexity. Results from this application provide deeper understanding of word knowledge than could be attained from simpler models and show that word knowledge is multidimensional and depends on word characteristics and instructional contexts.

Keywords

binary longitudinal datadoubly multilevel datageneralized linear mixed modelslearningpsycholinguistic dataword learning

Information

Type
Original paper
Information
Psychometrika , Volume 82 , Issue 3 , September 2017 , pp. 846 - 870
Copyright
Copyright © 2016 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Akaike, M.. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19, 415–438.CrossRefGoogle Scholar
Anderson, R. C., Freebody, P.Vocabulary knowledge 1981 Newark: International Reading Association.Google Scholar
Baayen, R. H., (2008). Analyzing linguistic data: A practical introduction to statistics using R. New York: Cambridge University Pressdoi:10.1017/CBO9780511801686.CrossRefGoogle Scholar
Baayen, R., Davidson, D., Bates, D.. (2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412. doi:10.1016/j.jml.2007.12.005.CrossRefGoogle Scholar
Bartolucci, F., Pennoni, F., Vittadini, G.. (2011). Assessment of school performance through a multilevel latent Markov Rasch model. Journal of Educational and Behavioral Statistics, 36, 491–522. doi:10.3102/1076998610381396.CrossRefGoogle Scholar
Bates, D. M., (2010). lme4: Mixed-effects modeling with R. New York: Springer.Google Scholar
Bates, D., Maechler, M., & Bolker, B. (2011). lme4: linear mixed-effects models using S4 classes. R package version 0.999375-39. http://cran.rproject.org/package=lme4.Google Scholar
Becker, W. C., Dixon, R., Anderson-Inman, L.Morphographic and root word analysis of 26,000 high frequency words 1980 Eugene: College of Education: University of Oregon Follow Through Project.Google Scholar
Bryk, A. S., Raudenbush, S. W., (1992). Hierarchical linear models in social and behavioral research: Applications and data analysis methods. 1Newbury Park, CA: Sage Publications.Google Scholar
Carey, S., Bartlett, E.. (1978). Acquiring a single new word. Papers and Reports on Child Language Development, 15, 17–29.Google Scholar
Carlisle, J. F., Katz, L. A.. (2006). Effects of word and morpheme familiarity on reading of derived words. Reading and Writing, 19, 669–693. doi:10.1007/s11145-005-5766-2.CrossRefGoogle Scholar
Cho, S-J, De Boeck, P., Embretson, S., Rabe-Hesketh, S.. (2014). Additive multilevel item structure models with random residuals: Item modeling for explanation and item generation. Psychometrika, 79, 84–104. doi:10.1007/s11336-013-9360-2.CrossRefGoogle ScholarPubMed
Cho, S-J, Gilbert, J. K., Goodwin, A. P.. (2013). Explanatory multidimensional multilevel random item response model: An application to simultaneous investigation of word and person contributions to multidimensional lexical quality. Psychometrika, 78, 830–855. doi:10.1007/s11336-013-9333-5.CrossRefGoogle Scholar
Cho, S-J, Partchev, I., De Boeck, P.. (2012). Parameter estimation of multiple item profiles models. British Journal of Mathematical and Statistical Psychology, 65, 438–466. doi:10.1111/j.2044-8317.2011.02036.x.CrossRefGoogle Scholar
De Boeck, P., Wilson, M.Explanatory item response models: A generalized linear and nonlinear approach 2004 New York: Springerdoi:10.1007/978-1-4757-3990-9.CrossRefGoogle Scholar
Elleman, A. M., Lindo, E. J., Morphy, P., Compton, D. L.. (2009). The impact of vocabulary instruction on passage-level comprehension of school-age children: A meta-analysis. Journal of Research on Educational Effectiveness, 2, 1–44. doi:10.1080/19345740802539200.CrossRefGoogle Scholar
Embretson, S. E.. (1991). A multidimensional latent trait model for measuring learning and change. Psychometrika, 56, 495–515. doi:10.1007/BF02294487.CrossRefGoogle Scholar
Fox, J-PBayesian item response modeling 2010 New York: Springerdoi:10.1007/978-1-4419-0742-4.CrossRefGoogle Scholar
Geerlings, H., Glas, C. A. W., van der Linden, W. J.. (2011). Modeling rule-based item generation. Psychometrika, 76, 337–359. doi:10.1007/s11336-011-9204-x.CrossRefGoogle Scholar
Gelman, A., Su, Y. S., Yajima, M., Hill, J., Pittau, M. G., Kerman, J., & Zheng, T. (2010). Data analysis using regression and multilevel/hierarchical models. R package version 1.3-06..Google Scholar
Glas, C. A. W., van der Linden, W. J.. (2003). Computerized adaptive testing with item cloning. Applied Psychological Measurement, 27, 247–261. doi:10.1177/0146621603027004001.CrossRefGoogle Scholar
González, J., De Boeck, P., Tuerlinckx, F.. (2014). Linear mixed modelling for data from a double mixed factorial design with covariates: A case-study on semantic categorization response times. Journal of Royal Statistical Soceity C, 63, 289–230. doi:10.1111/rssc.12031.CrossRefGoogle Scholar
Goodwin, A. P.. (2016). Effectiveness of word solving: Integrating morphological problem solving within comprehension instruction for middle school students. Reading and Writing: An International Journal, 29(1), 91–116. doi:10.1007/s11145-015-9581-0.CrossRefGoogle Scholar
Goodwin, A. P., Ahn, S.. (2010). A meta-analysis of morphological interventions: Effects on literacy achievement of children with literacy difficulties. Annals of Dyslexia, 60, 183–208. doi:10.1007/s11881-010-0041-x.CrossRefGoogle ScholarPubMed
Goodwin, A. P., Ahn, S.. (2013). A meta-analysis of morphological interventions in English: Effects on literacy outcomes for school-age children. Scientific Studies of Reading, 17, 257–285. doi:10.1080/10888438.2012.689791.CrossRefGoogle Scholar
Goodwin, A. P., Perkins, J.. (2015). Word detectives: Morphological Instruction that supports academic language. The Reading Teacher Journal, 68, 504–517.Google Scholar
Goodwin, A. P., Gilbert, J. K., Cho, S-J, Kearns, D. M.. (2014). Probing lexical representations: Simultaneous modeling of word and reader contributions to multidimensional lexical representations. Journal of Educational Psychology, 106, 448–468. doi:10.1037/a0034754.CrossRefGoogle Scholar
Graves, M. F.. (2007). Vocabulary instruction in the middle grades. Voices from the Middle, 15, 13–19.CrossRefGoogle Scholar
Harrell, F. (2015). Harrell miscellaneous. R package version 3.15-0..Google Scholar
Jak, S., Oort, F. J., Dolan, C. V.. (2013). A test for cluster bias: Detecting violations of measurement invariance across clusters in multilevel data. Structural Equation Modeling: A Multidisciplinary Journal, 20, 265–282. doi:10.1080/10705511.2013.769392.CrossRefGoogle Scholar
Kamata, A.. (2001). Item analysis by the hierarchical generalized linear model. Journal of Educational Measurement, 38, 79–93. doi:10.1111/j.1745-3984.2001.tb01117.x.CrossRefGoogle Scholar
Kieffer, M. J., Lesaux, N. K.. (2012). Knowledge of words, knowledge about words: Dimensions of vocabulary in first and second language learners in sixth grade. Reading and Writing, 25, 347–373. doi:10.1007/s11145-010-9272-9.CrossRefGoogle Scholar
Lesaux, N. K., Kieffer, M. J., Kelley, J. G., Russ Harris, J.. (2014). Effects of academic vocabulary instruction for linguistically diverse adolescents: Evidence from a randomized field trial. American Educational Research Journal, 51, 1159–1194. doi:10.3102/0002831214532165.CrossRefGoogle Scholar
Markson, L., Bloom, P.. (1997). Evidence against a dedicated system for vocabulary learning in Children. Nature, 385, 813–815. doi:10.1038/385813a0.CrossRefGoogle ScholarPubMed
Meade, A. W., Lautenschlager, G. J., Hecht, J. E.. (2005). Establishing measurement equivalence and invariance in longitudinal data with item response theory. International Journal of Testing, 5, 279–300. doi:10.1207/s15327574ijt0503_6.CrossRefGoogle Scholar
Milanzi, E., Molenberghs, G., Alonso, A., Verbeke, G., De Boeck, P.. (2015). Reliability measures in item response theory: Manifest versus latent correlation functions. British Journal of Mathematical and Statistical Psychology, 68, 43–64. doi:10.1111/bmsp.12033.CrossRefGoogle ScholarPubMed
Molenberghs, G., Verbeke, G.De Boeck, P., Wilson, M.. (2004). An introduction to generalized nonlinear mixed models. Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer 111–153. doi:10.1007/978-1-4757-3990-9_4.CrossRefGoogle Scholar
Muthén, B. O., Asparouhov, T., van der Linden, W. J., & Hambleton, R. K.. (2013). Item response modeling in Mplus: A multi-dimensional, multi-level, and multi-time point example. Handbook of item response theory, models, statistical tools, and applications. Boca Raton, FL: Chapman & Hall.Google Scholar
Nagy, W. E., Anderson, R. C., & Herman, P. A.. (1987). Learning word meanings from context during normal reading. American Educational Research Journal, 24, 237–270. doi:10.3102/00028312024002237.CrossRefGoogle Scholar
National Governors Association Center for Best Practices and Council of Chief State School. (2010). The common core standards: English language arts. Washington, D.C.: National Governors Association Center for Best Practices, Council of Chief State School Officers..Google Scholar
Ouellette, G. P.. (2006). What’s meaning got to do with it: The role of vocabulary in word reading and reading comprehension. Journal of Educational Psychology, 98, 554–566. doi:10.1037/0022-0663.98.3.554.CrossRefGoogle Scholar
Pearson, P. D., Hiebert, E. H., Kamil, M. L.. (2007). Vocabulary assessment: What we know and what we need to learn. Reading Research Quarterly, 42, 282–296. doi:10.1598/RRQ.42.2.4.CrossRefGoogle Scholar
Perfetti, C. A.. (2007). Reading ability: Lexical quality to comprehension. Scientific Studies of Reading, 11, 357–383. doi:10.1080/10888430701530730.CrossRefGoogle Scholar
R Development Core Team (2009). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-project.org.Google Scholar
Rijmen, F., Tuerlinckx, F., De Boeck, P., Kuppens, P.. (2003). A nonlinear mixed model framework for item response theory. Psychological Methods, 8, 185–205. doi:10.1037/1082-989X.8.2.185.CrossRefGoogle ScholarPubMed
Schreuder, R., Baayan, R. H.,Feldman, L.. (1995). Modeling morphological processing. Morphological aspects of language processing. Hillsdale, NJ: Lawrence Erlbaum 131–154.Google Scholar
Schwarz, G.. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464. doi:10.1214/aos/1176344136.CrossRefGoogle Scholar
Sinharay, S., Johnson, M. S., Williamson, D. M.. (2003). Calibrating item families and summarizing the results using family expected response functions. Journal of Educational and Behavioral Statistics, 28, 295–313. doi:10.3102/10769986028004295.CrossRefGoogle Scholar
Skrondal, A., Rabe-Hesketh, S.Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models 2004 Boca Raton, FL: Chapman & Halldoi:10.1201/9780203489437.CrossRefGoogle Scholar
Spiegelhalter, D. J., Thomas, A., Best, N. G., (2003). WinBUGS (Version, 1.4.)[Computer Program]. Cambridge, UK: MRC Biostatistics Unit, Institude of Public Health.Google Scholar
Stanovich, K. E.. (1986). Matthew effects in reading: Some consequences of individual differences in the acquisition of literacy. Reading Research Quarterly, 21, 360–107. doi:10.1598/RRQ.21.4.1.CrossRefGoogle Scholar
Tannenbaum, K. R., Torgesen, J. K., Wagner, R. K.. (2006). Relationships between word knowledge and reading comprehension in third-grade children. Scientific Studies of Reading, 10, 381–398. doi:10.1207/s1532799xssr1004_3.CrossRefGoogle Scholar
Zeno, S. M., Ivens, S. H., Millard, R. T., Duvvuri, R.The educator’s word frequency guide 1995 New York: Touchstone Applied Science Associates.Google Scholar