Traditional large-scale educational data are typically static and updated periodically, making it difficult to capture the dynamic changes in real time. However, recent technological advancements allow online exam platforms to collect students’ response data in real time. While item response theory (IRT) estimation methods are widely recognized for their accuracy, they are primarily designed for offline environments. When real-time data continuously arrives and online parameter estimation is required, these methods become computationally impractical. To address this challenge, we propose a recursive stochastic algorithm, i.e., truncated average stochastic Newton algorithm (TASNA), for the efficient online parameter estimation within the IRT framework. This algorithm significantly improves computational efficiency compared to the expectation–maximization (EM) algorithm implemented in the mirt package in R. The algorithm offers a powerful alternative to the traditional offline EM method. Furthermore, we investigate the asymptotic properties of the algorithm, proving its almost sure convergence and asymptotic normality. Numerical experiments using both simulated and real data demonstrate the practicality of the proposed method.

Various item selection algorithms have been proposed for cognitive diagnostic computerized adaptive testing (CD-CAT), with the goal of efficiently diagnosing examinees’ strengths and weaknesses. However, these algorithms often come with significant computational costs, which can hinder their practical implementation. A likelihood-based profile shrinkage (LBPS) algorithm is proposed to simplify the item selection process and reduce the computational cost in CD-CAT. Our simulation results indicate that incorporating LBPS into existing item selection methods yields substantial computational efficiency gains, with greater reductions in computation time as the number of attributes and test length increase. Additionally, LBPS maintains estimation accuracy at both the attribute and pattern levels. These findings suggest that LBPS is a scalable and effective solution for the item selection of CD-CAT in complex scenarios.

In item response theory (IRT), the conventional latent trait scale ($\theta $) is inherently arbitrary, lacking a fixed unit or origin and often tied to specific population distributional assumptions (e.g., standard normal). This limits the direct comparability and interpretability of scores across different tests, populations, or model estimation methods. This article introduces the “bit scale,” a novel metric transformation for unidimensional IRT scores derived from fundamental principles of information theory, specifically surprisal and entropy. Bit scores are anchored to the properties of the test items rather than the test-taker population. This item-based anchoring ensures the scale’s invariance to population assumptions and provides a consistent metric for comparing latent trait levels. We illustrate the utility of the bit scale through empirical examples: demonstrating consistent scoring when fitting models with different $\theta $ scale assumptions, and using anchor items to directly link scores from different test administrations. A simulation study confirms the desirable statistical properties (low bias and accurate standard errors) of Maximum Likelihood-estimated bit scores and their robustness to extreme scores. The bit scale offers a theoretically grounded, interpretable, and comparable metric for reporting and analyzing IRT-based assessment results. Software implementations in R (bitscale) and Python (IRTorch) are available and practical implications are discussed.

Over the past two decades, there has been growing interest in analyzing the effects of educational programs on outcomes using process data from computer-based testing and learning environments. However, most analyses focus on final outcomes at the end of a test or session, overlooking their functional nature over time and neglecting causal mechanisms. To address this gap, this article proposes a novel causal mediation framework for identifying and estimating functional natural direct effects, functional natural indirect effects, and functional total effects, along with their subgroup effects. We define these effects using potential outcomes and provide nonparametric identification strategies depending on whether post-treatment covariates are present or not. We then develop estimation methods using generalized additive models, a flexible and robust tool for analyzing functional data. Through a simulation study, we assess the finite-sample performance of the proposed approach by comparing it to parametric regression methods. We also demonstrate our approach by examining the effects of extended time accommodations on two functional outcomes using process data from the National Assessment of Educational Progress. Our mediation approach with functional outcomes effectively captures dynamic causal mechanisms underlying the program’s effects and pinpoints when and for whom each effect manifests throughout the testing period.

Longitudinal mental health assessments in mobile health (mHealth) settings are useful for monitoring subjects’ mental health statuses but are often difficult to analyze because they generally appear on an ordinal scale and at unequal time intervals. In this article, we explore the use of Gaussian processes (GPs) and hierarchical modeling techniques to understand mental health trajectories based on repeated multi-item mHealth surveys on a Likert scale. We introduce the GP model for health trajectories, which is based on item response theory. In the study of trajectories, a subject’s longitudinal collection of mHealth responses can be thought of as a single high-dimensional observation. We show how the GP is flexible enough to capture trends in individual trajectories even with the challenges associated with high-dimensional data. We also demonstrate how basis splines can be used to effectively capture nonlinear trends in the mean function of the GP. The high-dimension and ordinal nature of the data often make sampling from the posterior distribution in a Bayesian setting too slow to be practical. We show that using a Hilbert approximation for the GP trajectories can facilitate efficient sampling. We apply these methods to a longitudinal study that monitored college students’ self-esteem.

The association between a continuous and an ordinal variable is commonly modeled through the polyserial correlation model. However, this model, which is based on a partially-latent normality assumption, may be misspecified in practice, due to, for example (but not limited to), outliers or careless responses. The typically used maximum likelihood (ML) estimator is highly susceptible to such misspecification: One single observation not generated by partially-latent normality can suffice to produce arbitrarily poor estimates. As a remedy, we propose a novel estimator of the polyserial correlation model designed to be robust against the adverse effects of observations discrepant to that model. The estimator leverages density power divergence estimation to achieve robustness by implicitly downweighting such observations; the ensuing weights constitute a useful tool for pinpointing potential sources of model misspecification. The proposed estimator generalizes ML and is consistent as well as asymptotically Gaussian. As price for robustness, some efficiency must be sacrificed, but substantial robustness can be gained while maintaining more than 98% of ML efficiency. We demonstrate our estimator’s robustness and practical usefulness in simulation experiments and an empirical application in personality psychology where our estimator helps identify outliers. Finally, the proposed methodology is implemented in free open-source software.

In this paper, we generalize the multidimensional discrimination and difficulty parameters in the multidimensional two-parameter logistic model to account for nonidentity latent covariances and negatively keyed items. We apply Reckase’s maximum discrimination point method to define them in an arbitrary algebraic basis. Then, we define that basis to be a geometrical representation of the measured construct. This results in three different versions of the parameters: the original one, based on the item parameters solely; one that incorporates the covariance structure of the latent space; and one that uses the correlation structure instead. Importantly, we find that the items should be properly represented in a test space, distinct from the latent space. We also provide a procedure for the geometrical representation of the items in the test space and apply our results to examples from the literature to get a more accurate representation of the measurement properties of the items. We recommend using the covariance structure version for describing the properties of the parameters and the correlation structure version for graphical representation. Finally, we discuss the implications of this generalization for other multidimensional item response theory models and the parallels of our results in common factor model theory.

Psychological research has long centered around questionnaire assessments, but now digital devices, especially smartphones, enable the collection of real-world behavioral data through mobile sensing. While this data collection method offers unique opportunities, it also introduces new methodological challenges, as mobile-sensing data are highly complex and high in dimensionality (i.e., timestamped events with millisecond resolution), requiring advanced preprocessing to derive psychologically meaningful variables. This article highlights these challenges by reviewing the current state of data preprocessing based on app usage logs from smartphones. Afterward, it presents three preprocessing cases that vary in complexity across the dimensions of data enrichment—which involves adding context to raw data by integrating information from external and internal sources (including ecological momentary assessments)—and data aggregation—which entails summarizing data in different ways, from basic descriptive statistics to sophisticated machine-learning models. For each case, potential pitfalls are identified, and extensions are discussed to refine our preprocessing pipelines and accommodate different data types and research questions. By outlining these preprocessing strategies, this manuscript demonstrates the rich potential of mobile-sensing data for extracting nuanced behavioral variables beyond simple person-level summaries and aims to inspire the development of more advanced research questions based on sensing data.

Behavioral models are instrumental for studying human cognition, yet many inferences derived from such models fail to generalize. We argue that this is driven in part by the increasing complexity of behavioral models, where non-linearities and discontinuities create dynamic parameter interactions that limit the generalizability of inferences across different contexts, experiments, and datasets. We first demonstrate the problems that arise from parameter dependency. We then propose a new methodological framework for understanding the generalizability of behavioral modeling results using multivariate sampling distributions for the model parameters. We derive and validate novel sampling distributions for complex non-linear behavioral models by transforming the mimicry between different parameter values into the chances of one set of parameters being inferred from data generated by another set of parameters. Our approach is computationally scalable to evaluate how model estimates change across the parameter space and different experiments, which can limit the generalizability of experimental results. We then apply our approach to current behavioral models, revealing new theoretical insights. Using our approach, we reinterpret results from recent modeling work in decision-making and category learning. We conclude by discussing the implications of our proposed framework for building stronger, more generalizable psychological research and theory through behavioral modeling.

Differential item functioning (DIF) screening has long been suggested to ensure assessment fairness. Traditional DIF methods typically focus on the main effects of demographic variables on item parameters, overlooking the interactions among multiple identities. Drawing on the intersectionality framework, we define intersectional DIF as deviations in item parameters that arise from the interactions among demographic variables beyond their main effects and propose a novel item response theory (IRT) approach for detecting intersectional DIF. Under our framework, fixed effects are used to account for traditional DIF, while random item effects are introduced to capture intersectional DIF. We further introduce the concept of intersectional impact, which refers to interaction effects on group-level mean ability. Depending on which item parameters are affected and whether intersectional impact is considered, we propose four models, which aim to detect intersectional uniform DIF (UDIF), intersectional UDIF with intersectional impact, intersectional non-uniform DIF (NUDIF), and intersectional NUDIF with intersectional impact, respectively. For efficient model estimation, a regularized Gaussian variational expectation-maximization algorithm is developed. Simulation studies demonstrate that our methods can effectively detect intersectional UDIF, although their detection of intersectional NUDIF is more limited.

In psychometric sciences, such as social or behavioral sciences, and, similarly, in medical sciences, it is increasingly common to deal with longitudinal data organized as high-dimensional multidimensional arrays, also known as tensors. Within this framework, the time-continuous property of longitudinal data often implies a smooth functional structure on one of the tensor modes. To help researchers investigate such data, we introduce a new tensor decomposition approach based on the PARAFAC decomposition. Our approach allows researchers to represent a high-dimensional functional tensor as a low-dimensional set of functions and feature matrices. Furthermore, to capture the underlying randomness of the statistical setting more efficiently, we introduce a probabilistic latent model in the decomposition. A covariance-based block-relaxation algorithm is derived to obtain estimates of model parameters. Thanks to the covariance formulation of the solving procedure and thanks to the probabilistic modeling, the method can be used in sparse and irregular sampling schemes, making it applicable in numerous settings. Our approach is applied in the psychometric setting to help characterize multiple neurocognitive scores observed over time in the Alzheimer’s Disease Neuroimaging Initiative study. Finally, intensive simulations show a notable advantage of our method in reconstructing tensors.

Test fairness is a major concern in psychometric and educational research. A typical approach for ensuring test fairness is through differential item functioning (DIF) analysis. DIF arises when a test item functions differently across subgroups that are typically defined by the respondents’ demographic characteristics. Most of the existing research focuses on the statistical detection of DIF, yet less attention has been given to reducing or eliminating DIF. Simultaneously, the use of computer-based assessments has become increasingly popular. The data obtained from respondents interacting with an item are recorded in computer log files and are referred to as process data. In this article, we propose a novel method within the framework of generalized linear models that leverages process data to reduce and understand DIF. Specifically, we construct a nuisance trait surrogate with the features extracted from process data. With the constructed nuisance trait, we introduce a new scoring rule that incorporates respondents’ behaviors captured through process data on top of the target latent trait. We demonstrate the efficiency of our approach through extensive simulation experiments and an application to 13 Problem Solving in Technology-Rich Environments items from the 2012 Programme for the International Assessment of Adult Competencies assessment.

Understanding spatial navigation and memory formation is critical to exploring how humans learn and adapt in complex environments. To investigate these processes, we conducted an experiment using the Minecraft Memory and Navigation Task, collecting detailed three-dimensional (3D) path data in a virtual open-world setting. Statistically, we developed a novel methodology to convert complex high-dimensional 3D movement data into functional representations, enabling standardized comparisons and analyses across participants and environments. We applied techniques such as functional clustering and regression to identify navigation patterns and their relationships with cognitive map development and memory retention. Our analysis uncovered two significant insights: first, participants who adopted moderately exploratory behaviors during training demonstrated superior retention of object locations; second, inefficient navigation strategies were strongly linked to poorer spatial memory and navigation performance. These findings highlight the effectiveness of our methodology in advancing the study of navigation behaviors and cognitive processes in dynamic 3D environments.

Estimation in exploratory factor analysis often yields estimates on the boundary of the parameter space. Such occurrences, called Heywood cases, are characterized by non-positive variance estimates and can cause numerical instability, convergence failures, and misleading inferences. We derive sufficient conditions on the model and a penalty to the log-likelihood function that guarantee the existence of maximum penalized likelihood estimates in the interior of the parameter space, and that the corresponding estimators possess desirable asymptotic properties expected by the maximum likelihood estimator, namely, consistency and asymptotic normality. Consistency and asymptotic normality follow when penalization is soft enough, in a way that adapts to the information accumulation about the model parameters. We formally show, for the first time, that the penalties of Akaike (1987, Psychometrika, 52, 317–332) and Hirose et al. (2011, Journal of Data Science, 9, 243–259) to the log-likelihood of the normal linear factor model satisfy the conditions for existence, and, hence, deal with Heywood cases. Their vanilla versions, though, can result in questionable finite-sample properties in estimation, inference, and model selection. Our maximum softly-penalized likelihood (MSPL) framework ensures that the resulting estimation and inference procedures are asymptotically optimal. Through comprehensive simulation studies and real data analyses, we illustrate the desirable finite-sample properties of the MSPL estimators.

Social science researchers are generally accustomed to treating ordinal variables as though they are continuous. In this article, we consider how identification constraints in ordinal factor analysis can mimic the treatment of ordinal variables as continuous. We specifically describe model constraints that lead to latent variable predictions equaling the average of ordinal variables. This result leads us to propose minimal identification constraints, which we call integer constraints, that place the latent variables on the scale of the observed, integer-coded ordinal variables. The integer constraints lead to intuitive model parameterizations because researchers are already accustomed to thinking about ordinal variables as though they are continuous. We provide a proof that our proposed integer constraints are indeed minimal identification constraints, as well as illustrations of how integer constraints work with real data. We also provide simulation results indicating that integer constraints are similar to other identification constraints in terms of estimation convergence and admissibility.

This work establishes a new identifiability theory for a cornerstone of various cognitive diagnostic models (CDMs) popular in psychometrics: the Q-matrix. The key idea is a novel tensor-unfolding proof strategy. Representing the joint distribution of J categorical responses as a J-way tensor, we strategically unfold the tensor into matrices in multiple ways and use their rank properties to identify the unknown Q-matrix. This approach departs fundamentally from all prior identifiability analyses in CDMs. Our proof is constructive, elucidating a population-level procedure to exactly recover the Q-matrix within a parameter space where each latent attribute is measured by at least two “pure” items that solely measure this attribute. The theory has several desirable features: it can constructively identify both the Q-matrix and the number of latent attributes; it applies to broad classes of linear and nonlinear CDMs with main or all saturated effects of attributes; and it accommodates polytomous responses, extending beyond classical binary response settings. The new identifiability result unifies and strengthens identifiability guarantees across diverse CDMs. It provides rigorous theoretical foundations and indicates a future pathway toward using tensor unfolding for practical Q-matrix estimation.