Traditional large-scale educational data are typically static and updated periodically, making it difficult to capture the dynamic changes in real time. However, recent technological advancements allow online exam platforms to collect students’ response data in real time. While item response theory (IRT) estimation methods are widely recognized for their accuracy, they are primarily designed for offline environments. When real-time data continuously arrives and online parameter estimation is required, these methods become computationally impractical. To address this challenge, we propose a recursive stochastic algorithm, i.e., truncated average stochastic Newton algorithm (TASNA), for the efficient online parameter estimation within the IRT framework. This algorithm significantly improves computational efficiency compared to the expectation–maximization (EM) algorithm implemented in the mirt package in R. The algorithm offers a powerful alternative to the traditional offline EM method. Furthermore, we investigate the asymptotic properties of the algorithm, proving its almost sure convergence and asymptotic normality. Numerical experiments using both simulated and real data demonstrate the practicality of the proposed method.

Various item selection algorithms have been proposed for cognitive diagnostic computerized adaptive testing (CD-CAT), with the goal of efficiently diagnosing examinees’ strengths and weaknesses. However, these algorithms often come with significant computational costs, which can hinder their practical implementation. A likelihood-based profile shrinkage (LBPS) algorithm is proposed to simplify the item selection process and reduce the computational cost in CD-CAT. Our simulation results indicate that incorporating LBPS into existing item selection methods yields substantial computational efficiency gains, with greater reductions in computation time as the number of attributes and test length increase. Additionally, LBPS maintains estimation accuracy at both the attribute and pattern levels. These findings suggest that LBPS is a scalable and effective solution for the item selection of CD-CAT in complex scenarios.

In item response theory (IRT), the conventional latent trait scale ($\theta $) is inherently arbitrary, lacking a fixed unit or origin and often tied to specific population distributional assumptions (e.g., standard normal). This limits the direct comparability and interpretability of scores across different tests, populations, or model estimation methods. This article introduces the “bit scale,” a novel metric transformation for unidimensional IRT scores derived from fundamental principles of information theory, specifically surprisal and entropy. Bit scores are anchored to the properties of the test items rather than the test-taker population. This item-based anchoring ensures the scale’s invariance to population assumptions and provides a consistent metric for comparing latent trait levels. We illustrate the utility of the bit scale through empirical examples: demonstrating consistent scoring when fitting models with different $\theta $ scale assumptions, and using anchor items to directly link scores from different test administrations. A simulation study confirms the desirable statistical properties (low bias and accurate standard errors) of Maximum Likelihood-estimated bit scores and their robustness to extreme scores. The bit scale offers a theoretically grounded, interpretable, and comparable metric for reporting and analyzing IRT-based assessment results. Software implementations in R (bitscale) and Python (IRTorch) are available and practical implications are discussed.

Over the past two decades, there has been growing interest in analyzing the effects of educational programs on outcomes using process data from computer-based testing and learning environments. However, most analyses focus on final outcomes at the end of a test or session, overlooking their functional nature over time and neglecting causal mechanisms. To address this gap, this article proposes a novel causal mediation framework for identifying and estimating functional natural direct effects, functional natural indirect effects, and functional total effects, along with their subgroup effects. We define these effects using potential outcomes and provide nonparametric identification strategies depending on whether post-treatment covariates are present or not. We then develop estimation methods using generalized additive models, a flexible and robust tool for analyzing functional data. Through a simulation study, we assess the finite-sample performance of the proposed approach by comparing it to parametric regression methods. We also demonstrate our approach by examining the effects of extended time accommodations on two functional outcomes using process data from the National Assessment of Educational Progress. Our mediation approach with functional outcomes effectively captures dynamic causal mechanisms underlying the program’s effects and pinpoints when and for whom each effect manifests throughout the testing period.

The association between a continuous and an ordinal variable is commonly modeled through the polyserial correlation model. However, this model, which is based on a partially-latent normality assumption, may be misspecified in practice, due to, for example (but not limited to), outliers or careless responses. The typically used maximum likelihood (ML) estimator is highly susceptible to such misspecification: One single observation not generated by partially-latent normality can suffice to produce arbitrarily poor estimates. As a remedy, we propose a novel estimator of the polyserial correlation model designed to be robust against the adverse effects of observations discrepant to that model. The estimator leverages density power divergence estimation to achieve robustness by implicitly downweighting such observations; the ensuing weights constitute a useful tool for pinpointing potential sources of model misspecification. The proposed estimator generalizes ML and is consistent as well as asymptotically Gaussian. As price for robustness, some efficiency must be sacrificed, but substantial robustness can be gained while maintaining more than 98% of ML efficiency. We demonstrate our estimator’s robustness and practical usefulness in simulation experiments and an empirical application in personality psychology where our estimator helps identify outliers. Finally, the proposed methodology is implemented in free open-source software.

In this paper, we generalize the multidimensional discrimination and difficulty parameters in the multidimensional two-parameter logistic model to account for nonidentity latent covariances and negatively keyed items. We apply Reckase’s maximum discrimination point method to define them in an arbitrary algebraic basis. Then, we define that basis to be a geometrical representation of the measured construct. This results in three different versions of the parameters: the original one, based on the item parameters solely; one that incorporates the covariance structure of the latent space; and one that uses the correlation structure instead. Importantly, we find that the items should be properly represented in a test space, distinct from the latent space. We also provide a procedure for the geometrical representation of the items in the test space and apply our results to examples from the literature to get a more accurate representation of the measurement properties of the items. We recommend using the covariance structure version for describing the properties of the parameters and the correlation structure version for graphical representation. Finally, we discuss the implications of this generalization for other multidimensional item response theory models and the parallels of our results in common factor model theory.

Behavioral models are instrumental for studying human cognition, yet many inferences derived from such models fail to generalize. We argue that this is driven in part by the increasing complexity of behavioral models, where non-linearities and discontinuities create dynamic parameter interactions that limit the generalizability of inferences across different contexts, experiments, and datasets. We first demonstrate the problems that arise from parameter dependency. We then propose a new methodological framework for understanding the generalizability of behavioral modeling results using multivariate sampling distributions for the model parameters. We derive and validate novel sampling distributions for complex non-linear behavioral models by transforming the mimicry between different parameter values into the chances of one set of parameters being inferred from data generated by another set of parameters. Our approach is computationally scalable to evaluate how model estimates change across the parameter space and different experiments, which can limit the generalizability of experimental results. We then apply our approach to current behavioral models, revealing new theoretical insights. Using our approach, we reinterpret results from recent modeling work in decision-making and category learning. We conclude by discussing the implications of our proposed framework for building stronger, more generalizable psychological research and theory through behavioral modeling.

Differential item functioning (DIF) screening has long been suggested to ensure assessment fairness. Traditional DIF methods typically focus on the main effects of demographic variables on item parameters, overlooking the interactions among multiple identities. Drawing on the intersectionality framework, we define intersectional DIF as deviations in item parameters that arise from the interactions among demographic variables beyond their main effects and propose a novel item response theory (IRT) approach for detecting intersectional DIF. Under our framework, fixed effects are used to account for traditional DIF, while random item effects are introduced to capture intersectional DIF. We further introduce the concept of intersectional impact, which refers to interaction effects on group-level mean ability. Depending on which item parameters are affected and whether intersectional impact is considered, we propose four models, which aim to detect intersectional uniform DIF (UDIF), intersectional UDIF with intersectional impact, intersectional non-uniform DIF (NUDIF), and intersectional NUDIF with intersectional impact, respectively. For efficient model estimation, a regularized Gaussian variational expectation-maximization algorithm is developed. Simulation studies demonstrate that our methods can effectively detect intersectional UDIF, although their detection of intersectional NUDIF is more limited.

Test fairness is a major concern in psychometric and educational research. A typical approach for ensuring test fairness is through differential item functioning (DIF) analysis. DIF arises when a test item functions differently across subgroups that are typically defined by the respondents’ demographic characteristics. Most of the existing research focuses on the statistical detection of DIF, yet less attention has been given to reducing or eliminating DIF. Simultaneously, the use of computer-based assessments has become increasingly popular. The data obtained from respondents interacting with an item are recorded in computer log files and are referred to as process data. In this article, we propose a novel method within the framework of generalized linear models that leverages process data to reduce and understand DIF. Specifically, we construct a nuisance trait surrogate with the features extracted from process data. With the constructed nuisance trait, we introduce a new scoring rule that incorporates respondents’ behaviors captured through process data on top of the target latent trait. We demonstrate the efficiency of our approach through extensive simulation experiments and an application to 13 Problem Solving in Technology-Rich Environments items from the 2012 Programme for the International Assessment of Adult Competencies assessment.

Estimation in exploratory factor analysis often yields estimates on the boundary of the parameter space. Such occurrences, called Heywood cases, are characterized by non-positive variance estimates and can cause numerical instability, convergence failures, and misleading inferences. We derive sufficient conditions on the model and a penalty to the log-likelihood function that guarantee the existence of maximum penalized likelihood estimates in the interior of the parameter space, and that the corresponding estimators possess desirable asymptotic properties expected by the maximum likelihood estimator, namely, consistency and asymptotic normality. Consistency and asymptotic normality follow when penalization is soft enough, in a way that adapts to the information accumulation about the model parameters. We formally show, for the first time, that the penalties of Akaike (1987, Psychometrika, 52, 317–332) and Hirose et al. (2011, Journal of Data Science, 9, 243–259) to the log-likelihood of the normal linear factor model satisfy the conditions for existence, and, hence, deal with Heywood cases. Their vanilla versions, though, can result in questionable finite-sample properties in estimation, inference, and model selection. Our maximum softly-penalized likelihood (MSPL) framework ensures that the resulting estimation and inference procedures are asymptotically optimal. Through comprehensive simulation studies and real data analyses, we illustrate the desirable finite-sample properties of the MSPL estimators.

Social science researchers are generally accustomed to treating ordinal variables as though they are continuous. In this article, we consider how identification constraints in ordinal factor analysis can mimic the treatment of ordinal variables as continuous. We specifically describe model constraints that lead to latent variable predictions equaling the average of ordinal variables. This result leads us to propose minimal identification constraints, which we call integer constraints, that place the latent variables on the scale of the observed, integer-coded ordinal variables. The integer constraints lead to intuitive model parameterizations because researchers are already accustomed to thinking about ordinal variables as though they are continuous. We provide a proof that our proposed integer constraints are indeed minimal identification constraints, as well as illustrations of how integer constraints work with real data. We also provide simulation results indicating that integer constraints are similar to other identification constraints in terms of estimation convergence and admissibility.

This work establishes a new identifiability theory for a cornerstone of various cognitive diagnostic models (CDMs) popular in psychometrics: the Q-matrix. The key idea is a novel tensor-unfolding proof strategy. Representing the joint distribution of J categorical responses as a J-way tensor, we strategically unfold the tensor into matrices in multiple ways and use their rank properties to identify the unknown Q-matrix. This approach departs fundamentally from all prior identifiability analyses in CDMs. Our proof is constructive, elucidating a population-level procedure to exactly recover the Q-matrix within a parameter space where each latent attribute is measured by at least two “pure” items that solely measure this attribute. The theory has several desirable features: it can constructively identify both the Q-matrix and the number of latent attributes; it applies to broad classes of linear and nonlinear CDMs with main or all saturated effects of attributes; and it accommodates polytomous responses, extending beyond classical binary response settings. The new identifiability result unifies and strengthens identifiability guarantees across diverse CDMs. It provides rigorous theoretical foundations and indicates a future pathway toward using tensor unfolding for practical Q-matrix estimation.

Deep generative modeling is a powerful framework in modern machine learning, renowned for its ability to use latent representations to predict and generate complex high-dimensional data. Its advantages have also been recognized in psychometrics. In this article, we substantially extend the deep cognitive diagnostic models (DeepCDMs) in Gu (Psychometrika, 89:118–150, 2024) to challenging exploratory scenarios with deeper structures and all $\mathbf {Q}$-matrices unknown. The exploratory DeepCDMs can be viewed as an adaptation of deep generative models (DGMs) toward diagnostic purposes. Compared to classic DGMs, exploratory DeepCDMs enjoy critical advantages, including identifiability, interpretability, parsimony, and sparsity, which are all necessary for diagnostic modeling. We propose a novel layer-wise expectation–maximization (EM) algorithm for parameter estimation, incorporating layer-wise nonlinear spectral initialization and $L_1$ penalty terms to promote sparsity. From a parameter estimation standpoint, this algorithm reduces sensitivity to initial values and mitigates estimation bias that commonly affects classical approaches for deep latent variable models. Meanwhile, from an algorithmic perspective, our method presents an original layer-wise EM framework, inspired by modular training in DGMs but uniquely designed for the structural and interpretability demands of diagnostic modeling. Extensive simulation studies and real data applications illustrate the effectiveness and efficiency of the proposed method.

Factor analysis models explain dependence among observed variables by a smaller number of unobserved factors. A main challenge in confirmatory factor analysis is determining whether the factor loading matrix is identifiable from the observed covariance matrix. The factor loading matrix captures the linear effects of the factors and, if unrestricted, can only be identified up to an orthogonal transformation of the factors. However, in many applications, the factor loadings exhibit an interesting sparsity pattern that may lead to identifiability up to column signs. We study this phenomenon by connecting sparse confirmatory factor analysis models to bipartite graphs and providing sufficient graphical conditions for identifiability of the factor loading matrix up to column signs. In contrast to previous work, our main contribution, the matching criterion, exploits sparsity by operating locally on the graph structure, thereby improving existing conditions. Our criterion is efficiently decidable in time that is polynomial in the size of the graph, when restricting the search steps to sets of bounded size.

The multiple-choice (MC) item format has been adapted to the cognitive diagnosis (CD) framework. Early approaches simply dichotomized the responses and analyzed them with a CD model for binary responses. Obviously, this strategy cannot exploit the additional diagnostic information provided by MC items. De la Torre’s (2009, Applied Psychological Measurement, 33, 163–183) MC-DINA model was the first for the explicit analysis of MC items. However, the q-vectors of the distractors were constrained to be nested within the key and each other, which imposes serious restrictions on item development. Relaxing the nestedness-constraint, comes at a price. First, distractors may become redundant: they do not improve the classification of examinees beyond the response options already available for an item. Second, undesirable diagnostic ambiguity can arise from distractors that are equally likely to be chosen by an examinee, but have distinct attribute profiles pointing at different diagnostic classifications. In this article, two criteria, plausible and proper, are developed for detecting these problematic cases. Two theorems that permit for the detection and amendment of improper and implausible items are presented. An R function serving this purpose is used in several practical applications. Results of simulation studies and real data analysis are also reported.

Identifying causal directions among variables via data-driven approaches is a research hotspot. Researchers now focus on detecting causal direction heterogeneity among multiple variables (variables more than two) when covariates cause such heterogeneity. This study combines the structural equation likelihood function (SELF) method with a recursive partitioning method to achieve an interpretable model of multivariate causal direction heterogeneity in multivariable settings. Through simulation, we compared the performance of the SELF-Tree model in terms of the identification about heterogeneous causal direction under different conditions. Using a public drug consumption dataset, we demonstrated its real data application. The SELF-Tree model offers researchers a new way to understand variable causal direction heterogeneity.

Cultural consensus theory (CCT) leverages shared knowledge between individuals to optimally aggregate answers to questions for which the underlying truth is unknown. Existing CCT models have predominantly focused on unidimensional point truths using dichotomous, polytomous, or continuous response formats. However, certain domains, such as risk assessment or interpretation of verbal quantifiers, may require a consensus focused on intervals, capturing a range of relevant values. We introduce the interval consensus model (ICM), a novel extension of CCT designed to estimate consensus intervals from continuous bounded interval responses. We use a Bayesian hierarchical modeling approach to estimate latent consensus intervals. In a simulation study, we show that, under the conditions studied, the ICM performs better than using simple means and medians of the responses. We then apply the model to empirical judgments of verbal quantifiers.

Multiple response (MR) items—such as multiple true-false, multiple-select, and select-N items—are increasingly used in assessments to identify partial knowledge and differentiate latent abilities more accurately. Allowing multiple selections, MR items provide richer information and reduce guessing effects compared to single-answer multiple-choice items. However, traditional scoring methods (e.g., Dichotomous, Ripkey, Partial scoring) compress response combination (RC) data, losing valuable information and ignoring issues like local dependence and incompatibility across item types. To address these challenges, we introduce a novel psychometric model framework: the Multiple Response Model with Inter-option Local Dependencies (MRM-LD), and its simplified version, the Multiple Response Model (MRM). These models preserve RC data across MR item types, offering a more comprehensive understanding for MR assessment. Parameters for MRM-LD and MRM were estimated using Markov chain Monte Carlo algorithms in Stan and R. Empirical data from an eighth-grade physics test showed that MRM-LD and MRM outperform Graded Response Model and Nominal Response Model combined with three scoring methods, by retaining more test information, improving reliability and validity, and providing more detailed analysis of item characteristics. Simulation studies confirmed the proposed models perform robustly under various conditions, including small samples and few items, demonstrating their applicability across diverse testing scenarios.