Cognitive diagnosis models (CDMs) are an important psychometric framework for classifying students in terms of attribute and/or skill mastery. The $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\varvec{Q}$$\end{document} matrix, which specifies the required attributes for each item, is central to implementing CDMs. The general unavailability of $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\varvec{Q}$$\end{document} for most content areas and datasets poses a barrier to widespread applications of CDMs, and recent research accordingly developed fully exploratory methods to estimate Q. However, current methods do not always offer clear interpretations of the uncovered skills and existing exploratory methods do not use expert knowledge to estimate Q. We consider Bayesian estimation of $\mathit{Q}$\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\varvec{Q}$$\end{document} using a prior based upon expert knowledge using a fully Bayesian formulation for a general diagnostic model. The developed method can be used to validate which of the underlying attributes are predicted by experts and to identify residual attributes that remain unexplained by expert knowledge. We report Monte Carlo evidence about the accuracy of selecting active expert-predictors and present an application using Tatsuoka’s fraction-subtraction dataset.

Fully Bayesian estimation of item response theory models with logistic link functions suffers from low computational efficiency due to posterior density functions that do not have known forms. To improve algorithmic computational efficiency, this paper proposes a Bayesian estimation method by adopting a new data-augmentation strategy in uni- and multidimensional IRT models. The strategy is based on the Pólya–Gamma family of distributions which provides a closed-form posterior distribution for logistic-based models. In this paper, an overview of Pólya–Gamma distributions is described within a logistic regression framework. In addition, we provide details about deriving conditional distributions of IRT, incorporating Pólya–Gamma distributions into the conditional distributions for Bayesian samplers’ construction, and random drawing from the samplers such that a faster convergence can be achieved. Simulation studies and applications to real datasets were conducted to demonstrate the efficiency and utility of the proposed method.

The compatibility of computerized adaptive testing (CAT) with response revision has been a topic of debate in psychometrics for many years. The problem is to provide test takers opportunities to change their answers during the test, while discouraging deceptive strategies from their side and preserving the statistical efficiency of the traditional CAT. The estimating approach proposed in Wang et al. (Stat Sin 27(4):1987–2010, 2017), based on the nominal response model, allows test takers to provide more than one answer to each item during the test, which they all contribute to the interim and final ability estimation. This approach is here reformulated, extended to incorporate a larger class of polytomous and dichotomous item response theory models, and investigated with simulation studies under different test-taking strategies.

Knowledge space theory (KST) structures are introduced within item response theory (IRT) as a possible way to model local dependence between items. The aim of this paper is threefold: firstly, to generalize the usual characterization of local independence without introducing new parameters; secondly, to merge the information provided by the IRT and KST perspectives; and thirdly, to contribute to the literature that bridges continuous and discrete theories of assessment. In detail, connections are established between the KST simple learning model (SLM) and the IRT General Graded Response Model, and between the KST Basic Local Independence Model and IRT models in general. As a consequence, local independence is generalized to account for the existence of prerequisite relations between the items, IRT models become a subset of KST models, IRT likelihood functions can be generalized to broader families, and the issues of local dependence and dimensionality are partially disentangled. Models are discussed for both dichotomous and polytomous items and conclusions are drawn on their interpretation. Considerations on possible consequences in terms of model identifiability and estimation procedures are also provided.

Researchers often use regressions with a logarithmic link function to evaluate the effects of a treatment on a count variable. In order to judge the average effectiveness of the treatment on the original count scale, they compute average treatment effects, which are defined as the average difference between the expected outcomes under treatment and under control. Current practice is to evaluate the expected differences at every observation and use the sample mean of these differences as a point estimate of the average effect. The standard error for this average effect estimate is based on the implicit assumption that covariate values are fixed, i.e., do not vary across different samples. In this paper, we present a new way of analytically computing average effects based on regressions with log link using stochastic covariates and develop new formulas to obtain standard errors for the average effect. In a simulation study, we evaluate the statistical performance of our new estimator and compare it with the traditional approach. Our findings suggest that the new approach gives unbiased effect estimates and standard errors and outperforms the traditional approach when strong interaction and/or a skewed covariate is present.

The inverse probability of treatment weighted (IPTW) estimator can be used to make causal inferences under two assumptions: (1) no unobserved confounders (ignorability) and (2) positive probability of treatment and of control at every level of the confounders (positivity), but is vulnerable to bias if by chance, the proportion of the sample assigned to treatment, or proportion of control, is zero at certain levels of the confounders. We propose to deal with this sampling zero problem, also known as practical violation of the positivity assumption, in a setting where the observed confounder is cluster identity, i.e., treatment assignment is ignorable within clusters. Specifically, based on a random coefficient model assumed for the potential outcome, we augment the IPTW estimating function with the estimated potential outcomes of treatment (or of control) for clusters that have no observation of treatment (or control). If the cluster-specific potential outcomes are estimated correctly, the augmented estimating function can be shown to converge in expectation to zero and therefore yield consistent causal estimates. The proposed method can be implemented in the existing software, and it performs well in simulated data as well as with real-world data from a teacher preparation evaluation study.

Cognitive diagnosis models (CDMs) are useful statistical tools in cognitive diagnosis assessment. However, as many other latent variable models, the CDMs often suffer from the non-identifiability issue. This work gives the sufficient and necessary condition for identifiability of the basic DINA model, which not only addresses the open problem in Xu and Zhang (Psychometrika 81:625–649, 2016) on the minimal requirement for identifiability, but also sheds light on the study of more general CDMs, which often cover DINA as a submodel. Moreover, we show the identifiability condition ensures the consistent estimation of the model parameters. From a practical perspective, the identifiability condition only depends on the Q-matrix structure and is easy to verify, which would provide a guideline for designing statistically valid and estimable cognitive diagnosis tests.

In educational and psychological measurement, researchers and/or practitioners are often interested in examining whether the ability of an examinee is the same over two sets of items. Such problems can arise in measurement of change, detection of cheating on unproctored tests, erasure analysis, detection of item preknowledge, etc. Traditional frequentist approaches that are used in such problems include the Wald test, the likelihood ratio test, and the score test (e.g., Fischer, Appl Psychol Meas 27:3–26, 2003; Finkelman, Weiss, & Kim-Kang, Appl Psychol Meas 34:238–254, 2010; Glas & Dagohoy, Psychometrika 72:159–180, 2007; Guo & Drasgow, Int J Sel Assess 18:351–364, 2010; Klauer & Rettig, Br J Math Stat Psychol 43:193–206, 1990; Sinharay, J Educ Behav Stat 42:46–68, 2017). This paper shows that approaches based on higher-order asymptotics (e.g., Barndorff-Nielsen & Cox, Inference and asymptotics. Springer, London, 1994; Ghosh, Higher order asymptotics. Institute of Mathematical Statistics, Hayward, 1994) can also be used to test for the equality of the examinee ability over two sets of items. The modified signed likelihood ratio test (e.g., Barndorff-Nielsen, Biometrika 73:307–322, 1986) and the Lugannani–Rice approximation (Lugannani & Rice, Adv Appl Prob 12:475–490, 1980), both of which are based on higher-order asymptotics, are shown to provide some improvement over the traditional frequentist approaches in three simulations. Two real data examples are also provided.

Using the well-known strategy in which parameters are linked to the sampling distribution via an identification analysis, we offer an interpretation of the item parameters in the one-parameter logistic with guessing model (1PL-G) and the nested Rasch model. The interpretations are based on measures of informativeness that are defined in terms of odds of correctly answering the items. It is shown that the interpretation of what is called the difficulty parameter in the random-effects 1PL-G model differs from that of the item parameter in a random-effects Rasch model. It is also shown that the traditional interpretation of the guessing parameter in the 1PL-G model changes, depending on whether fixed-effects or random-effects versions of both models are considered.

In item response theory (IRT), it is often necessary to perform restricted recalibration (RR) of the model: A set of (focal) parameters is estimated holding a set of (nuisance) parameters fixed. Typical applications of RR include expanding an existing item bank, linking multiple test forms, and associating constructs measured by separately calibrated tests. In the current work, we provide full statistical theory for RR of IRT models under the framework of pseudo-maximum likelihood estimation. We describe the standard error calculation for the focal parameters, the assessment of overall goodness-of-fit (GOF) of the model, and the identification of misfitting items. We report a simulation study to evaluate the performance of these methods in the scenario of adding a new item to an existing test. Parameter recovery for the focal parameters as well as Type I error and power of the proposed tests are examined. An empirical example is also included, in which we validate the pediatric fatigue short-form scale in the Patient-Reported Outcome Measurement Information System (PROMIS), compute global and local GOF statistics, and update parameters for the misfitting items.

Previous influential simulation studies investigate the effect of underlying non-normality in ordinal data using the Vale–Maurelli (VM) simulation method. We show that discretized data stemming from the VM method with a prescribed target covariance matrix are usually numerically equal to data stemming from discretizing a multivariate normal vector. This normal vector has, however, a different covariance matrix than the target. It follows that these simulation studies have in fact studied data stemming from normal data with a possibly misspecified covariance structure. This observation affects the interpretation of previous simulation studies.

In this paper, we present the academic genealogy of presidents of the Psychometric Society by constructing a genealogical tree, in which Ph.D. students are encoded as descendants of their advisors. Results show that most of the presidents belong to five distinct lineages that can be traced to Wilhelm Wundt, James Angell, William James, Albert Michotte or Carl Friedrich Gauss. Important psychometricians Lee Cronbach and Charles Spearman play only a marginal role. The genealogy systematizes important historical knowledge that can be used to inform studies on the history of psychometrics and exposes the rich and multidisciplinary background of the Psychometric Society.

Covariate-adjusted treatment effects are commonly estimated in non-randomized studies. It has been shown that measurement error in covariates can bias treatment effect estimates when not appropriately accounted for. So far, these delineations primarily assumed a true data generating model that included just one single covariate. It is, however, more plausible that the true model consists of more than one covariate. We evaluate when a further covariate may reduce bias due to measurement error in another covariate and in which cases it is not recommended to include a further covariate. We analytically derive the amount of bias related to the fallible covariate’s reliability and systematically disentangle bias compensation and amplification due to an additional covariate. With a fallible covariate, it is not always beneficial to include an additional covariate for adjustment, as the additional covariate can extensively increase the bias. The mechanisms for an increased bias due to an additional covariate can be complex, even in a simple setting of just two covariates. A high reliability of the fallible covariate or a high correlation between the covariates cannot in general prevent from substantial bias. We show distorting effects of a fallible covariate in an empirical example and discuss adjustment for latent covariates as a possible solution.

In the study of human dynamics, the behavior under study is often operationalized by tallying the frequencies and intensities of a collection of lower-order processes. For instance, the higher-order construct of negative affect may be indicated by the occurrence of crying, frowning, and other verbal and nonverbal expressions of distress, fear, anger, and other negative feelings. However, because of idiosyncratic differences in how negative affect is expressed, some of the lower-order processes may be characterized by sparse occurrences in some individuals. To aid the recovery of the true dynamics of a system in cases where there may be an inflation of such “zero responses,” we propose adding a regime (unobserved phase) of “non-occurrence” to a bivariate Ornstein–Uhlenbeck (OU) model to account for the high instances of non-occurrence in some individuals while simultaneously allowing for multivariate dynamic representation of the processes of interest under nonzero responses. The transition between the occurrence (i.e., active) and non-occurrence (i.e., inactive) regimes is represented using a novel latent Markovian transition model with dependencies on latent variables and person-specific covariates to account for inter-individual heterogeneity of the processes. Bayesian estimation and inference are based on Markov chain Monte Carlo algorithms implemented using the JAGS software. We demonstrate the utility of the proposed zero-inflated regime-switching OU model to a study of young children’s self-regulation at 36 and 48 months.