1 Introduction
Most item response theory (IRT) models implicitly assume that the respondents are a random sample from the population envisaged. These IRT models assume one or possibly more latent variables only at the level of the respondent, and we refer to these IRT models as single-level IRT models. However, in many practical situations, the respondents are nested in groups. For example, students nested in school classes rating their teacher’s instructional quality (Scherer et al., Reference Scherer, Nilsen and Jansen2016), employees of the same department assessing humor in the workplace climate (Cann et al., Reference Cann, Watson and Bridgewater2014), or nurses within the same intensive care unit evaluating collaboration (Dougherty & Larson, Reference Dougherty and Larson2010). In such situations, it is inappropriate to assume that the respondents are a random sample due to the group effect. It is therefore reasonable to use IRT models with a latent variable both on the respondent level and the group level (e.g., De Jong & Steenkamp, Reference De Jong and Steenkamp2010; Fox, Reference Fox2007; Fox & Glas, Reference Fox and Glas2001). We refer to these IRT models as two-level IRT models. This paper investigates the measurement properties of a general nonparametric two-level IRT model, which was proposed by Snijders (Reference Snijders, Boomsma, van Duijn and Snijders2001), and which can be considered a two-level generalization of the single-level nonparametric IRT models proposed by Mokken (Reference Mokken1969) and Holland and Rosenbaum (Reference Holland and Rosenbaum1986).
Assume that a test consists of I items, indexed by i (
$i=1,2,\ldots ,I$
), and each item has
$m+1$
ordered item scores
$0,1,\ldots ,m$
. Assume that this test is administered to R randomly selected non-nested respondents, indexed by r (
$r = 1,\ldots ,R$
). Note that index r refers to the rth respondent in the sample. Before sampling, it is not known which respondent from the population will be the rth respondent in the sample. Therefore,
$X_{ri}$
— defined as the score of the randomly selected rth respondent in the sample on item i—is a random variable. In this paper, variables will be denoted by uppercase letters, and their realizations by lower case letters. Hence, the realization of
$X_{ri}$
is denoted by
$x_{ri}$
. For each respondent, the I item scores can be collected in a vector
$\mathbf {X}_r=(X_{r1},X_{r2},\ldots ,X_{rI})$
. Because the respondents are randomly and independently sampled, we consider the R vectors
$\mathbf {X}_r$
independent and identically distributed (i.i.d.) for all r. As the respondents are non-nested, a single-level IRT model may be appropriate as a measurement model. Let
$\Theta _r$
be a random latent variable of the rth randomly sampled respondent. Analogous to
$X_{ri}$
,
$\Theta _r$
is a random variable, because before sampling it is not known which respondent from the population will be the rth respondent in the sample. Because the respondents are randomly and independently sampled, the R variables
$\Theta _r$
are i.i.d. for all r. Let
$\theta _r$
be a value of respondent r on the random latent variable
$\Theta _r$
. For respondent r, the expected value on item i is
$E(X_{ri}|{\Theta _r =}\ \theta _r) = \sum _{x=1}^m P(X_{ri} \geq x | {\Theta _r =}\ \theta _r)$
. The expectation of
$X_{ri}$
as a function of
$\Theta _r$
,
$E(X_{ri}|\Theta _r)$
, is referred to as the item response function (IRF; Chang & Mazzeo, Reference Chang and Mazzeo1994). Most single-level IRT models are defined by at least these three assumptions:
-
1. Unidimensionality (UN): Latent variable
$\Theta _r$
is unidimensional -
2. Local independence (LI): Item scores
$X_{ri}$
are independent given
$\theta _r$
-
3. Monotonicity (MO):
$P(X_{ri}\geq x|\Theta _r = \theta _r)$
is nondecreasing in
$\theta _r$
, for all i and for
$x=1,\ldots ,m$
These assumptions are necessary to restrict the distribution of
$\mathbf {X}_r$
(Junker & Ellis, Reference Junker and Ellis1997). The combination of UN, LI, and MO is also referred to as the monotone homogeneity model (MHM, Mokken, Reference Mokken1971; Sijtsma & Molenaar, Reference Sijtsma and Molenaar2002; a.k.a. monotone unidimensional representation, Junker, Reference Junker1993; Junker & Ellis, Reference Junker and Ellis1997; unidimensional monotone latent variable model, Holland & Rosenbaum, Reference Holland and Rosenbaum1986; and nonparametric graded response model, Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1996, Reference Hemker, Sijtsma, Molenaar and Junker1997). The MHM does not use parameters to model the distribution of
$\Theta $
and the relation between the item scores and
$\Theta _r$
. The MHM is therefore called a nonparametric IRT model.
A fourth assumption in nonparametric IRT is invariant item ordering. Suppose that the I items are ordered by mean item score and numbered accordingly; that is, if
$i < j$
, then
$E(X_{ri}) \leq E(X_{rj})$
for all
$i \neq j$
. Then,
-
4. Invariant item ordering (IIO):
$E(X_{ri}|{\Theta _r=}\ \theta _r)\leq E(X_{rj}|{\Theta _r=}\ \theta _r)$
for all
$\theta _r$
(Ligtvoet et al., Reference Ligtvoet, Van der Ark, Bergsma and Sijtsma2011; Sijtsma & Hemker, Reference Sijtsma and Hemker1998; Sijtsma & Junker, Reference Sijtsma and Junker1996). IIO means that the order in difficulty is identical across all values of the latent variable. IIO allows the stochastic ordering of the items using the mean item scores. For applications of IIO, we refer to Sijtsma et al. (Reference Sijtsma, Meijer and Van der Ark2011). Following Sijtsma and Van der Ark (Reference Sijtsma and Van der Ark2017, Reference Sijtsma and Van der Ark2020, pp. 156–158; also see the Discussion), we call the model that assumes UN, LI, MO, and IIO the double monotonicity model (DMM).
The MHM has several ordering properties. The MHM implies stochastic ordering of the manifest variable by the latent variable (Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1996, Reference Hemker, Sijtsma, Molenaar and Junker1997), which implies that latent variable can be used stochastically to order the respondents on the unweighted sum score. More importantly, for dichotomous items, the MHM implies monotone likelihood ratio (MLR; Grayson, Reference Grayson1988; Huynh, Reference Huynh1994; Ünlü, Reference Ünlü2008), which implies the property of stochastic ordering of the latent variable by the sum score across the items (SOL; Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997). Measurement properties MLR and SOL imply that the sum score can be used to (stochastically) order respondents on the latent variable. For polytomous items, the MHM does not imply MLR and SOL (Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1996, Reference Hemker, Sijtsma, Molenaar and Junker1997); however, the MHM implies the measurement property of weak SOL (Van der Ark & Bergsma, Reference Van der Ark and Bergsma2010), which can be used for pairwise ordering of respondents or groups on the latent variable.
These theoretical results justify ordinal person measurement by means of sum score
$X_{r+}$
if the MHM holds. Suppose that two respondents have sum scores a and b, respectively (
$a < b$
), then for dichotomous items, due to the SOL property, the MHM implies
$E(\Theta _r|X_{r+} = a) \leq E(\Theta _r|X_{r+} = b)$
; for polytomous items, due to the weak SOL property, the MHM implies
$E(\Theta _r|X_{r+} < a) \leq E(\Theta _r|X_{r+} \geq a)$
. Hence, the sum score stochastically orders the respondents on
$\Theta _r$
. Alternatively, suppose that two respondents have latent variable values t and u, respectively (
$t<u$
), then due to the monotonicity assumption the MHM implies
$E(X_{r+}|\Theta _r = t) \leq E(X_{r+}|\Theta _r = u)$
. Hence the latent variable values stochastically orders the respondents on the sum score, a property sometimes referred to a stochastic ordering of the manifest variable by
$\Theta _r$
(SOM; Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997). These mutual ordering properties of
$X_{r+}$
and
$\Theta _r$
make
$X_{r+}$
an attractive estimator of
$\Theta _r$
. Under the MHM,
$X_{r+}$
is a consistent asymptotic normal estimator of
$\Theta _r$
(Junker, Reference Junker1991; Stout, Reference Stout1990).
The simple sum score is more intuitive for non-psychometricians than, for example, an estimated latent variable because the sum score is defined on the scale of the test. Therefore, a higher sum score has a fairly straightforward interpretation, such as responded to more items correctly or responded more extreme to the items (Sijtsma & Hemker, Reference Sijtsma and Hemker2000). In addition, using the sum score in scientific research avoids sample-specific transformations, which benefits comparability across studies and contributes to the replicability of results across studies (Edelsbrunner, Reference Edelsbrunner2022; Widaman & Revelle, Reference Widaman and Revelle2022). Hence, providing justification for using the sum score is relevant for psychometric research and testing practice, even when the estimated latent variable is used for test construction and measurement evaluation (Hemker et al., Reference Hemker, Van der Ark and Sijtsma2001).
The DMM implies an ordinal scale for both person and item measurement. Hence, besides using the respondent sum score to order respondents on a latent variable, the mean item score can be used to order the items on a latent difficulty scale. Using the mean item score has similar advantages as the sum score for psychometric and testing practice: They have an intuitive interpretation, such as the proportion correct or average extremeness in the sample. In addition, estimating a latent difficulty is not straightforward and can have various interpretations that do not necessarily relate to the difficulty in practice (Sijtsma & Hemker, Reference Sijtsma and Hemker2000; Sijtsma & Meijer, Reference Sijtsma and Meijer2001).
All popular unidimensional IRT models, such as the Rasch Model (Rasch, Reference Rasch1960), the two- and three-parameter logistic models (Birnbaum, Reference Birnbaum, Lord and Novick1968), the graded response model (Samejima, Reference Samejima1969), the rating scale model (Andrich, Reference Andrich1978), the partial credit model (Masters, Reference Masters1982), and the sequential model (Tutz, Reference Tutz1990), are special cases of the MHM (Van der Ark, Reference Van der Ark2001). Hence, if the goal of the test is to order respondents, the MHM is preferred over popular parametric IRT models because, by definition, the MHM fits better to the data than these parametric IRT models. If the goal of the test is estimating the respondents’ scores on
$\Theta _r$
, alternative methods are required, such as a smoothing procedure or estimating a parametric IRT model (e.g., Ramsay, Reference Ramsay1991; Sijtsma & Van der Ark, Reference Sijtsma and Van der Ark2020, Chapter 4, respectively). However, as these parametric IRT models are a special case of the MHM, investigating the fit of the MHM is still useful because if the MHM does not fit, neither do the parametric IRT models.
The MHM poses testable restrictions on the data, referred to as observable properties. For example, the MHM implies nonnegative inter-item covariances (e.g., Sijtsma & Molenaar, Reference Sijtsma and Molenaar2002, pp. 155–156). Observable properties can be investigated in data to find evidence against the MHM assumptions. Holland and Rosenbaum (Reference Holland and Rosenbaum1986) showed that the MHM implies conditional association (CA). Let
$\textbf {Y}_r$
and
$\textbf {Z}_r$
be two mutually exclusive and exhaustive subsets of
$\textbf {X}_r$
. CA holds if for every partitioning
$\textbf {X}_r = (\textbf {Y}_r,\textbf {Z}_r)$
and for all functions h and for all nondecreasing functions
$g_1$
and
$g_2$
$$ \begin{align} Cov[g_1(\textbf{Y}_r),g_2(\textbf{Y}_r) | h(\textbf{Z}_r)] \geq 0. \end{align} $$
The observable property CA is too comprohensive for a single testing procedure (see Ellis & Sijtsma, Reference Ellis and Sijtsma2023), but special cases of CA, including testing for nonnegative covariances, have been proposed to test the MHM. We focus on manifest monotonicity (MM; Sijtsma & Hemker, Reference Sijtsma and Hemker2000) and a testing procedure to identify locally dependent item sets using three cases of CA (Straat et al., Reference Straat, Van der Ark and Sijtsma2016). For dichotomous items, CA implies MM (Ligtvoet, Reference Ligtvoet2022). Let
$X_{r(i)}=\sum _{j\neq i}^I X_{ri}$
be the rest score of item i, then MM means that
$$ \begin{align} E(X_{ri}|X_{r(i)}) \text{ is nondecreasing in } X_{r(i)}. \end{align} $$
Hence, MM is the MO assumption with latent variable
$\Theta _r$
replaced by an observable proxy
$X_{r(i)}$
. Note that for polytomous items, the MHM does not imply MM. Straat et al. proposed testing
$Cov(X_{ri},X_{rj})\geq 0$
,
$Cov(X_{ri},X_{rj}|X_{rk})\geq 0$
, and
$Cov(X_{ri},X_{rj}|X_{r(ij)})\geq 0$
, where
$X_{r(ij)}=\sum _{k\neq i,j} X_{rk}$
. We refer to these three inequalities as nonnegative inter-item covariances (NNIIC). As these three inequalities of NNIIC are special cases of CA with
$g_1(\textbf {Y}_r)=X_{ri}$
, with
$g_2(\textbf {Y}_r)=X_{rj}$
, and with
$h(\textbf {Z}_r)=\emptyset $
,
$h(\textbf {Z}_r)=X_{rk}$
, and
$h(\textbf {Z}_r)=X_{r(ij)}$
, respectively, the MHM implies the three inequalities. Other CA-based observable properties have been proposed by, for example, Ellis (Reference Ellis2014) and Ligtvoet (Reference Ligtvoet2022). Ellis and Sijtsma (Reference Ellis and Sijtsma2023) noted that these CA-based observable properties cannot distinguish between unidimensional and multidimensional models, and these authors suggested using (also CA-based) conditioning on added regression predictions (CARP) inequalities to investigate UN.
The DMM poses additional observable properties (see Ligtvoet et al., Reference Ligtvoet, Van der Ark, Bergsma and Sijtsma2011). We focus on manifest invariant item ordering (MIIO), which holds if for
$E(X_{ri})~<~E(X_{rj})$
,
$$ \begin{align} E(X_{ri}|X_{r(ij)}=y)\leq E(X_{rj}|X_{r(ij)}=y) \text{ for all } y \text{ and all }i<j. \end{align} $$
Note that MIIO is the IIO assumption with latent variable
$\Theta _r$
replaced by
$X_{r(ij)}=\sum _{k\neq i,j}X_{rk}$
. Other observable properties of IIO have been proposed; for example, by Tijmstra et al. (Reference Tijmstra, Hessen, Van der Heijden and Sijtsma2011).
The assumptions (UD, LI, MO, and IIO) discussed in this paragraph have not been formally defined for two-level IRT models, and as a result, it is also unknown how these assumptions should be investigated in test data. Also, the measurement properties MLR, SOL, and SOM nor the observable properties MM, CA, and MIIO have been defined for two-level IRT models, and as a result, it is unknown whether two-level IRT models imply these measurement properties in the same way as single-level IRT models do. In the remainder of this paper, we generalize the MHM and DMM to two-level data on both the respondent level and the group level. We build on the work of Snijders (Reference Snijders, Boomsma, van Duijn and Snijders2001), who proposed a two-level nonparametric IRT model for scaling subjects (e.g., persons or groups) scored by multiple respondents (i.e., multi-rater measurement) using dichotomous items. For the proposed models, we establish which stochastic ordering properties and observable data properties are implied, and how they are related. Note that the proofs have been diverted to the Appendix. Implications and recommendations for practice and further research are discussed.
2 Two-level nonparametric IRT
Suppose a measurement instrument consists of I items, indexed by i or j (
$i,j = 1, 2, \ldots , I; j\neq i$
). Suppose there are S groups, indexed by s (
$s = 1,2,\ldots ,S$
), each consisting of
$R_s$
respondents, indexed by r
$(r = 1, 2, \ldots , R_s)$
. Note that index s refers to the sth group, and index r refers to the rth respondent in group s. Before sampling, it is not known which group from the population of groups will be the sth group in the sample, nor which respondent from the population of respondents will be the rth respondent in group s. The groups are assumed to be a random sample from a population of groups, and the respondents within a group are assumed to be a random sample from a population of respondents. Without loss of generality, we assume the number of respondents per group is the same; that is,
$R_1=R_2=\cdots =R_S=R$
. Let
$X_{sri}$
denote the score on item i of respondent r in group s, with realization
$x_{sri}$
(
$x_{sri} \in 0,\ldots ,m$
). For dichotomous items,
$m=1$
and
$x_{sri}$
takes on value 1 if item i is endorsed or answered correctly by respondent r in group s, and 0 otherwise. Let
$X_{sr+}=\sum _{i=1}^I X_{sri}$
denote the respondent-level sum score. Let
$X_{si} = R^{-1}\sum _{r=1}^R X_{sri}$
denote the group-level score on item i (i.e., the mean score over respondents’ scores on item i within group s), with realization
$x_{si}$
.
$X_{si}$
can take on
$Rm+1$
values with a minimum of 0 and a maximum of m. Let
$X_{s+}=\sum _{i=1}^I X_{si}$
denote the group-level sum score. The vector of item scores for respondent r in group s is denoted
$\mathbf {X}_{sr}=(X_{sr1}, \ldots , X_{srI})$
, with realization
$\mathbf {x}_{sr}=(x_{sr1},\ldots ,x_{srI})$
. Because the respondents within a group are randomly and independently sampled, the R vectors
$\mathbf {X}_{sr}$
are considered i.i.d. within each s for all r. The vector of all item scores for group s is denoted
$\mathbf {X}_s=(\mathbf {X}_{s1}, \ldots , \mathbf {X}_{sR}) = (X_{sr1}, \ldots , X_{sRI})$
, with realization
$\mathbf {x}_s=(\mathbf {x}_{s1}, \ldots , \mathbf {x}_{sR}) = (x_{sr1}, \ldots , x_{sRI})$
. Because the groups are randomly and independently sampled, the S vectors
$\mathbf {X}_s$
are considered i.i.d. for all s.
Let
$\Theta _{sr}$
,
$\Gamma _s \Delta _{sr}$
be random latent variables of the rth randomly sampled respondent in the sth randomly sampled group. Analogous to
$\Theta _r$
in the single-level situation, these are random variables because before the groups and respondents have been sampled, it is unknown which group from the population groups will be the sth group, and which respondent from the population of respondents belonging to the sth group will be the rth respondent. Variable
$\Gamma _s$
is considered a common group component,
$\Delta _{sr}$
is a combination of an individual (random) respondent effect and a group by respondent interaction effect, and
$\Theta _{sr}$
is the sum of these effects; that is,
$$ \begin{align} \Theta_{sr}=\Gamma_s+\Delta_{sr}, \end{align} $$
(Snijders, Reference Snijders, Boomsma, van Duijn and Snijders2001). Let
$\varepsilon _{sri}$
be a random latent variable that may be interpreted as an error term. Assumption 1 is a basic assumption(B) about the relation between the latent variables and the observed score
$X_{sri}$
using function
$f_i$
.
Assumption 1. Basic assumption of item scores and latent variables.
-
(B)
$X_{sri}=f_i(\Gamma _s+\Delta _{sr}, \varepsilon _{sri})$
. For all
$s,r,$
and i,
$\Gamma _s$
,
$\Delta _{sr}$
, and
$\varepsilon _{sri}$
are independent. Furthermore, all
$\Gamma _s$
$(s=1,\ldots ,S)$
are identically distributed, and all
$\Delta _{sr}$
(
$s=1,\ldots ,S; r=1,\ldots ,R$
) are identically distributed, with
$E(\Delta _{sr})=0$
.
It follows from B that
$\Theta _{sr}$
are identically distributed for all
$s,r$
, and that for a fixed item i, all
$\varepsilon _{sri}$
are identically distributed for all
$s,r$
. B is assumed throughout this paper. The variances of
$\Theta _{sr}$
,
$\Gamma _s$
, and
$\Delta _{sr}$
are denoted var(
$\Theta _{sr}$
), var(
$\Gamma _s$
), and var(
$\Delta _{sr}$
), respectively. Because
$\Gamma _s$
and
$\Delta _{sr}$
are assumed independent,
$\text {var}(\Theta _{sr})=\text {var}(\Gamma _s)+\text {var}(\Delta _{sr})$
, for all s and all r. Let
$\theta _{{sr}}$
be a group-respondent combination value on
$\Theta _{sr}$
of respondent r in group s,
$\gamma _s$
a value on
$\Gamma _s$
for group s, and
$\delta _{sr}$
a value on
$\Delta _{sr}$
for respondent r in group s. Hence, for respondent r in group s, we assume there exist value
$\theta _{sr}=\gamma _s+\delta _{sr}$
.
Let
$P(\mathbf {X}_{sr}=\mathbf {x} | \Gamma _s, \Delta _{sr})$
denote the probability of obtaining item-score pattern
$\mathbf {x}$
given
$\Gamma _s$
and
$\Delta _{sr}$
. Throughout the rest of the paper, we assume homogeneity of
$\Gamma _s,\Delta _{sr}$
and
$\Theta _{sr}$
:
Assumption 2. Homogeneity assumption of
$\Gamma _s,\Delta _{sr}$
and
$\Theta _{sr}$
-
(H) Homogeneity of the response probablities holds for
$\Gamma _s,\Delta _{sr}$
and
$\Theta _{sr}$
, hence,
$P(\mathbf {X}_{sr}=\mathbf {x} | \Theta _{sr}) = P({\mathbf {X}}_{sr}={\mathbf {x}} | \Gamma _s, \Delta _{sr})$
Let
$$ \begin{align} P(X_{sri}\geq x | \Theta_{sr})= \sum_{y=x}^m P(X_{sri}=y | \Theta_{sr}), \end{align} $$
denote the probability of obtaining at least score x on item i given
$\Theta _{sr}$
, which we refer to as the respondent-level item-step response function. For respondent r in group s, the expected item score is
$E(X_{sri}|\Theta _r=\theta _r)=\sum _{x=1}^m P(X_{sri}\geq x | \Theta _{sr}=\theta _{sr})$
. In two-level test data, we distinguish between a respondent-level IRF (IRF-1, denoted
$E_i(\cdot )$
) and a group-level IRF (IRF-2, denoted
$\mathcal {E}_i(\cdot )$
). IRF-1 is defined as
$$ \begin{align} \begin{array}{lcl} {E_i}(\Theta_{sr}) &=& E(X_{sri} | \Theta_{sr}) \\ &=&\sum_{x=1}^m P(X_{sri}\geq x | \Theta_{sr}), \end{array} \end{align} $$
where
$E(X_{sri}|\Theta _{sr})$
equals the expected item score
$\Theta _{sr}$
.
Let
$P(X_{sri}\geq x | \Gamma _s)$
denote the probability of obtaining at least score x on item i given
$\Gamma _s$
, which we refer to as the group-level item-step response function. By H and the law of total expectation (e.g., Rice, Reference Rice2006, p 149), the item-step response function can be formulated as
$$ \begin{align} \renewcommand{\arraystretch}{1.3} \begin{array}{lcll} P(X_{sri}\geq x | \Gamma_s) &=& {E[P(X_{sri}\geq x | \Theta_{sr}, \Gamma_s) | \Gamma_s]} \\ &=& {E[P(X_{sri}\geq x | \Gamma_s+\Delta_s, \Gamma_s) | \Gamma_s]} \\ &=& {E[P(X_{sri}\geq x | \Delta_s, \Gamma_s) | \Gamma_s]} \\ &=& E[P(X_{sri}\geq x | \Theta_{sr}) | \Gamma_s]. \end{array} \end{align} $$
For a randomly selected respondent in group s, the expected item score is
$E(X_{sri}|{\Gamma _s=\gamma _s})=\sum _{x=1}^m P(X_{sri}\geq x | \Gamma _s=\gamma _s)$
. IRF-2 is defined as
$$ \begin{align} \renewcommand{\arraystretch}{1.3} \begin{array}{lcll} {\mathcal{E}_i}(\Gamma_s) &=& E(X_{sri}|\Gamma_s) & \\ &=& \sum_{x=1}^m P(X_{sri}\geq x | \Gamma_s) \\ &=& E[\sum_{x=1}^m P(X_{sri}\geq x | \Theta_{sr}) | \Gamma_s] & \text{(Eq. 7)} \\ &=& E[{E_i}(\Theta_{sr}) | \Gamma_s] & \text{(Eq. 6)},\\ \end{array} \end{align} $$
where
$E(X_{sri}|\Gamma _s)$
equals the expected item score as a function of
$\Gamma _s$
. Note that because
$\Delta _{sr}$
variables are assumed i.i.d.,
$E(X_{si}|\Gamma _s)=R^{-1}\sum _{r=1}^R {\mathcal {E}_i}(\Gamma _s)={\mathcal {E}_i}(\Gamma _s)$
. Hence, the expected group-level item score for group s is the value of the IRF-2 for
$\Gamma _s=\gamma _s$
. Figure 1 shows an hypothetical IRF-1 and IRF-2. Because IRF-2 is the expectation of IRF-1 with respect to
$\Delta _{sr}$
(Equation 8), IRF-2 is flatter than function IRF-1.
Figure 1 An IRF-1 (
$E_i(\Theta _{sr})$
; solid curve) and an IRF-2 (
$\mathcal {E}_i(\Gamma _s)$
; dashed curve) depicted on the same
$\Theta _{sr}$
scale. The horizontal axis shows one hypothetical group value
$\gamma _s$
, plus the
$\theta _{sr}$
values of 10 randomly drawn respondents (
$r=1,\ldots ,10$
) from group s. Note that
$\delta _{sr}$
is represented by the length of the line segment between
$\gamma _s$
and the
$\theta _{sr}$
values on the horizontal axis.
2.1 Definitions of possible model assumptions
Besides the basic and homogeneity assumption (B and H, respectively), multiple assumptions of nonparametric IRT for two-level data can be defined at level 1 (the respondent level) and at level 2 (the group level).
Definition 1. Unidimensionality (UN).
-
(UN-1) Unidimensionality at level 1 holds if
$\Theta _{sr}$
is a unidimensional variable. -
(UN-2) Unidimensionality at level 2 holds if
$\Gamma _s$
is a unidimensional variable.
UN-1 and UN-2 mean that the item scores on the test or questionnaire are modeled using one latent variable.
Definition 2. Local independence (LI).
-
(LI-1) Local independence at level 1 holds if
(9)
$$ \begin{align} \begin{array}{lcl} P(\mathbf{X}_{sr}=\mathbf{x}_{sr}|{\Theta_{sr}=}\ {\theta_{sr}}) &=&\prod_{i=1}^I P(X_{sri}=x_{sri}| {\Theta_{sr}=}\ {\theta_{sr}}). \end{array} \end{align} $$
-
(LI-2) Local independence at level 2 holds if
(10)
$$ \begin{align} \begin{array}{lcl} {P(\mathbf{X}_s = \mathbf{x}_s | {\Gamma_s =}\ \gamma_s) = \prod_{r=1}^R P(\mathbf{X}_{sr}=\mathbf{x}_{sr}| {\Gamma_s =}\ \gamma_s)}. \end{array} \end{align} $$
LI-1 means that respondent-level item scores (
$X_{sri}$
) are independent given
$\theta _{sr}$
. LI-2 means that the response vectors of respondents are independent given
$\gamma _s$
. LI-2 implies that between respondents, the respondent-level item scores
$X_{sri}$
and
$X_{spj}$
(
$i\neq j$
;
$r\neq p$
) are independent given
$\gamma _s$
. However, within respondents, respondent-level item scores
$X_{sri}$
and
$X_{srj}$
(
$i\neq j$
) are not independent given
$\gamma _s$
.
Definition 3. Monotonicity (MO).
-
(MO-1) Monotonicity at level 1 holds if
$P(X_{sri}\geq x|\Theta _{sr}=\theta _{sr})$
is nondecreasing in
$\theta _{sr}$
, for all i and
$x=1,\ldots ,m$
. -
(MO-2) Monotonicity at level 2 holds if
$P(X_{sri}\geq x|\Gamma _s=\gamma _s)$
is nondecreasing in
$\gamma _s$
, for all i and
$x=1,\ldots ,m$
.
MO-1 implies that, for each item, IRF-1 (Equation 6) is nondecreasing in
$\Theta _{sr}$
, and MO-2 implies that, for each item, IRF-2 (Equation 8) is nondecreasing in
$\Gamma _s$
. Note that in Figure 1, IRF-1 satisfies MO-1 and IRF-2 satisfies MO-2.
Definition 4. Invariant item ordering (IIO). For a set of I items with
$m+1$
ordered item-score categories, for which the items are ordered and numbered such that
$E(X_{sri}) \leq E(X_{srj})$
for all
$i<j$
, then
-
(IIO-1) Invariant item ordering at level 1 holds if
$E(X_{sri}|\Theta _{sr}=\theta _{sr}) \leq E(X_{srj}|\Theta _{sr}=\theta _{sr})$
for all
$\theta _{sr}$
. -
(IIO-2) Invariant item ordering at level 2 holds if
$E(X_{sri}|\Gamma _s=\gamma _s) \leq E(X_{srj}|\Gamma _s=\gamma _s)$
for all
$\gamma _s$
.
IIO-1 means that the IRF-1s of different items do not intersect. IIO-2 means that the IRF-2s of different items do not intersect. Note that the definition of IIO-1 and IIO-2 allows for ties, such that for some values of the latent variable items may be equally difficult.
2.2 Relation between level 1 and level 2 assumptions
Theorem 1 gives the relations between the basic assumption and local independence at both levels.
Theorem 1. B implies LI-1 and LI-2.
The assumptions UN, LI, MO, and IIO were defined at both Level 1 and Level 2 (Definitions 1 to 4). However, Theorem 1 shows that B implies both LI-1 and LI-2, and as a result, LI is no longer a necessary assumption, as in all remaining proofs LI-1 and LI-2 may be replaced by B and H.
Theorem 2 gives the relations between the assumptions at level 1 and the assumptions at level 2.
Theorem 2. Under B and H, UN-1, MO-1, and IIO-1 imply UN-2, MO-2, and IIO-2, respectively.
Theorem 2 shows that the level-1 assumptions imply their level-2 assumptions, but not the other way around. Hence, the level-2 assumptions do not imply the level-1 assumptions. For example, if respondent-level item scores depend both on
$\Gamma _s$
and on
$\Delta _{sr}$
and var(
$\Delta _{sr}$
)
$>0$
, in general
$\Theta _{sr}\neq \Gamma _s$
,
$P(X_{sri}\geq x|{\Theta _{sr}})\neq P(X_{sri}\geq x|\Gamma _s)$
, and
$E_i(\Theta _{sr})\neq \mathcal {E}_i(\Gamma _s)$
. As a result, UN-1, MO-1, and IIO-1 are not equal to UN-2, MO-2, and IIO-2, respectively. It may be noted that because of the homogeneity assumption H, the level-1 assumptions (UN-1, LI-1, MO-1, and IIO-1) are equivalent to the single-level nonparametric-IRT assumptions (UN, LI, MO, and IIO), when
$X_{sri}$
is replaced by
$X_{ri}$
and
$\theta _{sr}$
by
$\theta _r$
.
2.3 Models
Two-level nonparametric IRT assumptions can be used to define several nonparametric IRT models. Analogous to the single-level nonparametric IRT models, we distinguish between the MHM and the DMM, but in addition we also distinguish between the level on which they can be defined. Snijders (Reference Snijders, Boomsma, van Duijn and Snijders2001) defined a two-level nonparametric IRT model for scaling groups with dichotomous item scores using assumptions UN-1, LI-1, MO-1, and IIO-1. We present four models that allow for both dichotomous and polytomous items. As mentioned before, for all models B and H are assumed.
The first respondent-level model is the MHM-1, defined by assuming UN-1, LI-1, and MO-1 (Table 1, first row). The MHM-1 consists of level-1 assumptions, which imply UN-2, LI-2, and MO-2 (Theorem 2). The second respondent-level model is the DMM-1, defined by assuming UN-1, LI-1, MO-1, and IIO-1, implying UN-2, LI-2, MO-2, and IIO-2 (Table 1, second row). The first group-level model is the MHM-2, defined by assuming UN-2, LI-2, and MO-2 (Table 1, third row). The second group-level model is the DMM-2, defined by assuming UN-2, LI-2, MO-2, and IIO-2 (Table 1, fourth row). Note that, for all models, LI-1 and LI-2 are implied by B, but we explicitly incorporate them into the models, such that the models align more obviously to the single-level models.
Table 1 Assumptions of the two-level nonparametric IRT models
Note: A = assumed, I = implied.
Figure 2 shows the hierarchical structure of the four models, where an arrow indicates an implication. The MHM-2 is the most general model, of which the other three are special cases. The DMM-1 is the most restrictive model, implying the other three models. In the next sections, we derive some ordering and observable properties implied by these models.
Figure 2 Hierarchical structure of the two-level nonparametric IRT models.
3 Ordering properties of two-level nonparametric IRT models
We investigated four possible ordering properties for sum score
$X_{sr+}$
at level 1 and sum score
$X_{s+}$
at level 2: MLR, SOM, SOL, and weak SOL.
Definition 5. Monotone likelihood ratio (MLR; Ferguson, Reference Ferguson1967, p. 208).
-
(MLR-1) Monotone likelihood ratio at level 1 holds if, for
$a < b$
, the probability ratio (11)
$$ \begin{align} \dfrac{P(X_{sr+} = b|\Theta_{sr})}{P(X_{sr+} = a|\Theta_{sr})} \text{ is nondecreasing in } \Theta_{sr}. \end{align} $$
-
(MLR-2) Monotone likelihood ratio at level 2 holds if, for
$a < b$
, the probability ratio (12)
$$ \begin{align} \dfrac{P(X_{s+} = b|\Gamma_s)}{P(X_{s+} = a|\Gamma_s)} \text{ is nondecreasing in } \Gamma_s. \end{align} $$
Definition 6. Stochastic ordering of the manifest score by the latent variable (SOM; Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997).
-
(SOM-1) Stochastic ordering of the manifest score by the latent variable at level 1 holds if, for any value x and
$t<u$
(13)
$$ \begin{align} P(X_{sr+} \geq x| \Theta_{sr} = t)\leq P(X_{sr+} \geq x|\Theta_{sr} = u). \end{align} $$
-
(SOM-2) Stochastic ordering of the manifest score by the latent variable at level 2 holds if, for any value x and
$t<u$
(14)
$$ \begin{align} P(X_{s+} \geq x| \Gamma_s = t)\leq P(X_{s+} \geq x|\Gamma_s = u). \end{align} $$
Definition 7. Stochastic ordering of the latent variable by the manifest score (SOL; Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997).
-
(SOL-1) Stochastic ordering of the latent variable by the manifest score at level 1 holds if, for any value t and
$a<b$
, (15)
$$ \begin{align} P(\Theta_{sr}>t|X_{sr+}=a)\leq P(\Theta_{sr}>t|X_{sr+}=b). \end{align} $$
-
(SOL-2) Stochastic ordering of the latent variable by the manifest score at level 2 holds if, for any value t and
$a<b$
, (16)
$$ \begin{align} P(\Gamma_s>t|X_{s+}=a)\leq P(\Gamma_s>t|X_{s+}=b). \end{align} $$
Definition 8. Weak SOL (WSOL; Van der Ark & Bergsma, Reference Van der Ark and Bergsma2010).
-
(WSOL-1) Weak SOL at level 1 holds if, for any t and a
(17)
$$ \begin{align} P(\Theta_{sr}>t|X_{sr+}<a)\leq P(\Theta_{sr}>t|X_{sr+}\geq a). \end{align} $$
-
(WSOL-2) Weak SOL at level 2 holds if, for any t and a
(18)
$$ \begin{align} P(\Gamma_s> t|X_{s+}<a)\leq P(\Gamma_s>t|X_{s+}\geq a). \end{align} $$
In general, ordering property MLR implies SOM, SOL, and WSOL, and SOL implies WSOL (Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997; Lehmann, Reference Lehmann1986, p. 85; Van der Ark & Bergsma, Reference Van der Ark and Bergsma2010). Hence, ordering property MLR-1 implies SOM-1, SOL-1, and WSOL-1, whereas ordering property MLR-2 implies SOM-2, SOL-2, and WSOL-2. The MLR, SOM, SOL, and WSOL results are valid for any monotone nondecreasing item summary within respondents (e.g., all-correct score, rest-scores, subscores; Rosenbaum, Reference Rosenbaum1984).
For two-level test data, it is unknown whether MLR, SOM, SOL, or WSOL are implied by the two-level nonparametric IRT models. Theorem 3 gives the result for the strongest ordering property (MLR) for the least restrictive models (MHM-1 and MHM-2) and generalizes to weaker ordering properties and more restrictive models.
Theorem 3.
-
(a) For dichotomous item scores, the MHM-1 implies MLR-1.
-
(b) For dichotomous item scores, for
$R\geq I$
, the MHM-2 implies MLR-2.
MLR is symmetric in its argument, so the statement
$X_{sr+}$
has MLR in
$\Theta _{sr}$
means that
$\Theta _{sr}$
also has MLR in
$X_{sr+}$
. Theorem 3 implies that for dichotomous items, under the MHM-1
$X_{sr+}$
is stochastically ordered by
$\Theta _{sr}$
(SOM-1) and
$\Theta _{sr}$
is stochastically ordered by
$X_{sr+}$
(SOL-1). It may be noted that Theorem 3(a) is very similar to the result obtained by Grayson (Reference Grayson1988) who proved for single-level dichotomous item scores that the MHM implies MLR. Under the MHM-2, for
$R\geq I$
, group-level item score
$X_{s+}$
is stochastically ordered by
$\Gamma _s$
(SOM-2) and
$\Gamma _s$
is stochastically ordered by
$X_{s+}$
(SOL-2). Note that for
$R<I$
, MLR-2 is implied for the sum score of any random subset of items of size
$I^*$
, for which
$I^*< R$
. Because the DMM-1 is a special case of the MHM-1 (see Figure 2), Theorem 3(a) also applies to the DMM-1. Similarly, the MHM-1, the DMM-1, and the DMM-2 are special cases of the MHM-2 (see Figure 2), Theorem 3(b) applies to these models as well.
For polytomous items, the single-level MHM and DMM generally do not imply MLR and SOL (see Hemker et al., Reference Hemker, Van der Ark and Sijtsma2001, for counter examples), but these models do imply SOM (Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1996, Reference Hemker, Sijtsma, Molenaar and Junker1997) and weak SOL (Van der Ark & Bergsma, Reference Van der Ark and Bergsma2010). Theorem 4 and 5 show that these results generalize to two-level models.
Theorem 4.
-
(a) The MHM-1 implies SOM-1.
-
(b) The MHM-2 implies SOM-2.
Theorem 4 implies that under the MHM-1
$X_{sr+}$
is stochastically ordered by
$\Theta _{sr}$
(SOM-1) and under the MHM-2,
$X_{s+}$
is stochastically ordered by
$\Gamma _s$
(SOM-2). Because the DMM-1 is a special case of the MHM-1, it also implies SOM-1. Also, because the MHM-1, the DMM-1, and the DMM-2 are special cases of MHM-2, these models imply SOM-2.
Theorem 5.
-
(a) The MHM-1 implies WSOL-1.
-
(b) The MHM-2 implies WSOL-2.
Let
$\mathbf {1}(X_{sr+} \geq k)$
denote the dichotomized respondent-level sum score that takes on value 1 if
$X_{sr+} \geq k$
, and 0 otherwise, and let
$\mathbf {1}(X_{s+} \geq k)$
denote the dichotomized group-level sum score that takes on value 1 if
$X_{s+} \geq k$
, and 0 otherwise. Then, Theorem 5 implies that under the MHM-1,
$\Theta _{sr}$
is stochastically ordered by
$\mathbf {1}(X_{sr+} \geq k)$
(WSOL-1), and that under the MHM-2,
$\Gamma _s$
is stochastically ordered by
$\mathbf {1}(X_{s+} \geq k)$
(WSOL-2). Because DMM-1 is a special case of MHM-1, this model also implies WSOL-1. Also, because the MHM-1, the DMM-1, and the DMM-2 are special cases of the MHM-2, these models imply WSOL-2.
4 Observable properties of two-level nonparametric IRT models
We define observable properties CA, MM, and MIIO for two-level IRT models. For single-level IRT models, rest score
$X_{r(i)}$
was used in MM, and rest score
$X_{r(ij)}$
was used in NNIIC and in MIIO. These rest scores are proxies for the latent variable that must be independent of the variables under investigation. Because of the independence requirement, for two-level IRT models, these rest scores become more involved. Table 2 provides an overview of these rest scores for classified by observable property and level. The rest scores at Level 1 can be considered within-respondent rest scores, and the rest scores at Level 2 can be considered between-level rest scores.
Table 2 Overview of rest scores used in observable properties in single-level and two-level IRT models
Note: a Pertains to the NNIIC given the rest score. The other two NNIIC inequalities do not use a rest score.
Definition 9 defines CA for two-level IRT models. First, partition
$\textbf {X}_{sr}$
into two mutually exclusive and exhaustive sets
$\textbf {Y}_{sr}$
and
$\textbf {Z}_{sr}$
. For example,
$\mathbf {Y}_{sr}$
may contain
$X_{sr1}$
and
$X_{sr2}$
and
$\mathbf {Z}_{sr}$
the remaining item scores. Second, partition the response vectors of the R respondents in group s —
$\textbf {X}_{s1},\ldots ,\textbf {X}_{sR}$
— which are collected in
$\textbf {X}_s$
, into three mutually exclusive and exhaustive sets:
$\textbf {Y}_{s1}$
,
$\textbf {Y}_{s2}$
, and
$\textbf {Z}_{s}$
. For example,
$\textbf {Y}_{s1}$
could contain just
$\textbf {X}_{sr}$
,
$\textbf {Y}_{s1}$
could contain just
$\textbf {X}_{s2}$
, and
$\textbf {Z}_{s}$
could contain the remaining response vectors from
$\textbf {X}_s$
. Note that all scores of the same respondent are in the same set.
Definition 9. Conditional association (CA; Holland & Rosenbaum, Reference Holland and Rosenbaum1986; Rosenbaum, Reference Rosenbaum1988).
-
(CA-1) Conditional association at level 1 holds if
(19)
$$ \begin{align} Cov[g_1(\mathbf{Y}_{sr}), g_2(\mathbf{Y}_{sr}) | h(\mathbf{Z}_{sr})]\geq 0. \end{align} $$
-
(CA-2) Conditional association at level 2 holds if, for
$r\neq p$
, (20)
$$ \begin{align} Cov[g_1(\mathbf{Y}_{sr}), g_2(\mathbf{Y}_{sp}) | h(\mathbf{Z}_{s(rp)})]\geq 0. \end{align} $$
CA-1 is conditional association of the scores within respondents, whereas CA-2 is conditional association of the scores between respondents in the same group (see, also, Rosenbaum, Reference Rosenbaum1988). As for CA, CA-1, and CA-2 are too comprehensive for a single test procedure. The testing procedure to identify locally dependent item sets using NNIIC (Straat et al., Reference Straat, Van der Ark and Sijtsma2016) can be readily generalized to two-level models: For Level 1, the three inequalities in NNIIC generalize to
$Cov(X_{sri},X_{srj})\geq 0$
,
$Cov(X_{sri},X_{srj}|X_{srk})\geq 0$
, and
$Cov(X_{sri},X_{srj}|X_{sr(ij)})\geq 0$
. For Level 2, let p, q, and r index three different respondents. The three inequalities generalize to
$Cov(X_{sri},X_{spj})\geq 0$
,
$Cov(X_{sri},X_{spj}|X_{sqk})\geq 0$
, and
$Cov(X_{sri},X_{spj}|X_{s(rp,ij)})\geq 0$
. Rest scores
$X_{sr(ij)}$
and
$X_{s(rp,ij)}$
have been defined in Table 2.
Definition 10 defines MM for two-level IRT models. The rest scores used in Definition 10 have been defined in Table 2
Definition 10. Manifest monotonicity (MM; Junker, Reference Junker1993; Sijtsma & Hemker, Reference Sijtsma and Hemker2000).
-
(MM-1) Manifest monotonicity at level 1 holds if the within-respondent item-rest regression
$E(X_{sri} | X_{sr(i)})$
is nondecreasing in
$X_{sr(i)}$
. -
(MM-2) Manifest monotonicity at level 2 holds if the between-respondent item-rest regression
$E(X_{sri} | X_{s(r,i)})$
is nondecreasing in
$X_{s(r,i)}$
.
(first row)
Definition 11 defines MM for two-level IRT models. The rest scores used in Definition 11 have been defined in Table 2
Definition 11. Manifest invariant item ordering (MIIO; Ligtvoet et al., Reference Ligtvoet, Van der Ark, Te Marvelde and Sijtsma2010).
-
(MIIO-1) Manifest invariant item ordering at level 1 holds if, for
$E(X_{sri})<E(X_{srj})$
,
$E(X_{sri}|X_{sr(ij)}=y)\leq E(X_{srj}|X_{sr(ij)}=y)$
for all y and all
$i<j$
. -
(MIIO-2) Manifest invariant item ordering at level 2 holds if, for
$E(X_{sri})<E(X_{srj})$
,
$E(X_{sri}|X_{s(r,ij)}=y)\leq E(X_{srj}|X_{s(r,ij)}=y)$
for all y and all
$i<j$
.
(second row).
In Theorem 7, 6, and 8, we state which two-level models imply the observable properties CA, MM, and MIIO, respectively.
Theorem 6.
-
(a) The MHM-1 implies CA-1.
-
(b) The MHM-2 implies CA-2.
Because the DMM-1 is a special case of the MHM-1, it also implies CA-1. Also, because the MHM-1, the DMM-1, and the DMM-2 are special cases of MHM-2, these models imply CA-2.
Theorem 7.
-
(a) For dichotomous items, the MHM-1 implies MM-1.
-
(b) For dichotomous items, the MHM-2 implies MM-2.
Because the DMM-1 is a special case of the MHM-1, it also implies MM-1. Also, because the MHM-1, the DMM-1, and the DMM-2 are special cases of MHM-2, these models imply MM-2. As for single-level IRT models, MM does not necessarily hold for polytomous items. However, MM-1 and MM-2 may still provide heuristic evidence for or against the MHM-1 and/or the MHM-2 (cf., Sijtsma & Van der Ark, Reference Sijtsma and Van der Ark2020, p. 151). Alternatively, if polytomous items are dichotomized, Theorem 7 holds (Junker & Sijtsma, Reference Junker and Sijtsma2000).
Theorem 8.
-
(a) The DMM-1 implies MIIO-1.
-
(b) The DMM-2 implies MIIO-2.
Because the DMM-1 is a special case of the DMM-2, it also implies CA-2.
5 Relations between models and properties
In the previous sections, we defined four models, eight ordering properties, and six observable properties. In addition, we provided proofs for which model implied which property, for the least restrictive model and strongest property possible. Because more restrictive models are special cases of models with fewer restrictions, they are defined with at least the same assumptions that imply the property (see Figure 2). Hence, more restrictive models imply the same properties as the more general models.
Table 3 provides an overview of the most important implications for each model. The MHM-1 (Table 3, first column) implies (W)SOL-1 and (W)SOL-2. Hence, the MHM-1 implies an ordinal respondent-level scale, on which respondents may be stochastically ordered on
$\Theta _{sr}$
using
$X_{sr+}$
, and an ordinal group-level scale, on which groups may be stochastically ordered on
$\Gamma _s$
using
$X_{s+}$
. Methods for investigating the model fit of the MHM-1 are MM-1, CA-1, MM-2, and CA-2. In addition to the implications by the MHM-1, the DMM-1 (Table 3, second column) also implies an ordinal item scale on which items may be stochastically ordered on their latent difficulty using the mean scores on the items. Methods MIIO-1 and MIIO-2 can be used for investigating model fit of the DMM-1 in addition to the methods of the MHM-1.
Table 3 Implied properties of the two-level nonparametric IRT models
Note: A = property is implied for dichotomous and polytomous items, D = property is implied for dichotomous or dichotomized items only.
The MHM-2 (Table 3, third column) implies (W)SOL-2. Hence, the MHM-2 implies an ordinal group-level scale on which groups may be stochastically ordered on
$\Gamma _s$
using
$X_{s+}$
. Methods for investigating the model fit of the MHM-2 are MM-2 and CA-2. In addition to the implications by the MHM-2, the DMM-2 (Table 3, fourth column) also implies an ordinal item scale on which items may be stochastically ordered on their latent difficulty using the mean scores on the items. Methods MIIO-1 and MIIO-2 can be used for investigating model fit of the DMM-2 in addition to the methods of the MHM-2.
The two-level nonparametric IRT models are defined on either or both the respondent level and the group level. Depending on the interest of the researcher, one or both levels are relevant for scaling. If the goal is to scale the respondents, it is sufficient to mainly focus on checking the respondent-level assumptions of the MHM-1 or DMM-1. If the goal is to only scale the groups, as is the case in multi-rater data, the group-level assumptions are of key interest. For example, if a group-level IRF is flat, an item does not discriminate between low and high values of
$\Gamma _s$
. Such an item does not contribute to accurate measurement on the group level. In addition, the respondent-level assumptions are informative for investigating, for example, whether the respondents may also be ordered using their sum score, or how the results relate to each other across levels. Therefore, even though investigating the MHM-2 or DMM-2 is sufficient to determine model fit at the group level, investigating the MHM-1 or the DMM-1 by checking assumptions on both level 1 and level 2 is suggested. If model violations occur at level 1, it is still possible that there are no violations at level 2, and the MHM-2 or the DMM-2 fits the data.
6 Discussion
The main contribution of this paper is the establishment of ordering properties and observable properties for two-level nonparametric IRT models. Ordering properties MLR-1, MLR-2, SOL-1.SOL-2, weak SOL-1, weak-SOL-2 SOM-1, and SOM-2 justify ordinal measurement using two-level nonparametric IRT models, in a way that is similar to ordinal measurement in the more popular single-level nonparametric IRT models. In addition, the observable properties MM-1, MM-2, CA-1, CA-2, MIIO-1, and MIIO-2 allow researchers to investigate the fit of the two-level nonparametric IRT models. Combined, these newly established ordering properties and observable properties enable the practical use of the two-level measurement models.
Building on previous work by Snijders (Reference Snijders, Boomsma, van Duijn and Snijders2001), we introduced four models for two-level test data. For level 1, we introduced the MHM-1, which allows ordering nested respondents on latent variable
$\Theta _{sr}$
using manifest variable
$X_{sr+}$
, and the DMM-1, which allows ordering nested respondents and items on
$\Theta _{sr}$
using
$X_{sr+}$
and
$E(X_{sri})$
, respectively. For level 2, we introduced the MHM-2, which allows ordering groups on latent variable
$\Gamma _s$
using manifest variable
$X_{s+}$
, and the DMM-2, which allows ordering groups and items on
$\Gamma _s$
using
$X_{s+}$
and
$E(X_{si})$
, respectively. The hierarchical relations among the four models show that the DMM-1 implies all other models and that the MHM-2 is the most general model (see Figure 2).
In addition, we derived observable data properties implied by the models, which can be used to investigate the model fit for a given data set. Specifically, we generalized the properties manifest monotonicity, conditional association, and manifest invariant item ordering for the respondent level and the group level. Theorem 7(b) showed the perhaps surprising result that, for a test consisting of dichotomous items, even though group-level item scores are not dichotomous (because they combine the item scores across respondents), still the strong results for dichotomous nonparametric IRT models hold. In deriving level-2 properties from level-1 properties, assuming the individual respondent-variables
$\Delta _{sr}$
are i.i.d. proved to be a key ingredient. Assuming i.i.d. in test data is usually based on the sampling design or data collection conditions in relation to the latent variable. However, finding support for the i.i.d. assumption based on observable properties on the group level may be a valuable topic for future research.
The properties derived in this paper apply at the population level. Koopman et al. (Reference Koopman, Zijlstra and Van der Ark2023) suggested statistical tests for MO-1, MO-2, IIO-1, and IIO-2 using observable properties MM-1, MM-2, MIIO-1, and MIIO-2, respectively. Using simulated data, these authors found that the tests for MO-1, IIO-1 and IIO-2 had satisfactory Type-1 error rates and power, whereas the tests for MO-2 had satisfactory Type-1 error rates but insufficient power (see also, Koopman, Reference Koopman, Wiberg, Molenaar, Gonzalez and Kim2023). Note that both procedures deviated slightly from the results in this paper, because they used level-2 item scores rather than the between-respondent item scores that were used in the MM-2 and MIIO-2 definitions in this paper. Perhaps these latter item scores increases the power of the significance test of MO-2.
Note that Molenaar (Reference Molenaar, van der Linden and Hambleton1997) originally defined the DMM nonintersecting item-step response functions
$P(X_i\geq x |\Theta )$
rather than an IRT model having nonintersecting item-response functions. As investigating properties of items can be considered more relevant than investigating properties of item-steps, the new definition of the DMM in terms of nonintersecting IRFs can be considered more useful. In addition, the property of IIO is defined in terms of conditional expected item scores and fits better to the new definition of the DMM than to the original definition. If there is reason to require an invariant item-step order, an alternative DMM-like model may be proposed including this assumption. However, one should realize that an invariant item-step order not necessarily implies an invariant item order (Sijtsma & Hemker, Reference Sijtsma and Hemker1998).
In this paper, we chose to expand on work by Snijders (Reference Snijders, Boomsma, van Duijn and Snijders2001) because of its strong link to the one-level MHM and DMM. However, other generalizations of the MHM and DMM are possible. Within the framework of this paper, one may also consider a within-group model, in which the IRFs are assumed to be increasing only in
$\delta _{sr}$
. Such a model may be useful if the focus is on within-group comparison only rather than comparison of all respondents, or if items contain a relative component in relation to a group aspect. Properties and applications of this model are yet unknown. Outside the framework proposed in this paper, Koopman, Zijlstra, De Rooij, and Van der Ark (Reference Koopman, Zijlstra, De Rooij and Van der Ark2020) proposed the nonparametric hierarchical rater model, a nonparametric version of the (parametric) hierarchical rater model (Patz et al., Reference Patz, Junker, Johnson and Mariano2002). Possibly other two-level parametric IRT models may be redefined as a nonparametric model, such as the multiple raters model (Verhelst & Verstralen, Reference Verhelst, Verstralen, Boomsma, van Duijn and Snijders2001) or the rater bundle model (Wilson & Hoskens, Reference Wilson and Hoskens2001). Alternatively, the nonparametric partial credit model or nonparametric sequential model (Hemker et al., Reference Hemker, Sijtsma, Molenaar and Junker1997, Reference Hemker, Van der Ark and Sijtsma2001, respectively) may be generalized to a two-level framework.
The presented models in this paper are unidimensional models. Hence, for MHM-1 and DMM-1, it is assumed that respondents across groups may be located on the same latent variable. This is quite a strict assumption and whether this is sensible should be investigated, for example, by analysis on differential item functioning (Holland & Wainer, Reference Holland and Wainer1993). Known methods within nonparametric IRT are comparing scales and scale properties across groups (Sijtsma & Van der Ark, Reference Sijtsma and Van der Ark2017; Van der Ark et al., Reference Van der Ark, Croon and Sijtsma2008) and performing an IIO analysis (Sijtsma & Junker, Reference Sijtsma and Junker1996). Two-level IRT modeling may benefit from multidimensional generalizations for developing scales that explicitly separate a respondent and group dimension. How these alternative models hierarchically relate to the models presented in this paper, and what properties they imply, is a topic for further investigation.
The developments presented in this paper are part of a larger project to make all elements of Mokken scale analysis available for two-level test data (Koopman, Zijlstra, & Van der Ark, Reference Koopman, Zijlstra and Van der Ark2020; Koopman et al., Reference Koopman, Zijlstra and Van der Ark2022). Next steps in development should be aimed at developing group-level item selection procedures and at allowing more complex research designs, such as a cross nested design in which respondents score multiple groups.
Funding statement
This study was funded by the Netherlands Organisation for Scientic Research (NWO) (grant number 406.16.554).
Competing interests
The authors declare none.
APPENDIX
Lemma A.1. UN-1 implies UN-2.
Proof. Equation 4 defines
$\Theta _{sr}=\Gamma _s+\Delta _{sr}$
, which implies
$\Gamma _s = \Theta _{sr}-\Delta _{sr}$
. However, as
$\Gamma _s$
does not depend on r,
$\Gamma _s = E(\Theta _{sr}-\Delta _{sr}) = E(\Theta _{sr})-E(\Delta _{sr})$
, where
$E(\Theta _{sr})$
denotes the expectation of
$\Theta _{sr}$
over the respondents in randomly selected s. Because
$E(\Delta _{sr})=0$
, it follows that
$\Gamma _s = E(\Theta _{sr})$
within group s. If
$\Theta _{sr}$
is unidimensional, its expectation
$E(\Theta _{sr})$
is also unidimensional. Variable
$\Theta _{sr}$
is unidimensional by UN-1; hence, variable
$\Gamma _s$
is also unidimensional.
Lemma A.2. MO-1 implies MO-2.
Proof. Let
$P(\Delta _{sr})$
denote the probability density function of the distribution of
$\Delta _{sr}$
. By H, the group-level item-step response function is
$$ \begin{align} \begin{array}{lcl} {P(X_{sri} \geq x | \Gamma_s)} &=& {E[P(X_{sri} \geq x | \Theta_{sr}) | \Gamma_s]} \text{ (Equation 7)} \\ &=&\int {P(X_{sri} \geq x | \Theta_{sr})} P(\Delta_{sr}|\Gamma_s)d{\Delta_{sr}}. \\ \end{array} \end{align} $$
As
$\Delta _{sr}$
and
$\Gamma _s$
are independent by B,
$P(\Delta _{sr}|\Gamma _s)=P(\Delta _{sr})$
, and the last term of Equation A.1 reduces to
$$ \begin{align} \begin{array}{lcl} \int {P(X_{sri} \geq x | \Theta_{sr})} P(\Delta_{sr})d{\Delta_{sr}}. \\ \end{array} \end{align} $$
By MO-1,
$P(X_{sri} \geq x | \Theta _{sr}=\theta _{sr})$
is nondecreasing in
$\theta _{sr}$
. Hence, Equation A.1 is nondecreasing in
$\gamma _s$
, which equals the definition of MO-2.
Lemma A.3. IIO-1 implies IIO-2.
Proof. By IIO-1
$$ \begin{align} \begin{array}{llcll} &E(X_{sri}|\Theta_{sr}=\theta_{sr}) &\leq &E(X_{srj}|\Theta_{sr}=\theta_{sr}) \text{ for all }\theta_{sr}\\ \Leftrightarrow&E_i(\Theta_{sr})&\leq& E_j(\Theta_{sr})\\ \Leftrightarrow & \int E_i(\Theta_{sr})P(\Delta_{sr}|\Gamma_s)d{\Delta_{sr}} &\leq &\int E_j(\Theta_{sr})P(\Delta_{sr}|\Gamma_s)d{\Delta_{sr}} \\ \Leftrightarrow & E[E_i(\Theta_{sr})|\Gamma_s] & \leq & E[E_j(\Theta_{sr})|\Gamma_s] \\ \Leftrightarrow & \mathcal{E}_i(\Gamma_s) & \leq & \mathcal{E}_j(\Gamma_s) \text{ ({by H,} Eq. 8)} \\ \Leftrightarrow &E(X_{sri}|\Gamma_s=\gamma_s) &\leq& E(X_{srj}|\Gamma_s=\gamma_s) \text{ for all }\gamma_s\\ \end{array}. \end{align} $$
The final result in Equation A.3 equals the definition of IIO-2.
Lemma A.4. MHM-1 implies that
$E(g(\mathbf {X}_{sr})|\Theta _{sr}=\theta _{sr})$
is nondecreasing in
$\theta _{sr}$
for any bounded, nondecreasing function
$g(\cdot )$
.
Proof. By LI-1, scores
$X_{sri}$
within
$\mathbf {X}_{sr}$
are independent given
$\Theta _{sr}$
. By MO-1,
$X_{sri}$
is stochastically ordered in
$\Theta _{sr}$
; that is, for
$t < u$
,
$P(X_{sri}\geq x | \Theta _{sr}= t) \leq P(X_{sri}\geq x | \Theta _{sr} = u)$
for all i and all x. For a set of independent variables the stochastic ordering is preserved under convolutions, for any bounded, nondecreasing function
$g(\cdot )$
(Shanthikumar e.g., Reference Shaked and Shanthikumar2007, Theorem 1.A.3(b); see also Ahmed et al. Reference Ahmed, Leon and Proschan1981, Lemma 3.3; Holland and Rosenbaum Reference Holland and Rosenbaum1986, Lemma 2). Hence
$$ \begin{align} E[g(\mathbf{X}_{sr})|\Theta_{sr} = t] \leq E[g(\mathbf{X}_{sr})|\Theta_{sr} = u]. \end{align} $$
Lemma A.5. MHM-2 implies
$E(g(\mathbf {X}_s)|\Gamma _s=\gamma _s)$
is nondecreasing in
$\gamma _s$
for any bounded, nondecreasing function
$g(\cdot )$
.
Proof. Assumptions UN-2, LI-2, and MO-2 are equivalent to Rosenbaum’s (Reference Rosenbaum1988) assumptions (1), (6), and (7), respectively, which collectively define the item-bundel model. In his Lemma 1, Rosenbaum showed that for any bounded, nondecreasing function
$g(\cdot )$
for which UN-2, LI-2, and MO-2 holds,
$E(g(\mathbf {X}_s)|\Gamma _s=\gamma _s)$
is nondecreasing in
$\gamma _s$
(see, also, Kamae et al., Reference Kamae, Krengel and O’Brien1977, Proposition 1).
Proof of Theorem 1. (B implies LI-1 and LI-2)
Proof. The independence of the
$\Gamma _s$
,
$\Delta _{sr}$
, and
$\varepsilon _{sr}$
by B implies that the
$\varepsilon _{sr}$
are independent given
$\Gamma _s+\Delta _{sr}=\Theta _{sr}$
, and that the (
$\Delta _{sr}$
,
$\varepsilon _{sr}$
) are independent given
$\Gamma _s$
. This, combined with
$X_{sri}=f_i(\Gamma _s+\Delta _{sr}, \varepsilon _{sr})$
, implies LI-1 and LI-2 in the following way. For each
$(s,r)$
, given
$\theta _{sr}$
, the
$X_{sri}$
are a function of
$\varepsilon _{sri}$
. Because for each
$(s,r)$
, the
$\varepsilon _{sri}$
are independent given
$\theta _{sr}$
, the
$X_{sri}$
are independent given
$\theta _{sr}$
, and LI-1 is implied. Furthermore, the
$\Delta _{sr},\varepsilon _{sri}$
are independent given
$\Gamma _s$
. Because
$X_{sri}$
are a function of
$(\Gamma _s+\Delta _{sr},\varepsilon _{sri})$
, given
$\Gamma _s$
the
$X_{sri}$
are a function of
$(\Delta _{sr},\varepsilon _{sri})$
. Hence,
$\mathbf {X}_{sr}$
are independent given
$\Gamma _s$
, and LI-2 is implied.
Proof of Theorem 2. (Under B and H, UN-1, MO-1, and IIO-1 imply UN-2, MO-2, and IIO-2, respectively.)
Proof. First, we consider the extreme case of no respondent variance: If var
$(\Delta _{sr})$
= 0, then
$\Delta _{sr} = 0$
and
$\Theta _{sr} = \Gamma _s$
for all r and all s. As a result,
$P(X_{sri} = x|\Theta _{sr})=P(X_{sri} = x|\Gamma _s)$
,
$P(X_{sri}\geq x|\Theta _{sr})=P(X_{sri}\geq x|\Gamma _s)$
, and
$E(X_{sri}|\Theta _{sr}=\theta _{sr}) = E(X_{sri}|\Gamma _s=\gamma _s)$
. Hence, UN-1 = UN-2 (Definiton 1), MO-1 = MO-2 (Definition 3), and IIO-1 = IIO-2 (Definition 4). Second, for var
$(\Delta _{sr})> 0$
, Lemma A.1 proves that UN-1 implies UN-2, Lemma A.2 proves that MO-1 implies MO-2, and Lemma A.3 proves that IIO-1 implies IIO-2.
Proof of Theorem 3.
(For dichotomous item scores (a) the MHM-1 implies MLR-1 and (b) for
$R_s\geq I$
, the MHM-2 implies MLR-2.)
Proof.
-
(a) The assumptions in MHM-1 are identical to the assumptions used by Grayson (Reference Grayson1988, Theorem 2) and Huynh, Reference Huynh1994 to establish MLR of the sum score in
$\Theta _{sr}$
, hence their proof can be applied. -
(b) For clarity, we give the proof for
$R=I$
, but it can straightforwardly be generalized for
$R>I$
. Let
$D = R!$
be the number of ways that respondents
$1, \ldots , R$
can be ordered, and let
$d (d = 1, \ldots , D)$
be an index of possible respondent orderings. Furthermore, let
$d_r (r = 1, \ldots , R)$
denote the the position of respondent r in respondent-ordering d. For
$R>I$
, the same method can be applied, but each permutation contains only I respondents, hence the number of permutations
$D = \frac {R!}{(R - I)!}$
.Let
$X^d_{s+} = X_{sd_{1}1} + X_{sd_{2}2} + ... + X_{sd_{r}r} + ... + X_{sd_{R}R}$
denote the group-level sum score in which each item score is taken from a different respondent, with realization
$x_{s+}^d$
. Let
$\mathbf {X}_{s}^d= (X_{s1}^d, X_{s2}^d, \ldots , X_{sI}^d)$
denote the vector of item scores from the respondent order from the dth permutation, with realization
$\mathbf {x}_{s}^d$
. For a given permutation *, let
$\sum _{\{{\mathbf {x}}_{s}^{*}| \mathbf {x}_{s}^{*'}\mathbf {1}=x_{s+}^*\}}$
denote the sum over all possible patterns of I item scores that sum to
$x_{s+}^*$
. Let
$P_i(\gamma _s)=P(X_{sri}~=~1|\Gamma _s~=~\gamma _s)$
and let
$Q_i(\gamma _s)=1-P_i(\gamma _s)$
. By LI-2, for
$r\neq p$
, item scores
${X}_{sri}$
and
${X}_{spj}$
are independent conditional on
$\gamma _s$
. Hence, for dichotomous items, the probability of obtaining group-level sum score
$x_{s+}^*$
is (A.5)Because
$$ \begin{align} \begin{array}{lcl} P(X_{s+}^*=x_{s+}^*|\Gamma_s=\gamma_s) &= & \sum_{\{\mathbf{x}_{s}^*| \mathbf{x}_{s}^{*'}\mathbf{1}=x_{s+}^*\}} \prod_{i=1}^I P_i(\gamma_s)^{x_{si}^*}Q_i(\gamma_s)^{(1-x_{si}^*)} \\ & = & \sum_{\{\mathbf{x}_{s}^*| \mathbf{x}_{s}^{*'}\mathbf{1}=x_{s+}^*\}} \prod_{i=1}^I Q_{i}(\gamma_s)[\frac{P_i(\gamma_s)}{Q_i(\gamma_s)}]^{x_{si}^*}. \end{array} \end{align} $$
$\prod _{i=1}^I Q_i(\gamma _s)$
is constant across each item-score pattern
$\mathbf {x}$
, Equation A.5 is identical to (A.6)The form of the right-hand side in Equation A.6 is equal to the form used by Grayson (Reference Grayson1988, Theorem 2) and Huynh, (Reference Huynh1994). Hence, their methods can be applied to establish MLR of the sum score
$$ \begin{align} \begin{array}{lcl} P(X_{s+}^*=x_{s+}^*|\Gamma_s=\gamma_s) &=& \prod_{i=1}^I Q_i(\gamma_s)\sum_{\{\mathbf{x}_{s}^*| \mathbf{x}_{s}^{*'}\mathbf{1}=x_{s+}^*\}} \prod_{i=1}^{I} \big[\frac{P_i(\gamma_s)}{Q_i(\gamma_s)}\big]^{x_{si}^*}.\\ \end{array} \end{align} $$
$X_{s+}^*$
in
$\gamma _s$
. Because
$E(X_{si}^* |\Gamma _s=\gamma _s) = E(X_{sri} |\Gamma _s=\gamma _s)$
, (A.7)it follows that MLR also holds for
$$ \begin{align} \begin{array}{ccl} E(X_{s+}^*| \Gamma_s=\gamma_s) &=& \sum_{i=1}^I E(X_{si}^* |\Gamma_s=\gamma_s) \\ &=& \sum_{i=1}^I E(X_{sri} |\Gamma_s=\gamma_s) \\ &=& E(X_{s+}|\Gamma_s=\gamma_s), \\ \end{array} \end{align} $$
$X_{s+}$
in
$\Gamma _s$
.
Proof of Theorem 4. ((a) The MHM-1 implies SOM-1 and (b) the MHM-2 implies SOM-2.)
Proof.
-
(a) SOM-1 follows from the general result presented in Lemma A.4. First, note that the respondent-level sum score
$X_{sr+}$
is a nondecreasing function of
$\mathbf {X}_{sr}$
(e.g., Rosenbaum, Reference Rosenbaum1984). Hence, by Lemma A.4,
$X_{sr+}$
is nondecreasing in
$\theta _{sr}$
, which is the definition of SOM-1. -
(b) SOM-2 follows from the general result presented in Lemma A.5. First, note that the group-level sum score
$X_{s+}$
is a nondecreasing function of
$\mathbf {X}_{s}$
. Hence, by Lemma A.5,
$X_{s+}$
is nondecreasing in
$\gamma _s$
, which is the definition of SOM-2.
Proof of Theorem 5. ((a) The MHM-1 implies WSOL-1 and (b) the MHM-2 implies WSOL-2.)
Proof.
-
(a) MHM-1 implies SOM-1 of
$X_{sr+}$
by
$\Theta _{sr}$
(Theorem 4(a)). Van der Ark & Bergsma, (Reference Van der Ark and Bergsma2010, Theorem) showed that SOM implies WSOL, hence WSOL of
$\Theta _{sr}$
by
$X_{sr+}$
is implied. -
(b) Similar to the proof in (a), the MHM-2 implies SOM-2 of
$X_{s+}$
by
$\Gamma _s$
, hence, WSOL of
$\Gamma _s$
by
$X_{s+}$
is implied.
Proof of Theorem 6. (For dichotomous items (a) the MHM-1 implies MM-1 and (b) the MHM-2 implies MM-2.)
Proof.
-
(a) The proof is analogous to the proof in Proposition 4.1a Junker, (Reference Junker1993). By the law of total expectation (e.g., Rice, Reference Rice2006, p 149) and LI-1
(A.8)
$$ \begin{align} \begin{array}{llll} E(X_{sri}|X_{sr(i)}) & = & E[E(X_{sri}|X_{sr(i)},\Theta_{sr})|X_{sr(i)}] & \\ & = & E[E_i(\Theta_{sr})|X_{sr(i)}]. & \\ \end{array} \end{align} $$
For dichotomous items, under the MHM-1, by Theorem 3(a),
$\Theta _{sr}$
is nondecreasing in
$X_{sr(i)}$
(SOL). Because
$\Theta _{sr}$
is stochastically ordered in
$X_{sr(i)}$
, so is any nondecreasing function of
$\Theta _{sr}$
, such as
$E_i(\Theta _{sr})$
(Equation 6; Shaked & Shanthikumar Reference Shaked and Shanthikumar2007, Theorem 1.A.3.(a)), which completes the proof. -
(a) This proof is parallel to the proof in (a), which holds when substituting
$\Theta _{sr}$
by
$\Gamma _s$
,
$X_{sr(i)}$
by
$X_{s({r,}i)}$
,
$E_i(\cdot )$
by
$\mathcal {E}(\cdot )$
, LI-1 by LI-2, and MHM-1 by MHM-2.
Proof of Theorem 7. ((a) The MHM-1 implies CA-1 and (b) the MHM-2 implies CA-2.)
Proof.
-
(a) The proof is similar to the proof of Theorem 1 by (Rosenbaum, Reference Rosenbaum1984; see also Holland and Rosenbaum (Reference Holland and Rosenbaum1986, Theorem 6). If CA-1 holds, the conditional covariance
$Cov[g_1(\mathbf {Y}_{sr}),g_2(\mathbf {Y}_{sr})|h(\mathbf {Z}_{sr})]\geq 0$
(Definition 9). Using standard algebra, it can be shown that this statement is equivalent to (A.9)(e.g., Rice, Reference Rice2006, p. 138). Hence, we prove that under the MHM-1 Equation A.9 holds. By the law of total expectation,
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] \geq E[g_1(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})]E[g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})], \end{align} $$
(A.10)(e.g., Rice, Reference Rice2006, p. 138). By LI-1,
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] = E\{E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}),\theta]|h(\mathbf{Z}_{sr})\}, \end{align} $$
$\mathbf {Y}_{sr}$
and
$\mathbf {Z}_{sr}$
are independent given
$\theta _{sr}$
. Hence, (A.11)
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] = E\{E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|\theta_{sr}]|h(\mathbf{Z}_{sr})\}. \end{align} $$
Because, by LI-1, the values in
$\mathbf {Y}_{sr}$
are independent given
$\theta _{sr}$
, they are associated (Esary et al., Reference Esary, Proschan and Walkup1967, Theorem 2.1). Therefore, (A.12)
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] \geq E\{E[g_1(\mathbf{Y}_{sr})|\theta_{sr}]E[g_2(\mathbf{Y}_{sr})|\theta_{sr}]|h(\mathbf{Z}_{sr})\}. \end{align} $$
By Lemma A.4,
$E(g_1(\mathbf {Y}_{sr}|\theta _{sr})$
and
$E(g_2(\mathbf {Y}_{sr}|\theta _{sr})$
are nondecreasing in
$\theta _{sr}$
, hence, they are associated (Esary et al., Reference Esary, Proschan and Walkup1967,
$P_4$
). In addition, by UN-1,
$\theta _{sr}$
is a scalar and therefore associated (Esary et al., Reference Esary, Proschan and Walkup1967,
$P_3$
). Hence, it follows that (A.13)
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] \geq E\{E[g_1(\mathbf{Y}_{sr})|\theta_{sr}]|h(\mathbf{Z}_{sr})\}E\{E[g_2(\mathbf{Y}_{sr})|\theta_{sr}]|h(\mathbf{Z}_{sr})\}. \end{align} $$
By the law of total expectation, the statement in Equation A.13 is equivalent to
(A.14)which completes the proof.
$$ \begin{align} E[g_1(\mathbf{Y}_{sr})g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})] \geq E[g_1(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})]E[g_2(\mathbf{Y}_{sr})|h(\mathbf{Z}_{sr})], \end{align} $$
-
(b) The proof is similar to the proof in (a). Throughout the proof,
$g_1(\mathbf {Y}_{sr})$
is kept the same, but
$g_2(\mathbf {Y}_{sr})$
is replaced by
$g_2(\mathbf {Y}_{sp})$
, with
$r\neq p$
. Hence,
$g_1$
and
$g_2$
apply to different respondents within the same group. Furthermore,
$\mathbf {Z}_{sr}$
is replaced by
$\mathbf {Z}_{s(rp)}$
, hence to the vector that contains all item scores in group s, except the scores of respondents r and p. Finally,
$\theta _{sr}$
is replaced by
$\gamma _s$
, LI-1 by LI-2, and Lemma A.4 by Lemma A.5, which gives the proof for (b) (see, also, Rosenbaum, Reference Rosenbaum1988).
Proof of Theorem 8. ((a) The DMM-1 implies MIIO-1 and (b) the DMM-2 implies MIIO-2.)
Proof.
-
(a) This proof is similar to the proof of the Corollary by Ligtvoet et al., (Reference Ligtvoet, Van der Ark, Bergsma and Sijtsma2011). By IIO-1,
$E(X_{sri}|\Theta _{sr}=\theta _{sr}) \leq E(X_{srj}|\Theta _{sr}=\theta _{sr})$
. By Equation 5,
$E(X_{sri}|\Theta _{sr}=\theta _{sr}) \leq E(X_{srj}|\Theta _{sr}=\theta _{sr})$
equals(A.15)By LI-1,
$X_{sri}$
, and
$X_{sr(ij)}$
are independent given
$\theta _{sr}$
, and their joint probability equals the product of their marginal conditional probabilities (e.g., Rice, Reference Rice2006, p. 84). Hence, Equation A15 equals (A.16)
$$ \begin{align} \begin{array}{lrlll} & \sum_{x=1}^m P(X_{sri}\geq x, X_{sr(ij)}=y|\Theta_{sr}=\theta_{sr}) &\leq& \sum_{x=1}^m P(X_{srj}\geq x, X_{sr(ij)}=y|\Theta_{sr}=\theta_{sr}). & \end{array} \end{align} $$
Let
$F(\Theta _{sr})$
denote the cumulative distribution function of
$\Theta _{sr}$
. Integrating both sides of Equation A.16 over
$\Theta _{sr}$
yields (A.17)for all y and all
$i<j$
, completing the proof.
-
(b) The proof is parallel to the proof in (a). Replacing
$\Theta _{sr}$
by
$\Gamma _s$
,
$\theta _{sr}$
by
$\gamma _s$
, IIO-1 by IIO-2, LI-1 by LI-2,
$X_{sr(ij)}$
by
$X_{s({r,}ij)}$
, proofs that the MHM-2 implies MIIO-2.
Open access
