2.1 Model Setup of CDMs with Binary Responses and Attributes

Consider a CDM where each subject is associated with J observed variables ${\mathbf {R}} = (R_1,\ldots ,R_J)$ as responses to J test items and K latent attributes ${\mathbf {A}}=(A_1,\ldots ,A_K)$ . We start by considering the most commonly used CDMs with binary responses ${\mathbf {R}}\in \{0,1\}^J$ and binary attributes ${\mathbf {A}}\in \{0,1\}^K$ , and later will generalize our result to polytomous responses and attributes. A key structure in a CDM is the Q-matrix introduced in Tatsuoka (Reference Tatsuoka1983), which specifies the relationship between the observed responses and the latent attributes. The Q-matrix is a $J\times K$ matrix ${\mathbf {Q}}=(q_{jk})$ with binary entries, with rows indexed by the J items and columns by the K attributes. The entry $q_{jk}=1$ or $0$ indicates whether or not the jth item requires/measures the kth latent attribute. The row vectors of the Q-matrix are also called the $\mathbf q$ -vectors and denoted by ${\mathbf {q}}_j$ for each $j\in [J]$ . Here, for any positive integer M, we denote $[M]=\{1,2,\ldots ,M\}$ .

To model the conditional distribution of the observed ${\mathbf {R}}$ given the latent ${\mathbf {A}}$ , we use the general notation of the item parameters

$$ \begin{align*} \theta_{j,\boldsymbol{\alpha}} = {\mathbb{P}}(R_j=1\mid {\mathbf{A}}=\boldsymbol{\alpha}),\quad j=1,\ldots,J;\quad\boldsymbol{\alpha}\in\{0,1\}^K, \end{align*} $$

and denote the collection of them by $\boldsymbol \Theta $ . These item parameters are subject to certain equality constraints imposed by the Q-matrix in different ways under different models; see Examples 1 and 2 for concrete examples. For the latent attributes, we consider the most commonly adopted saturated attribute model with proportion parameters ${\mathbf {p}}=(p_{\boldsymbol {\alpha }}; \boldsymbol {\alpha }\in \{0,1\}^K)$ , where

$$ \begin{align*} p_{\boldsymbol{\alpha}} = {\mathbb{P}}({\mathbf{A}}=\boldsymbol{\alpha}),\quad \forall\boldsymbol{\alpha}\in\{0,1\}^K. \end{align*} $$

The proportion parameters ${\mathbf {p}}$ satisfy that $p_{\boldsymbol {\alpha }}\geq 0$ for all $\boldsymbol {\alpha }$ and $\sum _{\boldsymbol {\alpha }\in \{0,1\}^K} p_{\boldsymbol {\alpha }}=1$ . We use $\mathbf e_k$ to denote the K-dimensional canonical basis vector with an “1” in the kth entry and “0” in all other entries.

Under the above general setup, various CDMs were proposed with different diagnostic modeling assumptions and different link functions. In this work, we focus on the flexible main-effect or all-saturated-effect CDMs, which are also called the multi-parameter restricted latent class models in Gu and Xu (Reference Gu and Xu2021) and encompass a wide range of diagnostic models.

In the following, we review concrete examples of multi-parameter CDMs in the following.

Example 1 (Main-Effect Cognitive Diagnosis Models).

An important family of cognitive diagnosis models assumes that the $\theta _{j,\boldsymbol {\alpha }}$ depends on the main effects of those attributes required by item j, but not their interactions. This family includes the popular reduced Reparameterized Unified Model (rRUM; DiBello et al., Reference DiBello, Stout and Roussos2012), Additive Cognitive Diagnosis Models (ACDM; de la Torre, Reference de la Torre2011), the Linear Logistic Model (LLM; Maris, Reference Maris1999), and the General Diagnostic Model (GDM; von Davier, Reference von Davier2008). We call them the Main-Effect Cognitive Diagnosis Models. In particular, under the rRUM,

$$ \begin{align*}{\mathbb{P}}^{\,\mathrm{rRUM}}(R_j = 1\mid {\mathbf{A}}=\boldsymbol{\alpha},Q)= \theta^+_j \prod_{k=1}^K r_{jk}^{q_{jk}(1-\alpha_k)}, \end{align*} $$

where $\theta ^+_j={\mathbb {P}}(R_j=1 \mid A_k\geq q_{jk} \text { for all }k=1,\ldots ,K)$ represents the positive response probability of a capable subject of item j (i.e., a subject mastering all required skills of item j), and $r_{jk}\in (0,1)$ is the parameter penalizing not mastering attribute k required by item j. Equivalently, the item parameter in the rRUM can be written as $\log \theta ^{\,rRUM}_{j,\boldsymbol {\alpha }} = \beta _{j0} + \sum _{k=1}^K\beta _{jk}q_{jk}\alpha _k$ . Similarly, the ACDM assumes the parameter $\theta _{j,\boldsymbol {\alpha }}$ can be written as a linear combination of the main effects of the required attributes:

$$ \begin{align*}{\mathbb{P}}^{\,\mathrm{ACDM}}(R_j = 1\mid {\mathbf{A}}=\boldsymbol{\alpha},Q) = \beta_{j0} + \sum_{k=1}^K \beta_{jk}q_{jk}\alpha_k. \end{align*} $$

The Linear Logistic Model assumes a logistic link function with

$$ \begin{align*}{\mathbb{P}}^{\,\mathrm{LLM}}(R_j = 1\mid {\mathbf{A}}=\boldsymbol{\alpha},Q) = \sigma\Big(\beta_{j0} + \sum_{k=1}^K\beta_{jk} q_{jk}\alpha_k\Big), \end{align*} $$

where $\sigma (x) = 1/(1+e^{-x})$ is the logistic function.

Example 2 (All-saturated-effect Cognitive Diagnosis Models).

Another popular type of cognitive diagnostic model assumes that the positive response probability depends on the main effects and all of the interaction effects of the required attributes of the item. We call these models all-saturated-effect cognitive diagnosis models. The GDINA model (de la Torre, Reference de la Torre2011), the log-linear cognitive diagnosis models (LCDM; Henson et al., Reference Henson, Templin and Willse2009), and the general diagnostic model (GDM; von Davier, Reference von Davier2008) are popular examples of all-saturated-effect models. The item parameters (conditional positive response probabilities) under these models are

(1) $$ \begin{align} {\mathbb{P}}^{\,\mathrm{All-Effect}} (R_j = 1\mid {\mathbf{A}}=\boldsymbol{\alpha}) =&~ f\Big(\beta_{j0} + \sum_{k=1}^K\beta_{jk} q_{jk}\alpha_k + \sum_{1\leq k_1<k_2 \leq K} \beta_{j,\{k_1,k_2\}} q_{j,k_1} q_{j,k_2} \alpha_{k_1} \alpha_{k_2} \\ &~ + \cdots + \beta_{j,\{1,2,\cdots, K\}} \prod_{k=1}^K q_{jk} \prod_{k=1}^K \alpha_{k}\Big).\nonumber \end{align} $$

When f is the identity link $f(x)=x$ , the above gives the GDINA model, and when f is the inverse logit link $f(x) = \sigma (x) = 1/(1+e^{-x})$ , the above gives the LCDM. von Davier (Reference von Davier2014) showed that the GDINA and LCDM can be rewritten as GDMs with an extended skill space.

The local independence assumption is usually imposed in CDMs, meaning that the observed responses $R_1,\ldots ,R_J$ are conditionally independent given the latent attributes ${\mathbf {A}}$ . With this assumption, we can write the joint distribution of the observed ${\mathbf {R}}$ by marginalizing out the latent ${\mathbf {A}}$ in a CDM with binary responses and binary attributes:

(2) $$ \begin{align} {\mathbb{P}}(R_1 = r_1, \ldots, R_J = r_J\mid Q,\boldsymbol \Theta,{\mathbf{p}}) &= \sum_{\boldsymbol{\alpha}\in\{0,1\}^K} {\mathbb{P}}({\mathbf{A}} = \boldsymbol{\alpha}) \prod_{j=1}^J {\mathbb{P}}(R_j = r_j\mid {\mathbf{A}}=\boldsymbol{\alpha},Q)\\ &= \sum_{\boldsymbol{\alpha}\in\{0,1\}^K} p_{\boldsymbol{\alpha}} \prod_{j=1}^J \theta_{j,\boldsymbol{\alpha}}^{r_j} (1-\theta_{j,\boldsymbol{\alpha}})^{1-r_j}, \nonumber \end{align} $$

for any response pattern $\mathbf r=(r_1,\ldots ,r_J)\in \{0,1\}^J$ , where the Q-matrix-induced constraints on parameters $\theta _{j,\boldsymbol {\alpha }}$ are made implicit. The Q-matrix is said to be identifiable, if it can be uniquely identified from the population distribution of ${\mathbf {R}}$ in (2) up to the column permutation of Q (Liu et al., Reference Liu, Xu and Ying2013).

2.2 Population Distribution Tensor under a CDM

Studying the identifiability in CDMs often requires one to take the tensor perspective. This subsection introduces the population tensor under a CDM and explains its importance in studying model identifiability.

We first introduce the notation of the population distribution tensor summarizing the population distribution of the observed response vector $\mathbf R$ . The distribution of ${\mathbf {R}}$ in (2) can be characterized by a Jth-order tensor $\mathcal T = (t_{r_1,r_2,\ldots ,r_J})$ of size $2\times 2 \times \cdots \times 2$ with entries

(3) $$ \begin{align} t_{r_1,r_2,\ldots,r_J} = {\mathbb{P}}(R_1 = r_1, \ldots, R_J = r_J\mid Q,\boldsymbol \Theta,{\mathbf{p}}),\quad \forall r_1,r_2,\ldots,r_J\in\{0,1\} \end{align} $$

denotes the marginal probability of the response pattern $\mathbf r = (r_1,r_2,\ldots ,r_J)$ under the specified CDM. Tensor $\mathcal T$ is said to have J modes indexed by the items $j=1,\ldots ,J$ . We call $\mathcal T$ the population distribution tensor of ${\mathbf {R}}$ . For a vector ${\mathbf {R}}=(R_1,\ldots , R_M)$ and a subset $S\subseteq [M]$ , denote ${\mathbf {R}}_S = (R_j; j\in S)$ as a $|S|$ -dimensional subvector of ${\mathbf {R}}$ that collects all entries indexed by S.

When establishing the identifiability of the Q-matrix or continuous parameters in main-effect or all-saturated-effect CDMs, previous proof approaches based on Kruskal’s Theorem often involve a step of rewriting this Jth-order tensor $\mathcal T$ into a three-way tensor (third-order tensor) by concatenating certain modes together (see, e.g. Culpepper, Reference Culpepper2019; Fang et al., Reference Fang, Liu and Ying2019). Even for studies that do not explicitly invoke Kruskal’s Theorem (see, e.g. Gu and Xu, Reference Gu and Xu2020; Xu, Reference Xu2017; Xu and Shang, Reference Xu and Shang2018), the identifiability proofs still implicitly partition the J binary responses into three subsets and investigate the polynomial moment equations arising from this partitioning. In this work, we take a fundamentally different approach but still investigates the population tensor $\mathcal T$ . Our approach avoids looking at three-way tensor decompositions but examines two-way tensor unfoldings and hence delivers a strictly more relaxed identifiability condition for multiparameter CDMs than existing studies; the next subsection presents details of the tensor unfolding.

2.3 Tensor Unfoldings

This subsection introduces the key tensor unfolding idea behind our new identifiability proof. Unfolding or flattening a tensor (third-order or higher-order array) means reshaping it into a lower-order tensor or a matrix (Kolda and Bader, Reference Kolda and Bader2009). To prove the identifiability result for the Q-matrix, we show that it suffices to consider unfolding the Jth-order tensor ${\mathcal {T}}$ to various matrices. For this purpose, define the row group of indices as $S\subseteq [J]$ and the column group as $[J]\setminus S$ . We denote the matrix resulting from the unfolded tensor as

(4) $$ \begin{align} \texttt{unfold}(\mathcal T;~S,~[J]\setminus S) =:[{\mathcal{T}}]_{S,:}, \end{align} $$

which is a $2^{|S|}\times 2^{J-|S|}$ matrix with rows indexed by the $2^{|S|}$ configurations of ${\mathbf {R}}_{S} \in \{0,1\}^{|S|}$ and columns by the $2^{J-|S|}$ configurations of ${\mathbf {R}}_{[J]\setminus S}\in \{0,1\}^{J-|S|}$ . The entries in $[{\mathcal {T}}]_{S,:}$ are still those marginal response probabilities $t_{r_1,\ldots ,r_J}$ in (3). A rather surprising fact that we will uncover shortly is that the ranks of such matrices (4) contain rich information about the unknown Q-matrix. For example, when S contains two items, the rank of $[\mathcal T]_{S,:}$ reflects whether these two items solely measure the same latent attribute; see Section 2.4 for details. So, we will strategically unfold the tensor ${\mathcal {T}}$ and use the rank properties of the resulting matrices as certificates to identify and recover the unknown Q-matrix.

We also introduce the notion of marginal tensor and marginal unfolding. Given a subset $S\subseteq [J]$ , define the marginal probability tensor for ${\mathbf {R}}_S$ as a $|S|$ -way tensor $\texttt {marginal}(\mathcal T, S)$ with entries specifying the joint PMF of the random vector ${\mathbf {R}}_S$ . This marginal tensor can be obtained from the aforementioned full tensor $\mathcal T$ by appropriately summing up its entries: each new entry in $[\mathcal T]_{S}$ takes the form of $\sum _{j\not \in S}\sum _{r_j=0,1}t_{r_1,\ldots ,r_J}$ . So, $\mathcal T$ and S uniquely defines the marginal probability tensor for ${\mathbf {R}}_S$ and we denote it by $\texttt {marginal}(\mathcal T, S).$ For two disjoint index sets $S_1, S_2\subseteq [J]$ whose union does not equal $[J]$ , we define a marginal unfolding

$$ \begin{align*}\texttt{unfold}(\texttt{marginal}(\mathcal T, S_1\cup S_2),\; S_1,\; S_2) =: [\mathcal T]_{S_1, S_2}, \end{align*} $$

which characterizes the joint probability table between random vectors ${\mathbf {R}}_{S_1}$ and ${\mathbf {R}}_{S_2}$ . The entries in $[\mathcal T]_{S_1, S_2}$ are summations of $t_{r_1,\ldots ,r_J}$ in the form of $\sum _{j\not \in S_1\cup S_2}\sum _{r_j=0,1}t_{r_1,\ldots ,r_J}$ , corresponding to the PMF of ${\mathbf {R}}_{S_1\cup S_2}$ by marginalizing out other random variables ${\mathbf {R}}_{[J]\setminus (S_1\cup S_2)}$ . We introduce the marginal tensor $[{\mathcal {T}}]_{S_1, S_2}$ in order to focus on the items belonging to $S_1\cup S_2$ and exclude the effects of latent attributes not manifested in $S_1\cup S_2$ .

We introduce the definition of a conditional probability table (CPT). Let ${\mathbb {P}}(R_j\mid {\mathbf {A}}_{L})$ denote the CPT that specifies the conditional distribution of $R_j$ given the latent attributes $A_k$ for $k\in L$ . In the general case where each observed $R_j$ has C categories, the CPT ${\mathbb {P}}(R_j\mid {\mathbf {A}}_{L})$ is a matrix with size $C\times 2^{|L|}$ . Its $2^{|L|}$ columns are indexed by all possible configurations of the latent vector ${\mathbf {A}}_{L}$ ranging in $\{0,1\}^{|L|}$ , and its C rows are indexed by the C categories of $R_j$ in $\{0,1,\ldots ,C-1\}$ .

Finally, we introduce a useful notation of the joint probability table between two random vectors. Consider general categorical latent $A_1,\ldots ,A_K\in \{0,1\}$ . For two sets $S_1, S_2\subseteq [K]$ , we describe the joint distribution of ${\mathbf {A}}_{S_1}=(A_k)_{k\in S_1}$ and ${\mathbf {A}}_{S_2}=(A_k)_{k\in S_2}$ using

$$ \begin{align*}{\mathbb{P}}({\mathbf{A}}_{S_1}, {\mathbf{A}}_{S_2}): \text{ a } 2^{|S_1|} \times 2^{|S_2|} \text{ matrix}. \end{align*} $$

The rows of ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_2})$ are indexed by the configurations of ${\mathbf {A}}_{S_1}\in \{0,1\}^{|S_1|}$ , and columns by those of ${\mathbf {A}}_{S_2}\in \{0,1\}^{|S_2|}$ . Importantly, we emphasize that $S_1$ and $S_2$ do not need to be disjoint and can overlap. At a high level, allowing overlapping sets will enable us to study the rank properties of unfoldings potentially involving multi-attribute items. When $S_1\cap S_2\neq \varnothing $ , ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_2})$ has some zero entries corresponding to impossible configurations of ${\mathbf {A}}_{S_1\cup S_2}$ . An extreme example is if $S_1=S_2$ , then ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_1})$ is a $2^{|S_1|} \times 2^{|S_1|}$ diagonal matrix, because ${\mathbb {P}}({\mathbf {A}}_{S_1}=\mathbf a, {\mathbf {A}}_{S_1}=\mathbf b) = 0$ for any $\mathbf a\neq \mathbf b$ . In this case, the diagonal entries of ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_1})$ are given by the PMF of ${\mathbf {A}}_{S_1}$ . Another example is when $S_1 \subseteq S_2$ , then the matrix ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_2})$ has orthogonal row vectors because ${\mathbb {P}}({\mathbf {A}}_{S_1}=\mathbf a, {\mathbf {A}}_{S_2}=\mathbf b) \neq 0$ only if $\mathbf a$ is a subvector of $\mathbf b$ indexed by integers in $S_1$ . This general matrix notation of ${\mathbb {P}}({\mathbf {A}}_{S_1}, {\mathbf {A}}_{S_2})$ for potentially overlapping sets $S_1$ and $S_2$ turns out to be very useful to facilitate our identifiability proofs.

2.4 A Toy Example Illustrating the Tensor Unfolding Insight

The next toy example illustrates the tensor unfolding idea and reveals useful insights on how to utilize unfoldings to identify and recover the Q-matrix.

Consider $J=5$ , $K=2$ , and the following Q-matrix

$$ \begin{align*}Q\ {=}\ \begin{pmatrix} 1 & 0 \\ 1 & 0 \\ 0 & 1 \\ 0 & 1 \\ 1 & 1 \end{pmatrix}. \end{align*} $$

If considering binary responses ${\mathbf {R}}\in \{0,1\}^5$ , then the population distribution tensor ${\mathcal {T}}$ has size $2\times 2\times 2\times 2\times 2$ . Following the definition of CDMs in Section 2.1, we can write

(5) $$ \begin{align} &{\mathbb{P}}(R_1=r_1, R_2=r_2, R_3=r_3, R_4=r_4, R_5=r_5) \nonumber \\ &\quad=\sum_{(a_1,a_2)\in\{0,1\}^2} {\mathbb{P}}({\mathbf{A}}=(a_1,a_2)) \prod_{j=1}^5 {\mathbb{P}}(R_j=r_j\mid {\mathbf{A}}=(a_1,a_2)) \nonumber\\ &\quad=\quad \sum_{(a_1,a_2)\in\{0,1\}^2} {\mathbb{P}}({\mathbf{A}} = (a_1,a_2)) {\mathbb{P}}({\mathbf{R}}_{1,2} = {\mathbf{r}}_{1,2}\mid A_1=a_1) {\mathbb{P}}({\mathbf{R}}_{3,4,5} = {\mathbf{r}}_{3,4,5}\mid A_1=a_1,A_2=a_2) \nonumber \\ &\quad=\quad \sum_{(a_1,a_2)\in\{0,1\}^2} \sum_{a_3\in\{0,1\}} {\mathbb{P}}({\mathbf{A}} = (a_1,a_2), A_1=a_3) {\mathbb{P}}({\mathbf{R}}_{1,2} = {\mathbf{r}}_{1,2}\mid A_1=a_3) \nonumber \\ & \qquad \times{\mathbb{P}}({\mathbf{R}}_{3,4,5} = {\mathbf{r}}_{3,4,5}\mid A_1=a_1,A_2=a_2), \end{align} $$

where we have purposefully introduced the term ${\mathbb {P}}({\mathbf {A}} = (a_1,a_2), A_1=a_3)$ to facilitate the following matrix factorization perspective for the unfolded tensor. To be more specific, the last equality in the above expression holds because although $a_3$ can freely range in $\{0,1\}$ , the probability ${\mathbb {P}}({\mathbf {A}} = (a_1,a_2), A_1=a_3)$ will be zero for any $a_3 \neq a_1$ .

First, we unfold the tensor ${\mathcal {T}}$ so that the row group contains item indices $\{1,2\}$ , and the column group contains item indices $\{3,4,5\}$ . Let $t_{ab,cde} = {\mathbb {P}}(R_1=a, R_2=b, R_3=c, R_4=d, R_5=e)$ denote entries from the joint distribution of the five observed variables, where $a,b,c,d,e\in \{0,1\}$ . With the row group $S=\{1,2\}$ and column group $[J]\setminus S=\{3,4,5\}$ , the unfolded tensor is a $2^2 \times 2^3$ joint probability table of ${\mathbf {R}}_{1,2} = (R_1,R_2)$ and ${\mathbf {R}}_{3,4,5} = (R_3, R_4, R_5)$ :

(6) $$ \begin{align} & \texttt{unfold}(\mathcal T;~S,~[J]\setminus S)=[{\mathcal{T}}]_{\{1,2\},\; :} \\ &\quad= \begin{pmatrix} t_{00, 000} &~ t_{00, 001} &~ t_{00, 010} &~ t_{00, 011} &~ t_{00, 100} &~ t_{00, 101} &~ t_{00, 110} &~ t_{00, 111} \\ t_{01, 000} &~ t_{01, 001} &~ t_{01, 010} &~ t_{01, 011} &~ t_{01, 100} &~ t_{01, 101} &~ t_{01, 110} &~ t_{01, 111} \\ t_{10, 000} &~ t_{10, 001} &~ t_{10, 010} &~ t_{10, 011} &~ t_{10, 100} &~ t_{10, 101} &~ t_{10, 110} &~ t_{10, 111} \\ t_{11, 000} &~ t_{11, 001} &~ t_{11, 010} &~ t_{11, 011} &~ t_{11, 100} &~ t_{11, 101} &~ t_{11, 110} &~ t_{11, 111} \end{pmatrix} \nonumber \\ &\quad= \underbrace{\mathbb P({\mathbf{R}}_{1,2}\mid A_1)}_{2^2 \times 2} \cdot \underbrace{\mathbb P(A_1, {\mathbf{A}}_{1,2})}_{2\times 2^2} \cdot \underbrace{\mathbb P({\mathbf{R}}_{3,4,5}\mid {\mathbf{A}}_{1,2})^\top}_{2^2\times 2^3},\quad \text{(following from the decomposition}\ (2)) \nonumber \\ &\quad \Longrightarrow \mathrm{{rank}}([{\mathcal{T}}]_{\{1,2\},\; :} ) \leq 2. \nonumber \end{align} $$

Specifically, $\mathbb P({\mathbf {R}}_{1,2}\mid A_1)$ is a $2^2\times 2$ matrix that collects the following conditional probabilities:

$$ \begin{align*} \mathbb P({\mathbf{R}}_{1,2}\mid A_1) = \begin{pmatrix} {\mathbb{P}}({\mathbf{R}}_{1,2} = (0,0)\mid A_1 = 0) & {\mathbb{P}}({\mathbf{R}}_{1,2} = (0,0)\mid A_1 = 1) \\ {\mathbb{P}}({\mathbf{R}}_{1,2} = (0,1)\mid A_1 = 0) & {\mathbb{P}}({\mathbf{R}}_{1,2} = (0,1)\mid A_1 = 1) \\ {\mathbb{P}}({\mathbf{R}}_{1,2} = (1,0)\mid A_1 = 0) & {\mathbb{P}}({\mathbf{R}}_{1,2} = (1,0)\mid A_1 = 1) \\ {\mathbb{P}}({\mathbf{R}}_{1,2} = (1,1)\mid A_1 = 0) & {\mathbb{P}}({\mathbf{R}}_{1,2} = (1,1)\mid A_1 = 1) \end{pmatrix}_{4\times 2}, \end{align*} $$

and the $2^3\times 2^2$ matrix $\mathbb P({\mathbf {R}}_{3,4,5}\mid {\mathbf {A}}_{1,2})$ is defined similarly. In summary, the above derivation shows that we can factorize $\texttt {unfold}(\mathcal T;~\{1,2\},~\{3,4,5\})$ into the product of three matrices, which have size $4\times 2$ , $2\times 4$ , and $4\times 8$ , respectively. Therefore, the rank of the $4\times 8$ matrix $\texttt {unfold}(\mathcal T;~\{1,2\},~\{3,4,5\})$ is at most 2. Similarly, $\text {{rank}}(\texttt {unfold}(\mathcal T;~\{3,4\},~\{1,2,5\})) \leq 2$ also holds due to symmetry because items 3 and 4 both solely measure the second attribute.

In contrast, if two items $j_1,j_2$ do not solely measure the same latent attribute, the unfolding $[{\mathcal {T}}]_{\{j_1,j_2\},:}$ will show distinct rank behavior. Specifically, if we unfold the tensor ${\mathcal {T}}$ with the row group being $\{1,3\}$ and the column group being $\{2,4,5\}$ , then as will be shown rigorously in the proof of the main theorem, the unfolding $\texttt {unfold}(\mathcal T;~\{1,3\},~\{2,4,5\})$ has rank strictly larger than 2; more generally, $\text {{rank}}(\texttt {unfold}(\{j_1,j_2\},\; [J]\setminus \{j_1,j_2\} ))>2$ holds as long as items $j_1$ and $j_2$ are not solely measuring the same attribute. In other words, whether the rank of $\texttt {unfold}(\{j_1,j_2\},\; [J]\setminus \{j_1,j_2\} )$ exceeds two is a perfect certificate that reveals whether $j_1$ and $j_2$ are measuring the same latent attribute. By exhaustively examining the rank of the unfoldings with the row group consisting of every pair of items, we can exactly tell which items are solely measuring the same attribute as well as the number of latent attributes: items $1,2$ solely measure one latent attribute, items $3,4$ solely measure the other latent attribute, and there are two latent attributes. Note that if there is an additional item 6 that solely measures the first latent attribute, then $\texttt {unfold}(\{1,2\},\; [J]\setminus \{1,2\} )$ , $\texttt {unfold}(\{1,6\},\; [J]\setminus \{1,6\} )$ , and $\texttt {unfold}(\{2,6\},\; [J]\setminus \{2,6\} )$ all have ranks at most two, so all three items 1, 2, and 6 will be correctly assigned as solely measuring the same latent attribute.

Since item 5 with ${\mathbf {q}}_5=(1,1)$ measures both latent attributes, identifying such ${\mathbf {q}}$ -vectors calls for a more challenging analysis than the pairwise unfolding illustrated in the previous paragraph. But surprisingly and fortunately, we rigorously prove (in Theorems 1 and 2 in Section 3) that examining strategically constructed unfolded matrices’ ranks can still identify the ${\mathbf {q}}$ -vectors that vary arbitrarily in $\{0,1\}^K$ . The proof for these multi-attribute items is much more technical than the single-attribute items described above, so we omit the details here and refer interested readers to the proof of the theorems.

In summary, the toy example above illustrates that the ranks of unfolded tensors can help reveal the underlying Q-matrix. Our theorem applies to general Q-matrix dimensions and structures far beyond this toy example, as will be laid out in the next section.

3.1 CDMs with Binary Responses

We next rigorously formalize the mild technical assumptions we impose for identifiability. If an item j solely measures attribute k (that is, $\mathbf q_j= \mathbf e_k$ where $\mathbf e_k$ is the kth canonical basis vector), then we call it a single-attribute item.

Requiring an item to satisfy Definition 1 is a mild assumption. Mathematically, if item j solely requires attribute k, then ${\mathbb {P}}(R_j\mid \mathbf A) \equiv {\mathbb {P}}(R_j\mid A_k)$ and the $2\times 2$ conditional probability table is:

$$ \begin{align*}\begin{pmatrix} {\mathbb{P}}(R_j=0\mid A_k=0) & {\mathbb{P}}(R_j=0\mid A_k=1) \\ {\mathbb{P}}(R_j=1\mid A_k=0) & {\mathbb{P}}(R_j=1\mid A_k=1) \end{pmatrix}. \end{align*} $$

In this case, requiring ${\mathbb {P}}(R_j\mid A_k)$ to have full rank 2 as specified in Definition 1 is equivalent to requiring $R_j$ not to be independent of $A_k$ . We next present the first main theorem.

Theorem 1. Consider any CDM in Examples 1 and 2 with binary responses. Consider the following identifiable parameter space for the Q-matrix:

(7) $$ \begin{align} \mathcal Q = \{Q\in\{0,1\}^{J\times K}: P Q = (I_K,I_K,Q^{*\top})^\top\text{ for some }J\times J \text{ permutation matrix }P\}, \end{align} $$

where $I_K$ is the $K\times K$ identity matrix and $Q^*$ is an arbitrary binary matrix or an empty matrix (meaning $J=2K$ ). Assume all single-attribute items are non-degenerate and $p_{\boldsymbol {\alpha }}> 0$ for all $\boldsymbol {\alpha }\in \{0,1\}^K$ . If the true and alternative Q-matrices $Q_{\text {true}}, Q_{\text {alternative}}\in \mathcal Q$ lead to the same marginal distribution of the observed response vector $\mathbf R$ , then $Q_{\text {true}}$ is identifiable and $Q_{\text {alternative}}$ must be identical to $Q_{\text {true}}$ up to a column permutation. Moreover, $Q_{\text {true}}$ can be constructively identified from the population distribution tensor ${\mathcal {T}}$ via strategically unfolding it and examining the ranks.

The identifiable Q-matrix space in (7) includes all $J\times K$ binary matrices that contain at least two identity submatrices after some column and row permutation. Specifically, we prove Theorem 1 by establishing the following two key technical propositions, which explicitly lay out the tensor unfolding procedures along with the corresponding rank certificates to identify and recover the Q-matrix.

Proposition 1. Under the condition in Theorem 1, for arbitrary $j_1\neq j_2\in [J]$ , it holds that

$$ \begin{align*}\mathrm{{rank}}(\underbrace{[{\mathcal{T}}]_{\{j_1,j_2\},\; :}}_{2^2\times 2^{J-2}}) \leq 2 \quad\text{if and only if}\quad \mathbf q_{j_1} = \mathbf q_{j_2} = \mathbf e_k\text{ for some } k\in[K].\end{align*} $$

Proposition 1 formalizes the intuition explained in the toy example in Section 2.4, and precisely characterizes how to identify the ${\mathbf {q}}$ -vectors for all single-attribute items in the Q-matrix. In words, Proposition 1 states that the rank of $[\mathcal T]_{\{j_1,j_2\},:}$ reflects whether the two items $j_1$ and $j_2$ solely measure the same latent attribute. Interestingly, Proposition 1 also implies we can constructively identify K, the number of latent attributes: K is equal to the maximum number of mutually disjoint item sets $S_1, S_2, \ldots \subseteq [J]$ such that the rank of the unfoldings $[{\mathcal {T}}]_{\{j_1,j_2\},:}$ is at most two when considering all $j_1,j_2\in S_k$ for each k.

Having identified K and all single-attribute items, the next proposition characterizes a more nuanced tensor unfolding strategy along with their rank certificates to identify the challenging multi-attribute items. These items can have arbitrary corresponding ${\mathbf {q}}$ -vectors ranging in $\{0,1\}^K\setminus \{{\mathbf {e}}_1,\ldots ,{\mathbf {e}}_K\}$ .

Proposition 2. Under the condition in Theorem 1, assume without loss of generality that the first $2K$ rows of the Q-matrix are identified to be $( I_K;~ I_K)^\top $ . The following holds for any attribute $k\in [K]$ and any item j that requires at least two attributes (i.e., with $\sum _{k=1}^K q_{jk} \geq 2$ ):

$$ \begin{align*} \mathrm{{rank}}(\underbrace{[\mathcal T]_{([K]\setminus\{k\}) \cup \{j\},~ \{K+1,\ldots,2K\}}}_{2^K \times 2^K})> 2^{K-1} \quad\text{if and only if}\quad q_{jk}=1. \end{align*} $$

The proof of Proposition 2 is challenging. The difficulties include: first, to come up with an appropriate row group $S_1$ and a corresponding column group $S_2$ of variables from $R_1,\ldots , R_J$ such that the marginal unfolding $[\mathcal T]_{S_1,S_2}$ contains as much as possible useful information about the multi-attribute ${\mathbf {q}}$ -vectors; and second, to carefully investigate the rank of these unfoldings to establish the certificate for recovering those ${\mathbf {q}}$ -vectors.

The identifiability condition in Theorem 1 is weaker than existing strict identifiability conditions on the Q-matrix for main-effect or all-saturated-effect, binary-response or polytomous response CDMs (e.g., Culpepper, Reference Culpepper2019; Fang et al., Reference Fang, Liu and Ying2019; Xu and Shang, Reference Xu and Shang2018). Specifically, the condition in Theorem 1 is satisfied if, after some row permutation, Q takes the form of $Q = (I_K, I_K)^\top $ with exactly $J=2K$ items; that is, when each latent attribute is exactly measured by two pure items, and there are no additional items in the test. To our best knowledge, such a Q-matrix does not satisfy the identifiability condition in the existing studies for general CDMs. The high-level intuition behind why our condition relaxes existing conditions in the literature is that, existing studies either explicitly or implicitly leverage the three-way decompositions of the population distribution tensor to establish identifiability. In contrast, we work with the rank of matrices to establish identifiability despite starting with tensors, and these matrices essentially describe the joint distributions of two random vectors instead of three random vectors. This intuitively explains why our approach delivers a weaker identifiability condition that only requires each latent attribute to be measured twice in the Q-matrix.

It is worth noting that Culpepper (Reference Culpepper2023) relaxed the condition that Q needs to contain identity submatrices by introducing and studying the notion of “dyads”, which consist of pairs of items. Specifically, the identifiability theorem in Culpepper (Reference Culpepper2023) applies even if no single-attribute items exist by allowing saturated dyads. This significantly broadens the applicability of the identifiability theory to practical cognitive diagnostic assessment settings. However, that work still leverages Kruskal’s Theorem on the uniqueness of three-way tensor decompositions to establish identifiability with the dyad structure and requires $J>2K$ . So, the theorem in Culpepper (Reference Culpepper2023) does not apply to the case where the Q-matrix contains exactly $2K$ rows; moreover, the proof of that theorem is still an existence proof, instead of a constructive proof like our tensor unfolding approach.

3.2 CDMs with Polytomous Responses

In this section, we consider CDMs with polytomous item responses. Suppose item $R_j\in \{0,1,\ldots ,C_j-1\}$ has $C_j$ categories for each $j\in \{1,\ldots ,J\}$ , where $C_j\geq 2$ is a general integer. Here, we allow different items to have different numbers of response categories (i.e., $C_j$ across different j can be different), similar to the setting considered in Liu and Culpepper (Reference Liu and Culpepper2024) and Wayman et al. (Reference Wayman, Culpepper, Douglas and Bowers2025). We still use the same notation for the proportion parameters ${\mathbf {p}} = (p_{\boldsymbol {\alpha }})_{\boldsymbol {\alpha }\in \{0,1\}^K}$ to describe the joint distribution of the binary latent attributes. To describe the conditional distribution of the responses ${\mathbf {R}}$ given the latent attributes ${\mathbf {A}}$ , we introduce item parameters $\boldsymbol \Lambda =(\lambda _{j,c,\boldsymbol {\alpha }})$ :

$$ \begin{align*} \lambda_{j,c,\boldsymbol{\alpha}} = {\mathbb{P}}(R_j = r_j\mid {\mathbf{A}}=\boldsymbol{\alpha}),\quad j\in[J],~~ r_j\in\{0,1,\ldots,C_j-1\}, ~~ \boldsymbol{\alpha}\in\{0,1\}^K. \end{align*} $$

The Q-matrix induces constraints on the $\boldsymbol \Lambda $ parameters similar to the binary CDM case; here, $\lambda _{j,c,\boldsymbol {\alpha }}$ depends only on those attributes that are measured by the jth item according to the Q-matrix. Mathematically, this means

$$ \begin{align*} {\mathbb{P}}(R_j = r_j\mid {\mathbf{A}} = \boldsymbol{\alpha}) \equiv {\mathbb{P}}(R_j = r_j\mid \{A_k=\alpha_k\}_{q_{jk}=1}),\quad \forall j\in[J],~ \boldsymbol{\alpha}\in\{0,1\}^K. \end{align*} $$

The above general assumption summarizes the key property shared by many polytomous-response CDMs. Our considered setting hence covers existing models for polytomous responses, such as Chen and de la Torre (Reference Chen and de la Torre2018), Fang et al. (Reference Fang, Liu and Ying2019), Culpepper (Reference Culpepper2019), and Liu and Culpepper (Reference Liu and Culpepper2024).

The population distribution of the observed response vector ${\mathbf {R}}$ can be written as

$$ \begin{align*} {\mathbb{P}}(R_1 = r_1, \ldots, R_J = r_J\mid Q, \boldsymbol \Lambda, {\mathbf{p}}) &= \sum_{\boldsymbol{\alpha}\in\{0,1\}^K} {\mathbb{P}}({\mathbf{A}}=\boldsymbol{\alpha}) \prod_{j=1}^J {\mathbb{P}}(R_j = r_j\mid {\mathbf{A}} = \boldsymbol{\alpha}) \\ &= \sum_{\boldsymbol{\alpha}\in\{0,1\}^K} p_{\boldsymbol{\alpha}} \prod_{j=1}^J \prod_{c=0}^{C_j-1} \lambda_{j,c,\boldsymbol{\alpha}}^{\mathbb{1}(r_j=c)}, \end{align*} $$

for any multivariate polytomous response pattern ${\mathbf {r}} = (r_1,\ldots ,r_J)$ with $r_j\in \{0,\ldots ,C-1\}$ for each j. The J-way population distribution tensor $\mathcal T=(t_{r_1,r_2,\ldots ,r_J})$ characterizing the distribution of ${\mathbf {R}}$ has size $C_1 \times C_2\times \cdots \times C_J$ , with entries

$$ \begin{align*} t_{r_1,r_2,\ldots,r_J} = {\mathbb{P}}(R_1 = r_1, \ldots, R_J = r_J\mid Q,\boldsymbol \Lambda, {\mathbf{p}}),\quad \forall r_j\in\{0,1,\ldots,C_j-1\},\ j=1,\ldots,J. \end{align*} $$

We will unfold this tensor similarly to the binary CDM case. Consider the row group of indices as $S\subseteq [J]$ and the column group as $[J]\setminus S$ , then the matrix resulting from the unfolded tensor is

$$ \begin{align*} [{\mathcal{T}}]_{S,:}=\texttt{unfold}(\mathcal T;~S,~[J]\setminus S): \text{ size } \prod_{j\in S} C_j \times \prod_{j\in [J]\setminus S} C_j, \end{align*} $$

The notations of marginal tensors and marginal unfoldings are defined similarly.

To formalize the requirements on the polytomous CDMs, we introduce the following two definitions.

Definition 2 (Non-degenerate Single-attribute Items in Polytomous-CDMs).

A single-attribute item j solely measuring attribute k is said to be non-degenerate if the following conditional probability table, denoted by ${\mathbb {P}}(R_j\mid A_k)$ , has full column rank 2:

$$ \begin{align*} \begin{pmatrix} {\mathbb{P}}(R_j=0\mid A_k=0) & {\mathbb{P}}(R_j=0\mid A_k=1) \\ \vdots & \vdots \\ {\mathbb{P}}(R_j=C_j-1\mid A_k=0) & {\mathbb{P}}(R_j=C_j-1\mid A_k=1) \end{pmatrix} \end{align*} $$

Definition 2 is a direct extension of the earlier Definition 1 to the polytomous CDMs and covers that as a special case.

As mentioned earlier in Section 2.1, we do not consider the Boolean-product-based CDMs (e.g., DINA or DINO models) in this work. We next formalize the non-DINA/DINO assumption in the polytomous CDM.

We have the following main theorem for polytomous CDMs with essentially the same identifiability condition as the binary CDMs, again proved via tensor unfoldings.

Theorem 2 is proved by establishing the following proposition on the ranks of the unfolded tensors in polytomous CDMs.

The above proposition generalizes Proposition 1 from the binary setting to the general polytomous setting. The two scenarios (a) and (b) in Proposition 3 give explicit and precise characterizations of the ${\mathbf {q}}$ -vectors of single-attribute items and multiple-attribute items, respectively. These conditions are simple rank constraints of the matrices resulting from unfolding the $C_1\times C_2\times \cdots \times C_J$ tensor $\mathcal T$ , and they serve as certificates to reveal the unknown $\mathbf q$ -vectors for all J items.

It is worth mentioning that prior works have studied generic identifiability in CDMs, which is a slightly weaker notion than strict identifiability studied in this work. Generic identifiability means the parameters are almost everywhere identifiable in the parameter space, possibly excluding a Lebesgue mesasure zero set where identifiability breaks down. For main-effect or all-saturated-effect CDMs, generic identifiablity has been established under weaker conditions on the Q-matrix (e.g., Chen et al., Reference Chen, Culpepper and Liang2020; Gu and Xu, Reference Gu and Xu2020), without requiring Q to contain two identity submatrices $I_K$ . We remark that it is not technically straightforward to establish generic identifiability using the tensor unfolding technique. To clarify, most previous studies that established generic identifiability for CDMs mainly utilized Kruskal’s Theorem by adapting the technique popularized by Allman et al. (Reference Allman, Matias and Rhodes2009). Roughly speaking, in those works, generic identifiability is often proved by finding one possible configuration of the parameters and showing that identifiability holds under this configuration using Kruskal’s Theorem; then, generic identifiability follows by concluding that the identifiable parameter configuration is “generic” in the parameter space, so that non-identifiability occurs only in a negligible subset of the parameter space. Therefore, generic identifiability is inherently tied to the Kruskal Theorem-based existence proofs, while not entirely aligned with the tensor-unfolding-based constructive proofs. Relaxing the condition of “Q contains at least two $I_K$ matrices” would distort the rank properties of the unfolded tensor, which are important certificates to recover the Q-matrix in our proof.