2.1 Derivation

In the standard multidimensional IRT model (Clinton, Jackman, and Rivers Reference Clinton, Jackman and Rivers2004), the binary response of subject i’s to question q is defined as

$$\begin{align*}y_{iq} = \begin{cases} 1 \; \text{if} \; \boldsymbol{\beta}_q'\boldsymbol{\theta}_{i} + \epsilon_{iq}> \alpha_q\\ 0 \; \text{otherwise.} \end{cases} \end{align*}$$

In this model, $\boldsymbol {\theta }_{i} \in \mathbb {R}^D$ is the ideal point vector representing subject i’s position in D-dimensional space. The difficulty  $\alpha _q \in \mathbb {R}$ represents the threshold required for a positive response on question q. The discrimination vector $\boldsymbol {\beta }_q \in \mathbb {R}^D$ captures the how strongly the response $y_{iq}$ depends on subject i’s ideal point. $\epsilon _{iq}$ is an observation-specific utility shock. Under the conventional identification restriction $\epsilon _{iq} \sim \mathcal {N}(0, 1)$ , the probability of a positive response is

$$\begin{align*}\text{Pr}(y_{iq} = 1 \mid \boldsymbol{\alpha}_q, \boldsymbol{\beta}_q, \boldsymbol{\theta}_{i}) = \Phi (\boldsymbol{\beta}_q'{\boldsymbol{\theta}}_{i} - \alpha_q). \end{align*}$$

This is the individual-level probit IRT model.

The MODGIRT model is intended for situations where the target of inference is the mean ideal point at some higher level of aggregation, such as a country. One option is to estimate this quantity directly by making it a parameter of the model (e.g., McGann Reference McGann2014; Solt Reference Solt2020). Alternatively, the mean can be estimated indirectly by parameterizing the model at a lower level of aggregation, such as age categories with country, and then poststratifying the parameter estimates to match the population of interest (Caughey and Warshaw Reference Caughey and Warshaw2015; cf. Park, Gelman, and Bafumi Reference Park, Gelman and Bafumi2004).

Under either approach, estimating mean ideal points without estimating the ideal points of individual subjects requires an assumption about the distribution of ideal points within groups. The assumption we make is that ideal points are distributed multivariate normal around a group-specific mean vector $\bar {\boldsymbol {\theta }}_{g}$ with a common variance–covariance matrix $\boldsymbol {\Sigma }_{\theta }$ :

$$\begin{align*}\boldsymbol{\theta}_{g[i]} \sim \mathcal{N}_D(\bar{\boldsymbol{\theta}}_{g}, \boldsymbol{\Sigma}_{\theta}). \end{align*}$$

This assumption allows us to derive $p_{gq} = \text {Pr}(y_{iq} = 1 \mid \alpha _q, \boldsymbol {\beta }_q, \bar {\boldsymbol {\theta }}_{g[i]}, \boldsymbol {\Sigma }_{\theta })$ . Affine transformations of the multivariate normal distribution operate such that if $X \sim \mathcal {N}(\boldsymbol {\mu },\boldsymbol {\Sigma })$ and $\mathbf {Y = BX + c}$ , then $\mathbf {Y} \sim \mathcal {N}(\mathbf {B}\boldsymbol {\mu } + \mathbf {c},\mathbf {B}\boldsymbol {\Sigma }\mathbf {B}')$ . By this rule, the distribution of $\boldsymbol {\beta }_q'\boldsymbol {\theta }_{g[i]} - \alpha _q$ can be written

$$\begin{align*}\boldsymbol{\beta}_q'\boldsymbol{\theta}_{g[i]} - \alpha_q \sim \mathcal{N}(\boldsymbol{\beta}_q'\bar{\boldsymbol{\theta}}_{g} - \alpha_q, \boldsymbol{\beta}_q'\boldsymbol{\Sigma}_{\theta}\boldsymbol{\beta}_q). \end{align*}$$

Moreover, since $\epsilon _{iq}$ is an independent standard normal variable,

$$\begin{align*}\boldsymbol{\beta}_q'\boldsymbol{\theta}_{g[i]} - \alpha_q + \epsilon_{iq}\sim \mathcal{N}(\boldsymbol{\beta}_q'\bar{\boldsymbol{\theta}}_{g} - \alpha_q, \boldsymbol{\beta}_q'\boldsymbol{\Sigma}_{\theta}\boldsymbol{\beta}_q + 1). \end{align*}$$

The probability that subject i randomly sampled from group g gives a positive answer to question q is therefore

(1) $$ \begin{align} p_{gq} &= \text{Pr} (\boldsymbol{\beta}_q'\boldsymbol{\theta}_{g[i]} - \alpha_q + \epsilon_{iq}> 0) \end{align} $$

(2) $$ \begin{align} &= \Phi \left( \frac{\boldsymbol{\beta}_q'\bar{\boldsymbol{\theta}}_{g} - \alpha_q}{\sqrt{\boldsymbol{\beta}_q' \boldsymbol{\Sigma}_{\theta}\boldsymbol{\beta}_q + 1}}\right), \end{align} $$

where $\Phi $ is the standard normal cumulative distribution function.

If responses are conditionally independent,Footnote ² the number of positive answers to item q in group g is distributed

$$\begin{align*}s_{gq} \sim \text{Binomial}(n_{gq},p_{gq}).\end{align*}$$

As shown in the Supplemental Information, the binary group-level IRT model just described can be extended to multiple ordered response categories using an ordinal cumulative model (Samejima Reference Samejima, Linden and Hambleton1997).

2.1.1 Priors

We make this model dynamic by allowing the group mean ideal points to change across periods ${t \in 1 \dots T}$ . We smooth the changes across periods with a “random walk” prior (Martin and Quinn Reference Martin and Quinn2002; cf. Kołczyńska and Bürkner Reference Kołczyńska and Bürkner2024),

$$\begin{align*}\bar{\boldsymbol{\theta}}_{gt} \sim \mathcal{N}_D(\bar{\boldsymbol{\theta}}_{g,t-1}, \Omega), \end{align*}$$

where $\Omega $ is a $D \times D$ variance–covariance matrix.

To complete the Bayesian specification, we assign standard-normal priors to $\boldsymbol {\beta }_q$ , $\alpha _{q}$ , and $\bar {\boldsymbol {\theta }}_{g1}$ . We define each of the covariance matrices $\boldsymbol {\Sigma }_{\theta }$ and $\Omega $ as the product of a correlation matrix, which is given a LKJ(2) prior, and a variance vector ( $\boldsymbol {\sigma }_{\theta } \sim \text {Cauchy}_+(0, 1)$ and $\boldsymbol {\omega } \sim \text {Cauchy}_+(0, 0.1)$ , respectively).

2.2 Identification

Like all latent variable models, identifying the MODGIRT likelihood requires restrictions on the parameter space. If the model has D latent factors, then $D(D+1)$ independent restrictions are required for local identification, and a further D restrictions for global identification (Rivers Reference Rivers2003). One set of $D(D+2)$ restrictions theoretically sufficient for global identification is the following:

1. 1. Zero-mean group ideal points (D): $\sum _g \bar {\theta }_{gdt} = 0\; \forall d$ and for one t

2. 2. Unit-variance group ideal points (D): $\sum _g \bar {\theta }_{gdt}^2 = 1\; \forall d$ and for one t

3. 3. Orthogonal factors $\left (D[D-1]/2\right )$ : $\sum _g \bar {\theta }_{gdt}\bar {\theta }_{gd't} = 0\; \forall d \neq d'$ and for one t

4. 4. Rotation invariance Footnote ³ $\left (D[D-1]/2\right )$ : $\beta _{qd} = 0\; \forall i < d$

5. 5. Sign invariance (D): $\beta _{qd}> 0$ or $\beta _{qd} < 0$ for some item i on each factor d.

Note that because the model is dynamic, we impose restrictions 1–3 in a single time period only (by default, $t=1$ ).

Though theoretically well understood, identification restrictions can be tricky to apply to models estimated with Monte Carlo simulation. In particular, point and sign restrictions like (4) and (5) above must be chosen carefully. Condition 4, for example, restricts the first item to load only on the first factor, but if this item happens to have no strong loadings, then identification will be lost.Footnote ⁴ While this problem can be avoided by reordering the items, practical difficulties such as these present challenges to automating the identification of Bayesian IRT models.

Our solution to these problems, which can be applied to any IRT or factor-analytic model, involves a combination of within-estimation and post-estimation transformations. We impose restrictions 1–3 during the estimation process. We do so by first drawing a “raw” matrix $\bar {\boldsymbol {\theta }}^z$ from its prior distribution. We then transform this matrix by de-meaning its columns (restriction 1) and then (restrictions 2 and 3) “whitening” the centered matrix $\bar {\boldsymbol {\theta }}^0$ to produce $\bar {\boldsymbol {\theta }}^*$ :

$$\begin{align*}\bar{\boldsymbol{\theta}}^* = L' \bar{\boldsymbol{\theta}}^0, \end{align*}$$

where L is the Cholesky decomposition of the inverse of the column-wise covariance matrix of $\bar {\boldsymbol {\theta }}^0$ . These transformations force each draw of the group ideal points to have zero mean and unit variance in each dimension and to be uncorrelated across dimensions.

Even with these restrictions imposed, the likelihood is still invariant to rotation and/or signed permutation of the item discriminations $\beta _{qd}$ . As noted above, one solution is to impose restrictions 4 and 5 during estimation, and the software we develop offers this as an option for the user. Through careful selection of items, these restrictions enable users to define the substantive meaning of the factors and their polarity.Footnote ⁵ In practice, however, we have found that sampling is more efficient if we impose the remaining $D(D+1)/2$ identification restrictions by post-processing the Markov Chain Monte Carlo (MCMC) draws.

To do so, we employ the Rotation-Sign-Permutation (RSP) algorithm proposed by Papastamoulis and Ntzoufras (Reference Papastamoulis and Ntzoufras2022). Rather than setting selected item loadings to 0, the RSP algorithm first applies a varimax rotation to each $\boldsymbol {\beta }$ draw.Footnote ⁶ This step resolves rotation invariance but not invariance to permutation of the factor labels or reversals in their polarity. The RSP algorithm’s second step is find the draw-specific sign and permutation matrices that harmonize the draws to a single mode. The RSP algorithm yields a collection of identified draws of $\boldsymbol {\beta }$ . As a byproduct, it also produces three $D \times D$ rotation matrices, R (varimax rotation), S (sign), and P (permutation). With these in hand, we can apply the same rotations to each draw $\bar {\boldsymbol {\theta }}^*_{(s)}$ , thus identifying the ideal points as well:

$$\begin{align*}\bar{\boldsymbol{\theta}}_{(s)} = \bar{\boldsymbol{\theta}}^*_{(s)}RSP. \end{align*}$$

Similarly, the covariance matrices can be identified with the transformation

$$\begin{align*}{\boldsymbol{\Omega}}_{(s)} = (RSP)^\top{\boldsymbol{\Omega}}^*_{(s)}RSP. \end{align*}$$

Due to the varimax rotation, the identified $\boldsymbol {\beta }$ estimates will have a relatively “simple” structure, aiding interpretation.Footnote ⁷ If desired, however, the identified draws can be further transformed with a different rotation criterion, including ones that result in oblique rather than orthogonal factors (for an overview of rotation criteria, see Sass and Schmitt Reference Sass and Schmitt2010).Footnote ⁸ Alternatively, $\boldsymbol {\beta }$ can be rotated to match a matrix of “target” loadings (Bernaards and Jennrich Reference Bernaards and Jennrich2005). For example, restriction 4 can be implemented by setting the requisite elements of the target matrix to 0.

2.3 Estimation

We use the R package cmdstanr (Gabry, Češnovar, and Johnson Reference Gabry, Češnovar and Johnson2023) to fit this model in the Bayesian software program Stan, which samples from the posterior distribution using Hamiltonian Monte Carlo (HMC).Footnote ⁹ The SI contains the Stan code for the model. The R package accompanying this paper contains functions implementing the pre-processing and post-processing steps needed to prepare the data and process the posterior draws.

3.1 Baseline Scenario

For the baseline scenario, we set the number of groups and items to $G = Q = 50$ and total responses to $R=$ 100,000.Footnote ¹¹ Under the assumed data-generating process, each individual answers $A = 1$ randomly selected item (local independence) and the within-group variance-covariance matrix $\boldsymbol {\Sigma _{\theta }}$ is constant across groups (homoskedasticity).

Table 1 summarizes the average mean-squared error, correlations between simulated and estimated parameter values, and coverage of $90\%$ credible intervals for each of the three types of target parameters ( $\boldsymbol {\bar {\theta }_g}$ , $\boldsymbol {\beta _q}$ , and $\alpha _q$ ).Footnote ¹² Correlations are all above $0.99$ , and average MSE ranges between 0.003 and 0.013. Coverage for the difficulty parameters $\alpha _q$ is close to the expected $90\%$ , while posterior intervals for group ideal points $\boldsymbol {\bar {\theta }_g}$ and item discrimination parameters $\boldsymbol {\beta _q}$ are too wide. These simulations demonstrate that under the assumed data-generating process our Stan code correctly recovers the true parameters.

Table 1 Model performance under assumed data-generating process (baseline scenario).

3.2 Violating Assumptions and Varying Conditions

We can adapt the process shown in Figure 1 to examine both violations of model assumptions and variation in modeling conditions. We examine two potential violations of assumptions: breakdowns in local independence and in the homoskedasticity of within-group ideal points. To test violations of local independence, we allow our simulated subjects to answer increasingly large numbers of items (A). To examine the effect of heteroskedasticity, we let $\boldsymbol {\Sigma _{\theta }}$ vary across groups.

The main consequence of violating either assumption is on coverage of credible intervals. The coverage rate for group ideal points declines when local independence is violated by allowing each subject to answer more than one question. After more than $A=$ 20 item responses per individual, the average coverage of 90% credible intervals starts to dip below 90%, and by $A=$ 40 intervals cover the true value of the ideal point only 85% of the time. However, in a real dataset with $A=$ 40, it likely would be feasible to estimate an individual-level IRT model.

The MODGIRT model is similarly robust to moderate heteroskedasticity, but the more $\boldsymbol {\Sigma _{\theta }}$ varies across groups the worse coverage becomes. Coverage starts to become anticonservative when within-group standard deviations range more than $50\%$ above or below the average standard deviation (see SI for full results). But while violating local independence makes coverage decline only for ideal point estimates, heteroskedasticity affects the coverage of intervals for all parameters types.

We also conduct simulations that reduce the number of groups (G) and vary the number of individuals in each group ( $N_g$ ) while keeping all other simulation conditions the same as in the baseline scenario. Results from these simulations indicate that MSE decreases the higher the total number of responses R and the greater the number of individuals per group $N_g$ . Less variability in group size also produces more accurate results, but the effects are less significant than absolute changes in R and $N_g$ . Finally, having fewer groups affects model performance only when it also reduces the total number of responses. However, unlike violating model assumptions such as local independence and homoskedasticity, shrinking the number and size of groups does not result in lower coverage, so credible intervals can still be trusted to quantify uncertainty.

In summary, the simulations confirm that the MODGIRT model performs extremely well when the data-generating process follows the model’s assumptions, and that when the assumptions of homoskedasticity and local independence are violated, the model remains largely robust. The main effect is on the frequentist properties of the Bayesian credible intervals, which are conservative (i.e., overly wide) under the assumed model but start to become anticonservative when the within-group variances differ greatly or subjects each answer more than 40% of items.

4.1 Preferences for Redistribution in Great Britain

In our first example application, we fit the MODGIRT model in a context where individual-level scaling is feasible, thus permitting us to validate the results against an individual-level analysis. Specifically, we reanalyze a study by Cavaillé and Trump (Reference Cavaillé and Trump2015), who use ordinal factor analysis to investigate the structure of attitudes towards economic redistribution in the Great Britain.Footnote ¹³ Examining 32 items from the 2004 wave of the British Social Attitudes (BSA) Survey, Cavaillé and Trump (CT) conclude that attitudes in this domain are structured by two dimensions, one concerning “redistribution from” the rich and the other “redistribution to” the poor.

To replicate CT’s analysis of the 2004 BSA, we fit a MODGIRT model with 30 groups, defined by the cross-classification of income quintile, education (non-university/university), and country (England/Scotland/Wales). Like the original authors, we characterize the latent space as two-dimensional.Footnote ¹⁴ We use the RSP algorithm to identify the raw parameter draws. We order the dimensions by the sum of squared discriminations and orient them so that larger values of $\bar {\theta }$ indicate greater opposition to redistribution.

Figure 2 plots the MODGIRT discrimination against the varimax-rotated factor loadings reported in Cavaillé and Trump (Reference Cavaillé and Trump2015, 152–53, Table 1). Estimates on the first dimension (redistribution from) are on the left, and the second dimension (redistribution to) is on the right. Both sets of estimates have correlations above 0.9. Despite being estimated at the level of (rather coarse) groups rather than individuals, the MODGIRT model projects items onto a latent space very similar to individual-level factor analysis.

Figure 2 Comparison of MODGIRT discriminations with the factor loadings of Cavaillé and Trump (Reference Cavaillé and Trump2015).

CT also conduct a longitudinal analysis of the years 1986–2011, focusing on differences across income quintiles (Cavaillé and Trump Reference Cavaillé and Trump2015, 151–155). The irregularity of items’ availability makes it difficult to apply factor analysis to the whole time series, so the authors instead create additive indices of the item subsets available in all years (four items for redistribution from, six items for redistribution to). Since the MODGIRT model has no trouble with differing item sets across periods, we include all available items in a single pooled model.Footnote ¹⁵ Unlike additive scales, MODGIRT does not categorize items to one dimension or the other, but rather allows items to contribute, in varying degrees, to inferences about both dimensions.

We fit the MODGIRT model to data from all BSA surveys fielded between 1986 and 2011, using the same group definitions and post-estimation rotations as above. To replicate CT’s analysis of income quintiles, we first average (within draws) the $\bar {\theta }_{gtd}$ estimates of the groups composing each quintile, poststratifying them by the groups’ estimated population proportions. These weighted averages are estimates of the mean ideal point in each income quintile at each point in time.

Our analysis, summarized in Figure 3, largely reproduces CT’s main conclusions. Like CT, we find that redistribution from preferences have become more conservative, especially since the mid-1990s, and that at each point in time conservatism on this dimension increases monotonically with income. As for the redistribution to dimension, we corroborate CT’s finding that the British public also has become more conservative since the early 1990s. Where we differ somewhat is with their conclusion that “[a]ttitudes regarding redistribution to the poor…are not consistently predicted by income” (Cavaillé and Trump Reference Cavaillé and Trump2015, 154). Although differences by income are much smaller on this dimension, we are able to conclude with high confidence that in nearly every year after 1994, the middle income quintile (Q3) exhibited greater conservatism on this dimension than both the highest and lowest quintiles.Footnote ¹⁶

Figure 3 Estimated mean ideal points and 50% credible intervals for the first, third, and fifth income quintiles, by dimension. Solid points indicate estimates in years without surveys, which are interpolated by the dynamic model.

In sum, our re-analysis of the 2004 BSA suggests that a MODGIRT model fit to aggregate data can uncover the same latent space as a similar scaling model fit at the individual level. Our longitudinal analysis further shows that the ease with which MODGIRT accommodates missing items allows it to make use of more data than methods requiring complete data, yielding more precise inferences. It is worth noting that CT’s dataset includes an unusually large number of items per respondent on a given topic, which is what makes it possible to perform multidimensional scaling at the level of individuals (though only at a single point in time). The following section considers a more typical application, in which individual-level scaling is infeasible.

4.2 Policy Ideology in Europe

In this section, we use the MODGIRT model to reanalyze a recent application of a group-level IRT model to cross-national data: Caughey, O’Grady, and Warshaw’s (COW’s) 2019 study of mass policy ideology in Europe. COW fit a one-dimensional model in each of four issue domains: “absolute economic,” “relative economic,” “social,” and “immigration.”Footnote ¹⁷ The resulting domain-specific measures of mass conservatism exhibit “exhibit contrasting cross-sectional cleavages and distinct temporal dynamics” (Caughey, O’Grady, and Warshaw Reference Caughey, O’Grady and Warshaw2019a, 674). In particular, while immigration and social conservatism are strongly correlated across countries, both are uncorrelated with absolute economic conservatism and negatively correlated with relative economic conservatism.

We reanalyze a subset of the COW data, focusing the years 1999–2016.Footnote ¹⁸ Like COW, we define groups as the interaction of country, sex, and three-category age and allow group ideal points to differ across biennia (1999–2000, 2001–2002, etc.). Being culled from many distinct surveys, the COW dataset is very rich, including 27 countries, 100 items, and 1,817,270 individual responses.Footnote ¹⁹ Nevertheless, it is also relatively sparse. Of the 158,922 possible biennium-group-item combinations, 80% are missing any responses. Of course, the individual-level data are sparser still. This dataset is thus typical of the sort of cross-national application for which a group-level IRT model is well suited.

COW’s model is strongly confirmatory in its restriction that each item load on no more than one factor. By contrast, our reanalysis is exploratory in orientation. Unlike COW, we pool together items from all four issue domains, modeling them jointly as a function of four latent factors. We identify the parameter draws using the RSP algorithm. Since the COW dataset assigns conservative responses higher values, we arrange the factors so that the average loading on each dimension is positive, thus orienting them from left to right.

The RSP algorithm’s varimax rotation yields one dominant factor, one secondary one, and two minor ones. The sum of squared discriminations, which is proportional to the variance explained by a given factor, is around 5 on the first dimension, 2.7 on the second, and just above 1 on the remaining two. The factors have approximately equal within-group standard deviations, all around one-tenth the between-group standard deviation ( $\sigma _{\theta ,d} \approx 0.1$ ).

Figure 4 plots the distribution of discrimination estimates by factor and issue domain.Footnote ²⁰ As the leftmost panel indicates, Factor 1 is predominantly defined by immigration and social issues. Given that larger values indicate more rightwing responses, this factor can thus be interpreted as representing conservatism in these domains. The discriminations of economic items tend to be smaller in absolute magnitude and to have the opposite sign, especially in the relative economic domain. This is consistent with COW’s observation that social/immigration conservatism is negatively correlated with economic, especially relative economic, conservatism.

Figure 4 Box plots of the distribution of varimax-rotated discrimination estimates by domain and factor.

Factor 2 is dominated by economic items, especially absolute ones. This factor may therefore be interpreted as capturing variation in economic conservatism orthogonal to Factor 1.Footnote ²¹ Social items tend to load negatively on this dimension, again indicating their inverse relationship with economic preferences. Factor 3, being dominated by immigration items, represents distinctive preferences on this domain orthogonal to the other factors. Factor 4 represents something similar on social issues, particularly those related to gay rights and gender relations.

Figure 5 Predictors of varimax-rotated group ideal points. Each dot represents the average difference in $\bar {\theta }_{gt}$ between groups with the indicated attribute and those in the baseline category (hollow square).

Figure 5, which plots the predictors of group ideal points, provides further insight into the meaning of each factor. First, as the top row of panels indicators, three of the four factors have trended lower over time. The exception is Factor 3 (immigration-specific conservatism), which underwent a durable increase in the early 2000s. The second row shows that older Europeans are to the right of younger ones on Factor 1 (immigration and social issues) and to a lesser extent Factor 4 (social), but are slightly to their left on Factor 2 (economics). Gender differences (third row) are absent on Factor 1 (social/immigration) but evident on Factor 2 (economics) as well as on sexuality-related Factor 4, with men being more conservative than women.

The bottom row Figure 5 displays country differences relative to Germany (the country whose estimates are the most precise). On Factor 1, Sweden and other Nordic countries anchor the left wing, whereas the right wing is dominated by countries in Southern and Eastern Europe, such as Cyprus. This pattern is reversed on Factor 2, with countries such as Greece and Spain on the left and Denmark and the Netherlands on the right. On Factor 3, Switzerland is the most leftwing and Hungary the most rightwing, reflecting these countries’ distinctive positions on immigration given their general social and immigration conservatism captured in Factor 1. Finally, the Factor 4 scores indicate the relatively progressive sexual attitudes of the French and the relative conservatism of Lithuanians.

Following COW, we can trace out the dynamics of national publics as a whole by poststratifying the group estimates to match the composition of national populations at each point in time. Figure 6 plots the resulting estimates for one illustrative county on each dimension. In the top-left panel (Factor 1), Germany illustrates the general Europe-wide trend to the left on social and immigration issues. The top-right panel (Factor 2) displays the more idiosyncratic dynamics of the Czech public, whose economic conservatism declined in the wake of the 2008 financial crisis before increasing sharply after 2012. The bottom-left (Factor 3) shows Italy’s steady trend to the right on the immigration-specific dimension, while the bottom-right (Factor 4) plots the decline in Spaniards’ sexual traditionalism since the turn of the century.

Figure 6 Poststratified group estimates matching the composition of national populations at each point in time.

Most of the specific patterns we uncover, including the cleavages across time, age, gender, and geographic region, are consistent with those reported by COW. Nevertheless, the MODGIRT model offers a meaningfully different perspective on European public opinion compared to COW’s domain-specific measures. First, although items in different domains load differentially on the four factors we uncover (see Figure 4), few items load solely on a single factor. In fact, of the 100 items we analyze, 86 have discrimination parameters distinguishable from zero on at least two factors.

More specifically, we find little evidence to support COW’s distinction between absolute and relative economic items. While economic items do vary substantially in the magnitude and even the sign of their discrimination on a given factor, almost none of this variation is explained by classification as absolute or relative.Footnote ²² The fact that Factors 3 and 4, respectively pick up variation specific to immigration and social issues provides greater support for a distinction between these domains. Nevertheless, variation in both domains is still dominated by Factor 1, on which immigration and social items load equally strongly.

In sum, our exploratory approach does not recover the sharp distinctions across issue domains assumed by COW’s confirmatory approach. Rather, it points to a dominant factor related primarily to immigration and social issues, a secondary factor related to economic issues, and two minor factors related to immigration and sexuality.

Whether an exploratory or a confirmatory approach is superior may depend on the research purpose. An important advantage of COW’s unidimensional domain-specific scales is ease of interpretation. Since unidimensional scales capture variation common to items in a given domain, they can be more straightforwardly interpreted as indicating leftwing versus rightwing attitudes in that domain. By contrast, unless the factors have a truly simple structure, MODGIRT factor scores should not be interpreted in isolation but rather in conjunction with scores on other dimensions.

It is possible to use MODGIRT in a confirmatory mode similar to COW’s. One way to do so is by constraining items to load only on a subset of factors (cf. restriction 4 in Section 2.2). For example, COW’s confirmatory analysis could be reproduced by fitting a four-dimensional MODGIRT model and then applying an oblique rotation to a target matrix with zeroes to indicate the item-dimension loadings ruled out by assumption. An alternative approach is to use the MODGIRT model to predict the probability of a rightwing response on each item. These probabilities could then be averaged across items in a given domain to produce a domain-specific analog to the “conservative vote probabilities” suggested by Fowler and Hall (Reference Fowler and Hall2017).

5.1 Dirichlet-Multinomial Response Distribution

The binomial version of the MODGIRT model defines group g’s probability at time t of responding positively to item q as

$$\begin{align*}p_{gqt} = \Phi \left( \frac{\boldsymbol{\beta}_q'\bar{\boldsymbol{\theta}}_{gt} -\alpha_{q}} {\sqrt{\boldsymbol{\beta}_q' \boldsymbol{\Sigma}_{\theta}\boldsymbol{\beta}_q + 1}} \right). \end{align*}$$

An alternative approach, suggested by McGann (Reference McGann2014), is to instead treat $p_{gqt}$ as the mean of a Beta prior for the actual response probability $\pi _{gqt} = \text {Pr}(y_{gqt} = 1)$ :

$$\begin{align*}\pi_{gqt} \sim \text{Beta} \left(\phi p_{gqt},\; \phi(1 - p_{gqt}) \right), \end{align*}$$

where the dispersion parameter $\phi $ governs the precision of the prior. For a multinomial model, the analog to the Beta is the Dirichlet distribution,

$$\begin{align*}\boldsymbol{\pi}_{gqt} \sim \text{Dirichlet} (A \mathbf{p}_{gqt}), \end{align*}$$

where A plays the same role as $\phi $ .

The advantage of the Dirichlet-multinomial formulation of the model is that it allows for overdispersion. Specifically, it allows each $\pi _{gqt}$ to deviate from its expected value to a degree inversely proportional to A, a parameter to be estimated. Claassen (Reference Claassen2019) presents evidence that this sort of model provides better uncertainty estimates than a model without such a prior.

5.2 Differential Item Functioning

A related concern is differential item functioning (Brady Reference Brady1985). One dimension along which items might differ in their relationship to a latent trait is across time. For example, in the United States attitudes towards homosexuality and abortion are both strong indicators of social conservatism, but since the 1970s the former have liberalized much more than the latter. In other words, the “difficulty” of expressing conservative views on homosexuality has increased relative to abortion. These sorts of item-specific time trends can be accommodated by allowing $\alpha _{qt}$ to evolve across periods like $\bar {\theta }_{gt}$ does (e.g., Caughey and Warshaw Reference Caughey and Warshaw2016).

Items can also function differently across countries (Stegmueller Reference Stegmueller2011). For example, if residents of France interpret “politics” as referring specifically to partisan politics while Finns interpret it more expansively, then expressions of “interest in politics” will be biased downward in the former relative to the latter (Tarrow Reference Tarrow1971). Claassen (Reference Claassen2019) proposes to address this possibility by defining each item difficulty as the sum of a population-level mean and a random effect that differs across countries (see also Solt Reference Solt2020). In a MODGIRT model estimated on subnational groups, this approach may be generalized by modeling variation in item difficulties across other factors, such as race or gender, in addition to or instead of geographical units.

5.3 Hierarchical Model for Group Means

A final extension is to enrich the dynamic model for $\bar {\theta }_{gt}$ with additional covariates $\mathbf {x}_g$ , yielding a model of the form

$$\begin{align*}\bar{\theta}_{gt} \sim \mathcal{N}(\mu + \rho \bar{\theta}_{g,t-1} + \mathbf{x}_g\boldsymbol{\gamma}_t). \end{align*}$$

As has been shown in the context of multilevel regression and poststratification (Lax and Phillips Reference Lax and Phillips2009), hierarchical models of this sort are of greatest use in contexts where the data are relatively sparse, resulting in the sample including few or no respondents from some groups. Models along these lines are used in Caughey and Warshaw (Reference Caughey and Warshaw2015) and Caughey, O’Grady, and Warshaw (Reference Caughey, O’Grady and Warshaw2019a). Caughey and Warshaw (Reference Caughey and Warshaw2015) additionally model the coefficients $\boldsymbol {\gamma }_t$ corresponding to geographic units with attributes of those units (e.g., partisan vote share).