Skip to main content

Dynamic Estimation of Latent Opinion Using a Hierarchical Group-Level IRT Model

  • Devin Caughey (a1) and Christopher Warshaw (a2)

Over the past eight decades, millions of people have been surveyed on their political opinions. Until recently, however, polls rarely included enough questions in a given domain to apply scaling techniques such as IRT models at the individual level, preventing scholars from taking full advantage of historical survey data. To address this problem, we develop a Bayesian group-level IRT approach that models latent traits at the level of demographic and/or geographic groups rather than individuals. We use a hierarchical model to borrow strength cross-sectionally and dynamic linear models to do so across time. The group-level estimates can be weighted to generate estimates for geographic units. This framework opens up vast new areas of research on historical public opinion, especially at the subnational level. We illustrate this potential by estimating the average policy liberalism of citizens in each U.S. state in each year between 1972 and 2012.

Corresponding author
e-mail: (corresponding author)
Hide All

Authors' note: We are grateful to Kevin Quinn, Simon Jackman, and Teppei Yamamoto for their advice on the model derivation and validation, and to Bob Carpenter and Alex Storer for their assistance with coding and computation. We also received excellent feedback from Stephen Jessee, Bob Erikson, Mike Alvarez, John Jackson, and others at PolMeth 2013. Adam Berinsky, Eric Schickler, and Tom Clark were kind enough to share their data with us. We appreciate the research assistance of Stephen Brown, Justin de Benedictis-Kessner, and Melissa Meek. Supplementary materials for this article are available on the Political Analysis Web site.

Hide All
Adcock, Robert, and Collier, David. 2001. Measurement Validity: A Shared Standard for Qualitative and Quantitative Research. American Political Science Review 95(3): 529–46.
Ansolabehere, Stephen, Rodden, Jonathan, and Snyder, James M. Jr. 2008. The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting. American Political Science Review 102(2): 215–32.
Armstrong, David A., Bakker, Ryan, Carroll, Royce, Hare, Christopher, Poole, Keith T., and Rosenthal, Howard. 2014. Analyzing Spatial Models of Choice and Judgment with R. Boca Raton, FL: CRC Press.
Bafumi, Joseph, and Herron, Michael C. 2010. Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress. American Political Science Review 104(3): 519–42.
Bailey, Michael. 2001. Ideal Point Estimation with a Small Number of Votes: A Random-Effects Approach. Political Analysis 9(3): 192210.
Berry, William D., Ringquist, Evan J., Fording, Richard C., and Hanson, Russell L. 1998. Measuring Citizen and Government Ideology in the American States, 1960–93. American Journal of Political Science 42(1): 327–48.
Buttice, Matthew K., and Highton, Benjamin. 2013. How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys? Political Analysis 21(4): 449–67.
Caughey, Devin, and Warshaw, Christopher. 2014. Replication Data for: Dynamic Estimation of Latent Opinion from Sparse Survey Data Using a Group-Level IRT Model. Dataverse [Distributor] V1 [Version].
Clinton, Joshua, Jackman, Simon, and Rivers, Douglas. 2004. The Statistical Analysis of Roll Call Data. American Political Science Review 98(2): 355–70.
Ellis, Christopher, and Stimson, James A. 2012. Ideology in America. New York: Cambridge University Press.
Enns, Peter K., and Koch, Julianna. 2013. Public Opinion in the U.S. States: 1956 to 2010. State Politics and Policy Quarterly 13(3): 349–72.
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1993. Statehouse Democracy: Public Opinion and Policy in the American States. New York: Cambridge University Press.
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 2006. Public Opinion in the States: A Quarter Century of Change and Stability. In Public Opinion in State Politics, ed. Cohen, Jeffrey E., 229–53. Palo Alto, CA: Stanford University Press.
Fiorina, Morris P., and Abrams, Samuel J. 2008. Political Polarization in the American Public, Annual Review of Political Science 11(1): 563–88.
Fox, Jean-Paul. 2010. Bayesian Item Response Modeling: Theory and Applications. New York: Springer (PDF ebook).
Fox, Jean-Paul, and Glas, Cees A. W. 2001. Bayesian Estimation of a Multilevel IRT Model Using Gibbs Sampling. Psychometrika 66(2): 271–88.
Gelman, Andrew. 2007. Prior Distributions for Variance Parameters in Hierarchical Models. Bayesian Analysis 1(3): 515–33.
Ghitza, Yair, and Gelman, Andrew. 2013. Deep Interactions with MRP: Election Turnout and Voting Patterns among Small Electoral Subgroups. American Journal of Political Science 57(3): 762–76.
Hoffman, Matthew D., and Gelman, Andrew. Forthcoming. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research.
Jackman, Simon. 2005. Pooling the Polls over an Election Campaign. Australian Journal of Political Science 40(4): 499517.
Jackman, Simon. 2009. Bayesian Analysis for the Social Sciences. Hoboken, NJ: Wiley.
James, Gareth, Witten, Trevor Hastie, Daniela, and Tibshirani, Robert. 2013. An Introduction to Statistical Learning. New York: Springer (PDF ebook).
Jessee, Stephen A. 2009. Spatial Voting in the 2004 Presidential Election. American Political Science Review 103(1): 5981.
Kernell, Georgia. 2009. Giving Order to Districts: Estimating Voter Distributions with National Election Returns. Political Analysis 17(3): 215–35.
Lax, Jeffrey R., and Phillips, Justin H. 2009. How Should We Estimate Public Opinion in The States? American Journal of Political Science 53(1): 107–21.
Levendusky, Matthew S., Pope, Jeremy C., and Jackman, Simon D. 2008. Measuring District-Level Partisanship with Implications for the Analysis of US Elections. Journal of Politics 70(3): 736–53.
Lewis, Jeffrey B. 2001. Estimating Voter Preference Distributions from Individual-Level Voting Data. Political Analysis 9(3): 275–97.
Linzer, Drew A. 2013. Dynamic Bayesian Forecasting of Presidential Elections in the States. Journal of the American Statistical Association 108(501): 124–34.
Martin, Andrew D., and Quinn, Kevin M. 2002. Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999. Political Analysis 10(2): 134–53.
McGann, Anthony J. 2014. Estimating the Political Center from Aggregate Data: An Item Response Theory Alternative to the Stimson Dyad Ratios Algorithm. Political Analysis 22(1): 115–29.
Mislevy, Robert J. 1983. Item Response Models for Grouped Data. Journal of Educational Statistics 8(4): 271–88.
Park, David K., Gelman, Andrew, and Bafumi, Joseph. 2004. Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls. Political Analysis 12(4): 375–85.
Park, Jong Hee. 2012. A Unified Method for Dynamic and Cross-Sectional Heterogeneity: Introducing Hidden Markov Panel Models. American Journal of Political Science 56(4): 1040–54.
R Core Team. 2013. R: A Language and Environment for Statistical Computing. Vienna, Austria. R Foundation for Statistical Computing,
Ruggles, Steven J., Alexander, Trent, Genadek, Katie, Goeken, Ronald, Schroeder, Matthew B., and Sobek, Matthew. 2010. Integrated Public Use Microdata Series: Version 5.0 [Machine-readable database]. Minneapolis: University of Minnesota.
Stan Development Team. Stan: A C++ Library for Probability and Sampling, Version 1.3.
Stimson, James A. 1991. Public Opinion in America: Moods, Cycles, and Swings. Boulder, CO: Westview.
Stimson, James A. 1999. Public Opinion in America: Moods, Cycles, and Swings. 2nd ed. Boulder, CO: Westview.
Stimson, James A. 2012. On the Meaning & Measurement of Mood. Daedalus 141(4): 2334.
Tausanovitch, Chris, and Warshaw, Christopher. 2013. Measuring Constituent Policy Preferences in Congress, State Legislatures and Cities. Journal of Politics 75(2): 330–42.
Warshaw, Christopher, and Rodden, Jonathan. 2012. How Should We Measure District-Level Public Opinion on Individual Issues? Journal of Politics 74(1): 203–19.
Wawro, Gregory J., and Katznelson, Ira. 2013. Designing Historical Social Scientific Inquiry: How Parameter Heterogeneity Can Bridge the Methodological Divide between Quantitative and Qualitative Approaches. American Journal of Political Science 58(2): 526–46.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Analysis
  • ISSN: 1047-1987
  • EISSN: 1476-4989
  • URL: /core/journals/political-analysis
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
Type Description Title
Supplementary materials

Caughey and Warshaw supplementary material
Supplementary Material

 PDF (1.2 MB)
1.2 MB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed