Hostname: page-component-5db58dd55d-h5th4 Total loading time: 0 Render date: 2026-06-09T19:31:18.812Z Has data issue: false hasContentIssue false
  • English
  • Français

Dynamic Estimation of Latent Opinion Using a Hierarchical Group-Level IRT Model

Published online by Cambridge University Press:  04 January 2017

Devin Caughey  and
Christopher Warshaw
Devin Caughey*
Affiliation:
Department of Political Science, Massachusetts Institute of Technology, Cambridge, MA 02139-4301, USA
Christopher Warshaw
Affiliation:
Department of Political Science, Massachusetts Institute of Technology, Cambridge, MA 02139-4301, USA, e-mail: cwarshaw@mit.edu
*
e-mail: caughey@mit.edu (corresponding author)
Get access Rights & Permissions [Opens in a new window]

Abstract

Over the past eight decades, millions of people have been surveyed on their political opinions. Until recently, however, polls rarely included enough questions in a given domain to apply scaling techniques such as IRT models at the individual level, preventing scholars from taking full advantage of historical survey data. To address this problem, we develop a Bayesian group-level IRT approach that models latent traits at the level of demographic and/or geographic groups rather than individuals. We use a hierarchical model to borrow strength cross-sectionally and dynamic linear models to do so across time. The group-level estimates can be weighted to generate estimates for geographic units. This framework opens up vast new areas of research on historical public opinion, especially at the subnational level. We illustrate this potential by estimating the average policy liberalism of citizens in each U.S. state in each year between 1972 and 2012.

Information

Type
Articles
Information
Political Analysis , Volume 23 , Issue 2 , Spring 2015 , pp. 197 - 211
Copyright
Copyright © The Author 2015. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

Footnotes

Authors' note: We are grateful to Kevin Quinn, Simon Jackman, and Teppei Yamamoto for their advice on the model derivation and validation, and to Bob Carpenter and Alex Storer for their assistance with coding and computation. We also received excellent feedback from Stephen Jessee, Bob Erikson, Mike Alvarez, John Jackson, and others at PolMeth 2013. Adam Berinsky, Eric Schickler, and Tom Clark were kind enough to share their data with us. We appreciate the research assistance of Stephen Brown, Justin de Benedictis-Kessner, and Melissa Meek. Supplementary materials for this article are available on the Political Analysis Web site.

References

Adcock, Robert, and Collier, David. 2001. Measurement Validity: A Shared Standard for Qualitative and Quantitative Research. American Political Science Review 95(3): 529–46.10.1017/S0003055401003100CrossRefGoogle Scholar
Ansolabehere, Stephen, Rodden, Jonathan, and Snyder, James M. Jr. 2008. The Strength of Issues: Using Multiple Measures to Gauge Preference Stability, Ideological Constraint, and Issue Voting. American Political Science Review 102(2): 215–32.10.1017/S0003055408080210CrossRefGoogle Scholar
Armstrong, David A., Bakker, Ryan, Carroll, Royce, Hare, Christopher, Poole, Keith T., and Rosenthal, Howard. 2014. Analyzing Spatial Models of Choice and Judgment with R. Boca Raton, FL: CRC Press.10.1201/b16486CrossRefGoogle Scholar
Bafumi, Joseph, and Herron, Michael C. 2010. Leapfrog Representation and Extremism: A Study of American Voters and Their Members in Congress. American Political Science Review 104(3): 519–42.10.1017/S0003055410000316CrossRefGoogle Scholar
Bailey, Michael. 2001. Ideal Point Estimation with a Small Number of Votes: A Random-Effects Approach. Political Analysis 9(3): 192–210.10.1093/polana/9.3.192CrossRefGoogle Scholar
Berry, William D., Ringquist, Evan J., Fording, Richard C., and Hanson, Russell L. 1998. Measuring Citizen and Government Ideology in the American States, 1960–93. American Journal of Political Science 42(1): 327–48.CrossRefGoogle Scholar
Buttice, Matthew K., and Highton, Benjamin. 2013. How Does Multilevel Regression and Poststratification Perform with Conventional National Surveys? Political Analysis 21(4): 449–67.10.1093/pan/mpt017CrossRefGoogle Scholar
Caughey, Devin, and Warshaw, Christopher. 2014. Replication Data for: Dynamic Estimation of Latent Opinion from Sparse Survey Data Using a Group-Level IRT Model. http://dx.doi.org/10.7910/DVN/27899. Dataverse [Distributor] V1 [Version].CrossRefGoogle Scholar
Clinton, Joshua, Jackman, Simon, and Rivers, Douglas. 2004. The Statistical Analysis of Roll Call Data. American Political Science Review 98(2): 355–70.10.1017/S0003055404001194CrossRefGoogle Scholar
Ellis, Christopher, and Stimson, James A. 2012. Ideology in America. New York: Cambridge University Press.10.1017/CBO9781139094009CrossRefGoogle Scholar
Enns, Peter K., and Koch, Julianna. 2013. Public Opinion in the U.S. States: 1956 to 2010. State Politics and Policy Quarterly 13(3): 349–72.10.1177/1532440013496439CrossRefGoogle Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 1993. Statehouse Democracy: Public Opinion and Policy in the American States. New York: Cambridge University Press.Google Scholar
Erikson, Robert S., Wright, Gerald C., and McIver, John P. 2006. Public Opinion in the States: A Quarter Century of Change and Stability. In Public Opinion in State Politics, ed. Cohen, Jeffrey E., 229–53. Palo Alto, CA: Stanford University Press.Google Scholar
Fiorina, Morris P., and Abrams, Samuel J. 2008. Political Polarization in the American Public, Annual Review of Political Science 11(1): 563–88.CrossRefGoogle Scholar
Fox, Jean-Paul. 2010. Bayesian Item Response Modeling: Theory and Applications. New York: Springer (PDF ebook).10.1007/978-1-4419-0742-4CrossRefGoogle Scholar
Fox, Jean-Paul, and Glas, Cees A. W. 2001. Bayesian Estimation of a Multilevel IRT Model Using Gibbs Sampling. Psychometrika 66(2): 271–88.10.1007/BF02294839CrossRefGoogle Scholar
Gelman, Andrew. 2007. Prior Distributions for Variance Parameters in Hierarchical Models. Bayesian Analysis 1(3): 515–33.Google Scholar
Ghitza, Yair, and Gelman, Andrew. 2013. Deep Interactions with MRP: Election Turnout and Voting Patterns among Small Electoral Subgroups. American Journal of Political Science 57(3): 762–76.10.1111/ajps.12004CrossRefGoogle Scholar
Hoffman, Matthew D., and Gelman, Andrew. Forthcoming. The No-U-Turn Sampler: Adaptively Setting Path Lengths in Hamiltonian Monte Carlo. Journal of Machine Learning Research.Google Scholar
Jackman, Simon. 2005. Pooling the Polls over an Election Campaign. Australian Journal of Political Science 40(4): 499–517.10.1080/10361140500302472CrossRefGoogle Scholar
Jackman, Simon. 2009. Bayesian Analysis for the Social Sciences. Hoboken, NJ: Wiley.10.1002/9780470686621CrossRefGoogle Scholar
James, Gareth, Witten, Trevor Hastie, Daniela, and Tibshirani, Robert. 2013. An Introduction to Statistical Learning. New York: Springer (PDF ebook).CrossRefGoogle Scholar
Jessee, Stephen A. 2009. Spatial Voting in the 2004 Presidential Election. American Political Science Review 103(1): 59–81.10.1017/S000305540909008XCrossRefGoogle Scholar
Kernell, Georgia. 2009. Giving Order to Districts: Estimating Voter Distributions with National Election Returns. Political Analysis 17(3): 215–35.10.1093/pan/mpp002CrossRefGoogle Scholar
Lax, Jeffrey R., and Phillips, Justin H. 2009. How Should We Estimate Public Opinion in The States? American Journal of Political Science 53(1): 107–21.10.1111/j.1540-5907.2008.00360.xCrossRefGoogle Scholar
Levendusky, Matthew S., Pope, Jeremy C., and Jackman, Simon D. 2008. Measuring District-Level Partisanship with Implications for the Analysis of US Elections. Journal of Politics 70(3): 736–53.10.1017/S0022381608080729CrossRefGoogle Scholar
Lewis, Jeffrey B. 2001. Estimating Voter Preference Distributions from Individual-Level Voting Data. Political Analysis 9(3): 275–97.10.1093/polana/9.3.275CrossRefGoogle Scholar
Linzer, Drew A. 2013. Dynamic Bayesian Forecasting of Presidential Elections in the States. Journal of the American Statistical Association 108(501): 124–34.10.1080/01621459.2012.737735CrossRefGoogle Scholar
Martin, Andrew D., and Quinn, Kevin M. 2002. Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999. Political Analysis 10(2): 134–53.10.1093/pan/10.2.134CrossRefGoogle Scholar
McGann, Anthony J. 2014. Estimating the Political Center from Aggregate Data: An Item Response Theory Alternative to the Stimson Dyad Ratios Algorithm. Political Analysis 22(1): 115–29.10.1093/pan/mpt022CrossRefGoogle Scholar
Mislevy, Robert J. 1983. Item Response Models for Grouped Data. Journal of Educational Statistics 8(4): 271–88.10.3102/10769986008004271CrossRefGoogle Scholar
Park, David K., Gelman, Andrew, and Bafumi, Joseph. 2004. Bayesian Multilevel Estimation with Poststratification: State-Level Estimates from National Polls. Political Analysis 12(4): 375–85.10.1093/pan/mph024CrossRefGoogle Scholar
Park, Jong Hee. 2012. A Unified Method for Dynamic and Cross-Sectional Heterogeneity: Introducing Hidden Markov Panel Models. American Journal of Political Science 56(4): 1040–54.10.1111/j.1540-5907.2012.00590.xCrossRefGoogle Scholar
R Core Team. 2013. R: A Language and Environment for Statistical Computing. Vienna, Austria. R Foundation for Statistical Computing, http://www.R-project.org/.Google Scholar
Ruggles, Steven J., Alexander, Trent, Genadek, Katie, Goeken, Ronald, Schroeder, Matthew B., and Sobek, Matthew. 2010. Integrated Public Use Microdata Series: Version 5.0 [Machine-readable database]. Minneapolis: University of Minnesota.Google Scholar
Stan Development Team. Stan: A C++ Library for Probability and Sampling, Version 1.3. http://mc-stan.org/.Google Scholar
Stimson, James A. 1991. Public Opinion in America: Moods, Cycles, and Swings. Boulder, CO: Westview.Google Scholar
Stimson, James A. 1999. Public Opinion in America: Moods, Cycles, and Swings. 2nd ed. Boulder, CO: Westview.Google Scholar
Stimson, James A. 2012. On the Meaning & Measurement of Mood. Daedalus 141(4): 23–34.10.1162/DAED_a_00171CrossRefGoogle Scholar
Tausanovitch, Chris, and Warshaw, Christopher. 2013. Measuring Constituent Policy Preferences in Congress, State Legislatures and Cities. Journal of Politics 75(2): 330–42.10.1017/S0022381613000042CrossRefGoogle Scholar
Warshaw, Christopher, and Rodden, Jonathan. 2012. How Should We Measure District-Level Public Opinion on Individual Issues? Journal of Politics 74(1): 203–19.10.1017/S0022381611001204CrossRefGoogle Scholar
Wawro, Gregory J., and Katznelson, Ira. 2013. Designing Historical Social Scientific Inquiry: How Parameter Heterogeneity Can Bridge the Methodological Divide between Quantitative and Qualitative Approaches. American Journal of Political Science 58(2): 526–46.Google Scholar
Supplementary material: PDF

Caughey and Warshaw supplementary material

Supplementary Material

Download Caughey and Warshaw supplementary material(PDF)
PDF 1.2 MB