Skip to main content
×
×
Home

“Don’t Know” Responses, Personality, and the Measurement of Political Knowledge*

  • Stephen A. Jessee
Abstract

A prominent worry in the measurement of political knowledge is that respondents who say they don’t know the answer to a survey question may have partial knowledge about the topic—more than respondents who answer incorrectly but less than those who answer correctly. It has also been asserted that differentials in respondents’ willingness to guess, driven strongly by personality, can bias traditional knowledge measures. Using a multinomial probit item response model, I show that, contrary to previous claims that “don’t know” responses to political knowledge questions conceal a good deal of “hidden knowledge,” these responses are actually reflective of less knowledge, not only than correct responses but also than incorrect answers. Furthermore, arguments that the meaning of “don’t know” responses varies strongly by respondent personality type are incorrect. In fact, these results hold for high- and low-trait respondents on each of the five most commonly used core personality measures.

Copyright
Footnotes
Hide All
*

Stephen A. Jessee, Associate Professor, Department of Government, The University of Texas at Austin, 1 University Station A1800 Austin, TX 78712 (sjessee@utexas.edu). The author thanks John Bullock for extensive input and assistance throughout the course of this project and Rob McCulloch for sharing his code and providing extremely helpful advice. This project also benefited greatly from the input of Doug Rivers, Adam Bonica, James Scott, Scott Moser, Tse-min Lin, Chris Wlezian, Seth Hill, Alex Tahk, Christopher Fariss, and participants at the UCSD Methods Workshop and the University of Texas American Politics Workshop. To view supplementary material for this article, please visit http://dx.doi.org/10.1017/psrm.2015.23

Footnotes
References
Hide All
Anderson, Carolyn J., Verkuilen, Jay, and Peyton, Buddy L.. 2010. ‘Modeling Polytomous Item Responses Using Simultaneously Estimated Multinomial Logistic Regression Models’. Journal of Educational and Behavioral Statistics 35(4):422452.
Ansolabehere, Stephen, and Rivers, Douglas. 2013. ‘Cooperative Survey Research’. Annual Review of Political Science 16:307329.
Bafumi, Joeseph, and Shapiro, Ian. 2009. ‘A New Partisan Voter’. Journal of Politics 71(1):124.
Bock, Darrell R. 1972. ‘Estimating Item Parameters and Latent Ability When Responses are Scored in Two or More Nominal Categories’. Psychometrika 37(1):2951.
Boudreau, Cheryl, and Lupia, Arthur. 2011. ‘Political Knowledge’. In James N. Druckman, Donald P. Green, James H. Kuklinski, and Arthur Lupia (eds), Cambridge Handbook of Experimental Political Science, pp. 171--86. New York, USA: Cambridge University Press.
Clinton, Joshua, Jackman, Simon, and Rivers, Douglas. 2004. ‘The Statistical Analysis of Roll Call Data: A Unified Approach’. American Political Science Review 98:355370.
Delli Carpini, Michael X., and Keeter, Scott. 1993. ‘Measuring Political Knowledge: Putting First Things First’. American Journal of Political Science 37(4):11791206.
Delli Carpini, Michael X., and Keeter, Scott. 1996. What Americans Know About Politics and Why It Matters. New Haven, CT: Yale University Press.
Funder, David C. 2001. ‘Personality’. Annual Review of Psychology 52:197221.
Furnham, Adrian, and Medhurst, Sarah. 1995. ‘Personality Correlates of Academic Seminar Behaviour: A Study of Four Instruments’. Personality and Individual Differences 19(2):197208. Available at http://www.sciencedirect.com/science/article/B6V9F-3YB56P1-2H/2/80c0fee51b8e315e81617475020cd3f8.
Gerber, Alan S., Huber, Gregory A., and Washington, Ebonya. 2010. ‘Party Affiliation, Partisanship and Political Beliefs: A Field Experiment’. American Political Science Review 104(4):720745.
Gibson, James L., and Caldeira, Gregory A.. 2009. ‘Knowing the Supreme Court? A Reconsideration of Public Ignorance of the High Court’. Journal of Politics 71(2):429441.
Goldberg, Lewis R. 1990. ‘An Alternative “Description of Personality”: The Big-Five Factor Structure’. Journal of Personality and Social Psychology 59(6):12161229.
Gosling, Samuel D., Rentfrow, Peter J., and Swann, William B. Jr. 2003. ‘A Very Brief Measure of the Big-Five Personality Domains’. Journal of Research in Personality 37(6):504528.
Groseclose, Tim, and Milyo, Jefferey. 2005. ‘A Measure of Media Bias’. Quarterly Journal of Economics CXX(4):11911237.
Heckman, James J., and Snyder, James M.. 1997. ‘Linear Probability Models of the Demand for Attributes with an Empirical Application to Estimating the Preferences of Legislators’. RAND Journal of Economics 28:142189.
Iyengar, Shanto. 1986. ‘Whither Political Information?’ American National Election Studies: Pilot Study Report No. 002253. http://www.electionstudies.org/resources/papers/documents/nes002253.pdf
Jackman, Simon. 2000. ‘Estimation and Inference are Missing Data Problems: Unifying Social Science Statistics via Bayesian Simulation’. Political Analysis 8:307332.
Jackman, Simon. 2009. pscl: Political Science Computational Laboratory (R library).
Jessee, Stephen A. 2009. ‘Spatial Voting in the 2004 Presidential Election’. American Political Science Review 103(1):5981.
Jessee, Stephen A.. 2010. ‘Partisan Bias, Political Information and Spatial Voting in the 2008 Presidential Election’. Journal of Politics 72(2):327340.
Keane, Micheal P. 1992. ‘A Note on Identification in the Multinomial Probit Model’. Journal of Business & Economic Statistics 10(2):193200.
Krosnick, Jon A., Lupia, Arthur, DeBell, Matthew, and Donakowski, Darrell. 2008. ‘Problems with ANES Questions Measuring Political Knowledge’. ANES Report. Available at http: //electionstudies.org/announce/newsltr/20080324PoliticalKnowledgeMemo.pdf, accessed 8 May 2010.
Lawrence, Christopher N. 2009. ‘Should Voters Be Encyclopedias? Measuring Political Knowledge in the United States and the Netherlands’. Manuscript. Texas A&M International University. Available at http://www.cnlawrence.com/papers/irt-soph.pdf, accessed 27 December 2009.
Lovallo, William R., and Pishkin, Vladimir. 1980. ‘Type A Behavior, Self-Involvement, Autonomic Activity, and the Traits of Neuroticism and Extraversion’. Psychosomatic Medicine 42(3):329334.
Lupia, Arthur. 2006. ‘Elitism and the Study of Democratic Competence’. Critical Review 18(1–3):174.
Luskin, Robert. 2002. ‘From Denial to Extenuation (and Finally Beyond): Political Sophistication and Citizen Performance’. In James H. Kuklinski (ed.), Thinking About Political Psychology, pp. 217--52. New York, NY: Cambridge University Press.
Luskin, Robert, and Bullock, John. 2011. ‘“Don’t Know” Means “Don’t Know”: DK Responses and the Public’s Level of Political Knowledge’. Journal of Politics 73(2):547557.
Martin, Andrew D., and Quinn, Kevin M.. 2002. ‘Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the US Supreme Court, 1953–1999’. Political Analysis 10(2):134153.
Mondak, Jeffrey J. 1999. ‘Reconsidering the Measurement of Political Knowledge’. Political Analysis 8(1):5782.
Mondak, Jeffrey J. 2010. Personality and the Foundations of Political Behavior. New York, NY: Cambridge University Press.
Mondak, Jeffery J., and Halperin, Karen D.. 2008. ‘A Framework for the Study of Personality and Political Behaviour’. British Journal of Mathematical and Statistical Psychology 38(2):335362.
Nadeau, Richard, and Niemi, Richard G.. 1995. ‘Educated Guesses: The Process of Answering Factual Knowledge Questions in Surveys’. Public Opinion Quarterly 59(3):323346.
Noel, Hans. 2007. ‘Multinomial Ideal Point Estimation: When the Decision to Speak is as Important as What You Say’. Poster, Annual Meeting of the Society for Political Methodology, July 18--21, 2007, Pennsylvania State University.
Ozer, Daniel J., and Reise, Steven P.. 1994. ‘Personality Assessment’. Annual Review of Psychology 45:357388.
Poole, Keith T., and Rosenthal, Howard. 1985. ‘A Spatial Model for Legislative Roll Call Analysis’. American Journal of Political Science 29(2):357384.
Prior, Markus, and Lupia, Arthur. 2008. ‘Money, Time, and Political Knowledge: Distinguishing Quick Recall and Political Learning Skills’. American Journal of Political Science 52(1):169183.
R Core Team. 2013. R: A Language and Environment for Statistical Computing. Vienna, Austria: R Foundation for Statistical Computing. Available at http://www.R-project.org/.
Rasch, Georg. 1960a. Probabilistic Models for Some Intelligence and Attainment Tests. Copenhagen: Danmarks Paedogogische Institut.
Rasch, Georg. 1960b. Probabilistic Model for Some Intelligence and Attainment Tests. Copenhagen: Danmarks Paedogogische Institut.
Rossi, Peter. 2012. bayesm: Bayesian Infer ence for Marketing/Micro-Econometrics (R library).
Rossi, Peter E., Allenby, Greg M., and McCulloch, Robert. 2005. Bayesian Statistics and Marketing. New York: John Wiley & Sons.
Schnakenberg, Keith E., and Fariss, Christopher J.. 2014. ‘Dynamic Patterns of Human Rights Practices’. Political Science Research and Methods 2(1):131.
Shor, Boris. 2009. ‘All Together Now: Putting Congress, State Legislatures, and Individuals in a Common Ideological Space’. Working Paper, University of Chicago, Chicago, IL.
Sturgis, Patrick, Allum, Nick, and Smith, Patten. 2008. ‘An Experiment on the Measurement of Political Knowledge in Surveys’. Public Opinion Quarterly 85(1):90102.
Swann, William B. Jr., and Rentfrow, Peter J.. 2001. ‘Blirtatiousness: Cognitive, Behavioral, and Physiological Consequences of Rapid Responding’. Journal of Personality and Social Psychology 81(6):11601175.
Treier, Sean, and Jackman, Simon. 2008. ‘Democracy as a Latent Variable’. American Journal of Political Science 52(1):201217.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Political Science Research and Methods
  • ISSN: 2049-8470
  • EISSN: 2049-8489
  • URL: /core/journals/political-science-research-and-methods
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×
Type Description Title
PDF
Supplementary materials

Jessee supplementary material
Online Appendix

 PDF (117 KB)
117 KB
UNKNOWN
Supplementary materials

Jessee supplementary material
Jessee supplementary material 1

 Unknown (117 KB)
117 KB
UNKNOWN
Supplementary materials

Jessee dataset
Datasets

 Unknown

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 34
Total number of PDF views: 323 *
Loading metrics...

Abstract views

Total abstract views: 959 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 21st July 2018. This data will be updated every 24 hours.