Skip to main content Accessibility help
×
Home
Hostname: page-component-568f69f84b-xr9nb Total loading time: 0.208 Render date: 2021-09-22T06:26:11.994Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "metricsAbstractViews": false, "figures": true, "newCiteModal": false, "newCitedByModal": true, "newEcommerce": true, "newUsageEvents": true }

Response Options and the Measurement of Political Knowledge

Published online by Cambridge University Press:  22 June 2021

John G. Bullock*
Affiliation:
Department of Political Science, Northwestern University, Evanston, IL, USA
Kelly Rader
Affiliation:
Department of Political Science, Yale University, New Haven, CT, USA
*Corresponding
*Corresponding author. E-mail: john.bullock@northwestern.edu

Abstract

By many measures, the public knows little about politics. But just how little people seem to know depends on the questions that are put to them. In particular, knowledge levels seem higher when people are asked closed- rather than open-ended questions. In turn, differences between estimated knowledge levels are sometimes attributed to fundamental differences between these types of questions. Building on this previous research, the present study uses a pre-registered experiment conducted with a representative national sample to shed new light on the relationship between question form and knowledge measurement. The authors find that inferences about political knowledge depend less on fundamental differences between open- and closed-ended questions than on two little-appreciated aspects of survey design: the number and difficulty of the response options that accompany closed-ended questions. These aspects of survey design have large effects. Scholars who use the same questions with different response options may reach substantively different conclusions about the public's levels of knowledge.

Type
Letter
Copyright
Copyright © The Author(s), 2021. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ahler, DJ and Goggin, SN (2017) Assessing Political Knowledge: Problems and Solutions in Online Surveys. Working paper. Available from http://www.sgoggin.org/papers/AhlerGoggin.KnowledgePaper.pdf.Google Scholar
Boudreau, C and Lupia, A (2011) Political knowledge. In Druckman, JN et al. (eds), Cambridge Handbook of Experimental Political Science. New York: Cambridge University Press, pp. 171183.CrossRefGoogle Scholar
Bullock, JG and Lenz, G (2019) Partisan bias in surveys. Annual Review of Political Science 22, 325342.CrossRefGoogle Scholar
Bullock, JG and Rader, K (2021), “Replication Data for: Response Options and the Measurement of Political Knowledge”, https://doi.org/10.7910/DVN/YLEY0R, Harvard Dataverse, V1, UNF:6:H+VKzNAx5kyjS5vitBFHXw== [fileUNF]Google Scholar
Converse, PE (2000) Assessing the capacity of mass electorates. Annual Review of Political Science 3, 331353.CrossRefGoogle Scholar
Cor, MK and Sood, G (2016) Guessing and forgetting: a latent class model for measuring learning. Political Analysis 24 (2), 226242.Google Scholar
Delli Carpini, MX and Keeter, S (1996) What Americans Know about Politics and Why it Matters. New Haven, CT: Yale University Press.Google Scholar
Gibson, JL and Caldeira, GA (2009) Knowing the Supreme Court? A reconsideration of public ignorance of the high court. Journal of Politics 71 (2), 429441.CrossRefGoogle Scholar
Gummer, T and Kunz, T (Forthcoming) Relying on external information sources when answering knowledge questions in web surveys. Sociological Methods & Research.Google Scholar
Jessee, SA (2017) ‘Don't know’ responses, personality, and the measurement of political knowledge. Political Science Research and Methods 5 (4), 711731.CrossRefGoogle Scholar
Kinder, DR (1998) Opinion and action in the realm of politics. In Gilbert, DT and Fiske, ST (eds), The Handbook of Social Psychology. 4th ed. New York: McGraw-Hill, pp. 778867.Google Scholar
Krosnick, JA (1991) Response strategies for coping with the cognitive demands of attitude measures in surveys. Applied Cognitive Psychology 5 (3), 213236.CrossRefGoogle Scholar
Krosnick, JA, Visser, PS and Harder, J (2010) The psychological underpinnings of political behavior. In Fiske, ST, Gilbert, DT, and Lindzey, G (eds), The Handbook of Social Psychology. 5th ed. New York: Wiley, pp. 12881342.Google Scholar
Luskin, RC (1987) Measuring political sophistication. American Journal of Political Science 31 (4), 856899.CrossRefGoogle Scholar
Luskin, RC (2002) From denial to extenuation (and finally beyond): political sophistication and citizen performance. In Kuklinski, JH (ed.), Thinking About Political Psychology. New York: Cambridge University Press, pp. 281305.CrossRefGoogle Scholar
Luskin, RC and Bullock, JG (2011) ‘Don't know’ means ‘don't know’: DK responses and the public's level of political knowledge. Journal of Politics 73 (2), 547557.CrossRefGoogle Scholar
Mondak, JJ (2001) Developing valid knowledge scales. American Journal of Political Science 45 (1), 224238.CrossRefGoogle Scholar
Nadeau, R and Niemi, RG (1995) Educated guesses: the process of answering factual knowledge questions in surveys. Public Opinion Quarterly 59 (3), 323346.CrossRefGoogle Scholar
Prior, M (2005) News vs. entertainment: how increasing media choice widens gaps in political knowledge and turnout. American Journal of Political Science 49 (3), 577592.CrossRefGoogle Scholar
Robison, J (2015) Who knows? Question format and political knowledge. International Journal of Public Opinion Research 27 (1), 11.CrossRefGoogle Scholar
Rodriguez, MC (2005) Three options are optimal for multiple-choice items: a meta-analysis of 80 years of research. Educational Measurement: Issues and Practice 24 (2), 313.CrossRefGoogle Scholar
Sen, M (2017) How political signals affect public support for judicial nominations: evidence from a conjoint experiment. Political Research Quarterly 70 (2), 374393.CrossRefGoogle Scholar
Sturgis, P, Allum, N and Smith, P (2008) An experiment on the measure of political knowledge in surveys. Public Opinion Quarterly 85 (1), 90102.CrossRefGoogle Scholar
Tedin, KL and Murray, RW (1979) Public awareness of congressional representatives: recall versus recognition. American Politics Quarterly 7, 509517.CrossRefGoogle Scholar
Tourangeau, R, Rips, LJ and Rasinski, K (2000) The Psychology of Survey Response. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Zaller, J and Feldman, S (1992) A simple theory of the survey response: answering questions versus revealing preferences. American Journal of Political Science 36 (3), 579616.CrossRefGoogle Scholar
Supplementary material: PDF

Bullock and Rader supplementary material

Bullock and Rader supplementary material

Download Bullock and Rader supplementary material(PDF)
PDF 177 KB

Send article to Kindle

To send this article to your Kindle, first ensure no-reply@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

Note you can select to send to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Response Options and the Measurement of Political Knowledge
Available formats
×

Send article to Dropbox

To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

Response Options and the Measurement of Political Knowledge
Available formats
×

Send article to Google Drive

To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

Response Options and the Measurement of Political Knowledge
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *