Skip to main content

Measuring Bias against Female Political Leadership

  • Mark Setzler (a1)

Much research examining gender bias in politics analyzes responses to explicit survey questions asking individuals whether they prefer male over female leaders or agree that male political leaders are superior. Drawing insights from the measurement of other types of prejudice, this article explores the methodological shortcomings of a widely used question of this type. Analyzing the results of two surveys—one national and one state-level—I compare response patterns to a standard, highly explicit question that is frequently administered by the Pew Research Center with those for a modestly altered item that employs multiple strategies to reduce social desirability bias. Compared with the alternative measure, the conventional item seriously underreports prejudice against women leaders. Moreover, the underreporting of bias is especially prevalent among individuals belonging to groups that are strong advocates of gender equality.

Hide All

The author gratefully acknowledges the many insights provided by Alixandra B. Yanus and Jeff Seward. Martin J. Kifer and High Point University's Survey Research Center provided critical support to the project by fielding survey questions on my behalf. Data and replication syntax will be provided upon request.

Hide All
Alexander, Amy C. 2012. “Change in Women's Descriptive Representation and the Belief in Women's Ability to Govern: A Virtuous Cycle.” Politics & Gender 8 (4): 437–64.
Alexander, Amy C. 2015. “Big Jumps in Women's Presence in Parliaments: Are These Sufficient for Improving Beliefs in Women's Ability to Govern?Advancing Women in Leadership 35: 8297.
Alexander, Amy C., and Welzel, Christian. 2011. “Islam and Patriarchy: How Robust Is Muslim Support for Patriarchal Values?International Review of Sociology 21 (2): 249–76.
Bejarano, Christina E., Manzano, Sylvia, and Montoya, Celeste. 2011. “Tracking the Latino Gender Gap: Gender Attitudes across Sex, Borders, and Generations.” Politics & Gender 7 (4): 521–49.
Ben-Nun Bloom, Pazit. 2016. “State-Level Restriction of Religious Freedom and Women's Rights: A Global Analysis.” Political Studies 64 (4): 832–53.
Benson, Brett V., Merolla, Jennifer L., Geera, John G.. 2011. “Two Steps Forward, One Step Back? Bias in the 2008 Presidential Election.” Electoral Studies 30 (4): 607–20.
Bernstein, Robert, Chadha, Anita, and Montjoy, Robert. 2001. “Overreporting Voting: Why It Happens and Why It Matters.” Public Opinion Quarterly 65 (1): 2244.
Brint, Steven, and Abrutyn, Seth. 2010. “Who's Right about the Right? Comparing Competing Explanations of the Link between White Evangelicals and Conservative Politics in the United States.” Journal for the Scientific Study of Religion 49 (2): 328–50.
Burden, Barry C., Ono, Yoshikuni, and Yamada, Masahiro. 2017. “Reassessing Public Support for a Female President.” Journal of Politics 79 (3): 1073–78.
Calfano, Brian R., and Djupe, Paul A.. 2011. “Not in His Image: The Moderating Effect of Gender on Religious Appeals.” Politics and Religion 4 (2): 338–54.
Claassen, Ryan L. and Ryan, John Barry. 2016. “Social Desirability, Hidden Biases, and Support for Hillary Clinton.” PS: Political Science & Politics 49 (4): 730–35.
Coffé, Hilde, and Dilli, Selin. 2015. “The Gender Gap in Political Participation in Muslim-Majority Countries.” International Political Science Review 36 (5): 526–44.
Cunningham, William A., Nezlek, John B., and Banaji, Mahzarin R.. 2004. “Implicit and Explicit Ethnocentrism: Revisiting the Ideologies of Prejudice.” Personality and Social Psychology Bulletin 30 (10): 1332–46.
Deckman, Melissa. 2010. “Women as Leaders in Protestant Denominations.” In Gender and Women's Leadership: A Reference Handbook, ed. O'Connor, Karen. Thousand Oaks, CA: Sage, 544–54.
Dolan, Kathleen. 2010. “The Impact of Gender Stereotyped Evaluations on Support for Women Candidates.” Political Behavior 32 (1): 6988.
Dolan, Kathleen. 2014. When Does Gender Matter? Women Candidates and Gender Stereotypes in American Elections. New York: Oxford University Press.
Espírito-Santo, Ana. 2016. “Citizens’ Attitudes towards Descriptive Representation: The Case of Women in Portugal.” European Journal of Women's Studies 23 (1): 4359.
Gilens, Martin, Sniderman, Paul M., and Kuklinski, James H.. 1998. “Affirmative Action and the Politics of Realignment.” British Journal of Political Science 28 (1): 159–83.
Hadaway, C. Kirk, Marler, Penny Long, and Chaves, Mark. 1993. “What the Polls Don't Show: A Closer Look at U.S. Church Attendance.” American Sociological Review 58 (6): 741–52.
Harkness, Janet A., van de Vijver, Fons J. R., and Mohler, Peter Ph., eds. 2002. Cross-Cultural Survey Methods. New York: Wiley.
Hayes, Danny. 2011. “When Gender and Party Collide: Stereotyping in Candidate Trait Attribution.” Politics & Gender 7 (2): 133–65.
Hedlund, Ronald D., Freeman, Patricia K., Hamm, Keith E., and Stein, Robert E.. 1979. “The Electability of Women Candidates: The Effects of Sex Role Stereotypes.” Journal of Politics 41 (2): 513–24.
Heerwig, Jennifer A., and McCabe, Brian J.. 2009. “Education and Social Desirability Bias: The Case of a Black Presidential Candidate.” Social Science Quarterly 90 (3): 674–86.
Holman, Mirya, Merolla, Jennifer, and Zechmeister, Elizabeth. 2011. “Sex, Stereotypes, and Security: A Study of the Effects of Terrorist Threat on Assessments of Female Leadership.” Journal of Women, Politics & Policy 32 (3): 173–92.
Inglehart, Ronald, and Baker, Wayne. 2000. “Modernization, Cultural Change, and the Persistence of Traditional Values.” American Sociological Review 65 (1): 1951.
Inglehart, Ronald, and Norris, Pippa. 2003. Rising Tide: Gender Equality and Cultural Change Around the World. New York: Cambridge University Press.
Inglehart, Ronald, Norris, Pippa, and Welzel, Christian. 2002. “Gender Equality and Democracy.” Comparative Sociology 1 (3): 321–45.
Institute for Women's Policy Research. 2015. “Status of Women in the States.” (accessed June 1, 2018).
Jackman, Mary R. 1978. “General and Applied Tolerance: Does Education Increase Commitment to Racial Integration?American Journal of Political Science 22 (2): 302–24.
Janus, Alexander L. 2010. “The Influence of Social Desirability Pressures on Expressed Immigration Attitudes.” Social Science Quarterly 91 (4): 928–46.
Kahn, Kim Fridkin. 1996. The Political Consequences of Being a Woman. New York: Columbia University Press.
Kane, Emily W., and Macaulay, Laura J.. 1993. “Interviewer Gender and Gender Attitudes.” Public Opinion Quarterly 57 (1): 128.
Kawakami, Kerry, and Dovidio, John F.. 2001. “The Reliability of Implicit Stereotyping? Or Confronting Prejudice: Effects of Prejudice Labeling on Stereotype Activation.” Personality and Social Psychology Bulletin 27 (2): 212–25.
Kenski, Kate, and Falk, Erika. 2004. “Of What Is That Glass Ceiling Made? A Study of Attitudes about Women and the Oval Office.” Women & Politics 26 (2): 5780.
Kenski, Kate, and Falk, Erika. 2006. “Sexism versus Partisanship: A New Look at the Question of Whether America Is Ready for a Woman President.” Sex Roles 54 (7–8): 413–28.
Kerevel, Yann P., and Atkeson, Lonna Rae. 2015. “Reducing Stereotypes of Female Political Leaders in Mexico.” Political Research Quarterly 68 (4): 732–44.
Knoll, Benjamin R. 2013. “Assessing the Effect of Social Desirability on Nativism Attitude Responses.” Social Science Research 42 (6): 1587–98.
Krosnick, Jon A., and Presser, Stanley. 2010. “Question and Questionnaire Design.” In Handbook of Survey Research, 2nd ed., eds. Marsden, Peter V. and Wrights, James D.. Bingley, UK: Emerald Group Publishing, 263314.
Krupnikov, Yanna, Piston, Spencer, and Bauer, Nichole M.. 2016. “Saving Face: Identifying Voter Responses to Black Candidates and Female Candidates.” Political Psychology 37 (2): 253–73.
Kunovich, Sheri, and Paxton, Pamela. 2005. “Pathways to Power: The Role of Political Parties in Women's National Political Representation.” American Journal of Sociology 111 (2): 505–52.
Lawless, Jennifer L. 2004. “Women, War, and Winning Elections: Gender Stereotyping in the Post–September 11th Era.” Political Research Quarterly 57 (3): 479–90.
Mo, Cecilia Hyunjung. 2015. “The Consequences of Explicit and Implicit Gender Attitudes and Candidate Quality in the Calculations of Voters.” Political Behavior 37 (2): 357–95.
Morgan, Jana, and Buice, Melissa. 2013. “Latin American Attitudes towards Women in Politics: The Influence of Elite Cues, Female Advancement, and Individual Characteristics.” American Political Science Review 107 (4): 644–52.
Paxton, Pamela, and Hughes, Melanie M.. 2014. Women, Politics, and Power: A Global Perspective. 2nd ed. Thousand Oaks, CA: Sage.
Paxton, Pamela, and Kunovich, Sheri. 2003. “Women's Political Representation: The Importance of Ideology.” Social Forces 82 (1): 87114.
Pew Forum on Religion and Public Life. 2008. “U.S. Religious Landscape Survey.” (accessed August 27, 2018).
Pew Research Center. 2008. “A Paradox in Public Attitudes: Men or Women: Who's the Better Leader?” August 25. (accessed August 27, 2018).
Pew Research Center. 2015. “Women and Leadership: Public Says Women are Equally Qualified, but Barriers Persist.” January 15. (accessed August 27, 2018).
Powell, Richard J. 2013. “Social Desirability Bias in Polling on Same-Sex Marriage Ballot Measures.” American Politics Research 41 (1): 1052–70.
Rizzo, Helen, Abdel-Latif, Abdel-Hamid, and Meyer, Katherine. 2007. “The Relationship between Gender Equality and Democracy: A Comparison of Arab versus Non-Arab Muslim Societies.” Sociology 41 (6): 1151–70.
Rosenthal, Cindy Simon. 1995. “The Role of Gender in Descriptive Representation.” Political Research Quarterly 48 (3): 599611.
Sanbonmatsu, Kira. 2002. “Gender Stereotypes and Vote Choice.” American Journal of Political Science 46 (1): 2034.
Sanbonmatsu, Kira, and Dolan, Kathleen. 2009. “Do Gender Stereotypes Transcend Party?Political Research Quarterly 62 (3): 485–94.
Schaffner, Brian F. 2011. “Racial Salience and the Obama Vote.” Political Psychology 32 (6): 963–88.
Sears, David O., Sidanius, James, and Bobo, Lawrence D., eds. 2000. Racialized Politics: Values, Ideology, and Prejudice in American Public Opinion. Chicago: University of Chicago Press.
Seguino, Stephanie. 2011. “Help or Hindrance? Religion's Impact on Gender Inequality in Attitudes and Outcomes.” World Development 39 (8): 1308–29.
Setzler, Mark. 2015. “Does Religion Bias Individuals against Female Political Leadership in Latin America?The Latin Americanist 59 (4): 4772.
Setzler, Mark, and Yanus, Alixandra B.. 2015. “The Impact of Religion on Voting For Female Congressional Candidates.” Politics and Religion 8 (4): 679–98.
Setzler, Mark, and Yanus, Alixandra B.. 2016. “Evangelical Protestantism and Bias against Female Political Leaders.” Social Science Quarterly 97 (2): 119.
Streb, Matthew J., Burrell, Barbara, Frederick, Brian, and Genovese, Michael A.. 2008. “Social Desirability Effects and Support for a Female American President.” Public Opinion Quarterly 72 (1): 7689.
Tripp, Aili Marie, and Kang, Alice. 2008. “The Global Impact of Quotas: On the Fast Track to Increased Female Legislative Representation.” Comparative Political Studies 41 (3): 338–61.
Valdini, Melody Ellis. 2013. “Electoral Institutions and the Manifestation of Bias: The Effect of the Personal Vote on the Representation of Women.” Politics & Gender 9 (1): 7692.
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Politics & Gender
  • ISSN: 1743-923X
  • EISSN: 1743-9248
  • URL: /core/journals/politics-and-gender
Please enter your name
Please enter a valid email address
Who would you like to send this to? *


Type Description Title
Supplementary materials

Setzler supplementary material
Table A1

 Word (15 KB)
15 KB


Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 0 *
Loading metrics...

Abstract views

Total abstract views: 0 *
Loading metrics...

* Views captured on Cambridge Core between <date>. This data will be updated every 24 hours.

Usage data cannot currently be displayed