Nearly every political science survey collects data on gender. As societal understandings of gender have evolved, researchers are giving consideration to new measurement approaches (e.g., Albaugh et al. Reference Albaugh, Harell, Loewen, Rubenson and Stephenson2023; Padgett et al. Reference Padgett, Gutierrez, Wronski and Barari2024). Scholars in and outside political science have produced innovations, as well as lessons in what does not work. Yet, most research has focused on online or phone surveys in North America and Western Europe.

We shift the focus to face-to-face surveys, which present unique challenges for incorporating gender questions, and to the Latin America and Caribbean (LAC) region. Through in-person cognitive interviews, an online experiment, and a survey experiment, we identify and assess an effective approach to incorporating an expansive gender question in these contexts. In so doing, we consider potential trade-offs. Expansive gender questions can yield more accurate data on gender minorities, but asking people to self-identify their gender in face-to-face surveys—rather than interviewers recording their inference on respondent gender—could have the following downsides:

• • Causing discomfort (among cis and/or trans respondents and interviewers)

• • Introducing missingness in a key variable (gender)

• • Increasing attrition and item nonresponse elsewhere in the survey

• • Influencing responses to other questions (e.g., by provoking backlash against LGBTQ+ inclusivity)

We show that most of these concerns can be mitigated by using a carefully designed question. We present an expansive gender question that minimizes (though does not eliminate) discomfort, and we show that this question does not increase attrition or item nonresponse. It does introduce some missingness into the gender variable, but the missingness rate is low (1.4%), especially when weighed against the rate of error in interviewer-inferred gender (up to 0.6%). We find some evidence that the gender question has small effects on responses to questions about LGBTQ+ rights among certain subgroups. In assessing these trade-offs, we identify issues and approaches for researchers to consider as they extend efforts to ask expansive gender questions across place and time.

Expansive Gender Questions

Many opinion scholars, among others, are interested in moving beyond the gender binary, which requires that surveys adapt traditional instruments (Albaugh et al. Reference Albaugh, Harell, Loewen, Rubenson and Stephenson2024; Medeiros, Forest, and Öhberg Reference Medeiros, Forest and Öhberg2020). Asking expansive gender questions can reduce measurement error (Albaugh et al. Reference Albaugh, Harell, Loewen, Rubenson and Stephenson2023). Even when individual samples contain too few gender-minority respondents for analysis, it may be possible to pool multiple surveys to achieve leverage for valid statistical analyses (Lagos and Compton Reference Lagos and Compton2021). Some also consider that the inclusion of expansive gender questions has legitimizing power (Browne Reference Browne, Browne and Nash2016). Surveys serve both instrumental (disseminating facts) and symbolic (signaling public values) roles (Herbst Reference Herbst1993). As such, expansive gender questions can normalize and validate gender minorities. Although we focus on gender in this project, these arguments apply to measures of other LGBTQ+ identities and to comprehensive considerations of the LGBTQ+ community.

In developing expansive gender questions, researchers must consider context-specific understandings, lexicons, and norms. The terminology people use to describe their gender identity varies across subgroups (e.g., by age), time, and place (Puckett et al. Reference Puckett, Brown, Dunn, Mustanski and Newcomb2020). Individuals differ in how much they see sex and gender as distinct constructs (Reisner et al. Reference Reisner, Biello, Rosenberger, Austin, Haneuse, Perez-Brumer, Novak and Mimiaga2014). And contexts vary in terms of norms and laws that stigmatize and create risk for gender minorities (Holzberg et al. Reference Holzberg, Ellis, Kaplan, Virgile and Edgar2019).

To date, survey innovations related to respondent-provided gender identity have centered on industrialized Western countries, with many recommendations focused on phone or online surveys. Experts have recently shifted to approaches that decouple questions about gender identity from those about sex or transgender experience (NASEM 2022). Multi-item modules provide the benefit of identifying transgender respondents, and recent work has provided guidance on how to word such questions (e.g., Albaugh et al. Reference Albaugh, Harell, Loewen, Rubenson and Stephenson2023; see also Puckett et al. Reference Puckett, Brown, Dunn, Mustanski and Newcomb2020).

Scholars simultaneously have made strides in identifying best practices for gender questions. For example, Albaugh et al. (Reference Albaugh, Harell, Loewen, Rubenson and Stephenson2023) use data from the online Canadian Election Studies (CES) to show that response options matter: Including an “other” category that mentions “trans, nonbinary, two-spirit, etc.” lowers the proportion of individuals identifying as men or women because that wording nudges man- and woman-identifying transgender individuals into that catch-all category.

Research in Western contexts finds that asking about gender typically does not evoke substantial discomfort, backlash, or incomprehension (Bauer et al. Reference Bauer, Braimoh, Scheim and Dharma2017; Holzberg et al. Reference Holzberg, Ellis, Kaplan, Virgile and Edgar2019; Lagos and Compton Reference Lagos and Compton2021; Medeiros, Forest, and Öhberg Reference Medeiros, Forest and Öhberg2020). Yet, researchers recognize potential traveling limitations for scholarship that focuses on comparatively progressive, advanced-industrialized democracies (Bauer et al. Reference Bauer, Braimoh, Scheim and Dharma2017; NASEM 2022).

Looking across regions, European countries are among the most progressive on policies toward same-sex couples and gender recognition, and clusters of African and Asian countries are among the most conservative.Footnote ¹ As a region, the Americas occupies a shifting and heterogeneous middle ground on LGBTQ+ legal frameworks and public opinion (de Abreu Maia, Chiu, and Desposato Reference de Abreu Maia, Chiu and Desposato2023). As such, discomfort and confusion may be comparatively prevalent among Latin American respondents (see, e.g., Reisner et al. Reference Reisner, Biello, Rosenberger, Austin, Haneuse, Perez-Brumer, Novak and Mimiaga2014). By discomfort, we refer to a range of emotional unease that is associated with sensitive survey questions (McNeeley Reference McNeeley and Gideon2012).

Another important consideration is the face-to-face mode, which is common among surveys in less developed contexts (Lupu and Michelitch Reference Lupu and Michelitch2018). Until recently, most face-to-face surveys instructed interviewers to observe and record the respondent’s apparent gender, asking for clarification only if needed (Westbrook and Saperstein Reference Westbrook and Saperstein2015). Relying on interviewers to code various traits is not uncommon, yet interviewer-assessed gender may produce errors (Lagos and Compton Reference Lagos and Compton2021; Westbrook and Saperstein Reference Westbrook and Saperstein2015).

To extend the state of the field, we developed an expansive gender question for inclusion on face-to-face surveys. We focus first on its use in the LAC region and later discuss its broader applicability. As a subregion, Latin America leans socially conservative, but attitudes toward LGBTQ+ individuals have trended more positive in recent decades (Daby and Rau Reference Daby and Rau2024) and several countries have recently passed progressive LGBTQ+ rights legislation. The contemporary region is significantly diverse, with Central American countries (and Paraguay) comparatively anti-LGBTQ+ and the Southern Cone countries (Brazil, Argentina, and Chile, among others) more progressive legally and attitudinally (de Abreu Maia, Chiu, and Desposato Reference de Abreu Maia, Chiu and Desposato2023).

In assessing gender expansive questions, we focus primarily on discomfort. Discomfort is not uncommon in surveys that include “sensitive” questions: items that provoke worry (e.g., about repercussions from honest answers), motivate censoring, and/or result in emotional unease (McNeeley Reference McNeeley and Gideon2012). Notably, discomfort can have negative effects, including response bias, misreporting, increased satisficing, increased mode effects, and interviewer effects (Andreenkova and Javeline Reference Andreenkova, Javeline, Johnson, Pennell, Ineke and Dorer2018; Tourangeau and Yan Reference Tourangeau and Yan2007). We define discomfort as a feeling of unease that, in theory, may surface among interviewees or interviewers, and—in the case of gender questions—may affect gender minorities, cisgender individuals, or both. We further consider that including expansive gender questions may prime individuals on LGBTQ+ issues, whether positively or negatively, in ways that shape top-of-the-head answers they provide to downstream survey questions (see Medeiros, Forest, and Öhberg Reference Medeiros, Forest and Öhberg2020).

Research Design

Our goal was to design an expansive gender question for face-to-face surveys that would be as unobtrusive as possible and then assess whether it exerts more than minimal consequences on survey quality. We used a multistage, multimethod approach. Stage 1 (question design) comprised cognitive interviews with small convenience samples drawn from the local population by a survey firm. The objective was to assess two variants of an open-ended gender self-identification question with a diverse group, although there was no selection on LGBTQ+ identity to form this group. Stage 2 (online experiment) compared the option that worked better in the cognitive interviews with a two-question sex and gender module in an online experiment in Guatemala. Stage 3 (comparative study) included the gender self-identification question in face-to-face surveys in 22 LAC countries alongside a question order experiment in three of these countries (Argentina, Brazil, and Chile) to test whether inclusion of the self-identification question affected attrition and responses to other questions; we then extended this experiment to two more socially conservative countries (El Salvador and Honduras).

Comparative Survey

For the face-to-face AmericasBarometer surveys (22 countries,Footnote ⁴ N = 34,459), 37 individuals (0.11%) self-identified in a way that was coded “neither male/man or female/woman.” Of those who self-identified outside the binary, 19 were coded as men, 17 were coded as women, and 1 was coded as “don’t know” by the interviewer. Ten respondents were coded as “don’t know” by the interviewer but self-identified as either a man or woman. An additional 152 individuals who self-identified as women were initially coded by the interviewer as men or vice versa.Footnote ⁵ In total, the number of respondents either misclassified or marked as “don’t know” by the interviewer was 200, or 0.6% of the sample. The number who did not respond to the self-identification question was 488 respondents, or 1.4%. On the interviewer-assessed measure, the “don’t know” option was used only 12 times (in one of these cases, the respondent self-identified as nonbinary). These data yield two conclusions: The number of individuals self-identifying outside the conventional gender binary in LAC-region surveys is low, and some individuals appear to be miscoded by interviewer-determined measures of gender. We also see some nonresponse to the gender self-identification question, although not at rates higher than for other sensitive questions.Footnote ⁶

Question Order Experiment

To systematically test for unintended effects of the gender question, we ran a preregistered experiment in three countries: Argentina, Brazil, and Chile.Footnote ⁷ We subsequently extended the study to national surveys in El Salvador and Honduras.Footnote ⁸ Individuals were randomly assigned to one of two conditions: In the treatment condition, the survey asked about gender at the beginning alongside other basic demographic information, whereas in the control condition, respondents received the gender question at the end of the survey.

We examined whether the gender self-identification question reduced engagement with the survey—via attrition and item nonresponse rates—and whether (via a priming effect) it affected expressed attitudes about LGBTQ+ rights. For this latter test, we analyzed response patterns on six questions (of which each respondent received four). These questions came after the gender question in the treatment group and before it in the control group. Each respondent was asked their approval or disapproval (on a 0–10 scale) of gay individuals’ right to run for office and of same-sex marriage. Respondents were asked two additional questions about equal rights and adoption, with random assignment to whether those questions asked about sexual minorities (LGB) or gender minorities (T).Footnote ⁹

The data from the Southern Cone countries are from the 2023 AmericasBarometer. These countries—Argentina, Brazil, and Chile—have some of the most progressive laws and attitudes on LGBTQ+ issues in the LAC region. In the 2023 AmericasBarometer, support for equal rights for gender minorities was 64% in Argentina, 63% in Chile, and 60% in Brazil. In only three other LAC countries was there majority support (Lupu et al. Reference Lupu, Rodríguez, Wilson and Zechmeister2023). We might expect comparatively lower risk of backlash and attrition in such contexts. In 2024 we implemented the same experimental design in face-to-face national surveys conducted by LAPOP in El Salvador and Honduras. Central Americans hold more conservative attitudes on LGBTQ+ rights: Support for same-sex marriage in El Salvador and Honduras is only 14% and 16%, respectively, compared to 49% to 61% support in Argentina, Brazil, and Chile (Lupu et al. Reference Lupu, Rodríguez, Wilson and Zechmeister2023; see also de Abreu Maia, Chiu, and Desposato Reference de Abreu Maia, Chiu and Desposato2023). The 2024 surveys in Honduras and El Salvador did not include questions on LGBTQ+ rights, but they do enable us to test for attrition and nonresponse bias.

Attrition and Nonresponse

Survey attrition (aka breakoff or early termination of the survey) and item nonresponse reflect survey disengagement and can create survey bias (Roßmann, Blumenstiel, and Steinbrecher Reference Roßmann, Blumenstiel and Steinbrecher2015). Figure 1 shows the estimated treatment effect on attrition and overall nonresponse rates across all questions in the survey. The treatment did not increase attrition or item nonresponse in any of the five countries.Footnote ¹⁰

Figure 1

Attrition and Overall Nonresponse

Treatment effects are estimated from linear regressions, controlling for interviewer ID. The estimated effects are presented with 95% confidence intervals.

Attitudes toward LGBTQ+ Rights

We next consider whether the treatment affected attitudes toward LGBTQ+ rights. Figure 2 shows the estimated average treatment effect (ATE) for each outcome question in the Southern Cone countries. In only one instance does treatment affect aggregate opinion: It has a positive effect on Adoption (LGB) for Brazil.

Figure 2

Average Treatment Effects

Treatment effects are estimated from linear regressions, controlling for age, urban/rural residence, interviewer ID, and whether any other individuals were present at the time of interview. The estimated effects are presented with 95% confidence intervals.

We also preregistered analysis of heterogeneous effects by age and education. Whereas we do not find evidence that our gender identity question shifts aggregate opinion on most LGBTQ+ rights questions, we do find evidence that it affects some individual-level responses. We observe heterogeneous effects by age on four questions in Argentina and on one question in Brazil and in Chile. Across all LGBTQ+ rights questions, younger respondents are more supportive than older respondents. The treatment (receiving the gender identity question at the start of the survey) widens this gap: It causes younger respondents to express even more support for LGBTQ+ rights and older respondents to express even less support.Footnote ¹¹