Audit experiments are used to measure discrimination in a large number of domains (Employment: Bertrand et al. (2004); Legislator responsiveness: Butler et al. (2011); Housing: Fang et al. (2018)). Audit studies all have in common that they estimate the average difference in response rates depending on randomly varied characteristics (such as the race or gender) of a requester. Scholars conducting audit experiments often seek to extend their analyses beyond the effect on response to the effects on the quality of the response. Response is a consequence of treatment; answering these important questions well is complicated by post-treatment bias (Montgomery et al., 2018). In this note, I consider a common form of post-treatment bias that occurs in audit experiments.

One argument against secret ballots is that such procedures lead to more selfish voting behavior and that public voting can increase prosocial voting and the likelihood of prosocial outcomes when voters are not subject to intimidation and coercion from outside interests. We investigate this supposition as well as voter preferences over observability in voting in this context. We find that voters are significantly more likely to choose unselfishly when voting is public. These differences in behavior advantage prosocial choices in elections (by 27%) when voting is public. Moreover, voters appear to recognize these differences and a substantial minority of voters whose selfish preference is not the prosocial choice willingly choose public voting even though the likely outcome will be costly to themselves.

The Vietnam draft lottery exposed millions of men to risk of induction at a time when the Vietnam War was becoming increasingly unpopular. We study the long-term effects of draft risk on political attitudes and behaviors of men who were eligible for the draft in 1969–1971. Our 2014–2016 surveys of men who were eligible for the Vietnam draft lotteries reveal no appreciable effect of draft risk across a wide range of political attitudes. These findings are bolstered by analysis of a vast voter registration database, which shows no differences in voting rates or tendency to register with the Democratic or Republican parties. The pattern of weak long-term effects is in line with studies showing that the long-term economic effects of Vietnam draft risk dissipated over time and offers a counterweight to influential observational studies that report long-term persistence in the effects of early experiences on political attitudes.

Many foreign aid donors brand development interventions. How do citizens in the donor country react to seeing this branding in action? We test the proposition that citizens will express higher levels of support for foreign aid when they see a branded foreign aid project relative to seeing the same project without branding. We present results from a survey-based laboratory experiment conducted in the United Kingdom where subjects learned about a typical foreign aid project and received a randomized UK branding treatment. Our results suggest that the branding treatments increase the likelihood that donor country respondents believe that aid recipients can identify the source of the foreign aid. Only among conservative respondents, however, does the evidence imply that branding increases support for foreign aid. “UK aid” branding increases conservative opinion that aid dollars are well spent and increases support among this group for the expansion of foreign aid.

How does the fear of anarchy affect telephone survey behaviors? A survey experiment administered to a sample of Mogadishu residents—validated with a natural experiment—is used to assess this question. Randomly assigned reminders of anarchic violence conditioned differential effects on survey participation depending on subjects’ background level of security and welfare. Vulnerable subjects were more likely than non-vulnerable subjects to refuse to provide sensitive survey information after reminders of anarchy.

Attempts to correct political misperceptions often fail. The dominant theoretical explanation for this failure comes from psychological research on motivated reasoning. We identify a novel source of motivated reasoning in response to corrective information: the justification of socially undesirable preferences. Further, we demonstrate that this motivation can, under certain conditions, overpower the motivation to maintain congruence. Our empirical test is a national survey experiment that asks participants to reconcile partisan motivations and the motivation to justify voting against a racial minority candidate. Consistent with our argument, racially prejudiced participants dismiss corrections when misinformation is essential to justify voting against a black candidate of their own party, but accept corrections about an otherwise identical candidate of the opposing party. These results provide new insight into the persistence of certain forms of political misinformation.

A number of theoretical and empirical studies analyze the effect of inclement weather on voter turnout and in turn on parties’ vote share. However, empirical findings suggest that the effect of weather on parties’ vote share is greater than can be explained by its influence on voter turnout alone. This article provides experimental evidence of the effect of weather on vote choice between more- versus less-risky candidates. Findings show that bad weather significantly and sizeably depresses risk tolerance making voters less likely to vote for risky candidates. This article also provides evidence of a possible mechanism: unpleasant weather conditions depress agents’ mood, making agents less inclined to vote for candidates who are perceived as more risky.

Federal systems can also provide citizens with multiple avenues to obtain service representation. In shared issue areas, citizens are officially represented by two sets of politicians. When politicians are willing to cross jurisdictional boundaries, citizens might also obtain help or information from more than one set of politicians, even in areas of exclusive jurisdiction. We report an experiment designed to examine responses to requests for assistance in different issue areas. Our sample includes 202 Canadian politicians, each of whom received two requests for assistance from fictional constituents. We show that federal arrangements can enhance service representation. On average, politicians are as helpful on issues of shared jurisdiction as issues of exclusive jurisdiction. They are less helpful for issues outside of their jurisdiction. These results suggest that federal arrangements can work to provide citizens with multiple access points to their representative, even in areas that fall outside their representatives’ jurisdictional purviews.

Research commonly finds that citizens are loss averse: they dislike losses far more than similarly sized gains. One implication is that arguments for policy change framed in terms of losses to be avoided often have a larger impact on public opinion than arguments framed in terms of gains to be enjoyed. Although several scholars have observed this pattern with respect to public opinion, we know far less about the effect of loss- and gain-framed arguments on political activism. This is a critical omission given the disproportionate impact of political activists on the policymaking process. Using field and survey experiments, we investigate the impact of gain- and loss-framed arguments on climate change activism. We find that loss-framed arguments can be less mobilizing, even when they are otherwise more persuasive, than gain-framed arguments. Our results deepen our theoretical understanding of what motivates political activism, especially in an era of professionalized politics.

This paper considers the implications of construal level theory in the context of survey experiments probing foreign policy opinion formation. Psychology research demonstrates that people discount the long-term consequences of decisions, thinking about distal or hypothetical events more abstractly than immediate scenarios. I argue that this tendency introduces a bias into survey experiments on foreign policy opinion. Respondents reasoning about an impending military engagement are likelier to consider its costs than are those reasoning in the abstract hypothetical environment. I provide evidence of this bias by replicating a common audience costs experimental design and introducing a prompt to consider casualties. I find that priming respondents to articulate their expectations about casualties in a foreign intervention reduces support and dampens the experimental effect, thereby cutting the estimated absolute audience cost substantially. This result suggests a gap between how survey respondents approach hypothetical and real situations of military intervention.

Using a political-frame-free, lab-in-the-field experiment, we investigate the associations between employment status, self-reported political ideology, and preferences for redistribution. The experiment consists of a real-effort task, followed by a four-player dictator game. In one treatment, dictator game initial endowments depend on participants’ performance in the real-effort task, i.e., they are earned, in the other, they are randomly determined. We find that being employed or unemployed is associated with revealed redistributive preferences, while the political ideology of the employed and unemployed is not. In contrast, the revealed redistributive preferences of students are strongly associated with their political ideologies. The employed and right-leaning students redistribute earnings less than windfalls, the unemployed, and left-leaning students make no such distinction.

Do people judge some forms of wage discrimination to be more unfair than others? We report an experiment in an online labor market in which participants were paid based on discriminatory rules. We test hypotheses about fairness based on procedural justice, divisiveness, and affective polarization between partisans. Workers transcribed text and then learned that they earned more or less money than other workers for doing the same job. We manipulated whether the unequal pay was based on their political party, eye color, or an arbitrary choice between two doors. Consistent with the divisiveness hypothesis, participants judged discriminatory pay to be less fair when it was based on a stable characteristic, political party, or eye color, compared to a transient choice (between doors). We find mixed evidence about how affective polarization exacerbates the unfairness of partisan discrimination. We discuss implications for the procedural justice of wage discrimination.

Shared social disadvantage relative to Whites is assumed to motivate inter-minority political behavior but we lack causal evidence. Using a survey experiment of 1,200 African Americans, we prompt respondents to consider group social position when evaluating political commonality with Latinos. The experiment describes racial disparities in a randomized domain (education or housing), varies the description of inequality (either Black versus White, Latino versus White, or Black and Latino versus White), and offers half of the respondents a political cue to test whether shared social disadvantage causes Blacks’ perceptions of political commonality with Latinos. We find little evidence of a causal relationship. We conclude that cross-racial minority political coalitions may be more difficult to activate than previously thought.

This short report exploits a unique opportunity to investigate the implications of response bias in survey questions about voter turnout and vote choice in new democracies. We analyze data from a field experiment in Benin, where we gathered official election results and panel survey data representative at the village level, allowing us to directly compare average outcomes across both measurement instruments in a large number of units. We show that survey respondents consistently overreport turning out to vote and voting for the incumbent, and that the bias is large and worse in contexts where question sensitivity is higher. This has important implications for the inferences we draw about an experimental treatment, indicating that the response bias we identify is correlated with treatment. Although the results using the survey data suggest that the treatment had the hypothesized impact, they are also consistent with social desirability bias. By contrast, the administrative data lead to the conclusion that the treatment had no effect.

How do individuals respond to internal security threats in non-democracies? Does violence make individuals more supportive of a strong state? Are the effects of violence on individual attitudes uniform, or are they heterogeneous with respect to the identity of the perpetrators? We field an online survey experiment on a national sample of Chinese citizens, in which respondents were randomly selected to view reports on violent acts in China. We show that exposure to violence makes individuals more supportive of a strong state: respondents randomly exposed to violence are more likely to approve police use of lethal force, and this effect is particularly strong among the less wealthy Han Chinese. We also find suggestive evidence that individuals exhibit intergroup biases in their reaction to violence.

We investigate in an economic experiment how people choose sides in disputes. In an eight-player side-taking game, two disputants at a time fight over an indivisible resource and other group members choose sides. The player with more supporters wins the resource, which is worth real money. Conflicts occur spontaneously between any two individuals in the group. Players choose sides by ranking their loyalties to everyone else in the group, and they automatically support the disputant they ranked higher. We manipulate participants’ information about other players’ loyalties and also their ability to communicate with public chat messages. We find that participants spontaneously and quickly formed alliances, and more information about loyalties caused more alliance-building. Without communication, we observe little evidence of bandwagon or egalitarian strategies, but with communication, some groups invented rank rotation schemes to equalize payoffs while choosing the same side to avoid fighting costs.

Political scientists commonly use survey experiments–often conducted online–to study the attitudes of the mass public. In these experiments, compensation is usually small and researcher control is limited, which introduces the potential for low respondent effort and attention. This lack of engagement may result in noncompliance with experimental protocols, threatening causal inferences. However, in reviewing the literature, we find that despite the discipline’s general familiarity with experimental noncompliance, researchers rarely consider it when analyzing survey experiments. This oversight is important because it may unknowingly prevent researchers from estimating their causal quantities of greatest substantive interest. We urge scholars to address this particular manifestation of an otherwise familiar problem and suggest two strategies for formally measuring noncompliance in survey experiments: recording vignette screen time latency and repurposing manipulation checks. We demonstrate and discuss the substantive consequences of these recommendations by revisiting several published survey experiments.

What makes people take ethnic divisions into account when judging politics? We consider here the possible effect of language. We hypothesize that speaking a minority tongue primes ethnic divisions, leading people to interpret politics more heavily through this prism. In two survey experiments with bilingual adults, we demonstrate that subjects assigned to interview in a minority language are indeed more likely to evaluate politics based on ethnic considerations: they rank ethnic relations as a more important political issue and they are more likely to correctly identify the anti-minority party in their political system. These results suggest that people may think about politics differently depending on the language they use.