Email Mobilization Messages Suppress Turnout Among Black and Latino Voters: Experimental Evidence From the 2016 General Election

Abstract Email can deliver mobilization messages at considerably lower cost than direct mail. While voters’ email addresses are readily available, experimental work from 2007 to 2012 suggests that email mobilization is ineffective in most contexts. Here, we use public data to reexamine the effectiveness of email mobilization in the 2016 Florida general election. Unsolicited emails sent from a university professor and designed to increase turnout had the opposite effect: emails slightly demobilizing voters. While the overall decrease in turnout amounted to less than 1 percent of the margin of victory in the presidential race in the state, the demobilizing effect was particularly pronounced among minority voters. Compared to voters from the same group who were assigned to control, black voters assigned to receive emails were 2.2 percentage points less likely to turn out, and Latino voters were 1.0 percentage point less likely to turn out. These findings encourage both campaigns and researchers to think critically about the use and study of massive impersonal mobilization methods.

these messages. Experiments conducted between 2007 and 2012 are mixed in their findings about email's ability to generate votes (see Table 1). In this study, we re-examine email mobilization because, despite little experimental evidence in support of its effectiveness, campaigns continue to spend considerable capital to send emails to voters for support.
Here, we test the impact of email messages that prime social norms about voting (Gerber and Rogers 2009). The messages we send emanate from a publicly identifiable political science professor who is a member of our research team. We contact voters using a real identity to forestall concerns about deceptive messaging. While this experiment was designed to test the differential impact of injunctive and descriptive norms on turnout (Rivera, Hughes, and Gell-Redman 2016), we find no clear evidence that voters react differently to different treatment messages. Instead, we were very surprised to observe a small, but persistent decrease in turnout among voters assigned to receive a treatment message.
Experimental setting, approach, and data We target voters who provide a valid email in the publicly available October 10, 2016, Florida Division of Elections voter roll. Random assignment to receive an email message was blocked on congressional district and self-reported race (See Table 2 in the SI). Of the more than 12 million registered voters in Florida, 503,859 provide a valid email and are assigned to a condition (Rivera, Hughes and Gell-Redman 2020).
Of all votes cast during the 2016 general election in Florida, 39.7% were cast early in-person, 29.4% were by mail, and 30.5% were cast in-person on election day. Throughout the reported results, the analytic sample is the group of 328; 181 Florida voters who provide a valid email address and did not vote early in-person. We include those who voted by mail, because some of these voters could have sent in their ballot after receiving a message. We exclude voters who cast a ballot early in-person, because these votes were cast before the distribution of our treatment messages. The analytic sample is notably younger than the whole set of registered University administrators' email encouragement of students to register to vote slightly decreases registration rates.
Email messages sent from the registrar of voters can have small, but significant positive effects on turnout.
When email has been effective the senders were a trusted, official source (Malhotra, Michelson, and Valenzuela 2012 voters, but otherwise broadly similar in terms of party affiliation, gender identification, and ethnic/racial identification ( Table 2, columns 1 & 4). We contact voters via a single email sent from an active email address with a.edu top-level domain. All emails contain the subject heading "Please Remember to Vote Tomorrow," begin with a greeting, and close with a brief paragraph that directs questions about the voting process to the Florida Department of State. 1 In line with current campaign practices, we sent messages the morning of November 7, 2016, at 10am EST, one day before the election. Limitations on the sending infrastructure precluded the ability to measure whether a voter received or opened our emails (Hughes et al. 2019). All results are intent-to-treat effects.

Results
Outcome data are drawn from the June 14, 2017, official voter roll. Voters assigned to receive no contactthat is, the control groupturned out to vote at a rate of 78.7 percent (see SI Section 4.1.1). 2 Our main experimental finding, shown in Figure 1, is that those who were randomly assigned to receive any email message turned out at a 0.53 percentage point lower rate (τ 0:53, robust SE 0:17). 3 We further examine the effect of email messages within groups of self-identified black, Latino, and white voters. 4 Among the 36,518 black voters in the analytic sample, 64.0 percent assigned to the control group voted, compared to 61.8 percent The SI contains additional details about message language, sending infrastructure, randomization, and data capture.
2 At a target power of 0.8, this sample and design can detect a difference in the rates of turnout as small as 0.3 percentage pointsroughly half the size of the established effect size of direct-mail (see SI Section 3.5). 3 All estimates use robust standard errors and are reported as percentage points. Details of the estimation and accompanying regression results are provided in SI, Section 4.1.2 We find no evidence of differential effects across message variants (see SI Section 4.2.1 for details). Throughout the main text, we compare the turnout of voters assigned to control to those assigned any message, and denote this treatment as τ.

4
See SI Section 4.3 for details about estimation within subgroups. See SI Table 7 for a test for heterogeneous response to treatment between racial/ethnic groups. in the treatment group (τ b 2:168, robust SE 0:585). Among the 63,404 Latino voters in the analytic sample, 73.9 percent assigned to the control group voted, compared to 73.0 percent in the treatment group (τ l 0:970, robust SE 0:414). Finally, among the 204,054 white voters in the analytic sample, 84.0 percent assigned to the control group voted, compared to 83.7 percent in the treatment group (τ w 0:309, robust SE 0:193). Plainly stated, these messages may have kept as many as 1,389 voters from going to the polls (95% robust CI 2249; 531).

Discussion
Email mobilization has attracted interest in part because of its potential to deliver results at very low cost. Our results show that this low-cost investment for campaigns may yield zero or even negative returns for some groups of voters. Like all field experiments, ours has idiosyncratic context and design features that require care in extrapolating to both existing findings and future contexts (Bates and Glennerster 2017). Specifically, our treatment messages were sent in the competitive 2016 election by a university professor with no personal or institutional connection to voters in the study. The African American voters who received our messages live in the shadow of twin legacies of exclusion. Not only have black voters been shut out of the political process (Dawson 1995), but the black community has also been excluded from, and in the extreme, exploited by the research community (Brandt 1978). In fact, one explanation for the particularly sharp depressive effect of our message among these voters is a lingering mistrust of academic researchers, especially those who lack a personal connection to the community.
Ultimately, our design cannot support direct claims about why turnout was reduced. However, our preferred conjecture is that receiving a treatment message induced stress in some subjects. Approximately 700 voters chose to respond to  Table 4 and Table 6 in the SI.
our message; many of these people used worried or anxious language. Voters' decisions to reply occur after exposure to treatment, and so we are circumspect about the causal inferences permitted by these responses (Coppock 2019). It is possible, however, that when voters perceive the political environment to be hostile or unfavorable the circumstances of voting cause stress, and unsolicited communication may reinforce those negative impressions and lower voters' internal motivation to turnout (Hassell and Settle 2017). This perspective is consonant with careful observational work that demonstrates that black voters who perceive a lower chance of electing their preferred candidate are less responsive to mobilization efforts (McGowen 2010).
There is no clear consensus within political science over the ethical norms that should guide experimentation (e.g., Desposato 2015). Our own view is that the guiding principle of experimentation should be to avoid doing harm wherever possible and to balance any potential harm against the knowledge to be gained. We expected that email messages that prime social norms might provide a low cost means to encourage turnout, and we reasoned that for at least some voters this would occur by increasing negative emotions (Gerber and Rogers 2009). On the one hand, priming norms may lower voters' internal motivation (Hassell and Settle 2017) by increasing shame, stress, or anxiety (Marcus, Neuman, and MacKuen 2000;Panagopoulos 2010); on the other hand, priming norms may increase internal motivation by increasing pride (Panagopoulos 2010). The net effect of these and other forces cannot be known in advance. Because our messages decreased turnout, we must ask whether the study's scientific benefits outweighed the harm done. Our answer is that the learning strongly reinforces findings in the literature that emphasize the importance of personal contact and connection to the community, particularly among minority voters.