Hostname: page-component-78c5997874-mlc7c Total loading time: 0 Render date: 2024-10-30T12:55:01.523Z Has data issue: false hasContentIssue false

Email Mobilization Messages Suppress Turnout Among Black and Latino Voters: Experimental Evidence From the 2016 General Election

Published online by Cambridge University Press:  15 March 2022

Michael U. Rivera
Affiliation:
School of Information, University of California, Berkeley, CA, USA
D. Alex Hughes*
Affiliation:
School of Information, University of California, Berkeley, CA, USA
Micah Gell-Redman
Affiliation:
Department of International Affairs, University of Georgia, Athens, GA, USA Department of Health Policy & Management, University of Georgia, Athens, GA, USA
*
*Corresponding author. Email: d.alex.hughes@ischool.berkeley.edu
Rights & Permissions [Opens in a new window]

Abstract

Email can deliver mobilization messages at considerably lower cost than direct mail. While voters’ email addresses are readily available, experimental work from 2007 to 2012 suggests that email mobilization is ineffective in most contexts. Here, we use public data to reexamine the effectiveness of email mobilization in the 2016 Florida general election. Unsolicited emails sent from a university professor and designed to increase turnout had the opposite effect: emails slightly demobilizing voters. While the overall decrease in turnout amounted to less than 1 percent of the margin of victory in the presidential race in the state, the demobilizing effect was particularly pronounced among minority voters. Compared to voters from the same group who were assigned to control, black voters assigned to receive emails were 2.2 percentage points less likely to turn out, and Latino voters were 1.0 percentage point less likely to turn out. These findings encourage both campaigns and researchers to think critically about the use and study of massive impersonal mobilization methods.

Type
Research Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0/), which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Copyright
© The Author(s), 2022. Published by Cambridge University Press on behalf of The Experimental Research Section of the American Political Science Association

Email mobilization

What is the effect of an unsolicited email encouragement on voter turnout? Campaigns currently employ a broad strategy of contact that includes both text messages (Roose Reference Roose2018) and email (Astor Reference Astor2019) to generate financial and electoral support. Voters opt-in to provide their email addresses to issue-advocacy groups, and national campaigns spend considerable effort to manage, merge, and then re-distribute these lists for use by political allies (Evers-Hillstrom and Erickson Reference Evers-Hillstrom and Erickson2019). The consequence of these strategies is that voters receive electronic communication from campaigns, though they may not have explicitly “opted in” to receive these messages. Experiments conducted between 2007 and 2012 are mixed in their findings about email’s ability to generate votes (see Table 1). In this study, we re-examine email mobilization because, despite little experimental evidence in support of its effectiveness, campaigns continue to spend considerable capital to send emails to voters for support.

Table 1 Mixed Evidence for Email Mobilization

When email has been effective the senders were a trusted, official source (Malhotra, Michelson, and Valenzuela Reference Malhotra, Michelson and Valenzuela2012). Outside of email mobilization, there is evidence that some efforts meant to increase turnout can inadvertently have the opposite effect (e.g., Cornwall and Kessler Reference Cornwall and Kessler2012; Grose and Russell Reference Grose and Russell2008; Kousser and Mullin Reference Kousser and Mullin2007; McCabe and Michelson Reference McCabe and Michelson2015).

Here, we test the impact of email messages that prime social norms about voting (Gerber and Rogers Reference Gerber and Rogers2009). The messages we send emanate from a publicly identifiable political science professor who is a member of our research team. We contact voters using a real identity to forestall concerns about deceptive messaging. While this experiment was designed to test the differential impact of injunctive and descriptive norms on turnout (Rivera, Hughes, and Gell-Redman Reference Rivera, Hughes and Gell-Redman2016), we find no clear evidence that voters react differently to different treatment messages. Instead, we were very surprised to observe a small, but persistent decrease in turnout among voters assigned to receive a treatment message.

Experimental setting, approach, and data

We target voters who provide a valid email in the publicly available October 10, 2016, Florida Division of Elections voter roll. Random assignment to receive an email message was blocked on congressional district and self-reported race (See Table 2 in the SI). Of the more than 12 million registered voters in Florida, 503,859 provide a valid email and are assigned to a condition (Rivera, Hughes and Gell-Redman Reference Rivera, Hughes and Gell-Redman2020).

Table 2 Comparison of Email Providers

Notes: SEs are omitted here but reported in the SI. Proportion Democrat is the proportion of registered Democrats among those registered as a Democrat or Republican. Column 2, Provide Email includes registered voters with email addresses that could not be reconciled to be valid, Assigned A Condition, includes only voters with a valid email. The Analytic Sample, used throughout the reported analysis, excludes voters who voted early in-person.

Of all votes cast during the 2016 general election in Florida, 39.7% were cast early in-person, 29.4% were by mail, and 30.5% were cast in-person on election day. Throughout the reported results, the analytic sample is the group of $328,181$ Florida voters who provide a valid email address and did not vote early in-person. We include those who voted by mail, because some of these voters could have sent in their ballot after receiving a message. We exclude voters who cast a ballot early in-person, because these votes were cast before the distribution of our treatment messages. The analytic sample is notably younger than the whole set of registered voters, but otherwise broadly similar in terms of party affiliation, gender identification, and ethnic/racial identification (Table 2, columns 1 & 4).

We contact voters via a single email sent from an active email address with a.edu top-level domain. All emails contain the subject heading “Please Remember to Vote Tomorrow,” begin with a greeting, and close with a brief paragraph that directs questions about the voting process to the Florida Department of State. Footnote 1 In line with current campaign practices, we sent messages the morning of November 7, 2016, at 10am EST, one day before the election. Limitations on the sending infrastructure precluded the ability to measure whether a voter received or opened our emails (Hughes et al. Reference Hughes, Gell-Redman, Krishnaswami, Rodenberger, Monge and Crabtree2019). All results are intent-to-treat effects.

Results

Outcome data are drawn from the June 14, 2017, official voter roll. Voters assigned to receive no contact – that is, the control group – turned out to vote at a rate of 78.7 percent (see SI Section 4.1.1). Footnote 2 Our main experimental finding, shown in Figure 1, is that those who were randomly assigned to receive any email message turned out at a 0.53 percentage point lower rate ( ${\overline{\tau}} = - 0.53$ , robust $SE = 0.17$ ). Footnote 3

Figure 1 Reduction in rate of turnout among registered voters assigned to receive an email, compared to voters of the same race/ethnicity in the control group. All points are estimated in separate models, on data partitions noted on the x-axis. Thick lines are 1.64 times the robust standard error; thin lines are 1.96 times the robust standard error. See Table 4 and Table 6 in the SI.

We further examine the effect of email messages within groups of self-identified black, Latino, and white voters. Footnote 4 Among the 36,518 black voters in the analytic sample, 64.0 percent assigned to the control group voted, compared to 61.8 percent in the treatment group ( ${{\overline{\tau}}_b} = - 2.168$ , robust $SE = 0.585$ ). Among the 63,404 Latino voters in the analytic sample, 73.9 percent assigned to the control group voted, compared to 73.0 percent in the treatment group ( ${{\overline{\tau}}_l} = - 0.970$ , robust $SE = 0.414$ ). Finally, among the 204,054 white voters in the analytic sample, 84.0 percent assigned to the control group voted, compared to 83.7 percent in the treatment group ( ${{\overline{\tau}}_w} = 0.309$ , robust $SE = 0.193$ ). Plainly stated, these messages may have kept as many as 1,389 voters from going to the polls ( $95\%\, robust \,CI = [ - 2249, - 531]$ ).

Discussion

Email mobilization has attracted interest in part because of its potential to deliver results at very low cost. Our results show that this low-cost investment for campaigns may yield zero or even negative returns for some groups of voters. Like all field experiments, ours has idiosyncratic context and design features that require care in extrapolating to both existing findings and future contexts (Bates and Glennerster Reference Bates and Glennerster2017). Specifically, our treatment messages were sent in the competitive 2016 election by a university professor with no personal or institutional connection to voters in the study. The African American voters who received our messages live in the shadow of twin legacies of exclusion. Not only have black voters been shut out of the political process (Dawson Reference Dawson1995), but the black community has also been excluded from, and in the extreme, exploited by the research community (Brandt Reference Brandt1978). In fact, one explanation for the particularly sharp depressive effect of our message among these voters is a lingering mistrust of academic researchers, especially those who lack a personal connection to the community.

Ultimately, our design cannot support direct claims about why turnout was reduced. However, our preferred conjecture is that receiving a treatment message induced stress in some subjects. Approximately 700 voters chose to respond to our message; many of these people used worried or anxious language. Voters’ decisions to reply occur after exposure to treatment, and so we are circumspect about the causal inferences permitted by these responses (Coppock Reference Coppock2019). It is possible, however, that when voters perceive the political environment to be hostile or unfavorable the circumstances of voting cause stress, and unsolicited communication may reinforce those negative impressions and lower voters’ internal motivation to turnout (Hassell and Settle Reference Hassell and Settle2017). This perspective is consonant with careful observational work that demonstrates that black voters who perceive a lower chance of electing their preferred candidate are less responsive to mobilization efforts (McGowen Reference McGowen, White and Philpot2010).

There is no clear consensus within political science over the ethical norms that should guide experimentation (e.g., Desposato Reference Desposato2015). Our own view is that the guiding principle of experimentation should be to avoid doing harm wherever possible and to balance any potential harm against the knowledge to be gained. We expected that email messages that prime social norms might provide a low cost means to encourage turnout, and we reasoned that for at least some voters this would occur by increasing negative emotions (Gerber and Rogers Reference Gerber and Rogers2009). On the one hand, priming norms may lower voters’ internal motivation (Hassell and Settle Reference Hassell and Settle2017) by increasing shame, stress, or anxiety (Marcus, Neuman, and MacKuen Reference Marcus, Neuman and MacKuen2000; Panagopoulos Reference Panagopoulos2010); on the other hand, priming norms may increase internal motivation by increasing pride (Panagopoulos Reference Panagopoulos2010). The net effect of these and other forces cannot be known in advance. Because our messages decreased turnout, we must ask whether the study’s scientific benefits outweighed the harm done. Our answer is that the learning strongly reinforces findings in the literature that emphasize the importance of personal contact and connection to the community, particularly among minority voters.

Supplementary material

To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2021.34

Data availability statement

The data, code, and compute environment required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: https://doi.org/10.7910/DVN/YIZEA7.

Acknowledgements

We are grateful for comments from attendees of the 2017 State Politics and Policy Conference.

Conflicts of interest

The authors have no conflicts of interest to declare.

Footnotes

This article has earned badges for transparent research practices: Open Data and Open Materials. For details see the Data Availability Statement.

This article has been updated since its original publication. See https://doi.org/10.1017/XPS.2023.19

1 The SI contains additional details about message language, sending infrastructure, randomization, and data capture.

2 At a target power of 0.8, this sample and design can detect a difference in the rates of turnout as small as 0.3 percentage points – roughly half the size of the established effect size of direct-mail (see SI Section 3.5).

3 All estimates use robust standard errors and are reported as percentage points. Details of the estimation and accompanying regression results are provided in SI, Section 4.1.2 We find no evidence of differential effects across message variants (see SI Section 4.2.1 for details). Throughout the main text, we compare the turnout of voters assigned to control to those assigned any message, and denote this treatment as ${\overline{\tau}}$ .

4 See SI Section 4.3 for details about estimation within subgroups. See SI Table 7 for a test for heterogeneous response to treatment between racial/ethnic groups.

References

Astor, Maggie. 2019. “880 Emails Over 6 Weeks: Here’s How 2020 Democrats Portray Themselves.” The New York Times. URL: https://www.nytimes.com/2019/06/25/us/politics/democrats-2020-emails.html Google Scholar
Bates, Mary Ann and Glennerster, Rachel. 2017. “The generalizability puzzle.Stanford Social Innovation Review 15(3): 50–4.Google Scholar
Bennion, Elizabeth A. and Nickerson, David W.. 2011. “The cost of convenience: An experiment showing e-mail outreach decreases voter registration.Political Research Quarterly 64(4): 858–69.CrossRefGoogle Scholar
Brandt, Allan M. 1978. “Racism and research: the case of the Tuskegee Syphilis Study.” Hastings center report pp. 21–29.CrossRefGoogle Scholar
Coppock, Alexander. 2019. “Avoiding post-treatment bias in audit experiments.Journal of Experimental Political Science 6(1): 14.CrossRefGoogle Scholar
Cornwall, Tom and Kessler, Anke S.. 2012. “Does Misinformation Demobilize the Electorate? Measuring the Impact of Alleged ‘Robocalls’ in the 2011 Canadian Election.”.Google Scholar
Dawson, Michael C. 1995. Behind the mule: Race and class in African-American politics. Princeton, NJ: Princeton University Press.10.1515/9780691212982CrossRefGoogle Scholar
Desposato, Scott. 2015. Ethics and experiments: Problems and solutions for social scientists and policy professionals. Routledge.CrossRefGoogle Scholar
Evers-Hillstrom, Karl and Erickson, Camille. 2019. “Your email is for sale — and 2020 candidates are paying up.” The Center for Responsive Politics. URL: https://www.opensecrets.org/news/2019/06/email-list-for-sale-2020-candidates-arepaying/ Google Scholar
Gerber, Alan S and Rogers, Todd. 2009. “Descriptive social norms and motivation to vote: Everybody’s voting and so should you.The Journal of Politics 71(1): 178–91.CrossRefGoogle Scholar
Grose, Christian and Russell, Carrie. 2008. “Avoiding the Vote: A Theory and Field Experiment of the Social Costs of Public Political Participation.SSRN Electronic Journal, 1. doi: 10.2139/ssrn.1310868.Google Scholar
Hassell, Hans J. G. and Settle, Jaime E.. 2017. “The differential effects of stress on voter turnout.Political Psychology 38(3): 533–50.10.1111/pops.12344CrossRefGoogle Scholar
Hughes, D. Alex, Gell-Redman, Micah, Krishnaswami, Natarajan, Rodenberger, Diana, Monge, Guillermo and Crabtree, Charles. 2019. “Persistent Bias Among Local Election Officials.Journal of Experimental Political Science 7(3): 19.Google Scholar
Kousser, Thad and Mullin, Megan. 2007. “Does voting by mail increase participation? Using matching to analyze a natural experiment.Political Analysis 15(4): 428–45.CrossRefGoogle Scholar
Malhotra, Neil, Michelson, Melissa R. and Valenzuela, Ali Adam. 2012. “Emails from official sources can increase turnout.Quarterly Journal of Political Science 7(3): 321–32.10.1561/100.00011073CrossRefGoogle Scholar
Marcus, George E., Neuman, W. Russell and MacKuen, Michael. 2000. Affective intelligence and political judgment. Chicago, IL: University of Chicago Press.Google Scholar
McCabe, Deborah Brown and Michelson, Melissa R.. 2015. “Pushing too hard: Using doorin-the-face to get voters out the door.Journal of Political Marketing 14(4): 316–32.10.1080/15377857.2015.1086137CrossRefGoogle Scholar
McGowen, Ernest B. 2010. Depressing The Vote: The Environmental Determinants of Turnout In African-American Precincts in The 2000 Presidential Election. In African-American Political Psychology, eds. White, I. K. and Philpot, T. S. New York, NY: Palgrave Macmillan.Google Scholar
Nickerson, David W. 2007. “Does email boost turnout.Quarterly Journal of Political Science 2(4): 369–79.CrossRefGoogle Scholar
Panagopoulos, Costas. 2010. “Affect, social pressure and prosocial motivation: Field experimental evidence of the mobilizing effects of pride, shame and publicizing voting behavior.Political Behavior 32(3): 369–86.10.1007/s11109-010-9114-0CrossRefGoogle Scholar
Rivera, Michael U., Hughes, D. Alex and Gell-Redman, Micah. 2016. “Mobilizing Voters Using Email.” Evidence in Governance and Politics: Experimental Pre-Registration. Registration ID: 20161129AA. URL: http://egap.org/registration/2283 Google Scholar
Rivera, Michael U., Hughes, D. Alex and Gell-Redman, Micah. 2020. “Replication Data for: Email Mobilization Messages Suppress Turnout Among Black and Latino Voters: Experimental Evidence From the 2016 General Election.” URL: https://doi.org/10.7910/DVN/YIZEA7 CrossRefGoogle Scholar
Roose, Kevin. 2018. “Campaigns Enter Texting Era With a Plea: Will U Vote 4 Me?” The New York Times. URL: https://www.nytimes.com/2018/08/01/technology/campaign-text-messages.html Google Scholar
Figure 0

Table 1 Mixed Evidence for Email Mobilization

Figure 1

Table 2 Comparison of Email Providers

Figure 2

Figure 1 Reduction in rate of turnout among registered voters assigned to receive an email, compared to voters of the same race/ethnicity in the control group. All points are estimated in separate models, on data partitions noted on the x-axis. Thick lines are 1.64 times the robust standard error; thin lines are 1.96 times the robust standard error. See Table 4 and Table 6 in the SI.

Supplementary material: Link

Rivera et al. Dataset

Link
Supplementary material: PDF

Rivera et al. supplementary material

Rivera et al. supplementary material

Download Rivera et al. supplementary material(PDF)
PDF 764.5 KB