Over the last decade, journalists (e.g., the Washington Post) and activists (e.g., Mapping Police Violence) have labored to identify and publicize the total and annual numbers of people shot by the police in the US. Notable shootings of civilians by police and other incidents of police violence motivate them. They seek to make transparent and increase numeracy about police harms against civilians and to inform public debates and policymaking. Do information-driven, numeric-based initiatives about police harms move mass opinion regarding policing reform preferences?
Scholars debate whether the provision of information about the practices of government agents influences public opinion (Green et al., Reference Green, Aronow, Bergan, Greene, Paris and Weinberger2011; Hopkins, Sides, and Citrin, Reference Hopkins, Sides and Citrin2019), including public attitudes toward criminal justice policy, inclusive of policing (Pickett, Reference Pickett2019). Information-provision studies about policing and policy attitudes exist (Peyton, Weiss, and Vaughn, Reference Peyton, Weiss and Vaughn2022; McLean and Nix, Reference McLean and Nix2022; Mullinix and Norris, Reference Mullinix and Norris2019; Vaughn, Peyton, and Huber, Reference Vaughn, Peyton and Huber2022; Hansen and Navarro, Reference Hansen and Navarro2023). None examine whether factual numeric information about police shootings of civilians moves public preferences about policing reforms.
We conducted a preregistered Footnote 1 survey experiment in 2021 through YouGov. It employed factual data about police shootings we obtained from the official records of 10 municipal police departments in the US. We anticipated that giving subjects concrete statistics on police shootings in their cities would provide new information that would affect support for a variety of police reforms under consideration or implementation in cities across the nation. Providing statistics on police shootings did not change policy preferences. Instead, partisanship and ideology, linked with race, were strongly associated with police reform preferences.
Information, perceptions, and preferences
It is difficult to know the precise effect of information provision on public perceptions and preferences, partly because individuals have many options for receiving and consuming information. Survey experiments are a way to investigate causal relationships between information provision and preference movement. Regarding policing, multiple experiments have exposed subjects to factual and stylized information about policing (e.g., crime rates), along with photographs and videos, mock news stories, and other means, to identify whether information provision has discernible and lasting effects on public perceptions (e.g., public safety) and preferences (e.g., diversifying the racial composition of police officers) (Shi, Reference Shi2022; Larsen and Olsen, Reference Larsen and Olsen2020; Mummolo, Reference Mummolo2018; Wozniak, Drakulich, and Calfano, Reference Wozniak, Drakulich and Calfano2021; Mullinix, Bolsen, and Norris, Reference Mullinix, Bolsen and Norris2021; Reinka and Leach, Reference Reinka and Wayne Leach2017). Extant results provide mixed evidence regarding whether, and the degree to which, numeric information provision alters knowledge, views, and preferences regarding policing. For instance, correcting misperceptions about burglary rates greatly reduces misinformation about community-level crime (Larsen and Olsen, Reference Larsen and Olsen2020), while correcting misperceptions about trends in homicides may not alter criminal justice policy preferences (Esberg, Mummolo, and Westwood, Reference Esberg, Mummolo and Westwood2020).
If there is an emerging consensus from experiment-based studies of policing, it is twofold: “[A]cross a variety of contexts, individuals typically update their beliefs in the direction of the evidence they receive. The belief changes induced by information provision experiments do not, however, always have downstream effects on relevant attitudes and behaviors” (Peyton, Weiss, and Vaughn, Reference Peyton, Weiss and Vaughn2022, 3). Footnote 2 Substantive differences between detailed information about a single police-civilian encounter and broader statistical information about police behavior may account for the imbalance. This represents a distinction between episodic framing, which focuses on particular cases, and thematic framing, which emphasizes aggregate trends, general patterns, or the broader context (Iyengar, Reference Iyengar1994; Norris and Mullinix, Reference Norris and Mullinix2020). Many information provision experiments by scholars leverage episodic framing, while information-driven initiatives of journalists and activists to influence public opinion and police practices employ thematic framing.
Our study examines individual reactions to factual information about police behavior. It tests whether making aggregate, thematic information about police shootings transparent influences public preferences for reforming policing. In the experiment, factual statistics are indicators of local police behavior, providing benchmarks for considering the need to reform policing. Our primary hypothesis is that respondents who receive factual information about police shootings of civilians in their local communities will be more supportive of police reforms than respondents who do not receive such information.
We explore the effect of local police behavior on support for policing reforms, even as local political behavior and attitudes nationalize in the US (Hopkins, Reference Hopkins2018). Amid nationalization, localities still produce policies independent of national politics on a number of issues. Locally, there is “a variety of political issues with disparate spatial impacts, issues that have the potential to give rise to distinctive, localized political decisions” (14) and local political agendas and decision-making processes (Barari and Simko, Reference Barari and Simko2021). The issue of police shootings is one of them. Moreover, while national media coverage of police shootings nationalizes the subject, news coverage of local issues may abate nationalization of political knowledge and behavior (Moskowitz, Reference Moskowitz2021). Given that local media outlets in big cities routinely report on local police behavior and shootings, we assume that individuals form opinions about policing reform preferences partly based on local news consumption and from their knowledge of their local police more than their perceptions of police elsewhere, which is logical for issues involving numeracy and “objective’ indicators” (Wong, Reference Wong2007, 410).
Individuals may interpret factual information about local police shootings as evidence of excessive police violence (police excess) in their cities; others may interpret the same information as evidence that policing is dangerous work (police danger). Anticipating heterogeneity in individuals’ reactions to police shooting statistics, we also hypothesize that, relative to control, respondents learning there were more (fewer) shootings in their city will increase (decrease) their support for police reforms (police excess mechanism) or, alternatively, decrease (increase) their reform support (police danger mechanism).
Knowing encounters with and attitudes about policing vary by race (Jefferson, Neuner, and Pasek, Reference Jefferson, Neuner and Pasek2021; Porter, Wood, and Cohen, Reference Porter, Wood and Cohen2021; Pickett, Graham, and Cullen, Reference Pickett, Graham and Cullen2022), we expect that generally, Black Democrats will interpret factual information from a police excess perspective, while white Republicans will interpret information from a police danger view. Finally, we expect case-specific information about the individuals police shot (armed or unarmed) and/or the outcomes of their shootings (fatal or nonfatal for victims) to condition responses. Given widespread media attention to unarmed victims of police violence and popular interpretations of unarmed victims as “innocents” (McGowen and Wylie, Reference McGowen and Wylie2020; Burch, Reference Burch2022), we hypothesize a positive association between the unarmed shooting rate in respondents’ local communities and respondents’ support for police reforms. Our expectations regarding the receipt of information regarding fatal shootings are mixed.
SI Section A.2 elaborates on our preregistered hypotheses.
Empirical approach
Our survey experiment, administered through YouGov, included 2,575 adult residents of 10 large US cities: Dallas, Houston, Jacksonville, Los Angeles, New York City, Philadelphia, Phoenix, San Antonio, San Diego, and San Jose. Footnote 3 We obtained municipal records of police shootings of civilians in those cities in 2020.
We began by asking survey respondents to estimate the number of civilian police officers shot in their cities during 2020. Footnote 4 Afterward, respondents were randomly assigned to a control group or one of three treatment groups. Treatments included factual, city-specific information from our collection of police shooting records: (1) number of people shot in 2020, (2) number of people shot in 2020 and fatal and nonfatal shooting rates, or (3) number of people shot in 2020 and the rates of shootings of armed and unarmed civilians. The control group did not receive additional information.
We then asked respondents about their support for five common policing reforms US cities are considering or implementing: limiting police involvement in traffic stops, limiting police involvement in mental health emergencies, reducing police use of no-knock warrants, reducing qualified immunity protections, and giving civilian oversight boards powers to hold police officers accountable for misconduct.
SI Section A.3 describes our collection of police records and IRB-approved experimental design, including details about the sample, ethics, treatments, outcome measures, and analytic strategy.
Results
Providing police shooting statistics does not change policing reform preferences
We find that providing factual information on police shootings of civilians does not change support for police reforms. This is true regardless of the statistics provided, across reform proposals, and across cities. In particular, we observe no evidence that individuals update based on their priors as expected – on average, respondents who overestimated the number of shootings did not decrease their support for police reforms, and respondents who underestimated the number of shootings did not increase their support for police reforms. Figure 1 plots standardized treatment effect estimates and interactions of interest, showing none of the effects are statistically significant when accounting for multiple testing. Footnote 5 We determined statistical significance using the Benjamini–Hochberg correction on the ordered list of p-values for 190 estimates, with a false discovery rate of 0.05. Footnote 6 Additionally, the magnitudes of the main effect estimates are small, with most (89%) less than 0.2 standard deviations. Our results comport with prior findings that individuals do not update policy preferences according to relevant local conditions.
Even among the subsets of respondents we anticipated would exhibit evidence of the police excess and police danger mechanisms – Black Democrats and white Republicans, respectively – preferences were unresponsive to shooting statistics. In addition, respondents were unresponsive to the magnitude of statistics that, arguably, evince questionable police behavior, namely the rates of unarmed civilians shot and fatal shootings.
Probing the null results, we conducted exploratory tests aimed at uncovering treatment effects among subsets of individuals, particularly (a) respondents who provided high overestimates or low underestimates of police shootings in their cities, (b) attentive respondents Footnote 7 , and (c) respondents who self-identified as independents and may not have had strong partisan reasons for maintaining policy preferences in the face of new information. We find, as presented in SI Section A.5, that the informational treatments largely did not affect the police reform preferences of these individuals.
Not accounting for multiple testing, we found 7 statistically significant treatment effect estimates among the 190 tests used to evaluate our preregistered hypotheses (less than 4%). To assess whether these significant estimates occurred by chance, we conducted 100 simulations in which we performed all of our main tests on data constructed through random sampling from the control group. Across the 100 simulations, the average number of significant estimates was 5.7, and 25% found 7 or more significant results. This suggests that if our treatments truly had no effect on policy preferences, it would have been unsurprising to find the number of significant results that we found. Additionally, Figure 2 shows density plots of the standardized treatment effect estimates by outcome measure for our actual, observed data and for the simulated data. Figure 2 reveals that the distributions of the observed and simulated main effect estimates are very similar. Footnote 8 Because the simulated estimates were produced from only the (untreated) control group, the similarity in the distributions suggests that the observed treated respondents behaved, on average, no differently than they would have behaved in control without the informational interventions. Use of the nonparametric combination method (Caughey, Dafoe, and Seawright, Reference Caughey, Dafoe and Seawright2017) also supports the robustness of our null results (see SI Section A.3) The evidence from these methods, along with the fact that our experiment was sufficiently powered to detect effects as small as 0.15 standard deviations, suggests that information on police shootings of civilians does not impact individuals’ support for police reform.
Density plots of observed and simulated main effect estimates.
Exploratory analysis: what predicts preferences for police reform?
If factual information about police shootings does not move policing reform preferences, what predicts preferences? We find that partisanship and ideology relate to support for policing reforms. Table A16 in SI Section A.5 shows that Republicans and conservatives are less supportive of all five policing reform proposals. Furthermore, respondents that positively evaluate their local police departments are less likely to support policing reforms. Meanwhile, experience with police use of force is associated with increased support for three of five police reform proposals.
Generally, identity-based characteristics of respondents, including the race of respondents, had no effect on police reform preferences. The lack of associations by race, specifically for Black respondents, is surprising, especially in light of the demonstrated “racial divide in fear of the police” in the US (Pickett, Graham, and Cullen, Reference Pickett, Graham and Cullen2022). However, the power of partisanship may squelch the relationship between race and policing preferences. We therefore examined the association between partisanship (measured on a 1–7 scale from Strong Democrat to Strong Republican) and policy preferences separately for Black and non-Black respondents. While we again find evidence that partisanship is strongly associated with policing reform preferences, we now observe heterogeneous effects by race. As Figure 3 shows, support for police reforms declines as partisanship (Republican identification) increases. We observe this pattern for Black and non-Black respondents. But, the rate of decline is steeper for non-Black respondents, suggesting that partisanship has a stronger relationship to police reform attitudes for non-Black respondents than Black respondents.
Partisanship, race, and reform support.
Note: Full table of results with covariates available as SI Table A.17.
The differences are clearest among Republicans (partisanship scores of 6 or 7). Table A17 in SI Section A.5 reports corresponding results testing the interaction between partisanship and race. Consistent with the figure, and for three of five outcomes (traffic stops, qualified immunity, and civilian oversight), Republican identification is a significantly stronger, negative factor in support for policing reforms for non-Black respondents compared to Black respondents. Rather than solely determined by partisan cues or policy stances, Black respondents’ views on police reform may derive from additional personal and community-based factors. Still, the differences we observe may not be fully explained by disparate contact with law enforcement, as we find no difference in the association of personal or otherwise close contact with police use of force on the policing reform preferences of Black and non-Black respondents (p > 0.63 for all five outcomes).
Discussion
Our findings echo prior ones reporting moderate or no change in public opinions after numerical information provision (Hopkins, Sides, and Citrin, Reference Hopkins, Sides and Citrin2019; Joslyn and Haider-Markel, Reference Joslyn and Haider-Markel2018; Lawrence and Sides, Reference Lawrence and Sides2014). Still that the observed pattern of information-invariant preferences extends to policing, a governmental sphere with significant public attention, marked by calls for better public reporting about practices within it, is striking.
Our results suggest that moving public opinion on commonly proposed police reforms to reduce the number of police shootings is challenging. Our study presented information in a particular manner, and our conclusions are specific to our design. Therefore, our null results do not imply that nothing moves individual-level policing reform preferences, even as they suggest that reporting police shooting statistics may not significantly alter existing attitudes. Perhaps other types and sources of information may prompt attitude change. Accordingly, our results invite further testing of how information about police behavior may shape policing reform preferences.
It is possible that individuals do change their preferences in the face of some information about police behavior, albeit not the information we provided. Maybe it was too much to expect our study subjects to connect police shooting rates to policing reform proposals (Esberg, Mummolo, and Westwood, Reference Esberg, Mummolo and Westwood2020). Subjects may not have known how to weigh police shooting statistics against other factors bearing on policing in cities. Perhaps we would have found information-driven effects if we asked about a policy intervention more directly and immediately connected to police shootings (e.g., mandating de-escalation training). Important policy questions, however, do not always benefit from such directly connected information, and the information environment in which individuals acquire political facts and policy recommendations is rarely this simple.
Preferences for police reform may also respond more to episodic information about police shootings, rather than thematic information about them. Norris and Mullinix (Reference Norris and Mullinix2020) find, for instance, that narratives about individuals wrongfully convicted of crimes (episodic framing) move support for police reform, even as factual numbers of exonerations of criminal convictions (thematic framing) do not. Relatedly, narratives about specific police shootings and images associated with them may be more powerful than statistical information at moving police reform preferences (Jones and McBeth, Reference Jones and McBeth2010; Reinka and Leach, Reference Reinka and Wayne Leach2017). Indeed, the names and faces of civilians killed during encounters with police, along with narratives including the details of the type and amount of lethal force used against them, may explain the documented effects of such cases on opinions about policing generally (Mullinix, Bolsen, and Norris, Reference Mullinix, Bolsen and Norris2021). A future direction is to examine the impact of video- and audio-based information about police use of force on preferences for reforms.
Since we based our experimental treatments on reports by local police departments, it is also possible that subjects did not trust the statistics. If so, that is an opportunity for new experimental research into the forces that influence trust in police reports and official statistics about police shootings, such as the sources of this information. If participants are informed of police shooting statistics by local media outlets and civic organizations they trust, for example, perhaps their policing reform preferences change (or hold) (Jerit and Barabas, Reference Jerit and Barabas2012).
Furthermore, our findings suggest that policing reform preferences are a product of forces greater than the mere provision of facts about policing. Ideology and partisanship are two of the most powerful forces. They may strengthen and constrain preferences as political attitudes and behavior further nationalize. Because policing reform preferences are strongly tethered to ideology and partisanship, perhaps survey participants deemed the factual information we provided superfluous or dismissed it due to cognitive dissonance or motivated reasoning associated with ideology and partisanship.
Plus, we found evidence that evaluations of local police by individuals are important determinants of policing reform preferences, and negative experiences with police use of force correlate with support for reforms related to directly reducing use of force (i.e., ending kn-knock warrants) or holding police more accountable for it (i.e., ending qualified immunity). Given our evidence that individualized evaluations of local police create strong attachments to preferences about police reforms, experimentally exposing survey respondents to varying communal ratings of local police departments may yield important findings about movement (or lack thereof) in policing reform preferences. Future experimental research should also explore the conditions under which negative experiences with the police inform and influence support for policing reform preferences, as well as whether negative experiences trump or mediate partisanship or ideology.
In light of our findings, does accurately documenting and reporting the number of people police shoot annually in the US remain an important civic task? Yes. Our findings do not undermine that or invite scaling back or shutting down efforts by journalists and activists to reveal police harm against civilians. Cataloging instances of police harm against civilians to determine their substance, scope, and scale perform the important function of “mere description,” which is vital to knowing and understanding social phenomena (Gerring, Reference Gerring2012), including the tragic phenomenon of police shootings during police-civilian encounters and the durable disparities of police shootings of civilians by race, class, gender, and age.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/XPS.2023.30
Data availability
Support for this research was provided by the National Science Foundation (Award no. 1946768). The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: doi: 10.7910/DVN/LMWH17.
Acknowledgments
We thank Paul Zachary and Josh Tschantret for assisting with the development of this study. We also thank Gabe Lenz, Anna Harvey, Shanto Iyengar, Neil Malhotra, Justin Nix, and Justin Pickett for helpful their comments and feedback.
Competing interests
The authors declare none.
Ethics statement
This study was approved by the Institutional Review Board of Emory University (Study no. 00003543). The research adheres to APSA’s Principles and Guidance for Human Subjects Research. SI Section A.3.2 provides additional details on the study’s sample and ethics.
Open access
