Public Health Communication Reduces COVID-19 Misinformation Sharing and Boosts Self-Efficacy

During health crises, misinformation may spread rapidly on social media, leading to hesitancy towards health authorities. The COVID-19 pandemic prompted significant research on how communication from health authorities can effectively facilitate compliance with health-related behavioral advice such as distancing and vaccination. Far fewer studies have assessed whether and how public health communication can help citizens avoid the harmful consequences of exposure to COVID-19 misinformation, including passing it on to others. In two experiments in Denmark during the pandemic, the effectiveness of a 3-minute and a 15-second intervention from the Danish Health Authorities on social media was assessed, along with an accuracy nudge. The findings showed that the 3-minute intervention providing competences through concrete and actionable advice decreased sharing of COVID-19-related misinformation and boosted their sense of self-efficacy. These findings suggest that authorities can effectively invest in building citizens ’ competences in order to mitigate the spread of misinformation on social media.


Introduction
Misinformation about COVID-19 on social media has been a public concern during the COVID-19 pandemic.How can public health authorities communicate to mitigate the spread of misinformation?One line of research on countering COVID-19 misinformation suggests that subtly nudging people to think about accuracy reduces misinformation sharing (Pennycook et al. 2020).The strength of such interventions is their fast and frugal nature.At the same This article has earned badges for transparent research practices: Open Data and Open Materials.For details see the Data Availability Statement.
time, they are premised on the idea that people can discern between true and false, but "that when deciding what to share on social media, people are often distracted from considering the accuracy of the content" (Pennycook et al. 2021).As such, a key drawback of accuracy nudge interventions is that they simply remind people about accuracy, but leave individuals to rely on their own knowledge without providing tools or competences to deal with misinformation.The frugal nature of accuracy nudges also implies that their effects are small (Pennycook and Rand 2022), making recent studies question their effectiveness during the COVID-19 pandemic (Rathje et al. 2022;Roozenbeek et al. 2021;Gavin et al. 2022;Pretus et al. 2023).
Other interventions move beyond nudging accuracy motivations by seeking to equip people with better capabilities for identifying and avoiding sharing misinformation (Lee 2018;Guess et al. 2020;Hertwig and Grüne-Yanoff 2017;Sheeran and Orbell 2000;Sheeran et al. 2007;Van Der Linden 2022).This is consistent with research on risk communication that argues that feelings of competence are key in order to motivate people to respond effectively to risks by engaging in protective behaviors (Jørgensen et al. 2021b).People respond effectively when they are provided with trustworthy information about a threat, provided with actionable advice on how to respond to the threat and assured that this response will be efficient against the threat (Rogers 1975;Maddux and Rogers 1983;Rippetoe and Rogers 1987).
In this manuscript, we test the effectiveness of interventions that nudge accuracy to ones that additionally provide capabilities.Specifically, we test an accuracy nudge as well as two video-based, real-world interventions circulated by the Danish National Health Authority on social media in January 2021 during the COVID-19 pandemic.The accuracy nudge subtly primed people to think about their motivation to share accurate headlines, while the videosa 15-second and a 3-minute interventionprovided capabilities through concrete instructions on how to avoid sharing COVID-19 related to misinformation.
For study 1, we predicted that all three interventions would decrease false headline sharing, increase real headline sharing, and increase sharing discernment (i.e., the relative sharing of real compared to false headlines).The accuracy nudge, the 15-second intervention and the 3-minute intervention all significantly increased sharing discernment.Only the 3-minute intervention, however, directly and significantly decreased false headline sharing, but did not alter real headline sharing.Neither the 15-second intervention nor the accuracy nudge had a statistically significant effect on either false or real headline sharing.Consistent with a capability perspective, study 2 showed that the 3-minute intervention increased participants' sense of self-efficacy in dealing with online misinformation.The intervention did not influence other aspects often highlighted in research on risk communication, specifically, participants' sense of the threat from misinformation and the effectiveness of remedies against misinformation.Overall, these results suggest that when health authorities communicate elaborate and actionable advice on how to avoid sharing COVID-19 misinformation, such communication can reduce the spread of false headlines and enhance people's sense of personal competence.

Two approaches to misinformation interventions
We examine two types of interventions to reduce misinformation sharing.One type of intervention is "accuracy nudges" which has received significant research interest (Pennycook et al. 2021(Pennycook et al. ,2020;;Roozenbeek et al. 2021;Rathje et al. 2022;Gavin et al. 2022;Pretus et al. 2023).The psychological assumption behind accuracy nudges is that people are motivated to share accurate content on social media and are capable of distinguishing between true and false content.Yet, accuracy concerns do often not drive online sharing behavior because people are distracted from accuracy toward a desire to share emotionally engaging content and receive positive social feedback from friends on their sharing.Thus, reminding people to pay attention to accuracy through nudging should decrease the sharing of misinformation.Some research shows that subtle accuracy nudges where people are asked to rate the accuracy of a few news headlines decrease subsequent sharing of false headlines on social media (Pennycook et al. 2021(Pennycook et al. , 2020)).
Another type of intervention is capability interventions that seeks to mitigate misinformation sharing by building competences or resistance against the rhetorical techniques and strategies that are used to mislead people through misinformation (Roozenbeek and Van Der Linden 2022;Guess et al. 2020;Badrinathan 2021;Van Der Linden 2022;Lee 2018;Mo Jones-Jang et al. 2021).While these interventions are often rooted in distinct theoretical frameworkssuch as inoculation theory or digital media literacythe common denominator is that they go beyond merely priming accuracy motivations by providing concrete tools, actionable advice, or psychological competences to mitigate misinformation sharing.In other words, while the assumption behind accuracy nudges is that people already have the competence to avoid sharing misinformation and simply are in need of motivationoriented reminders, capability interventions go beyond the motivational component of accuracy nudges: They build capabilities by providing education (i.e., increasing knowledge) and training (i.e., imparting skills and tools) to avoid misinformation sharing.Where accuracy nudges rely on prompting a pre-existing motivation for accuracy, capability interventions aim to build reflective motivation where citizens contemplate and plan how to implement advice behaviorally.

Overview of studies
In two studies, we conduct a preregistered test of a 15-second and a 3-minute capability-oriented intervention from the Danish Health Authorities as well as an accuracy nudge (Pennycook et al. 2020) and compare them to a control group.Study 1 tested the effect of the interventions on sharing of false and real headlines while Study 2 assessed the effect of the interventions on self-efficacy, response efficacy, and threat appraisal. 1Whenever we report additional analyses that were not preregistered, we label them as exploratory.Table 1 provides an overview of the data collection.
In both studies, each participant was assigned to one of four conditions.In the control condition, participants were not exposed to any treatment prior to the respective dependent measures of Study 1 and Study 2. In the accuracy nudge condition, participants rated the accuracy of a single headline (unrelated to COVID-19) framed as a pretest mimicking prior studies of accuracy nudges (Pennycook et al. 2020;Roozenbeek et al. 2021).In the 15-second condition and the 3-minute condition, participants were shown videos titled "Can you trust what you read?" containing guidance on how to recognize and avoid sharing COVID-19 misinformation. 2Specifically, the actionable advice in the video is summed up in three questions that one should ask oneself when facing novel information on social media: (1) Who is saying it?(2) How many are saying it?(3) Is the content too far out?Besides the 3-minute video being longer than the 15-second video, there are two major differences between the interventions.First, while the 15-second video only contains text, the 3-minute video includes both text and audio, which makes the content more immersive.Second, the 3-minute video provides more elaborate advice.In other words, the advice provided by the 3-minute intervention is more concrete and actionable in terms of providing a plan for implementation, which is conducive to behavior change (Pearce et al. 2019;Sommestad et al. 2015).Through collaboration with the Danish Health  Participants were not able to skip the videos and 88% passed the attention check regarding the stimuli.We provide additional information and robustness tests regarding attention checks in section F of the supplementary material.
Authority, we have been given permission to use the actual videos which were shared on Facebook in 2020 and 2021. 3he studies were conducted in Denmark during in the Summer and Winter of 2021 during the COVID-19 pandemic.Denmark is characterized by high levels of interpersonal and institutional trust and low levels of political polarization.These factors materialized during the onset of the pandemic, as compliance with and support for governmental responses were high, while polarization was low, in contrast to other countries where governmental responses were more disputed (Lindholt et al. 2021;Van Bavel et al. 2022;Jørgensenet al. 2021b,a).Furthermore, it was widely accepted, even among political elites, that the COVID-19 virus constituted a significant public health threat that required public collaboration and responsiveness to contain.As policies and messages aimed at countering the COVID-19 pandemic are more effective when backed by cross-partisan coalitions of political elites (Flores et al. 2022), public health communication is more likely to be effective in Denmark, compared to countries where the nature of the COVID-19 pandemic as a health crisis was disputed.We further elaborate on the implications for the generalizability of the findings in the discussion section.

Study 1
We preregistered the following hypotheses for study 1.We expected that the interventions would reduce false headline sharing, and thus, we predicted that compared to the control condition, all three interventions decrease sharing of false headlines, but not real headlines about COVID-19 on social media (H1).Conversely, the intervention could work by increasing real headline sharing, and thus, we predicted that compared to the control condition, all three interventions increase sharing of real headlines, but not false headlines about COVID-19 on social media (H2).Furthermore, the interventions might prompt a relative increase in real versus false headlines, and thus, we predicted that compared to the control condition, the interventions increase sharing discernment (H3). 4Furthermore, we preregistered a range of robustness analyses of the treatment effects across covariates.We predicted that the treatment effect of 15-second and 3-minute interventions on citizens' likelihood of sharing false headlines is lower for respondents who have low trust in public institutions and government handling of the pandemic and low scores on cognitive reflection and attention to social comparison information, compared to respondents with high scores on these variables (H4).Next, we predicted a significant interaction between attention to social comparison information and all of the three interventions compared to the control condition on the willingness to share both real and false headlines, such that the effect is stronger for people who score high on attention to social comparison information (H5), and, finally, that the effect of all https://doi.org/10.1017/XPS.2024.2Published online by Cambridge University Press three interventions decays gradually with number of rating tasks completed (H6).We report briefly on all preregistered hypotheses in the main text and provide more details in Section A of the appendix.
Data were collected in collaboration with the market research institute YouGov through a two-wave panel among a national sample of the Danish population.YouGov sampled from their internet panels and employed quota sampling to match population characteristics on age, gender, region, and education.The power analysis suggested that 940 participants were required to have 90% power to replicate effect sizes from previous research on accuracy nudges (Pennycook et al. 2021) (see preregistration for formal power calculation).We recruited 2,541 participants between July 2 and 13, 2021 for the first wave.2,232 participants (88% of the original sample) completed the second wave between August 2 and 23.In the first wave, we collected psychological correlates of participants, while in the second wave, participants were exposed to a survey experiment.We did not record any posttreatment attrition in wave 2.
To assess the effectiveness of the interventions in study 1, participants completed a news-sharing task consisting of 15 real and 15 false headlines.Participants were informed that they would be exposed to a range of articles from the past year concerning COVID-19.We opted for headlines instead of full articles because people often share articles on social media without reading the full article (Gabielkov et al. 2016).The headlines were presented one at a time in random order, and respondents were asked whether they were willing to share them.
The main outcome is headline sharing measured through a standard item: "If you were to see the above article on social media, how likely would you be to share it?"(1: Extremely unlikely, 2: Moderately unlikely, 3: Slightly unlikely, 4: Slightly likely, 5: Moderately likely, 6: Extremely likely; re-scaled to 0-1) (Pennycook et al. 2020).Studies validating the measure suggest that people report higher sharing intentions for headlines in surveys that do indeed receive more shares on Twitter (Mosleh et al. 2021).We use the outcome to measure both real (M = 0.26, SD = 0.32) and false (M = 0.14, SD = 0.26) headline sharing respectively.Furthermore, we use it to assess sharing discernment (M = 0.13, SD = 0.20) which is defined as the difference in sharing intentions between real and false headlines where a higher discernment score indicates that people share more real relative to false news. 5

Results
We conduct the analyses using OLS regressions with standard errors clustered on subject and headline.Fig. 1 shows the effect of each intervention in three separate models with false and real headline sharing as well as sharing discernment, respectively, as the dependent variable (scaled 0-1)6 .5 Each individual indicates sharing intentions for 15 real and 15 false headlines.The discernment score is calculated as the respondent-level difference between real and false headline sharing, that is, sharing real sharing false .If a participant shared 12 out of 15 real headlines (0.8) and 6 out of 15 false headlines (0.4), their discernment score would be 0:8 0:4 0:4.We preregistered a range of robustness analyses of the treatment effects across covariates.We did not find that the treatment effect on sharing was significantly moderated by cognitive reflection, need for cognition, trust in government or health authorities (H4), and attention to social comparison (H5).Neither did the treatment effect significantly decay over time (H6) (See Section F in the appendix for details).

Study 2
In Study 1, we established that the 3-minute video intervention decreased sharing of false headlines.In Study 2, we used predictions derived from protection motivation theory to probe why the intervention worked.Protection motivation theory proposes that people protect themselves against risksin our case believing in and sharing false headlines on social mediabased on appraisals of (1) the threat from the risk and (2) their ability to cope with the risk (Rogers 1975; Maddux and Rogers 1983; Floyd et al. 2000;Sommestad et al. 2015;Pearce et al. 2019).Threat appraisals reflect the severity of the situation, the likelihood of a threat materializing, and individual vulnerability to the threat.Coping appraisals consist of two factors.The first factor is perceived "self-efficacy," understood as one's ability to carry out the recommended action and follow the advice successfully.The second factor is the perceived "response efficacy," understood as an individual's expectation that carrying out the recommended action and following the given advice will keep one safe from the threat.Research on protection motivation theory suggests that feelings of self-efficacy are, in general, the most important factor behind motivations to engage in protective behavior (Norman et al. 2015).Consistent with this, prior work on protection motivations in the context of the COVID-19 pandemic in Denmark has shown that compliance with public health authority advice is most strongly affected by self-efficacy (Jørgensen et al. 2021b).For study 2, we preregistered that compared to the control condition, the 3-minute intervention would increase threat appraisals (H7), feelings of self-efficacy (H8), and feelings of response efficacy (H9). 7As exploratory analyses, we also report the effects of the 15-second intervention and the accuracy nudge on these outcomes. 8 To determine sample size, we conducted a two-sided t-test power calculation as noted in the preregistration.Given 500 participants in each group, significance level at 0.05 an effect size of d = 0.19 can be estimated with power = 0.9, and d = 0.16 with power = 0.8.A national sample of 2,012 participants quota sampled to match population characteristics on age, gender, region, and education was collected in Denmark between December 17, 2021, and December 23, 2021, by YouGov.
Instead of the news-sharing task used in Study 1, the outcome measure in Study 2 was a battery of six protection motivation items that were combined to three measures (threat appraisal, self-efficacy, and response efficacy).Besides the change of outcome, the experimental protocol was the same.
Threat appraisal, self-efficacy, and response efficacy were measured in a battery of six items where each factor was measured as the mean of its two corresponding items.The following items were included.Threat appraisal (M = 0.55, SD = 0.21): (1) "I am exposed in terms of false information regarding COVID-19," (2) "False information regarding COVID-19 is a threat to the Danish society."Self-efficacy (M = 0.80, SD = 0.22): (3) "It is easy for me to avoid spreading false information about COVID-19," (4) "I am confident that I can avoid spreading false information about COVID-19 if I want to."Response efficacy (M = 0.73, SD = 0.24): ( 5) "If I avoid falling for false information, I will be in greater security during the Corona epidemic," (6) "If I avoid spreading false information, I take part in protecting others against COVID-19."For each item, participants were asked to what extent they agreed or disagreed with the statements on a scale from 1, "Strongly disagree," to 7, "Strongly agree," and re-scaled to vary between 0 and 1.

8
We did not predict specific hypotheses, but preregistered them as secondary analyses to test whether the accuracy nudge and the 15-second intervention affected threat appraisal, self-efficacy, and response efficacy.In sum, across the 3 interventions, we only found evidence of a statistically significant effect of the 3-minute intervention on self-efficacy.Across study 1 and study 2, this suggests that the 3-minute intervention both increase people's personal feelings of being competent in terms of avoiding sharing misinformation (in study 2) and reduce people's sharing of false headlines (study 1).In the early stages of the COVID-19 pandemic, there were extensive campaigns from the Danish Health Authorities about the threat of COVID-19, including misinformation.One potential explanation that we do not observe an effect on threat appraisal is pre-treatment effects from this communication from the public authorities.

Discussion
Are misinformation interventions effective against misinformation?The analyses showed that while the accuracy nudge and a 15-second capability-oriented intervention significantly increased sharing discernmentthat is, the relative sharing of real vs.false headlinesthey did not have a significant effect on neither false or real headline sharing compared to the control condition.The 3-minute capability-oriented intervention significantly increased sharing discernment and self-efficacy and reduced false headline sharing.In sum, we found mixed support for effectiveness of short capability-oriented messages and accuracy nudges against misinformation.These results add to recent academic research on the effectiveness of short messages or nudges on misinformation sharing (Rathje et al. 2022;Roozenbeek et al. 2021;Gavin et al. 2022;Pretus et al. 2023;Pennycook and Rand 2022).Notably, the 3-minute intervention was effective both in terms of boosting people's feelings of competence as well as reducing false headline sharing.The results expand our knowledge of the effectiveness of misinformation interventions by deploying different interventions in the same experimental framework.To assess the effectiveness of interventions, future research could adopt a similar approach in which they assess multiple interventions within the same experimental framework.Furthermore, while several studies assess whether interventions affect the belief in and sharing of misinformation (Pennycook et al. 2020;Roozenbeek et al. 2021;Guess et al. 2020;Badrinathan 2021;Bode and Vraga 2018;Jensen et al. 2022), few studies explicitly address the potential underlying mechanisms for why people alter their behavior (Lin et al. 2022;Altay et al. 2020).The finding that the most effective intervention also influenced participants' feelings of self-efficacy is consistent with the general finding in the risk communication literature that such feelings are key for motivating protective behavior (Norman et al. 2015).
In assessing these conclusions, it is worth noting important limitations of the study.While the study contributes by assessing interventions against misinformation beyond American or British samples (see also Badrinathan 2021;Guess et al. 2020;Gavin et al. 2022), it is important to note that the results are limited to a context where people are particularly responsive to government interventions.Danes held more trust in authorities, were more compliant, were less polarized, and were more concerned in the first stages of the COVID-19 pandemic than most other countries (Lindholt et al. 2021;Lieberoth et al. 2021).As a consequence, the Danes may be more responsive to messages from public health authorities.In other contexts with a higher degree of political polarization and skepticism regarding, for example, the threat of COVID-19, the effectiveness of interventions from public authorities may be more limited.To examine this proposition, future studies could use a comparative approach to assess the effectiveness of communication from public authorities across countries with varying degrees of trust, political polarization, and compliance with advice from authorities.
Practitioners should note that the interventions are mitigation strategies, rather than addressing the root cause of misinformation sharing.First, the effects of these types of interventions are small (Pennycook and Rand 2022).For instance, we included a shortened sharing task in study 2 and observed a smaller effect size than in study 1 for all interventions as the control condition was negatively affected by the protection motivation theory items asking questions like "False information regarding COVID-19 is a threat to the Danish society." 10 This in line with the literature suggesting that small nudges can boost sharing discernment (Pennycook et al. 2021).Nevertheless, the ephemeral effect sizes serve as an important reminder for researchers and practitioners alike that the effects of misinformation interventions often are small (Pennycook and Rand 2022), diminish over time (Carnahan et al. 2021;Carey et al. 2022), and thus require repeated intervention in practice (Ecker et al., 2022).Second, while a relatively small share of people share misinformation (Guess and Lyons 2020;Guess et al. 2019;Cinelli et al. 2021;Grinberg et al. 2019), recent studies suggest that social and political goals serve as important motivations (Uscinski et al. 2021;Osmundsen et al. 2021;Petersen et al. 2023;Rathje et al. 2023;Pickup et al. 2022) and some interventions may be less effective for people with certain political allegiances (Rathje et al. 2022).In Section F of the appendix, we conduct exploratory analysis of whether the interventions have heterogeneous treatment effects across trust in government, trust in health authorities, cognitive reflection, need for closure, attention to social comparison information, age, gender, income, education, and partisanship.In line with previous research, we find that partisanship predicts false headline sharing (Osmundsen et al. 2021), yet the effect of the 3-minute intervention on false headline sharing is consistent across partisanship.Furthermore, we do not find heterogeneous treatment effects for a range of other potentially relevant individual differences (specifically, cognitive reflection, need for closure, or attention to social comparison information, trust in government, or health authorities).In sum, while social, political, and accuracy motivations may shape people's overall propensity to share false headlines, the results suggest that the effect of the 3-minute video intervention is consistent.In other words, while interventions that provide competences do not address the root cause of misinformation sharing such as political motivations, they can be a reliable mitigation strategy with consistent effects across sub-populations.
The experimental design does not allow us to disentangle the effects of the length of the videos from the comprehensiveness of the advice.To be clear, our interpretation is that the 3-minute intervention is more effective in reducing false headline sharing and boosting self-efficacy, because it provides more elaborate guidance both in terms of providing information, tools, and devising actions to avoid sharing misinformation, compared to the 15-second intervention that only provides brief advice.Yet, further studies could benefit from testing interventions of similar length and assessing their efficacy.
The measures used in this study are based on sharing intentions, not actual online behavior, which clearly warrants concerns about whether the experimental results generalize beyond the experimental context.While sharing intentions is a standard way of measuring the experimental effects of misinformation interventions and it has 10 We specified in the preregistration that the protection motivation theory battery would treat respondents in the control condition who would otherwise be untreated.Therefore the sharing task could not be "considered a direct replication [of study 1]."Consistent with this expectation, the reduced effect size was due to participants in the control condition being less willing to share false headlines (the mean sharing of false headlines decreases from 0.156 in study 1 to 0.131 in study 2 amounting to a difference of 0.0248 (p < 0.001).Thus, the reason the effect size is lower in the shortened sharing task is that the mean of the control condition decreases in study 2. See Table 12 and Figure 8 in the appendix for details). 2 Did the interventions decrease false headline sharing (H1)?The 3-minute video intervention significantly decreased willingness to share false headlines (b = −0.055,95% CI = [−0.078,−0.031], p < 0.001, d = −0.222),while the 15-second video intervention (b = −0.00295% CI = [−0.027,0.023], p < 0.88, d = −0.008)and the accuracy nudge (b = −0.017,95% CI = [−0.041,0.008], p < 0.18, d = −0.064)did not, as shown in Fig. 1.In other words, only the 3-minute video intervention decreased false headline sharing.Did the interventions increase real headline sharing (H2)?The 3-minute video intervention (b = 0.028, 95% CI = [−0.002,0.052], p < 0.07, d = 0.089), the 15-second video intervention (b = 0.025, 95% CI = [−0.002,0.052], p < 0.08, d = 0.081) and the accuracy nudge (b = 0.026, 95% CI = [−0.001,0.054], p < 0.06, d = 0.084) did not significantly affect sharing of real headlines compared to the control condition.Did the interventions increase sharing discernment (H3)?All three interventions significantly increased sharing discernment.In other words, both the accuracy nudge (b = 0.043, 95% CI = [0.021,0.065], p < 0.001, d = 0.230), the 15-second video intervention (b = 0.027, 95% CI = [0.005,0.049], p < 0.02, d = 0.145), and the 3-minute video intervention (b = 0.082, 95% CI = [0.059,0.106], p < 0.001, d = 0.416) increased the relative sharing of real compared to false headlines.

Figure 1 .
Figure 1.Willingness to share real and false headlines.Note: Points are OLS estimates with 95% confidence interval bars based on clustered standard errors at the respondent level and headline level.The panels display estimates based on regressions of the interventions on false (n = 32,480) and real (n = 32,480) headline sharing as well as sharing discernment (n = 2,232) all re-scaled to 0-1.

Figure 2 .
Figure 2. Effect of interventions on threat appraisal, self-efficacy, and response efficacy.Note: Points are OLS estimates with 95% confidence interval bars based on clustered standard errors at the respondent level from three regressions.Each panel represents a regression of the treatment conditions on the respective pmt measure as the dependent variable.All regressions are based on samples of 2,012 respondents.

Table 1 .
Overview of data collection Note: † We conducted a pretest to validate the headlines for study 1 and study 2. Details can be found in Section C of the Supplementary Material.