INTRODUCTION
The duties of democratic citizenship place a premium not only on voter learning but on the assimilation of information into well-reasoned decisions. Under traditional theories of democratic accountability, voters must convert performance information into judgments of incumbent governments and cast their votes accordingly (Fearon Reference Fearon, Przeworski, Manin and Stokes1999). Likewise, the cause for widespread concern about misinformation is not merely that citizens confidently hold false beliefs, but that policy and candidate preferences are formed on their basis (Hochschild and Einstein Reference Hochschild and Einstein2015; Kuklinski et al. Reference Kuklinski, Quirk, Jerit, Schwieder and Rich2000).
In spite of mounting evidence that people generally update their beliefs in the direction of information and arguments (Arias et al. Reference Arias, Larreguy, Marshall and Querubín2022; Coppock Reference Coppock2023; Gerber and Green Reference Gerber and Green1999), studies have repeatedly found that these stimuli produce inconsistent or muted effects on downstream attitudes and behaviors. Information about inequality and government spending has been shown time after time to alter economic beliefs while generating minimal or inconsistent change in economic policy preferences (Ballard-Rosa et al. Reference Ballard-Rosa, Rogowski, Scheve and Thor2025; Barnes et al. Reference Barnes, Feller, Haselswerdt and Porter2018; Kuziemko et al. Reference Kuziemko, Norton, Saez and Stantcheva2015). Large-scale studies providing transparency and accountability information to citizens across varied contexts find little effect on evaluations of politicians and voting behavior (Dunning et al. Reference Dunning, Grossman, Humphreys, Hyde, McIntosh, Nellis and Adida2019). Successfully correcting false beliefs about vaccines, immigrants, welfare, and more begets no change in intention to vaccinate (Porter, Velez, and Wood Reference Porter, Velez and Wood2022), immigration attitudes (Hopkins, Sides, and Citrin Reference Hopkins, Sides and Citrin2019), prejudice (Yeung and Glasgow Reference Yeung and GlasgowForthcoming), or sentiment toward the candidates responsible for propagating the misinformation (Coppock et al. Reference Coppock, Gross, Porter, Thorson and Wood2023; Nyhan et al. Reference Nyhan, Porter, Reifler and Wood2020). To be sure, factual information has in some cases been found to induce meaningful attitude change (Carey et al. Reference Carey, Fogarty, Gehrke, Nyhan and Reifler2025; Kustov and Landgrave Reference Kustov and Landgrave2025; Sides Reference Sides2016; Thorson and Abdelaaty Reference Thorson and Abdelaaty2023). Yet the overall pattern remains inconsistent, and there is little guidance as to why the effects of counter-attitudinal information sometimes propagate to related judgments and sometimes do not.
A review of the literature further reveals little consensus for what this empirical pattern implies about mass political behavior. A distressing possibility—consistent with influential theories about the biased assimilation of evidence (Festinger Reference Festinger1957; Kunda Reference Kunda1990)—is that beliefs are the consequence of political attitudes rather than the cause (Lodge and Taber Reference Lodge and Taber2013). Beliefs may be little more than post-hoc rationalizations of attitudes conceived in order to shield prior commitments from incongruent evidence. An absence of attitude change in the presence of belief change should therefore come as no surprise.
Understanding why information and arguments sometimes shift beliefs without correspondingly moving attitudes, we argue, requires paying more careful attention to belief relevance as a crucial moderator. Studies in political persuasion typically target beliefs that academics or analysts perceive a priori as focal to a policy debate or momentarily pertinent due to widely circulating information. Yet to the mass public, a wholly different set of beliefs may matter. For one, the considerations citizens deem relevant to a policy debate are shaped by how politicians frame the debate, and thus may differ from what academics prescribe as relevant (Kuklinski et al. Reference Kuklinski, Quirk, Schwieder and Rich1998). This concern bears resemblance to Converse’s (Reference Converse and Apter1964, 8–10) familiar argument that mass ideology is more likely to exhibit “social” than “logical” constraint: voters’ ideas about “what goes with what” tend not to emerge organically as logical wholes, but rather are diffused from elites, broadly conceived.Footnote 1 Moreover, even if scholars identify and account for the most salient beliefs within a population or subgroup, doing so may mask substantial heterogeneity between voters (Fishbein and Ajzen Reference Fishbein and Ajzen2010, 102), which can mute downstream attitude change. Consequently, belief change may facilitate little to no attitude change when interventions target beliefs that respondents consider only weakly relevant.
Our argument proceeds in two steps. First, we show that prominent debates about change and rigidity in public opinion address why information fails to change beliefs without explaining how belief and attitude change could diverge. We offer conceptual clarity surrounding the distinction between beliefs and attitudes and summarize how an attitude may be modeled as the weighted sum of beliefs about an attitude object, where the influence of each belief is in direct proportion to its relevance. In doing so, this section illustrates why the concept of relevance enables scholars to reconcile widespread evidence of inconsistent or muted effects of information on attitudes with the standard intuition of belief–attitude models—“a reasonable and consistent link between information about an object and attitude toward that object” (Fishbein and Ajzen Reference Fishbein and Ajzen2010, 99).
To address the longstanding methodological challenge of identifying and targeting a respondent’s personally relevant beliefs, we developed a respondent-tailored design enabled by the large language model (LLM) GPT-4o. In the first stage, respondents engage in dialogue with a chatbot about a deeply held political attitude and their reasoning behind that view. GPT then synthesizes a summary attitude, a “focal” belief underlying that attitude, and a second “distal” belief generated without reference to chat input. In the second stage, respondents are randomly exposed to an intervention that targets a focal belief, distal belief, or placebo topic, in each case displaying an extended counterargument.Footnote 2 This design furthers the development of tailored experimental methods in the political persuasion literature (e.g., Velez and Liu Reference Velez and Liu2025).
Across the two studies, we provide evidence that belief relevance matters for attitude change. Tailored counterarguments were similarly effective at shifting focal and distal beliefs. However, focal belief counterarguments produced larger attitudinal effects than distal counterarguments on average: a random effects meta-analysis pooling 12 attitudinal outcomes yields a significant average difference of d = 0.054 (SE = 0.02;
$ p= $
0.014), with 11 of 12 individual comparisons directionally favoring the focal condition. Additionally, in follow-up surveys approximately 10 days after initial exposure, we find some persistence of attitudinal effects of the focal belief counterarguments but no significant effects of the distal belief counterarguments. These differences in attitude change despite comparable belief change testify to the importance of accounting for belief relevance at the respondent level. We conclude by exploring additional mechanisms and discussing implications for the study of persuasion and democratic politics. Specifically, we note the empirical challenges in distinguishing between a continuous model of belief relevance, where persuasive effects increase monotonically with relevance, and a categorical model, where persuasive effects plateau above a certain level of relevance.
EXPLAINING ATTITUDES’ RIGIDITY DESPITE BELIEF CHANGE
The tendency of deeply held attitudes to resist change even in the face of compelling counterarguments and information has long alarmed social scientists (e.g., Festinger, Riecken, and Schachter Reference Festinger, Riecken and Schachter1956). Although students of public opinion have written extensively on how citizens respond to new information, recent debates focus on whether and why information fails to alter beliefs, without addressing why attitudes may resist change even as beliefs move.
One widely cited explanation for the rigidity of beliefs and attitudes faults citizens as unwilling to pursue accuracy. Under the theory of motivated reasoning, people are often driven by “directional motives” to reach specific conclusions (Kunda Reference Kunda1990) and therefore accept and interpret evidence in a selective or skewed fashion (e.g., Bartels Reference Bartels2002; Taber and Lodge Reference Taber and Lodge2006). The principal rejoinder contends that these patterns can be reconciled with a Bayesian model of rational learning. People with different prior beliefs may update in divergent ways not due to motivation but because their priors generate different posterior beliefs about the same newly learned information (e.g., Bullock Reference Bullock2009; Gerber and Green Reference Gerber and Green1999). A slew of recent studies further contend that resistance is not the modal way in which people respond to political information. For example, people seem to consistently update beliefs in the direction of fact checks, irrespective of their priors, across settings and across time (Coppock et al. Reference Coppock, Gross, Porter, Thorson and Wood2023; Porter, Velez, and Wood Reference Porter, Velez and Wood2023; Walter et al. Reference Walter, Jonathan Cohen, Holbert and Morag2020).Footnote 3
Neither critique, however, addresses the puzzle animating this article. Gerber and Green (Reference Gerber and Green1999) emphasize that the Bayesian learning model is concerned with beliefs, “which measure a voter’s assessments of objective characteristics of the political world” (193), but not evaluations or preferences. As a result, the Bayesian learning debate offers little insight into why belief change and attitude change might diverge. Moreover, informational interventions that successfully change beliefs rarely have discernible effects on downstream attitudes or behaviors, even in well-powered studies (e.g., Nyhan et al. Reference Nyhan, Porter, Reifler and Wood2020; Porter, Velez, and Wood Reference Porter, Velez and Wood2022).
The literature therefore offers little consensus about how to interpret cases in which information fails to shape attitudes despite altering beliefs. A motivated reasoning account would suggest that people are more intensely motivated to preserve attitudes than to preserve beliefs and therefore may rationalize away the connection between newly learned information and their prior opinions. Studies that draw upon motivated reasoning have speculated, for instance, that “beliefs are not so much building blocks of attitudes but consequences of attitudes” (Lawrence and Sides Reference Lawrence and Sides2014, 2).Footnote 4 Other studies take a more conservative approach, drawing conclusions about domain-specific attitudes but not about persuasion in general.Footnote 5
We caution against drawing broad conclusions about persuasion from these kinds of experimental results, especially before belief relevance is taken into account. We begin by defining the difference between beliefs and attitudes and specifying a function that links belief change to attitude change. Doing so helps clarify why, although the Bayesian versus motivated reasoning debate does not address our puzzle, existing intuitions about information processing can be incorporated within a broader model of attitude change. Drawing on the expectancy value (EV) model, we argue that beliefs may take on different weights in this attitude function due to varying relevance to an attitude. Furthermore, we argue that typical designs ignore individual-level variation in belief relevance, contributing to the preemptive conclusion that attitudes are either orthogonal to or upstream of beliefs.
DISTINGUISHING BELIEFS FROM ATTITUDES
While studies of persuasion frequently measure relevant beliefs and attitudes in tandem,Footnote 6 the distinction is key. Whereas beliefs pertain to perceptions of facts or objective states of the world, attitudes lack inherent truth value, instead representing personal preferences, values, or normative judgments about what ought to be (Petty Reference Petty2024, 1276). One might hold the belief that a candidate is an avowed progressive (a descriptive claim) while viewing that characteristic negatively (an evaluative position).
According to Bayesian models, individuals possess prior beliefs about a state of the world and update these beliefs when presented with new information, weighing both the strength of their prior beliefs and the credibility of the new evidence. In the simplest model, the posterior probability that one’s view of a state of the world, b, is true given some evidence, e, depends on the prior probability of b (i.e.,
$ \Pr (b) $
) and the likelihood of e given b:
$$ \hskip1pc \Pr (b\mid e)\hskip2.77695pt =\begin{array}{ccc}\hskip2.77695pt \frac{\Pr (e\mid b)\hskip0.3em \Pr (b)}{\Pr (e\mid b)\hskip0.3em \Pr (b)\hskip2.77695pt +\hskip2.77695pt \Pr (e\mid \neg b)\hskip0.3em \Pr (\neg b)}.& & \end{array} $$
Attitudes, as evaluative judgments, do not hold truth value per se, but they often rest on a foundation of beliefs or considerations (Fishbein Reference Fishbein1979). Specifying a belief–attitude function enables us to incorporate Bayesian learning within a model of attitude change. Consider an individual forming an attitude A toward an object (e.g., a policy proposal). Suppose this person holds a set of n beliefs about various attributes of the policy, such as whether the policy is effective, necessary, or costly. Each belief
$ {b}_i $
can be represented as the perceived probability that the policy possesses attribute i. Using the Bayesian model,
$ \Pr ({b}_i\mid e) $
can be updated in the light of new evidence e. However, to translate those updated beliefs into an overall evaluative judgment, an additional function is necessary. Here,
$ f(\cdot ) $
is an aggregation rule that maps the updated belief probabilities into a summary evaluation:
$$ \begin{array}{rll}\hskip1pc A\hskip2.77695pt =\hskip2.77695pt f\left(\Pr ({b}_1),\hskip0.3em \Pr ({b}_2),\hskip0.3em \dots, \hskip0.3em \Pr ({b}_n)\right).& & \end{array} $$
The implications of a belief can vary substantially across individuals. One reason is that people may differ in their evaluation of the attribute implied by
$ {b}_i $
. Those who believe a candidate is progressive may hold positive, negative, or neutral affect toward progressives, changing the implications of those beliefs. This is the intuition behind psychology’s EV model (Fishbein and Ajzen Reference Fishbein and Ajzen1975; Reference Fishbein and Ajzen2010), one of several classic frameworks that conceive of attitudes as an aggregation of beliefs and evaluations.Footnote 7 A second reason is that politicians or media figures often shape the considerations citizens bring to bear on their political opinions through issue framing. In summative models like EV, this corresponds to shifting the weights associated with beliefs (Chong and Druckman Reference Chong and Druckman2007; Jerit Reference Jerit2009), though the precise mechanism behind changes in weights can vary (Nelson, Oxley, and Clawson Reference Nelson, Oxley and Clawson1997; Slothuus Reference Slothuus2008). A frame may increase the weight of a consideration either by increasing its cognitive accessibility or by changing the explicit importance we consciously assign it (Nelson and Oxley Reference Nelson and Oxley1999). Formally, this can be expressed as
$$ \begin{array}{rll}\hskip1pc A=f\left({w}_1\cdot \Pr ({b}_1),\hskip0.3em {w}_2\cdot \Pr ({b}_2),\hskip0.3em \dots, \hskip0.3em {w}_n\cdot \Pr ({b}_n)\right),& & \end{array} $$
where
$ {w}_i $
represents the weight assigned to each belief.Footnote 8
In short, the extent to which changes in beliefs result in shifts in attitudes depends fundamentally on the relevance of those beliefs to individuals’ judgments. Beliefs deemed irrelevant or peripheral are less likely to result in attitude change, even if convincingly rebutted, challenged, or corrected. Here, we use relevance to signify the weights on beliefs.
Though scholars of political persuasion seldom make explicit reference to attitudinal models like the one above, the basic intuition behind belief relevance underpins the field’s standard empirical designs. For placebo-controlled survey experiments that evaluate the persuasive impact of arguments or informational treatments, the conventional approach to selecting placebo stimuli involves identifying an excerpt that resembles the treatment stimulus in length and other attributes but lacks any bearing on the attitude being studied (Porter and Velez Reference Porter and Velez2022). Put another way, placebo selection in persuasion experiments relies on a presumption that the stimulus bears zero relevance in respondents’ heads to the attitude of interest.
Figure 1 presents the range of possible belief weights
$ {w}_i\in [0,{w}_{max}] $
for a hypothetical person. Arguments equally germane to a political issue in the abstract may nonetheless be located differently along this spectrum due to individual heterogeneity in accessibility and personal relevance. Where (a) an inert placebo argument about daylight saving time (or some other unrelated topic) falls on the lowest end of this linear scale, beliefs and arguments concerning (b) abortion restrictions increasing foster care placement, (c) bodily autonomy, and (d) health of the mother contain increasing amounts of relevance to this hypothetical individual. Their focal justification or reason for holding an issue stance may be (d), but they nonetheless may be able to retrieve additional considerations from memory such as (c) or might be familiar with arguments regarding (b).
Spatial Representation of Belief Relevance with Examples of Pro-Choice Beliefs
Empirical Implications
Depicting relevance along a continuum helps illustrate what we might learn about the role of beliefs in attitude formation from traditional and tailored designs. Persuasion studies often focus on the effects of popular or ubiquitous counterarguments, in essence comparing low or medium relevance beliefs (b and c) to a placebo condition (a). Such a design is helpful in identifying how arguments that are commonly encountered in politics shape attitudes on average and can be useful for gauging the effectiveness of political rhetoric. However, this design is unable to separate popularity (or aggregate salience) from personal relevance, as one must assume that beliefs that are widely held or frequently discussed are also beliefs that matter most for individual attitudes. In contrast, directly eliciting high
$ {w}_i $
beliefs may allow us to identify the beliefs that actually facilitate attitude change for each individual, without assuming that what is modally salient in a population is personally relevant. Our design tests these gradations of
$ {w}_i $
by directly comparing attitude change in response to counterarguments targeting low relevance beliefs, salient beliefs, and respondents’ own beliefs about a given issue domain.
CHALLENGES IN IDENTIFYING BELIEF RELEVANCE WEIGHTS
Researchers use various strategies to identify beliefs for empirical study, each with distinct theoretical implications. The first are what we deem population-level approaches. Misinformation scholars typically target beliefs based on their prevalence, drawing on fact-checking databases to identify false claims that have been widely circulated (see Porter, Velez, and Wood Reference Porter, Velez and Wood2023). Persuasion scholars often focus on salient arguments or campaign advertisements (Coppock, Hill, and Vavreck Reference Coppock, Hill and Vavreck2020). Network methods have also emerged as a tool for mapping the structure between beliefs and attitudes at the group level (e.g., Fishman and Davis Reference Fishman and Davis2022).
Population-level approaches to identifying influential beliefs—whether based on expert judgment, viral spread, or aggregate network patterns—are not well-suited for identifying the conditions under which beliefs affect attitudes at the individual level. One limitation lies with obscuring individual heterogeneity (Brandt and Morgan Reference Brandt and Scott Morgan2022). Beliefs about bodily autonomy may be irrelevant to a given voter’s abortion attitude even if they best predict pro-choice attitudes in the aggregate. A second challenge is construct validity. Selecting issues ex ante based on aggregate salience may miss idiosyncratic or less salient beliefs that are nonetheless central to attitudes. Recent studies that attempt to construct belief networks, even at the individual level, assess relationships between preselected political issues, which may mask people’s idiosyncratic connections (Brandt Reference Brandt2022). Our studies show, for instance, that a small handful of citizens with pro-choice attitudes cite a potential increase in unsafe procedures as their central justification. Failure to include relevant survey questions about unsafe procedures would produce only a partially observed network of considerations for these respondents.
These limitations complicate the inferences we draw about the relationship between beliefs and attitudes. If beliefs are low relevance for most in the sample due to individual heterogeneity or because they reflect “nonbeliefs,” scholars might conclude that beliefs have no effects on attitudes or that they are consequences of those attitudes. The inferential problem is thus not whether beliefs can influence attitudes, but whether the specific beliefs measured in any given study are the ones that actually matter to respondents.
We propose an alternative method: using LLMs to identify personally salient beliefs through direct dialogue with participants. Rather than assuming beliefs relevant at the population level will matter for all individuals, this approach elicits each participant’s own stated reasons for holding their attitudes. By engaging participants in an open-ended discussion about their views, we can identify the specific beliefs they spontaneously invoked to justify their positions—beliefs that are, on the respondent’s own terms, identified as relevant to their individual attitude structures. Upon identifying important attitude–relevant beliefs, we intervene on those beliefs to assess how this affects downstream attitudes.
Using our design, we elicit issue attitudes (e.g., supporting DACA, opposing restrictions on automatic firearms) and their justifications (e.g., improving the educational attainment of all Americans, protecting Second Amendment rights) for each individual in our study, which we use to produce tailored measures of attitude strength and belief strength (see Appendix A.1 of the Supplementary Material for question wording). We instruct an LLM—OpenAI’s GPT-4o—to create two descriptive statements for each respondent, one summarizing their own justifications (hereafter, focal beliefs) and a second capturing more distal arguments that support the issue attitude but that were not disclosed in the conversation (e.g., economic contributions of undocumented immigrants and their children).Footnote 9 This allows us to directly test whether heavily weighted considerations produce more attitude change than less heavily weighted considerations. It also mirrors the population-level approach in that it targets beliefs without first assessing how relevant those beliefs are to individual participants.
Our design contributes to a growing body of work that employs open-ended survey methods to study political persuasion. Thorson (Reference Thorson2024) made inroads on the inferential problem of population-based approaches by conducting pilot phone interviews to identify important misperceptions about government before measuring their prevalence through a representative panel survey. Our design simplifies but scales this interview-based approach so that—in the same survey wave and identical sample—we can elicit respondents’ beliefs before presenting tailored counterarguments. To ensure that respondents can meaningfully discuss the reasons behind their attitudes, we draw upon studies that use open-ended prompts to tailor public opinion surveys to citizens’ idiosyncratic issue priorities (Ryan and Ehlinger Reference Ryan and Ehlinger2023). This approach is increasingly common as political scientists pay closer attention to issue intensity and its moderating effects on attitudes and participation (Hill Reference Hill2022). Further, we advance a nascent method of embedding LLMs into online surveys (e.g., Velez and Liu Reference Velez and Liu2025).
As defined in our pre-analysis plan, our central research question concerns how belief relevance affects persuasive effects. Though not pre-registered as hypotheses, we propose the following sequential expectations to organize our empirical findings. These expectations build on prior work in information effects, motivated reasoning, and attitude formation.
Assessing the “First Stage” of Belief Change
Hypothesis 1a (Uniform Updating)
Studies of factual corrections and counter-attitudinal arguments more broadly have increasingly found that people consistently update their beliefs in the direction of information, irrespective of their prior beliefs (e.g., Coppock Reference Coppock2023; Porter, Velez, and Wood Reference Porter, Velez and Wood2022). These studies generate an expectation that focal and distal belief counterarguments will reduce focal and distal belief strength (respectively) relative to control conditions, and to similar degrees.
Individuals presented with a counterargument (either focal or distal) will update their beliefs toward the counter-attitudinal information to similar degrees.
Hypothesis 1b (Differential Updating)
Drawing on theories of motivated reasoning, individuals may process counterarguments differently depending on how central the challenged beliefs are to their broader belief system. While a strong form of motivated reasoning suggests complete resistance or backfire effects (Kunda Reference Kunda1990; Taber and Lodge Reference Taber and Lodge2006), people may also act as “cautious Bayesians” who update their beliefs to varying degrees based on the centrality and strength of their prior beliefs (Hill Reference Hill2017). Under this account:
The magnitude of belief updating will vary based on belief relevance, with focal beliefs showing either no movement or significantly smaller updates compared to distal beliefs. Specifically: (i) for focal beliefs, individuals will either maintain their prior beliefs or exhibit minimal updating in response to counterarguments; (ii) for distal beliefs, which are less central to attitudes, individuals will show larger shifts toward the counterarguments.
Assessing the Belief–Attitude Function
Hypothesis 2a (Equal Effectiveness)
It may be that any counterargument—whether targeting a focal or distal belief—has a comparable effect on attitudes, provided the individual updates and sees the counterargument as relevant enough to reshape their broader evaluation. This represents what seems to be the operating assumption within the literature.
Counterarguments to focal and distal beliefs will be equally effective at shifting downstream attitudes.
Hypothesis 2b (Focal Belief Dominance)
Alternatively, if focal beliefs are highly weighted in the formation of attitudes, changing these beliefs should be more influential in shifting attitudes than changing more peripheral, distal beliefs.
Counterarguments challenging focal beliefs (i.e., those identified by participants as central to their opinions) will produce greater changes in downstream attitudes than counterarguments challenging distal beliefs.
RESEARCH DESIGN
Our experimental design uses LLMs at various stages of the process to recover key theoretical quantities and generate experimental stimuli (see Figure 2 for a diagram). Text classification, summarization, and generation are among the strengths of LLMs (Argyle et al. Reference Argyle, Busby, Gubler, Lyman, Olcott, Pond and Wingate2025; Mellon et al. Reference Mellon, Bailey, Scott, Breckwoldt, Miori and Schmedeman2022), and previous work has shown that they can be productively integrated into surveys to measure deeply held beliefs and attitudes (Velez and Liu Reference Velez and Liu2025) and produce compelling arguments (Bai et al. Reference Bai, Voelkel, Muldowney, Eichstaedt and Willer2025). We leverage these strengths to (1) elicit issue attitudes; (2) capture justifications for those attitudes (“focal beliefs”); and (3) generate counterarguments that randomly target those “focal beliefs” versus unmentioned, but potentially relevant, beliefs that might support those attitudes (“distal beliefs”).Footnote 10
Overview of the Two-Wave Design and Experimental Procedures
Note: In Wave 1, participants first provide an issue they care deeply about (Stage 1A), then engage in a semi-structured dialogue with an LLM to clarify their justifications (Stage 1B). The LLM identifies participants’ “focal” beliefs, generates “distal” beliefs, and constructs tailored counterarguments (Stage 2). Pre-treatment measures are taken and participants are then randomly assigned to receive one of three messages (focal belief counterargument, distal belief counterargument, or placebo) before completing post-treatment measures. In Wave 2, approximately 10 days later, participants repeat the attitude and belief measures to assess the durability of belief and attitude change.
We carry out a pre–post experimental design that comprises three stages (Clifford, Sheagley, and Piston Reference Clifford, Sheagley and Piston2021). First, we elicit issue positions from participants. In Study 1, we randomly assigned participants to either a closed-ended slate of issue positions or open-ended question asking them to report a deeply held issue position. In Study 2, we solely relied on the latter question. Second, in both studies, we have all participants engage in a semi-structured discussion about their issue position and their justifications for holding the issue position.Footnote 11 Once the conversation is complete, these data are passed to an LLM that summarizes the participant’s issue attitude, summarizes their primary justification for holding that attitude (“focal belief”), generates an attitude-congenial, but unmentioned, justification for holding the attitude (“distal belief”), and produces three counterargument vignettes: (1) a vignette challenging the “focal belief”; (2) a vignette challenging the “distal belief”; and (3) a placebo vignette producing an argument about Daylight Saving Time. Participants were informed of their interactions with AI models in the debriefing materials, as well as in the chat itself (e.g., participants were shown text that an AI was “typing”).Footnote 12
Following this phase, we measure issue attitudes on a five-point Likert item ranging from “strongly disagree” to “strongly agree,” focal belief strength on a four-point scale ranging from “not at all accurate” to “very accurate,” and distal belief strength using the same scale. In Study 2, we also measure pretreatment scores on an “attitude defense” scale developed in Velez and Liu (Reference Velez and Liu2025). After the pretreatment measurement step, participants are randomly assigned to see one of the tailored vignettes, with participants equally likely to be assigned to a “focal belief counterargument,” “distal belief counterargument,” or “placebo” condition. These vignettes involved approximately 250-word arguments disputing the assigned belief. The LLM was instructed to craft persuasive messages that incorporated evidence.
In the post-treatment phase, we measured attitude strength and belief strength for the focal and distal beliefs. In Study 1, we additionally measured a multi-item “attitude extremity” scale developed in Velez and Liu (Reference Velez and Liu2025), whereas in Study 2, we measured “attitude extremity,” “attitude defense,” and certainty on a 101-point scale. In addition to attitudinal measures, we asked participants to report the relevance of their beliefs to their attitudes on a four-point scale ranging from “not at all important” to “very important” in Studies 1 and 2. In Study 2, we provided participants with a text entry box to indicate “omitted beliefs” that support their attitude along with a rating scale. This measure was included to capture whether participants selectively bolstered unmentioned beliefs as a defense mechanism in the face of counterarguments.
In both studies, we carry out a second wave approximately 10 days after the first wave. In this wave, we measure the full set of beliefs and attitudes measured in the initial wave, along with measures of argument recall. Specifically, we used GPT-4o-mini to extract two “definitive” sentences from the focal belief counterargument, distal belief counterargument, and placebo texts generated for each respondent, with six sentences in total. We asked participants to identify which sentences they recall seeing in the previous wave. This serves as a test of selective memory, such that participants may forget arguments and information that bear on their deeply held beliefs or attitudes.
Our first study was conducted on CloudResearch Connect (
$ n= $
2,494). Wave 1 was fielded through November 1–5, 2024, and wave 2 (
$ n=\mathrm{2,096} $
) through November 15–22. Study 2 sought to replicate results with a sample from Prolific. Wave 1 (
$ n=\mathrm{1,924} $
) was fielded through November 26–27, 2024, and wave 2 (
$ n=\mathrm{1,712} $
) through December 5–12. CloudResearch Connect and Prolific are both high-quality sample providers that have been used extensively in social science research (Stagnaro et al. Reference Stagnaro, Druckman, Berinsky, Arechar, Willer and Rand2024). Studies were pre-registered on AsPredicted.org (see Appendix E of the Supplementary Material).Footnote 13
Comparing Focal and Distal Beliefs
A centerpiece of our empirical strategy is identifying a distal belief that, while less personally relevant to the respondent, is nevertheless germane to their core issue. To evaluate the success of our design in practice, we identified the top five most commonly occurring issue positions across our two studies and, for each issue, identified common arguments mentioned in respondent’s own focal beliefs. We then used this list to categorize all focal and distal beliefs, paying special attention to cases in which the distal belief did not relate to common, predefined arguments. The distributions of arguments contained in the focal and distal beliefs—along with a description of the procedure combining handcoding with LLM classification, details about interrater reliability, LLM evaluation metrics, and a full list of coded beliefs—are found in Appendix B.1 of the Supplementary Material. For every issue, over 98% of distal beliefs were related to at least one common argument mentioned by other respondents, offering certainty that—unlike the placebo—distal beliefs generated by GPT possessed a baseline relevance to respondents’ core issues. This in turn gives us the confidence to interpret a difference in attitudinal effects between focal and distal counterarguments as evidence for beliefs’ personal relevance.
Can We Recover Strong Attitudes and Focal Beliefs?
We focus on identifying participants’ deeply held attitudes. Though this in theory may decrease our ability to detect belief and attitude change, we do so for two important reasons. Relative to “non-attitudes” or those that are not very salient to participants, deeply held attitudes ought to have a richer set of considerations that can be intervened on (Zaller Reference Zaller1992). Respondents will be more likely to call to mind relevant beliefs in conversation, enabling us to observe how intervening on a belief affects attitudes. By contrast, focusing on novel issue areas or weak attitudes would increase the likelihood that any observed effects primarily reflect learning or opinion formation rather than genuine belief change. Second, focusing on deeply held attitudes provides a “harder test” of our theoretical predictions, as any observed effects are likely to represent meaningful attitude change rather than shifts in superficial opinions.
To what extent can we recover deeply held attitudes? As shown in Table 1, we observe attitude strength scores of 4.85 (SE = 0.01) in Study 1 and 4.84 (SE = 0.02) in Study 2. The vast majority of participants select the maximum score on the five-point scale, indicating strong attitudes in the aggregate. Indeed, we find that 89% of participants in Study 1 select the maximum score on the Likert item, whereas 88% do so in Study 2. This dovetails with Velez and Liu (Reference Velez and Liu2025) who find that LLMs can successfully summarize issue positions and recover deeply held attitudes that persist over time.
Summary Statistics for Variables across Studies
Note: Mean (SE) and percentage selecting maximum option.
Next, we turn to belief strength. Our theoretical tests hinge on the capability of LLMs to properly summarize focal justifications for attitudes. In Study 1, we find a mean belief strength score of 3.64 (SE = 0.01), with 72% of participants selecting the maximum score; Study 2 replicates this general pattern, with a mean belief strength score of 3.67 (SE = 0.02) and 72% of participants selecting the maximum score. Directly comparing focal beliefs to distal beliefs, we observe a difference in belief strength of 0.32 scale points (SE = 0.02) in Study 1 (
$ p< $
0.001) and 0.31 scale points (SE = 0.04) in Study 2 (
$ p< $
0.001), indicating that the focal beliefs retrieved are significantly stronger than distal beliefs.
Additionally, we can assess whether participants see the focal beliefs as relevant to their attitudes. In Study 1, we observe a mean belief relevance score of 3.72 with 76% selecting the maximum option. In Study 2, the mean belief relevance score is 3.84 with 85% selecting the maximum. This can be compared to the distal belief, where scores are 0.27 scale points lower (SE = 0.02) in Study 1 (
$ p< $
0.001) and 0.23 scale points lower (SE = 0.04) in Study 2 (
$ p< $
0.001). Thus, focal beliefs are generally seen as relevant to attitudes and more relevant than distal beliefs.
We also carry out an exploratory analysis of belief stability in Appendix B.5 of the Supplementary Material. Focusing on placebo respondents who are not treated with any attitude–relevant information, we find that focal beliefs exhibit higher levels of stability relative to distal beliefs, with approximately three-quarters of participants maintaining the same level of belief strength across waves in both studies (compared to 64.1%–66.7% for the distal belief). Moreover, among those scoring at the maximum level of belief strength for the focal belief, 85%–86% of participants remain at the maximum, compared to 79%–80% of those responding to the distal belief question.
Finally, we assess the face validity of our procedure by presenting the user inputs and the LLM-generated measures. Recall that participants provide an issue stance that is used to initiate a subsequent conversation with a chatbot. This conversation then serves as the input for the construction of personalized measures and interventions. Table 2 presents the raw inputs with respect to open-ended responses and snippets from the subsequent conversation in the first two columns, with LLM-generated measures in the last three. Participants mentioned a variety of issues and provided discrete reasons for their issue positions. The LLM-generated focal beliefs consistently tracked these justifications and rationales (e.g., “extreme weather” for a participant worried about climate change and “drug and human trafficking” for a participant worried about immigration), whereas distal beliefs were issue stance-aligned but did not reflect the key justifications mentioned by the participant (e.g., “mass migration and geopolitical instability” for a climate-concerned participant who never explicitly mentioned these concerns).
Sample of LLM-Generated Measures across Diverse Policy Issues
Note: Chats are abbreviated with ellipses to show relevant excerpts from longer conversations.
Overall, the evidence suggests that we are successfully recovering deeply held attitudes and relevant beliefs. Moreover, the high levels of support and stability for the distal beliefs suggest that the LLM is also producing plausible beliefs that support participants’ attitudes, even if they are not as central to their reasoning. This provides reassuring evidence that our approach successfully identifies strong attitudes and focal beliefs.
Inducing Belief Change
Before examining effects on attitudes, we first assess whether counterarguments successfully shifted their targeted beliefs. This serves as a manipulation check: if counterarguments failed to move beliefs, downstream attitude effects would be difficult to attribute to persuasive effects on beliefs. Additionally, comparing belief change across focal and distal conditions allows us to rule out the alternative interpretation that focal counterarguments are simply more compelling arguments. Finding equivalent belief change alongside divergent attitude change makes a relevance-weight explanation more plausible than an alternative account based on differences in argument quality.
According to the differential updating hypothesis (H1b), we should observe resistance or even backfire effects when challenging focal beliefs that participants use to justify their attitudes, whereas a uniform updating process (H1a) might hold that counterarguments have similar efficacy regardless of belief strength.
To assess the effects of arguments on focal and distal belief strength, we estimate OLS regression models with HC2 standard errors. The models take the following form:
$$ \begin{array}{c}{Y}_i={\beta}_0+{\beta}_1{FocalBeliefCounterarg}_i\\ {}+{\beta}_2{DistalBeliefCounterarg}_i+{\boldsymbol{X}}_i^{\prime}\boldsymbol{\gamma} +{\epsilon}_i,& \end{array} $$
where
$ {Y}_i $
is post-treatment belief strength for participant i,
$ {\boldsymbol{X}}_i $
is a vector of pre-registered pretreatment covariates (pretreatment attitude strength, focal belief strength, and distal belief strength in Study 1; the same set of covariates, along with a pretreatment attitude defense measure in Study 2) and the omitted category is the placebo condition. The coefficients of interest are
$ {\beta}_1 $
and
$ {\beta}_2 $
, which capture the average treatment effects of focal and distal belief arguments relative to the placebo. In Study 1, we adjust for pretreatment attitude strength, focal belief strength, and distal belief strength. In Study 2, we adjust for the covariates, as well as pretreatment scores on a multi-item “attitude defense” scale.
We directly compare effect estimates across two outcomes: focal belief strength and distal belief strength. For each outcome, we estimate separate regression models where the dependent variable is either the focal or distal belief strength. This allows us to test our competing hypotheses in several ways.
First, to assess H1a, we compare
$ {\beta}_1 $
when focal belief strength is the outcome to
$ {\beta}_2 $
when distal belief strength is the outcome. Similar effect sizes would indicate that counter-attitudinal arguments are equally effective regardless of belief type—that is, focal belief arguments move focal beliefs just as much as distal belief arguments move distal beliefs. Second, for H1b, we examine whether focal beliefs are more resistant to change than distal beliefs. Under this hypothesis,
$ {\beta}_2 $
might be negative when distal belief strength is the outcome (indicating successful belief change), but
$ {\beta}_1 $
could be positive or null when focal belief strength is the outcome (indicating resistance or backfire). Alternatively,
$ {\beta}_1 $
could be negative but smaller in absolute magnitude than
$ {\beta}_2 $
, consistent with a “cautious Bayesian” view.
To facilitate the comparison of effect estimates, we rely on figures reporting effects of focal and distal counterarguments on focal and distal beliefs, respectively, along with differences in effect estimates between the two arms.
Turning to Figure 3, which presents our key effect estimates in Wave 1 for both studies, we find that the focal counterargument reduces focal belief strength by 0.32 scale points across both studies and distal belief strength by 0.13 and 0.17 scale points in Studies 1 and 2, respectively (in all cases, SE = 0.03;
$ p< $
0.001). The effect of the focal counterargument on the focal belief is approximately 0.20 scale points larger than the distal counterargument’s effect on the focal belief across both studies (
$ p< $
0.001), indicating that persuasive effects are stronger for the belief the counterargument is designed to target. Similarly, the distal counterargument reduced distal belief strength by 0.31 scale points in Study 1 and 0.28 in Study 2 (SE = 0.03;
$ p< $
0.001) and focal belief strength by 0.12 in both studies (Study 1 SE = 0.02; Study 2 SE = 0.03; all
$ p< $
0.001). In Study 1 (Study 2), the distal counterargument’s effect on distal belief strength was approximately 0.18 (0.11) scale points larger than the focal counterargument’s effect on distal belief strength (Study 1
$ p< $
0.001; Study 2
$ p< $
0.01). As is evident from the figure, the effect of each counterargument on its respective measure of belief strength was nearly identical, such that the focal counterargument was just as persuasive as the distal counterargument. Pooling estimates across the two studies, we estimate a difference in counterargument effectiveness between the two arms of −0.02 scale points (SE = 0.03; 95% CI = [−0.08, 0.04]). While the point estimate is close to zero, the uncertainty around estimates suggests we cannot rule out small effects in either direction.
Wave 1 Covariate-Adjusted Effect Estimates of Focal and Distal Counterargument Conditions on Focal Belief Strength, Distal Belief Strength, and Belief-Specific Counterargument Effects, with Corresponding 95% Confidence Intervals
Note: Final pooled estimate is a random effects meta-analytic (REML) estimate.
Importantly, these effects persist into the second wave of each study, approximately 10 days after the initial exposure (see Figure 4). Focal belief counterarguments continue to reduce focal belief strength by 0.24 scale points (SE = 0.04;
$ p< $
0.001) in Study 1 and 0.26 scale points (SE = 0.04;
$ p< $
0.001) in Study 2, whereas effects on distal belief strength decrease from −0.13 scale points to −0.10 scale points in Study 1 (SE = 0.04;
$ p< $
0.01) and drop from −0.17 scale points to −0.13 scale points in Study 2 (SE = 0.04;
$ p< $
0.01). Similarly, distal belief counterarguments maintain their impact on distal belief strength, though the effect is somewhat smaller in magnitude in Study 1, dropping from −0.31 scale points to −0.15 scale points. In Study 2, distal belief arguments experience a slight decrease from −0.28 scale points to −0.23 scale points. Effects on focal belief strength drop from −0.12 scale points to −0.07 scale points in Study 1 and from −0.12 scale points to −0.08 scale points in Study 2. The focal counterargument’s effect on focal belief strength remains approximately 0.17–0.18 scale points larger than the distal counterargument’s effect on focal belief strength going into Wave 2 (
$ p< $
0.001), whereas the distal counterargument’s effect on distal belief strength weakens slightly relative to the focal counterargument’s effect, with a difference in effect estimates of 0.05 scale points (
$ p= $
n.s.) in Study 1 and 0.10 scale points (
$ p< $
0.05) in Study 2.
Wave 2 Covariate-Adjusted Effect Estimates of Focal and Distal Counterargument Conditions on Focal Belief Strength, Distal Belief Strength, and Belief-Specific Counterargument Effects, with Corresponding 95% Confidence Intervals
Note: Final pooled estimate is a random effects meta-analytic (REML) estimate.
Turning to our comparison of counterargument effectiveness on the beliefs each counterargument targets, we observe slightly weaker effects in Wave 2 for the distal counterargument relative to the focal counterargument, with a pooled difference of −0.06 scale points (SE = 0.04; CI = [−0.14, 0.02]). The durability of effects across both studies suggests that counterarguments produce lasting changes in belief strength, rather than reflecting momentary shifts in beliefs due to within-wave demand effects. Moreover, the comparability of effect estimates suggests that focal and distal counterarguments are similarly persuasive.
Taken together, our evidence is inconsistent with H1b, which posits that belief change ought to be more pronounced for distal versus focal beliefs. Moreover, our finding that focal counterargument effects exhibit persistence whereas distal counterargument effects tend to decay at a slightly higher rate provides additional evidence against motivational accounts that predict greater resistance to updating focal beliefs (though this pattern is only observed in Study 1).
Inducing Attitude Change
We now assess whether counterarguments targeting focal beliefs exhibit larger effects on attitudes relative to those targeting distal beliefs. We review the findings of each test here before turning to a comprehensive meta-analytic test of the difference between focal and distal counterarguments.
We present the wave 1 findings regarding attitude strength in Figure 5. For attitude strength, in the first wave of Study 1 the focal counterargument reduces strength by 0.12 scale points (SE = 0.02;
$ p< $
0.001), while the distal argument has approximately half the effect, decreasing strength by 0.06 (SE = 0.02;
$ p< $
0.01). The difference in effect estimates between focal and distal is statistically significant (
$ p< $
0.05). In the first wave of Study 2, we largely replicate these findings: the focal argument reduces strength by 0.13 (SE = 0.03;
$ p< $
0.001) and the distal argument produces an effect of 0.09 (SE = 0.03;
$ p< $
0.001). The difference between conditions is not statistically significant. For attitude extremity, Study 1 shows a 0.05 scale point reduction (SE = 0.07; p = n.s.) for the focal argument and a 0.01 increase for the distal argument (SE = 0.07; p = n.s.); in Study 2, we find modest evidence that focal arguments reduce extremity by 0.12 (SE = 0.07;
$ p< $
0.10), while distal arguments have no discernible effect with estimates hovering around zero (SE = 0.07; p = n.s.). The difference between focal and distal is marginally significant (
$ p< $
0.10). For attitude certainty, we detect a −3.7 scale point reduction on a 101-point scale (SE = 0.68;
$ p< $
0.001) among those exposed to the focal argument, versus −2.16 (SE = 0.65;
$ p< $
0.001) among those exposed to the distal argument. The difference in effect estimates across the two conditions is statistically significant (
$ p< $
0.05). Finally, for attitude defense, we observe a −0.13 scale point reduction (SE = 0.05;
$ p< $
0.01) among those exposed to the focal argument, which is −0.04 scale points larger than the effect for the distal argument (
$ {\beta}_2 $
= −0.09; SE = 0.05;
$ p< $
0.10). This difference is not statistically significant (Figure 5).
Wave 1 Covariate-Adjusted Effect Estimates of Focal and Distal Counterargument Conditions on Attitudes, with Corresponding 95% Confidence Intervals
Overall, effect sizes tend to be small, reflecting the choice to target deeply held attitudes. Most importantly, however, we find consistent evidence that focal belief counterarguments produce larger immediate attitude change than distal belief counterarguments across multiple measures and studies.
Turning to the second wave (see Figure 6), effects generally weaken. For attitude strength, Study 1’s immediate effects dissipate (focal:
$ \beta =-0.01, $
SE = 0.03, p = n.s.; distal:
$ \beta =0.03 $
, SE = 0.03, p = n.s.), and in Study 2, the focal effect is likewise weak (
$ {\beta}_1=-0.06 $
, SE = 0.04, p = n.s.). Comparing across effect estimates, we estimate a larger effect for the focal condition in Wave 2 in Studies 1 and 2, but this difference is statistically significant only in the latter. For attitude extremity, Study 1 shows modest, nonsignificant reductions (focal:
$ \beta =-0.09 $
, SE = 0.07, p = n.s.; distal:
$ \beta =-0.10 $
, SE = 0.07, p = n.s.), while Study 2 shows attenuation with a marginal focal reduction (
$ {\beta}_1=-0.14 $
, SE = 0.08,
$ p<0.10 $
). The difference between the two effect estimates is marginally significant (
$ p< $
0.10). For attitude certainty (Study 2), focal arguments continue to reduce certainty by 2.57 points on the 101-point scale (SE = 0.71,
$ p<0.001 $
), whereas the distal effect is no longer significant (
$ {\beta}_2=-1.00 $
, SE = 0.72, p = n.s.). The focal and distal difference is statistically significant at the 0.05 level. For attitude defense (Study 2), both effects are small and nonsignificant (focal:
$ {\beta}_1=-0.10 $
, SE = 0.07, p = n.s.; distal:
$ {\beta}_2=-0.04 $
, SE = 0.07, p = n.s.). The difference in effect estimates is not statistically significant. Taken together, we observe mixed persistence, but generally larger Wave 2 estimates for focal counterarguments relative to distal ones (Figure 6).
Wave 2 Covariate-Adjusted Effect Estimates of Focal and Distal Counterargument Conditions on Attitudes, with Corresponding 95% Confidence Intervals
Assessing Differences in Attitude Change (Exploratory Analysis)
We now conduct an exploratory analysis to formally assess differences between focal and distal counterarguments on attitudes. While focal arguments generally produce larger effect estimates, examining each attitudinal measure separately may obscure the overall pattern of effects and reduce our ability to detect meaningful differences between argument types. A meta-analytic approach allows us to systematically compare the relative effectiveness of focal versus distal arguments while accounting for both within- and between-measure heterogeneity in effect sizes.
We employ a random effects meta-analysis for several reasons. First, our attitudinal measures—though related—capture distinct theoretical constructs ranging from strength to extremity to defense. We should therefore expect some variation in the true effect sizes across measures. Second, these measures operate on different scales (e.g., 4-point versus 101-point), which could produce mechanical differences in effect magnitudes. Third, we collected these measures across two independent studies and waves, introducing another potential source of effect heterogeneity. Figure 7 displays the results. For visual clarity, we reverse the signs so that a positive sign indicates a larger focal counterargument effect.
Results from a Random Effects Meta-Analysis with Correlated Sampling Errors Showing Mean Differences between the Focal and Distal Argument Conditions across Two Studies
Note: Points show the estimated effects and horizontal lines represent 95% confidence intervals. The diamond shows the overall random effects meta-analytic estimate. Appendix C.3 of the Supplementary Material presents full model results.
Pooling across all attitudinal measures, we find that focal arguments produce significantly larger effects than distal belief arguments. In 11 out of 12 comparisons, we observe larger effect sizes for the focal argument relative to the distal argument. The average difference in treatment effects is 0.054 control-group standard deviation units (SE = 0.02;
$ p= $
0.014), indicating a larger effect for focal belief counterarguments. We fail to reject the null hypothesis of effect homogeneity (Q(11) = 7.71, p = 0.74), suggesting that the effect of focal arguments is comparable across different attitudinal measures, waves, and studies. Looking at individual comparisons, the largest differences appear in attitude certainty, where focal arguments outperform distal arguments by roughly 0.10 standard deviation units in both Wave 1 (d = 0.104, SE = 0.050) and Wave 2 (d = 0.105, SE = 0.052) of Study 2. The consistency of positive differences between the focal and distal arguments provides support for H2b (Focal Belief Dominance) while suggesting that larger attitudinal effects due to targeting focal beliefs generalizes across different measures.
Benchmarking Effects (Exploratory Analysis)
We contextualize our effect sizes by comparing them to a study of fact-checking during the 2020 election (Coppock et al. Reference Coppock, Gross, Porter, Thorson and Wood2023). Though our intervention involves presenting counterarguments produced by language models, it is nonetheless useful to assess how personalized counter-attitudinal information compares against other informational interventions such as fact checks. Using identical four-point belief strength scales, Coppock and colleagues detect an average effect of −0.29 scale points when averaging over 21 political claims, which is just slightly smaller than the counterargument exposure effects we observe for focal and distal counterarguments (−0.31 scale points on average).Footnote 14
These similarities in effect sizes are striking, especially given the strength of focal and distal beliefs. Participants’ focal beliefs averaged 3.65 on the four-point scale, while distal beliefs averaged 3.34, surpassing the most widely believed claim in Coppock et al. (
$ \overline{x} =2.69 $
). Achieving similar effects while targeting more personally relevant beliefs highlights the capacity for counter-attitudinal information to shift even relevant political beliefs.
While Coppock et al. (Reference Coppock, Gross, Porter, Thorson and Wood2023) provide an ideal benchmark for belief change they lack comparable measures of attitudinal outcomes. Therefore, we draw on Coppock, Hill, and Vavreck (Reference Coppock, Hill and Vavreck2020), who analyzed 59 real-time randomized experiments manipulating exposure to 49 professional campaign videos. Their meta-analysis revealed a short-term average treatment effect of 0.056 scale points on a five-point scale. In contrast, we detect short-term effects between 0.12 and 0.13 scale points for the focal counterargument on a five-point scale, approximately twice the magnitude. While designs and outcomes differ across studies, these exploratory comparisons suggest that personalized counterarguments produce effects comparable to fact-checks and, in the short run, may be larger than standard persuasive advertisements.
Mechanisms of Attitude Change: Relevance and Recall
Belief Relevance
In addition to belief strength, we consider auxiliary outcomes that help clarify why focal arguments generally have larger and more persistent effects than distal arguments. First, we focus on belief relevance, which is measured on a four-point scale capturing the relevance of focal and distal beliefs to attitudes. We regress belief relevance for the two relevance outcomes on treatment indicators and pretreatment covariates with HC2 robust standard errors.
Turning to Table 3, we find evidence that participants decrease belief relevance when exposed to the corresponding argument. In Wave 1 of Study 1, focal arguments decrease focal belief relevance by −0.167 scale points (SE = 0.025;
$ p< $
0.001) and distal arguments decrease distal belief relevance by −0.205 (SE = 0.032;
$ p< $
0.001), with evidence of spillover from the focal counterargument onto distal relevance (
$ {\beta}_1 $
=−0.120; SE = 0.030;
$ p< $
0.001). We replicate these findings in Study 2, with reductions of 0.09 (0.13) scale points in focal (distal) relevance among those assigned to the focal (distal) argument (though the difference between these two effects is not statistically significant).
Effect of Interventions on Belief Relevance
Note: Appendix C.4 of the Supplementary Material presents full model results. Statistical significance: +
$ p< $
0.1, *
$ p< $
0.05, **
$ p< $
0.01, and ***
$ p< $
0.001.
On their face, these findings are consistent with a “bolstering” process (Jervis Reference Jervis2006, 653), where once salient considerations are supplanted by alternative considerations, leading individuals to discount their previous reliance on these beliefs when reporting their attitudes. However, it is worth noting that the self-reports of relevance and belief strength are intertwined, with moderate correlations between the two.Footnote 15 It might be the case that beliefs that are held with less certainty are also perceived as less relevant to an attitude.
To directly assess the bolstering mechanism, we included a text entry option in Study 2 when asking about belief relevance. In addition to presenting them with focal and distal beliefs, we told participants “if there are more important reasons why you hold this view that are not listed, please tell us using the Other text box.” If participants are simply substituting a core belief for another, we ought to see additional considerations being offered in the counterargument conditions compared to the control condition. This is coded as a binary variable capturing whether an additional consideration is disclosed in the text entry (
$ \overline{y} = 0.25 $
). This would provide direct evidence of the theorized process whereby individuals compensate for weakened beliefs by elevating the importance of alternative considerations. However, turning to our exploratory analysis in Table 4, we fail to find evidence that this is what participants are doing, with small estimates hovering around zero for both experimental conditions (
$ p> $
0.05). We also estimate an additional model treating the relevance scores for the additional consideration as the outcome (we bottom-code this variable as a 1 for those who do not disclose an additional consideration). This suggests that when participants’ beliefs are weakened by contradictory information, they simultaneously downgrade both their strength in these beliefs and their perceived relevance, without necessarily propping up alternative beliefs to fill this explanatory void.
Effect of Interventions on Stated Relevance of Alternative Beliefs (Study 2)
Note: Appendix C.4 of the Supplementary Material presents full model results. Statistical significance: +
$ p< $
0.1, *
$ p< $
0.05, **
$ p< $
0.01, and ***
$ p< $
0.001.
Argument Recall
Previous research on motivated reasoning has stressed the importance of biased recall, such that people more easily retrieve pro-attitudinal considerations than counter-attitudinal considerations (Taber and Lodge Reference Taber and Lodge2006). While our design did not directly ask participants to provide pro-attitudinal arguments, we can test whether participants exhibit selective memory when recalling arguments that bear on their focal beliefs. We assess this in the second waves of Studies 1 and 2 by asking participants to indicate whether they had seen two arguments drawn from their assigned condition and two from the other conditions each in the last month (see Table 5).Footnote 16 Our evidence suggests slightly higher rates of recall for the focal versus distal belief condition in both studies, such that participants are three to four percentage points more likely to recall an assigned argument in the focal condition (
$ {\beta}_1 $
= 0.09; SE = 0.02;
$ p< $
0.001 in Study 1;
$ {\beta}_1 $
= 0.09; SE = 0.03;
$ p< $
0.001 in Study 2) relative to the distal condition (
$ {\beta}_2 $
= 0.06; SE = 0.02;
$ p< $
0.05 in Study 1;
$ {\beta}_2 $
= 0.05; SE = 0.02;
$ p< $
0.05 in Study 2). This pattern of enhanced recall for focal belief counterarguments may help explain their greater persistence in producing attitude change relative to distal counterarguments. The heightened memorability of focal arguments suggests deeper engagement with information that challenges core justifications for attitudes, rather than a reflexive defensiveness against attitude-incongruent information (Table 5).
Effect of Interventions on Argument Recall
Note: Appendix C.4 of the Supplementary Material presents full model results. Statistical significance: +
$ p< $
0.1, *
$ p< $
0.05, **
$ p< $
0.01, and ***
$ p< $
0.001.
Continuous versus Categorical Relevance
Throughout the article, we carry out two theoretically motivated contrasts. First, we compare the attitudinal effects of a counterargument targeting a focal belief to a counterargument addressing an attitudinally irrelevant placebo related to Daylight Saving Time. This contrast captures the persuasive advantages of addressing a respondent-generated justification versus a low-relevance belief pertaining to an entirely different topic. In terms of the weighted aggregation model described earlier, this entails comparing a high
$ {w}_i $
belief to a low
$ {w}_i $
. Our second contrast compares the focal belief to an attitudinally congenial belief (the so-called “distal belief”). Here, we compare a high
$ {w}_i $
belief to a more evenly-matched belief with respect to attitudinal congruence, but one that is potentially lower in
$ {w}_i $
. Our comparisons of belief strength and self-reported relevance bear this out, with the “distal belief” scoring at the high end of these measures, but not as highly as the “focal belief.”
One possibility is that belief relevance is entirely determined by accessibility—because focal beliefs were elicited from respondents and typically the first justification that came to mind, they must also be highly accessible. We carried out a third study to try to disentangle relevance and accessibility (see Appendix A.6 of the Supplementary Material). Participants engaged in a semi-structured interview with an LLM, as in Studies 1 and 2, but the LLM was instructed to extract not only the focal consideration but also additional considerations that came to mind. Specifically, participants were asked to provide additional reasons for their issue stances and to disclose reasons for their issue stance held by others. The LLM then extracted a secondary belief, which we call the “elicited distal belief.”
This procedure recovered strong beliefs, with focal beliefs and elicited distal beliefs scoring 3.65 and 3.57 on the four-point belief strength scale, compared to 3.25 for the non-elicited distal belief. However, we found only modest differences in belief relevance, with a mean of 3.81 for the focal belief and 3.69 for the elicited distal belief (compared to 3.40 for the non-elicited distal belief). This small difference in belief relevance makes it difficult to disentangle relevance from accessibility.
After the belief elicitation stage, respondents were randomly assigned to a counterargument targeting the focal belief, the elicited distal belief, or the non-elicited distal belief. Notably, we omit a placebo control condition and make comparisons between these three counterarguments. Consistent with Studies 1 and 2, we find similar effects on belief strength across the two elicited belief arms: the focal counterargument produces −0.162 scale point decreases in focal belief strength relative to the non-elicited distal counterargument and the elicited distal counterargument reduces elicited distal belief strength by −0.151 scale points (two one-sided tests of treatment equivalence suggest the null hypothesis of equivalence cannot be rejected; p = 0.193). However, differences among the set of elicited considerations with respect to attitude change are small in magnitude, producing a meta-analytic estimate of
$ d=-0.01 $
(SE = 0.036). Given that belief relevance differed only modestly between the two arms, the weak experimental contrast makes this outcome unsurprising. Methodologically, it suggests that it is difficult to elicit beliefs from participants that they do not view as highly relevant to their own positions, indicating that an alternative design may be needed.
DISCUSSION
Our findings shed new light on the relationship between beliefs and attitudes. By using LLMs to recover and confront beliefs that participants identified as central to their policy views, we provide experimental evidence that attitude–relevant beliefs are capable of change. Across two studies, participants updated their beliefs in the direction of the belief-incongruent information, running counter to expectations from approaches that strongly emphasize resistance or bias in the face of exposure to belief-incongruent information. The durability of these shifts—observed more than 10 days later—provides further support for a view of persuasion vis-à-vis beliefs that does not align neatly with approaches predicting extreme resistance to persuasion.
Focal counterarguments exerted larger and more durable effects on downstream attitudes than arguments targeting relatively peripheral beliefs. We observed stronger immediate reductions in attitudinal measures for participants who received a counterargument targeting beliefs they had explicitly described as key justifications for their position. In contrast, counterarguments to more distal beliefs that were unmentioned by the participant but nonetheless relevant to the attitude yielded smaller or inconsistent attitudinal effects. This dovetails with the notion that beliefs differ in their relevance to broader evaluations: changing a belief that participants see as important to their political views appears to produce more discernible evidence of persuasion.
The greater memorability of focal counterarguments may help explain their persistence. Participants assigned to focal counterarguments recognized the arguments at higher rates 10 days after exposure, suggesting deeper engagement with these arguments. Throughout our studies, we found little evidence of widespread defensiveness. Participants in focal counterargument conditions did not uniformly strengthen alternative beliefs or discount the credibility of the argument. While we observed decreases in belief relevance, we did not find evidence that alternative considerations were “bolstered” to compensate for the weakened belief.
These findings help reconcile an apparent puzzle in the literature on political information: why does successful belief change often yield minimal attitude change? Our results suggest this pattern may stem partly from informational interventions that target malleable but relatively peripheral beliefs. Many popular fact-checks address viral claims on social media platforms but may not reflect the most relevant beliefs underlying individuals’ political attitudes. The methodological approach developed here—using LLMs to identify personally salient beliefs—provides a framework for designing more effective arguments by targeting beliefs that individuals themselves identify as central to their political evaluations. Our framework aligns with accumulating empirical findings about the persuasive advantages of chatbots that directly elicit concerns (Costello, Pennycook, and Rand Reference Costello, Pennycook and Rand2024; Reference Costello, Pennycook and Rand2025; Costello et al. Reference Costello, Pennycook, Willer and Rand2025; Offer-Westort, Rosenzweig, and Athey Reference Offer-Westort, Rosenzweig and Athey2024).
Social scientists have long been skeptical that people can meaningfully engage with—or even accurately articulate—the reasons underpinning their own ideological commitments. Though people may attribute their political attitudes to core beliefs or values, subjective understandings of personal behavior are famously misleading and incomplete (Kelley and Michela Reference Kelley and Michela1980; Nisbett and Wilson Reference Nisbett and Wilson1977). Our study offers some evidence that people are surprisingly capable of articulating beliefs with real bearing on their attitudes. In the process, our work also speaks to ongoing debates about the role of listening in interpersonal conversations. Hearing out the other side may improve citizens’ capacity to persuade their peers more by helping one identify relevant counterarguments than by signaling one’s own receptiveness, as theories of social psychology have long theorized (Santoro et al. Reference Santoro, Broockman, Kalla and Porat2025).
A limitation of our tailored design is that focal and distal counterarguments differ on multiple dimensions beyond belief relevance, including the specific text, the degree to which the counterargument echoes the respondent, and the familiarity of the targeted argument. While we cannot fully isolate relevance from these bundled features, several patterns in our data suggest relevance is operative: focal and distal counterarguments produce equivalent belief change, ruling out a simple “argument quality” explanation; adjusting for high-dimensional text features via the Imai and Nakamura (Reference Imai and NakamuraForthcoming) deconfounding method does not systematically attenuate our estimates (Appendix B.3 of the Supplementary Material); and our follow-up study comparing elicited focal and elicited distal beliefs, which share the “elicitation” feature, finds similar effects for both (Appendix A.6 of the Supplementary Material).
Several questions merit further investigation. First, while we focused on strongly held political attitudes to provide a stringent test of our theoretical expectations, future work might examine whether the relative effectiveness of focal and distal counterarguments generalizes to less crystallized attitudes or those that are grounded in identity (e.g., partisanship).Footnote 17 Second, our finding that arguments reduce both belief strength and perceived relevance raises questions about the relationship between these constructs and their distinct roles in attitude formation. Third, additional research could explore whether providing multiple arguments targeting different types of beliefs produces cumulative effects on attitudes, as argued by Porter and Wood (Reference Porter and Wood2024). Finally, our framework accommodates but does not directly address the possibility that evidence-based counterarguments may have muted effects depending on whether issue stances are moralized and grounded in more evaluative beliefs that are potentially less responsive to information (Carmines and Stimson Reference Carmines and Stimson1980).Footnote 18
Future studies could also identify designs that distinguish between continuous models—which reflect a monotonic relationship between relevance and attitude change—and categorical models of belief relevance, in which beliefs that cross some threshold of accessibility—whether the single most focal consideration or a small set of top-of-mind considerations—carry sufficient weight to make counterarguments targeting them similarly effective. Our third study did not find a difference in attitudinal effects between arguments targeting elicited focal and elicited distal beliefs, but distinguishing between highly accessible beliefs per the continuous model may require substantially larger samples or more sensitive measures than those employed here. Furthermore, future studies could decouple elicitation from argumentation by having separate elicitation and intervention waves. While our design mimics the sequence of eliciting and responding to beliefs that is common in political discourse, a decoupled design would help further isolate whether the elicitation procedure itself moderates the persuasive effect of arguments targeting elicited beliefs.
Our findings extend to pivotal questions about political cognition and attitude change, suggesting that citizens can meaningfully update both peripheral and central political beliefs when presented with belief-incongruent information. However, the translation of these changes in beliefs into attitude change appears to depend critically on the centrality of the targeted beliefs to downstream judgments. This highlights the importance of understanding how individuals weight different beliefs in forming political attitudes and points toward strategies for more effective belief-based persuasion.
SUPPLEMENTARY MATERIAL
The supplementary material for this article can be found at https://doi.org/10.1017/S0003055426101622.
DATA AVAILABILITY STATEMENT
Research documentation and data that support the findings of this study are openly available at the American Political Science Review Dataverse: https://doi.org/10.7910/DVN/BZOCDJ.
ACKNOWLEDGEMENTS
We thank Stanley Feldman, Tim Ryan, Jennifer Jerit, Isaac Mehlhaff, Brian Guay, Eric Groenendyk, and Ethan Porter for thoughtful comments on earlier versions of this article. We also benefited from feedback received at presentations at the University of North Carolina-Chapel Hill, New York University, City University of New York, Harvard Kennedy School, and the 2025 American Political Science Association conference.
CONFLICT OF INTEREST
The authors declare no ethical issues or conflicts of interest in this research.
ETHICAL STANDARDS
The authors declare that the human subjects research in this article was reviewed and approved by the Columbia Human Research Protection Office (IRB Protocol No. AAAU8308). The authors affirm that this article adheres to the principles concerning research with human participants laid out in APSA’s Principles and Guidance on Human Subject Research (2020).
Open access
Comments
No Comments have been published for this article.