1. Theory and hypotheses

DRD has two components. The first is ongoing republican consultation: intervening prospectively to “help [the legislator] understand how to better represent [the constituent] going forward.” The second is ongoing deliberative accountability: entering retrospectively “to help [the constituent] to better understand why [the legislator] represented [the constituent] the way [the legislator] did.” Our experiment explicitly taps these motivations using this exact verbiage.

We expect that motivating calls for participation will primarily act via political efficacy. People with higher efficacy participate more in politics; for example, those with higher external efficacy are more likely to vote (Abramson and Aldrich, Reference Abramson and Aldrich1982). Efficacy perceptions are susceptible to change with exogenous evidence. Women see increases in external efficacy when they observe more women in public office (Atkeson and Carillo, Reference Atkeson and Carillo2007), and participation in events designed along DRD principles caused increases in efficacy (Esterling et al., Reference Esterling, Neblo and Lazer2011). Prospective and retrospective frames may be taken as evidence of one’s efficacy, thereby increasing participation. We therefore expect opportunities for both prospective and retrospective communication with officials to motivate participation relative to control.

These frames may have different effects. Efficacy is related to perceived agency (Bandura Reference Bandura1982; Bandura, Reference Bandura2006), which is emphasized more by the prospective than the retrospective frame. Additionally, people may value being shown the respect of ex ante consultation. The retrospective treatment is also more familiar, as it hews closer to the role of elections. In contrast, prospective input may seem more efficient since electoral reward and punishment are delayed and bundle together many policy choices (Urbinati, Reference Urbinati2008; Hofmann, Reference Hofmann2019).

We do not directly measure efficacy perceptions in the study. Instead, we infer efficacy from observed behavioral responses: whether individuals accept the call and how long they remain on the line. These measures capture engagement as a downstream consequence of perceived agency. While not a direct test of an efficacy mechanism, this approach is consistent with prior work in which behavioral participation is interpreted as evidence of efficacy (e.g., Esterling et al., Reference Esterling, Neblo and Lazer2011). We acknowledge the inferential limits of this strategy and return to this point in the conclusion.

In terms of our primary measure—Call Acceptance by a phone number dialed for the events under study—our main (pre-registeredFootnote ¹) hypotheses are:

If these messages work by boosting efficacy, effects may depend on existing variation. For example, women feel less efficacious than men (Bennett and Bennett, Reference Bennett and Bennett1989; Bennett, Reference Bennett1997; Burns et al., Reference Burns, Schlozman and Verba2001; Thomas, Reference Thomas2012), driven partly by underlying differences in self-confidence (Wolak, Reference Wolak2020). As such, the effects of treatment may differ by gender. Under control, men may participate more than women if they perceive themselves as more efficacious. However, if men start with higher efficacy, the effect of a boost from (what we anticipate as) the milder, more familiar retrospective treatment may be higher for women. In contrast, higher efficacy in men could make them more susceptible to the prospective treatment.Footnote ² We therefore hypothesize:

We also preregistered several research questions. While the effects above focus on initial Call Acceptance, we leave it open whether treatments will influence Length of Time on Call. Treatments may be quickly overridden by the call itself (for those who actually accept the call), or they might set expectations in a way that conduces toward, or detracts from, that experience. We also have research questions about how effects depend on age, and our discussion of gender leaves open how treatments will affect net levels of participation.

2. Experimental design, measurement, and analysis

To test our hypotheses and address our research questions, Australian MP Andrew Leigh (coauthor on this paper) conducted field experiments in the federal electorate of Fenner, which encompasses the northernmost third of Canberra and the Jervis Bay Territory.Footnote ³ The MP was motivated by a desire to maximize participation in his events. Compared with the rest of Australia, the electorate is relatively affluent. Of Australia’s 151 House of Representatives electorates, the district ranks 11th on median household income ($107,432 per year) and 35th on median mortgage repayments ($24,444 per year). Twenty-nine percent of residents were born overseas, while 27 percent speak a language other than English at home (both figures are slightly above average).

In May 2021, the MP held a first event, administered by the firm Tele-Town Hall. All phone numbers on the firm’s list (30,126 numbers associated with 34,154 individuals) were randomly assigned to one of three groups (Prospective, Retrospective, and Control) with equal probabilities. The night before the event, numbers were contacted using an automatic system. If a person answered, they received the following message, with bolded passages indicating the experimental frames:

On the night of the event, a second call with a similar message was delivered to each number. Similar messages were used when calls went to voicemail, replacing “This is an automatic call” with “I’m calling.” Complete scripts appear in Supporting Materials. The telephone event itself contained both prospective and retrospective elements: the MP opened the call with a short speech about current events, and then the bulk of the event was devoted to answering constituent questions (participants pressed a key on their telephones to ask a question, and were then put into a queue). Given that the event took place in 2021, much of the discussion focused on the public health response to the COVID-19 pandemic, including vaccinations, movement restrictions, and support payments to workers and businesses.

In July 2021, the MP held a subsequent event, repeating this design. For July, the firm contacted a different list of phone numbers (39,891 numbers associated with 43,455 individuals). The lists overlap; 16,266 numbers (16,990 individuals, about 50% of those contacted in May) were contacted only in July.Footnote ⁴ About 95% of numbers included in both events were assigned to the same treatment. The substance of the July telephone town hall was similar to the May town hall described above. In keeping with preregistration, we analyze experiments separately with the same analysis plan. To address spillovers, we present analyses of July for the entire sample (as preregistered), the subsample of newly enrolled numbers, and the subsample excluding those whose treatment changed.Footnote ⁵

Our main unit of analysis is the individual (i.e., voting-eligible resident, or elector). Our primary outcomes are Call Acceptance and Length of Time on Call. The former equals 1 if a person picked up the second call, delivered on the night of the event, and 0 otherwise. The latter is the length of time the phone remained connected, measured in fifths of a second. At the individual-level, we have data on Age (five-category), Gender, and Suburb, and we merge in several more suburb-level covariates.Footnote ⁶ We checked balance on covariates using equivalence testing (Hartman and Hidalgo, Reference Hartman and Hidalgo2018) for each pair of conditions.Footnote ⁷ We rejected null hypotheses of differences larger than 0.36 SD for all comparisons with p < 0.001, corrected for false discovery.

Finally, there is another complication: we have data drawn from phone numbers, but we are interested in individual-level behavior. Individuals sometimes share phone numbers. Although we have no direct evidence as to why they do so, one obvious explanation is landlines. Although we lack household composition data, 56% of electors in our dataset (28,735/51,144) are uniquely associated with their phone number, which is slightly larger than the 63% of Australians who had no landline in 2022 (Australian Communications and Media Authority, 2022). The difference would include any single-person household with only a landline. For context, our data include 38,284 unique phone numbers, and 25% (9549) are shared. Of those, 7118 are associated with only two electors, and 82% of that subset includes multiple gender values, and so are potentially heterosexual couples. Of the multi-elector phone numbers, 31% are potentially multigenerational households, measured as being associated with two non-contiguous age groups (e.g., at least one 18–24 and at least one 65+).

We therefore weight each (individual-level) observation by the inverse of the number of people associated with the same number. For example, if two individuals share a number, each receives a weight of 0.5; if three share a number, each receive 0.33; etc. The average weight is 0.88 in May and 0.72 in July. We analyze evidence at the phone number-level for robustness (see SM Tables A26–A30).

We use regression for analysis. For Call Acceptance, the main model is

\begin{equation*}Call{\text{ }}Acceptanc{e_i} = \alpha + {\beta _1}\;Prospectiv{e_i} + {\beta _2}\;Retrospectiv{e_i} + {\text{ }}{\gamma _j} + {\text{ }}{\varepsilon _i}\end{equation*}

where individual i is associated with a number assigned to a treatment, and γ_j is a Suburb-level fixed effect. We interpret β ₁ and β ₂ as the average effects of the Prospective and Retrospective conditions relative to Control. To measure moderated effects, we use multiplicative interactions. We cluster standard errors by phone number. For Length of Time on Call, we preregistered four analyses. First, we use an analogous model to above. However, this outcome will be 0 for individuals associated with numbers that did not accept the call. Therefore, we also estimated a hurdle model that yokes together a logistic regression for Call Acceptance and a negative binomial model of Length of Time on Call; a tobit; and a linear model of the logged outcome.Footnote ⁸ For presentation, we focus on hurdle models. Results are similar for other specifications. All analyses appear in the SM.

4. Conclusion

We reported evidence that providing specific rationales does not motivate consultative participation. Rather, explicitly stipulating a retrospective rationale was demotivating relative to control. Yet specific rationales also increased the amount of time individuals spent participating. Finally, while we found no evidence of differential effects by gender, the prospective frame motivated the youngest cohort to participate. Although that result was replicated across both events, it was un-hypothesized and awaits future confirmation.

Although our approach infers efficacy from observed behavioral responses, these findings are consistent with the explanation that specifying motivation winnows participants, leaving only the more intrinsically motivated. Moreover, two further conjectures may explain these results. First, individuals may divide into groups differing on which rationale they find more motivating, and people in the control condition could project their preferred rationale onto the event. Second, individuals might find both rationales disingenuous, which could cause a backfire effect relative to simply extending an invitation. These conjectures have important practical and theoretical implications, warranting further research to adjudicate between them. If the projection explanation is correct, then people are interested in—not overly cynical about—opportunities for directly representative democratic innovations, as suggested in a U.S. context by Neblo et al. (Reference Neblo, Esterling, Kennedy, Lazer and Sokhey2010), but it remains to identify who finds which rationale more motivating (e.g., variation in efficacy, personality, identity, etc.). On the backlash explanation, however, future research should disentangle what motivated people who accepted the call (e.g., social norms, expressing grievances, attachment to the MP) and whether participation in the call changed beliefs about the value of such events and the MP’s sincerity. More generally, scholars should link inquiries in new forms of political participation to normative democratic theory to facilitate new technology serving human flourishing rather than further alienating citizens from government.

Beyond the specific academic and practical value of this study, our work illustrates the unique benefits that collaboration between scholars and practitioners can produce. Most political science claims to seek general knowledge, yet restricts itself to the status quo. Elected officials, too, seek to understand both how things are and how they might be—and though their understanding may not be scientifically grounded, it gets tested regularly at the ballot box. Scholars and practitioners, therefore, share an interest in understanding how nearby worlds differ from ours, as a precondition for realizing the efficiency and benefits of intentionally guided change (for example, around increasing political participation).