Theory: responsiveness to public opinion

There needs to be linkage between public opinion and government actions for a state to qualify as democratic (Dahl Reference Dahl2008). Hanna Pitkin (Reference Pitkin1967: 224) similarly suggests that ‘[t]he representative system must … be responsive to public opinion’. While accepting the normative necessity for responsiveness, Pitkin (Reference Pitkin1967), Jane Mansbridge (Reference Mansbridge2003), Andrew Sabl (Reference Sabl2015) and others have debated the appropriate level of responsiveness. Political science typically identifies three main types of responsiveness: (1) position responsiveness, where political elites adjust their ideological stance to align with public opinion (Hakhverdian Reference Hakhverdian2012; Wood and Lee Reference Wood and Lee2009); (2) policy responsiveness, which refers to substantive government actions, such as budgetary allocations or legislative decisions (Hobolt and Klemmensen Reference Hobolt and Klemmensen2008; Soroka and Wlezien Reference Soroka and Wlezien2005); and (3) agenda responsiveness, the focus of this study, which examines whether politicians shift their public communications in response to public concerns (Jones and Baumgartner Reference Jones and Baumgartner2004). This study builds on prior work by testing whether agenda responsiveness applies to letters from the public – a rarely examined but widely used form of political engagement.

There has been a large literature confirming that political leaders and governments are (broadly) responsive to public opinion (as measured by opinion polls) and that institutional factors impact the degree of responsiveness (Beyer and Hänni Reference Beyer and Hänni2018; Dassonneville et al. Reference Dassonneville, Feitosa, Hooghe and Oser2021; Hobolt and Klemmensen Reference Hobolt and Klemmensen2008; Sevenans Reference Sevenans2021; Wlezien Reference Wlezien1995). Similar findings have been made in Australia (Martin et al. Reference Martin, Dowding, Hindmoor and Gibbons2014). Much of these studies built on Warren Miller and Donald Stokes’s (Reference Miller and Stokes1963) ‘diamond’ model of the connection between the public and their leaders (Figure 1).

Source: Based on Miller and Stokes (Reference Miller and Stokes1963).

Figure 1. Connections between Publics’ Attitudes and Leader’s Behaviour

The responsiveness literature has increasingly moved towards contingent effects models that recognize variation in when and how political leaders respond to public opinion (Manza and Cook Reference Manza and Cook2002). Factors such as electoral incentives, issue salience and institutional constraints shape this dynamic. In other circumstances, public opinion can be led by the elites, rather than the other way around, where the policy agenda runs from above, with political elites leading and shaping the agenda and substantive policy – often called elite agenda-setting. This is especially likely to be the case where an issue is particularly new, or complex. Establishing which ‘conditions’ and which ‘issues’ are likely to establish strong or weak connections has since become the focus of research.

This article seeks to contribute to the questions asked by Benjamin Page (Reference Page1994: 25): ‘under what circumstances is this impact larger or smaller? … what about … other means of measuring public opinion?’ and associated questions about the transmission of public opinion, and the leader’s perception of that opinion. My adjustments to the Miller and Stokes (Reference Miller and Stokes1963) model (Figure 1) highlight that different groups will have different attitudes, each of which are likely to have different levels of intensity, which will have different impacts on leaders.

Classic representation theorists suggest that ‘the constituency that a representative reacts to is the constituency that he or she sees’ (Fenno Reference Fenno1977: 883), and that ‘which sources politicians (prefer to) consult, has a bearing on their perceptions of public opinion’ (Walgrave and Soontjens Reference Walgrave and Soontjens2023: 1). The literature finds that the wealthy and higher educated are more likely to participate; more likely to express their opinion; have different opinions and policy priorities; and that governments are more responsive to the demands of this group (Schlozman et al. Reference Schlozman, Verba and Brady2012; Verba and Nie Reference Verba and Nie1987; Verba et al. Reference Verba, Nie and Kim1978, Reference Verba, Schlozman and Brady1995).

This means that understanding what opinion politicians engage with is important. Politicians viewed ‘direct contact with citizens’ as the most useful way to understand the public’s preferences – and significantly more useful than opinion polls (Walgrave and Soontjens Reference Walgrave and Soontjens2023). Similarly, politicians think that writing to elected representatives is one of the most effective ways of participating in politics (after voting, media attention and being active in a political party) (Hooghe and Marien Reference Hooghe and Marien2012). While these findings show that direct contact is important, they do not tell us which groups’ opinions are the most important. Which opinions, and from which publics, politicians are likely to find most useful, will be driven by the institutional structures. Different levels of voter turnout (Bugarin and Portugal Reference Bugarin and Portugal2015; Karp et al. Reference Karp, Banducci and Bowler2008; Singh Reference Singh2019); majoritarian versus proportional representation (Calvo and Hellwig Reference Calvo and Hellwig2011; Cox Reference Cox1990; Ezrow et al. Reference Ezrow, De Vries, Steenbergen and Edwards2011); and alternative vote versus first-past-the-post (Benoit Reference Benoit2006; Reilly Reference Reilly2019) all impact political incentives and therefore drive different forms of responsiveness (Hobolt and Klemmensen Reference Hobolt and Klemmensen2008).

This presents two challenges. The first is identifying which opinion political leaders are listening to, and the second is ensuring that the public opinion used as an independent variable is (to paraphrase Miller and Stokes Reference Miller and Stokes1963: 50) ‘at least tolerably’ similar to the political leaders’ perception of that opinion. Unfortunately, as Karolin Soontjens and Stefaan Walgrave (Reference Soontjens and Walgrave2023: 500, emphasis added) argued, ‘scholarly work on policy responsiveness hinges on the assumption that politicians monitor public opinion’, and I would add the assumption that the public opinion being monitored by politicians is tolerably similar to the opinion used as the independent variable. These challenges are often brushed over when the research relies on opinion polls. As Scott Althaus et al. (Reference Althaus, Swigger, Chernykh, Hendry, Wals and Tiwald2011) argue, we tend to assume that the relevant causal information has been transmitted, and heard, by the relevant individual or group. We need to confirm that the relevant information has been transmitted, to ensure that the theorized causal link is actually present. Apart from experimental studies (Butler and Nickerson Reference Butler and Nickerson2011; Sevenans Reference Sevenans2021; Soontjens and Sevenans Reference Soontjens and Sevenans2022), responsiveness studies generally assume that their independent variable has been accurately transmitted to political leaders, and that political leaders have accurately perceived that opinion.Footnote ³ If we are not prepared to assume away the lack of evidence of transmission of opinion from the public to leaders, we need to turn to other forms of public opinion, where we can confirm that leaders have been exposed to that opinion.

The preceding paragraphs suggest that to better understand responsiveness, we must find measures of public opinion that accurately reflect the opinion as perceived by political leaders, and that reflect the opinion that politicians are incentivized to listen to – which will depend on the different institutional structures. The circular definition of V.O. Key (Reference Key1961: 14) is actually useful – public opinion is ‘those opinions held by private persons which governments find it prudent to heed’. If the purpose of public opinion research is to determine ‘the function and role of public opinion … in the operation of society’ (Blumer Reference Blumer1948: 543), then a key factor is which public(s) and which opinion(s) the policymaker considers.

This also implicitly reflects the ideas of Mansbridge (Reference Mansbridge2003) of ‘anticipatory representation’, where representatives seek to be responsive to (their perception of) future voters’ preferences at the time of the next election – rather than simply being held to promises made at the previous election (which she terms ‘promissory representation’). By moving temporal focus from the previous election to the next election, Mansbridge (Reference Mansbridge2003: 518–519) suggests that this creates ‘crucial elements of continuing communication’ between representatives and the represented, and ‘the better the communication … the better the representation’. Federico Russo (Reference Russo, Cotta and Russo2020: 314), building on Pitkin and Mansbridge, suggests that a key normative criterion for effective anticipatory representation is that ‘there must be institutional arrangements for citizens to express their demands’. It is therefore in the representatives’ interest to identify and use the best possible, and most nuanced, sources of public opinion.

In this vein, I turn to public opinion that has been deliberately and actively expressed by members of the public, in a way that we know that political leaders have heard. This decision to express a political opinion (whether it be through a letter, protest, donation, etc.) is a demonstration of the individual’s intensity of feelings about the issue (Gause Reference Gause2022; Hill Reference Hill2022). Understanding this issue intensity is vital, because the responsiveness driven by anticipatory representation is mediated by leaders’ expectations of the weight/intensity that voters place on an issue, and how that will impact their vote, rather than a simple headcount (Gause Reference Gause2022; Hill Reference Hill2022; Ryan and Ehlinger Reference Ryan and Ehlinger2023). As Paul Burstein (Reference Burstein2003: 30) noted, ‘citizens who care about an issue are especially likely to take elected officials’ actions on that issue into account on election day’.

I argue that letters from the public is an example of a form of public opinion that provide a practical measure of issue intensity (Hill Reference Hill2022). It is substantively different from answering a ‘most important issue’ or salience question in an opinion poll, because by writing, they have demonstrated that they are part of the ‘attentive public’ (Adler Reference Adler1984; Almond Reference Almond1950; Danley-Scott Reference Danley-Scott2006). If this is the case, political leaders are likely to be responsive to this form of public opinion, given that these letters reflect an intensity of opinion that is not evident in opinion polls; and leaders’ bias towards public opinion that is actively expressed and transmitted to them (Schlozman et al. Reference Schlozman, Verba and Brady2012):

One driver of responsiveness is politicians’ rational desire to avoid defeat at the next election. This should mean that the greater the risk of a loss of power, the greater the likelihood of responsiveness (Hakhverdian Reference Hakhverdian2012). Luca Bernardi (Reference Bernardi2020) notes, however, that there is conflicting evidence that electoral vulnerability or election proximity increase responsiveness. Focusing on budgetary policy, he finds limited evidence to support this thesis. Bernardi (Reference Bernardi2020: 183) concludes by suggesting further research, particularly ‘in more rhetorical policy venues’, which aligns with my approach. Therefore, I hypothesize:

Manza and Cook (Reference Manza and Cook2002) argue that different levels of responsiveness are related to the salience of an issue. Letter-writing reflects issue salience, but not all letters carry the same weight. Individuals who take the time to compose personal letters likely have stronger preferences than those who sign pro-forma campaign materials. If responsiveness is driven by intensity rather than volume alone, we should observe greater government attention to issues with a high proportion of personal letters relative to pro-forma campaigns. This hypothesis extends work on issue salience (Gause Reference Gause2022; Hill Reference Hill2022) by testing whether governments prioritize highly engaged citizens over mass mobilization efforts:

As noted above, there are also a range of studies that show that in some circumstances, public policy can lead public opinion, or in E.E. Schattschneider’s terms, ‘new policies create a new politics’ (quoted in Pierson Reference Pierson1993: 595). This is particularly likely to be the case for subjects/topics with lower levels of salience, issues that are new to the political agenda, or where external events have driven the agenda (Manza and Cook Reference Manza and Cook2002). Based on this:

Case selection, method, data and modelling

I focus on Australia, between 1996 and 2000, as a matter of data availability. Australia is a federal, bicameral, parliamentary system. The lower house, the House of Representatives, has 151 single-member electorates, elected by alternative vote. John Howard was prime minister from 1996 until 2007, making him Australia’s second longest-serving prime minister. There are two major parties, the centre-left Australian Labor Party (ALP) and the centre-right Liberal Party of Australia, which has been in (practical) permanent coalition with the agrarian National Party of Australia. Since Mr Howard left office, no prime minister has served more than four years.Footnote ⁴ This makes the period well suited to analysis, as there are no changes of government, or prime minster, to account for. The research period also excludes the significant change in communication methods brought in with email.

The importance of early hypothesis testing and research design justifies a single-country approach (Pepinsky Reference Pepinsky2019). It allows for additional levels of detail and in-depth description and analysis; however, I recognize that such a study may have limited generalizability.

The main independent variable is the volume of letters per topic per fortnight, based on newly accessed documentation from the National Archives of Australia (NAA). Each fortnight, Mr Howard received a brief from his department, giving the volume of letters received in the previous fortnight, as well as details of their topics.Footnote ⁵ Archival research and interviews have confirmed that provision of regular data to the prime minister’s office (PMO) about volume and topic of correspondence was standard practice since at least the mid-20th century (Casey Reference Casey2024a). Similarly, Brandon Rottinghaus’s (Reference Rottinghaus2007) study shows that some US presidents received briefings about their letters.

I obtained 69 of these briefs (see Appendix in the Supplementary Material), from the period March 1996 (when Mr Howard was first elected) to December 2000. The topics stated in the briefs were highly specific, such as ‘Cuts to funding for the Australian Broadcasting Corporation’ (ABC), or ‘Tasmania’s multi-purpose AFL stadium – pro’.Footnote ⁶ These topics were then manually coded against the Australian version of the Comparative Agendas Project (CAP)Footnote ⁷ (Bevan Reference Bevan, Baumgartner, Breunig and Grossman2019; Dowding and Martin Reference Dowding and Martin2017; Martin et al. Reference Martin, Dowding, Hindmoor and Gibbons2014) and intercoder reliability checking was undertaken.Footnote ⁸ Examples of how the letters were initially coded by public servants in PM&C, and then subsequently coded against the CAP are in the Appendix in the Supplementary Material. Consistent with Will Jennings and Peter John (Reference Jennings and John2009), only those topics where the proportion of both letters and speeches were above 2% were included in the analysis, resulting in four topics being removed.

To measure agenda responsiveness, this study examines all of Mr Howard’s public communications, including speeches, interviews, press releases and media appearances – more than 2,000 speeches totalling over 6 million words. This approach differs from previous studies, which typically rely on major speeches such as the Speech from the Throne or the Budget Speech (Hobolt and Klemmensen Reference Hobolt and Klemmensen2005; Jennings and John Reference Jennings and John2009; Wood and Lee Reference Wood and Lee2009). While these high-profile speeches are often compared with annual opinion polls, my analysis requires a higher-frequency dataset to align with fortnightly correspondence data. I coded these speeches into the same CAP topics, using the SeededLDA topic modelling package (Watanabe and Zhou Reference Watanabe and Zhou2020), which has been extensively used across social science (Curini and Vignoli Reference Curini and Vignoli2021; Fernandes Reference Fernandes2023; Levorato and Sguazzini Reference Levorato and Sguazzini2024; Macanovic Reference Macanovic2022). SeededLDA is semi-supervised; by incorporating a researcher-designed seed-word dictionary, it ensures that the topics identified by the model align with prior theoretical topics. My seed-word dictionary (see Appendix in the Supplementary Material) is based on the CAP codebook, following the process set out in Kohei Watanabe and Yuan Zhou (Reference Watanabe and Zhou2020), including fine-tuning using average feature entropy. I also undertook a range of post-hoc manual validation checks (Chan and Sältzer Reference Chan and Sältzer2020; Curini and Vignoli Reference Curini and Vignoli2021; Watanabe and Zhou Reference Watanabe and Zhou2020; Wesslen Reference Wesslen2018).Footnote ⁹ These tests resulted in two more topics being removed, leaving 15 topics within the analysis.

I also conducted semi-structured interviewsFootnote ¹⁰ with John Howard, senior members of his former staff and former public servants within the Department of Prime Minister and Cabinet (PM&C) (see Appendix in the Supplementary Material). My interviews have two core purposes. First, to establish the facts on how ministerial correspondence and the ministerial correspondence briefs were handled within PM&C and the PMO, to trace how public opinion information was transmitted to Mr Howard and the PMO. The second purpose is to understand how Mr Howard and the PMO interpreted and reacted to both the correspondence and the briefs, to understand how they ‘view[ed them] and the meaning that they attribute to [them]’ (Rubin and Rubin Reference Rubin and Rubin2005: 19). Both interviews and archival research ‘can provide excellent leverage for causal inference’ (Page Reference Page1994: 28). This helps directly address Hypotheses 1 and 4.

Modelling approaches

Various modelling approaches were used to test the hypotheses. To test H1, I use the approach of Sara Hobolt and Robert Klemmensen (Reference Hobolt and Klemmensen2008) and Armen Hakhverdian (Reference Hakhverdian2012). The response variable (Howard’s speeches) could be expressed as either a count (number of words per topic per fortnight), or a proportion. I use proportion, as the underlying premise of the CAP is that topics compete for a limited amount of space on the agenda (Dowding et al. Reference Dowding, Hindmoor and Martin2016). This also controls for other factors that impacted word count, including holiday periods (when word count dropped), election periods (when word count increased) and the annual increase in media engagements. The dependent variable was log transformed (to address non-normality of residuals), and the first difference taken, to address stationarity and autocorrelation (Parker Reference Parker2012),Footnote ¹¹ consistent with the time series regression literature (Ahmed et al. Reference Ahmed, Atiya, Gayar and El-Shishiny2010; Cai Reference Cai2002). Therefore, my model for each topic is:

\begin{equation*}\Delta \log \left( {speech\,proportion} \right) = \,\alpha + \,\beta {\left( {total\,mail} \right)_{t - 1}} + \,\varepsilon \end{equation*}

A range of other modelling approaches were tried, and the results were consistent.Footnote ¹²

To test H2, I use the same model, with the addition of an interaction term of the Newspoll results.Footnote ¹³ Using the change in Newspoll results both reflects the need to ensure that the data series is stationary but also suggests that politicians are more concerned with the trend, rather than the absolute figure.

\begin{align*}\Delta \log \left( {speech\,proportion} \right) & = \,\alpha + \,{\beta _1}{\left( {total\,mail} \right)_{t - 1}} + \,{\beta _2}\Delta {\left( {Newspoll} \right)_{\left( {t - 1} \right)}} \\ & \quad + {\beta _3}\left( {{{\left( {total\,mail} \right)}_{t - 1}}*\Delta {{\left( {Newspoll} \right)}_{\left( {t - 1} \right)}}} \right) + \,\varepsilon \end{align*}

For H3, a different modelling approach was required. Given the large number of fortnights where no letters were received on a particular topic, ordinary least squares (OLS) regression is not suitable, so the response variable is transformed into a binary and binary regression used.Footnote ¹⁴ The model includes the volume of letters in the previous fortnight, to control for autocorrelation (Hakhverdian Reference Hakhverdian2012; Stimson et al. Reference Stimson, Mackuen and Erikson1995).

\begin{align*}\ln \left( {\Pr \left( {Letters} \right)} \right) & = \,\alpha + \,{\beta _1}\left( {{{\left( {speech\,proportion} \right)}_{t - 1}} + {{\left( {speech\,proportion} \right)}_{t - 2}}} \right) \\ & \quad + {\beta _2}lag\left( {volume\,of\,letters} \right) + \,\varepsilon \end{align*}

Discussion and conclusion

This article examines the agenda responsiveness of Prime Minister Howard to the volume and topics of letters from the public. It demonstrates how expressions of political opinion can be traced from the public, directly to the prime minister. Using both quantitative and qualitative approaches, I find evidence of deliberate and systemic non-responsiveness. Mr Howard and his staff made an active decision to be non-responsive to this form of public opinion. These findings challenge conventional models of democratic responsiveness by highlighting how political leaders selectively engage with public input. Mr Howard and his staff actively dismissed letters as politically irrelevant. This suggests that not all forms of expressed opinion are perceived as electorally significant, reinforcing the argument that leaders prioritize engagement with sources they see as politically consequential (Walgrave and Soontjens Reference Walgrave and Soontjens2023).

Null findings provide important additions to our understanding and advance the discipline (Munafò and Neill Reference Munafò and Neill2016). They should force us to re-examine the theory, modifying it to address the new findings. This is what Manza and Cook (Reference Manza and Cook2002) suggest – that in some circumstances, we would find responsiveness, and in other circumstances we would not. They set the discipline a challenge of finding those different circumstances. It is in this framework that I situate these findings, helping to identifying those circumstances and situations where responsiveness is not expected to be found.

Hobolt and Klemmensen (Reference Hobolt and Klemmensen2008) show that there are very low levels of responsiveness in executive speeches in some countries, with ideology rather than public opinion being the driving force. Similarly, political elites are more likely to be responsive when there is a stronger electoral incentive (including to co-partisans in proportional electoral systems (Sevenans Reference Sevenans2021), or when they are electorally vulnerable (Hakhverdian Reference Hakhverdian2012; Soontjens and Sevenans Reference Soontjens and Sevenans2022)). This means that lower levels of responsiveness could be expected where there are reduced electoral incentives to be responsive to any particular form of public opinion or subgroup of constituents. To that extent, these findings may be generalizable.

Addressing the concern of James Druckman (Reference Druckman2014) that studies of responsiveness ‘typically fail to account for responsiveness to “what type of opinion”’, I help confirm the ‘contingent effects’ model, by suggesting that political elites are less likely to be responsive to opinion from those group(s) that do not pose an electoral threat. These findings similarly address the concerns of Herbert Blumer (Reference Blumer1948) and Key (Reference Key1961: 14) by identifying which opinions the government ‘finds prudent to heed’, and reinforce the importance of improving our understanding of the institutional impacts on responsiveness – which groups of voters they consider to be ‘pivotal’, and which (as in this case) they can choose to ignore.

Does this mean that writing letters is a waste of time? Not necessarily. Writers have a range of motivations and expectations. People writing to congratulate Mr Howard on an election victory are unlikely to expect an impact on the public agenda. Similarly, those who wrote seeking help with their social security or tax would not expect to impact Mr Howard’s speeches. The outcomes they are seeking would not be detectable by quantitative methods. More research is needed to better understand the impact of these individual letters.

In conclusion, this research makes a range of contributions to the literature. First, it sets out a unique dataset on the volume and topic of letters to a political leader. Like the study of protests (Gause Reference Gause2022), this helps to broaden the study of public opinion to better reflect how the public actually expresses opinion and how political elites engage with it. Second, it applies existing theories on responsiveness to a new empirical domain. The use of letters to political executives in quantitative political science is a novel methodological approach. Finally, by using briefs to the prime minister, it explores the causal linkages between public opinion and public agenda, where existing studies have often struggled.