Skip to main content Accessibility help
×
Home
Hostname: page-component-5c569c448b-s84wp Total loading time: 0.389 Render date: 2022-07-06T00:30:30.227Z Has data issue: true Feature Flags: { "shouldUseShareProductTool": true, "shouldUseHypothesis": true, "isUnsiloEnabled": true, "useRatesEcommerce": false, "useNewApi": true } hasContentIssue true

When Do Politicians Pursue More Policy Information?

Published online by Cambridge University Press:  20 January 2021

Peter John Loewen*
Affiliation:
Department of Political Science, University of Toronto, 1 Devonshire Place, Toronto, ON, M5S 3K7Canada, Twitter: @PeejLoewen, @mcandrewsjr
Daniel Rubenson
Affiliation:
Department of Politics & Public Administration, Ryerson University, Jorgenson Hall 729, 350 Victoria Street, Toronto, ON, M5B 2K3Canada, Twitter: @dktr_dr
John R. McAndrews
Affiliation:
Department of Political Science, University of Toronto, 1 Devonshire Place, Toronto, ON, M5S 3K7Canada, Twitter: @PeejLoewen, @mcandrewsjr
*
*Corresponding author. Email: peter.loewen@utoronto.ca
Rights & Permissions[Opens in a new window]

Abstract

We conducted a field experiment with 334 Canadian Members of Parliament exploring whether politicians seek out more information about an issue when they are farther offside the average opinion in their constituency on that issue. In the midst of a contentious national debate on the oil industry, we invited MPs and their staff to watch a webinar or read a written summary of the webinar created by experts and containing a variety of viewpoints on the issue. For politicians on either side, the information could prove useful in future debate and conversation. Some politicians were randomly assigned to information about the distribution of opinion in their constituency on the issue. We find no evidence that politicians are more likely to pursue policy information when they are offside their average constituency opinion, and none that this effect is enhanced when they learn about their relative position vis-a-vis constituency preferences.

Type
Research Article
Copyright
© The Author(s) 2021. Published by Cambridge University Press on behalf of The Experimental Research Section of the American Political Science Association

When do politicians seek out expert information on policy?Footnote 1 In theory, politicians should have at least two broad motivations for seeking out information. On the one hand, they may view themselves as policy-makers (Docherty Reference Docherty1997). The policy-making politician may make policy via a formal position – for example, as a cabinet minister. They may alternatively make policy as an entrepreneur, by persuading other elites or voters about the merits of some policy change (Kindgon 1997 [Reference Kingdon2003]). In either scenario, they should benefit from objective information about policy options. On the other hand, politicians may view themselves as persuaders, tasked with selling policy set by others (Grose et al. Reference Grose, Malhotra and Van Houweling2015; Malloy Reference Malloy1996; McGraw Reference McGraw1991). In this instance, knowing experts’ views on issues will make it easier for them to sell their own party’s position. Overlaying all of this is the reality that on some issues politicians will have an easy sell – voters will largely be in agreement with them and promoting their party’s position will be straightforward. On other issues, they will be far offside constituency opinion and will face a more difficult task in selling their own or their party’s position. We might expect politicians to be even more likely to seek out helpful information when this is the case.

In this paper, we explore whether politicians seek out more information about an issue when they are farther offside the average opinion in their constituency on that issue. To do so, we conducted a field experiment with 334 Canadian Members of Parliament (MPs). In the midst of a contentious national debate on federal government support for the oil industry, we invited MPs and their staff to watch a webinar or read a written summary of the webinar. The webinar was created by a school of public policy and contained a variety of viewpoints on the future prospects of oil extraction in Canada. For politicians on either side of this issue, the information could prove useful in future debate and conversation. Within our invitations, some politicians were randomly assigned to information about the distribution of opinion in their constituency on the issue of whether the government should be involved in actively helping the resource sector, including in the construction of pipelines.

Our analysis follows closely the logic of Butler and Nickerson (Reference Butler and Nickerson2011), in that we estimate the effect of constituency opinion on an MP taking action, conditional on the MP receiving information about that constituency opinion, though we focus more purposively on the degree to which opinion disagrees with a politician’s party position. We find that neither are politicians in the control group likely to be responsive to constituency disagreement in seeking information nor are those who receive a prime about constituency opinion more likely to seek out information conditional on disagreement. In short, politicians who are offside their constituency opinion do not appear more likely to seek out expert information on contentious policy issues.

In what follows, we provide some theoretical intuitions, followed by background information for our experiment. We describe our experimental design and present our results. We then note the limitations of our study and next steps.

Theoretical intuitions

Our experiment is guided by four simple theoretical intuitions. First, elected politicians have multiple roles to play – among them as policy-makers and as representatives of their parties (Docherty Reference Docherty2005; Fenno Reference Fenno1973; Franks Reference Franks1987). The degree to which they will play both roles varies both individually and systematically. A politician who sits on a policy-focused committee will have more opportunities to act as a policy-maker than one who does not. This is especially so if the politician is in the governing party. At the political system level, the extent to which politicians can influence actual policy development will depend on the openness of the policy-making system, the extent of party discipline, and the strength of the executive (e.g., Lijphart Reference Lijphart2012; Russell et al. Reference Russell, Gover and Wollter2016). As representatives of their parties, some individuals will be better persuaders than others and some will devote more resources to policy persuasion than others. This could vary based on their own skills or their marginality. At a systemic level, the degree to which politicians have to act as apologists for their party’s positions will depend on the accountability calculus in their own country, and on the degree to which their party is seen as responsible for policy-making. What matters most for our experiment is not the variation, however, but that each politician is likely to view their own role as at least somewhat about policy-making and somewhat about being a representative of their party’s positions.

Second, politicians at least sometimes want expert information on the policy issues before them. Expert information serves several functions, not least to help policy-makers understand the nature of a policy problem, the link between policy instrument and outcome, and the range of alternative policies that could have been chosen (Quirk and Bendix Reference Quirk, Bendix, Schickler and Lee2011). Moreover, understanding expert opinions on issues allows politicians to draw on arguments when they are in agreement with their position and develop responses against these arguments when they are counter to their position.

Third, politicians will sometimes know the policy preferences of their constituents, but these estimates are likely to be both biased and to have some error and/or uncertainty associated with them (Broockman and Skovron Reference Broockman and Skovron2018).

Fourth, politicians are responsive to constituency opinion (e.g., Soroka et al. Reference Soroka, Penner and Blidook2009), such that ceteris paribus, they are likely closer to their constituency’s mean opinion and they would like to know when they are likely to be perilously distant from constituency opinion.

Taken together, these four intuitions support the two principal hypotheses of our experiment. First, MPs whose political positions are farther away from their constituency mean will be more likely to seek out information on “offside” issues. Second, this will be more likely among MPs who are given accurate information about the distribution of opinion in their constituency. Below, we describe an experiment that tests both of these propositions.

Background on the case

Our experiment is centered around an ongoing issue of political contention in Canada: Federal government support for resource development, especially the extraction of oil and its export via pipelines. Canada is an oil-rich country, with reserves estimated in the top five globally. However, much of this oil is “landlocked,” in two senses. First, it exists not in free-flowing wells, but in “tar sands,” which require expensive and environmentally costly extraction technologies. Both of these increase the global barrel cost at which extracting oil is profitable. Second, the oil is mostly contained in the province of Alberta, which is a substantial distance from tidewater. To export this oil to refineries or end users requires either expensive and dangerous rail transportation or pipelines. The development of the oil sands and the pipelines necessary for their economic viability has been a major issue of political debate in Canada in the last decade.

Generally speaking, the leadership of each political party lines up on one side of the issue of energy extraction and pipeline construction. The Conservative Party and the Liberal Party of Canada both support the oil sands and the development of pipelines. The New Democratic Party (NDP), Bloc Québécois, and the Green Party all oppose this development.Footnote 2

There is heterogeneity within each party, however, at the level of their provincial counterparts and individual MPs. The Liberal Party is generally a broad party, with so-called business Liberals supporting the resources industry, while left or social Liberals support greater environmental restrictions. The NDP likewise houses heterogeneous opinion, as its labor wings are supportive of the jobs created through this industry, while its environmental and social justice wings oppose oil extraction. As importantly, there is substantial regional variation across the country, which is reflected in the distribution of opinions within each party (see Figure 1, below). The consequence of this is that it would be difficult to ascribe with certainty to any constituency the distribution of opinion or mean view based solely on its geographic location and the party of its MP. MPs face some uncertainty about their constituents’ opinions.

Figure 1The distribution of constituency opinion on federal assistance for energy development, by party and overall.

Experimental design

In May 2017, the University of Toronto School of Public Policy and Governance hosted a webinar about energy policy in Canada. It featured presentations from three academic experts, who also took audience questions. In May 2018, we posted online – via a Qualtrics survey – a video recording and written summary of this webinar.Footnote 3 We then invited MPs and their staff to access these materials through the survey. Bilingual invitations were sent by email to the offices of 334 of the 338 MPs in the Canadian House of Commons.Footnote 4 The invitations reminded MPs about the ongoing uncertainty surrounding the proposed expansion of the Trans Mountain Pipeline and offered a clickable link to the webinar materials.Footnote 5

An experimental treatment was embedded in the text of the email invitation.Footnote 6 We randomly assigned MPs to either a treatment group or a control group. MPs in the treatment group received information on their constituents’ policy preferences with respect to federal assistance for energy development, including oil pipelines. We obtained and computed these constituency-level preferences from a publicly available online survey of Canadians conducted by the Local Parliament Project (LPP) during the 2015 general election (Loewen et al. Reference Loewen, Rubenson and Koop2018). The 2015 LPP survey was the largest ever election study of Canadians and its sample size allowed us to aggregate individual-level responses into unusually precise constituency-level estimates. MPs in the control group did not receive any constituency preference data as part of their invitations. The texts of the invitations – both treatment and control – are reported in the supplementary material. An important aspect of the experiment is the timing of the study. Emails were sent to MPs on May 28, 2018, a few days before the well-publicized “deadline” for a decision on the Trans Mountain Pipeline expansion issued by Kinder Morgan, Canada, after which the company said it would walk away from the project. In other words, the stakes were high and the issue was very much on the political agenda.

At the level of the individual survey respondent, our measure of support for federal energy development assistance was a five-point scale of disagreement/agreement with the statement: “The federal government should do more to help Canada’s energy sector, including building oil pipelines.” This scale was coded from −2 to 2 (with 0 being “neither agree nor disagree”) and a mean was computed for each constituency.Footnote 7 The mean number of survey respondents per constituency was 102 (SD = 37). As we describe below, we later use this constituency-level measure of support for energy development to construct a measure of the degree to which each constituency disagreed with the party position of their MP on the matter.

Figure 1 shows the distribution of constituency support for federal energy development assistance. We note two points here. First, there is an association between constituency-level opinion and the party of the MP. As one might expect, constituencies represented by Conservative MPs are, on average, more supportive of energy development than constituencies with MPs from other parties (p < 0.05). Second, with the exception of the one-MP Green Party, there is a range of constituency opinions within all of the parties. This implies there are likely to be cross-pressured MPs in each party and that MPs from all parties face some degree of uncertainty about opinion in their constituency.

Drawing on the theoretical intuitions outlined in the previous section, we set out two hypotheses:

H1: MPs/their staff are more likely to access the webinar materials if the average preference of their constituents differs from the policy of their party.

H2: The relationship described in H1 will be stronger among those MPs/staff that received the treatment information – that is, where the average constituency preference was provided in the email invitation.Footnote 8

Put differently, we expect that cross-pressured or “offside” MPs are more likely to access the webinar material (H1) and that this relationship will be accentuated among the randomly assigned subset of MPs who were provided with clear estimates of their constituency’s aggregate opinions (H2). We preregistered our hypotheses and analysis plan with the Evidence in Governance and Politics (EGAP) design registry.

The outcome of interest is whether – upon receiving an invitation – an MP or a member of their staff accessed the webinar materials. MPs/staff were identified as having done so using unique survey links generated from Qualtrics.Footnote 9 A binary indicator of whether the MP/staff accessed the material serves as our dependent variable. A total of 16 MPs’ offices accessed the webinar material.Footnote 10

To test our hypotheses, we use two explanatory variables. The first is a continuous measure of the degree to which an MP’s party and their constituency disagreed about federal assistance for energy development. To obtain this measure, we started with the constituency mean scores. Since higher scores on this scale indicate greater support for energy development, we use this as our measure of disagreement for MPs from parties that opposed the expansion of the Trans Mountain Pipeline: namely, for the NDP, the Bloc Québécois/Groupe parlementaire québécois, and the Green Party. For parties that supported the Trans Mountain expansion – that is, the Liberals and the Conservatives – we flipped the scale by multiplying by −1 so as to arrive at a measure where higher scores indicate greater disagreement between constituency and party.Footnote 11 The second explanatory variable is a simple binary indicator of whether the MP received a control invitation or a treatment invitation.

Results

Table 1 reports two sets of OLS regression estimates.Footnote 12 Model 1 regresses the binary indicator of whether the MP/staff accessed the webinar material on our measure of constituency party disagreement, as well as on the indicator of whether the MP/staff received the treatment of constituency opinion embedded in their invitation. The Disagreement coefficient is not significant in a one-tailed test at the p < 0.05 level. In other words, we find no evidence to support our expectation in H1 that cross-pressured MPs are more likely to access the webinar materials. Indeed, if anything, this relationship runs in the opposite direction: The greater the disagreement between an MP and their party, the less likely they were to seek out the expert information provided in our webinar materials. Although we did not have expectations about the direct effect of the treatment, Model 1 shows that it is also statistically nonsignificant.

Table 1 OLS estimates of the impact of constituency party disagreement on the probability of accessing webinar material

NOTE: Standard errors in parentheses.

*p < 0.05, **p < 0.01, ***p < 0.001.

Model 2 shows the estimates of the impact of constituency party disagreement conditional on the treatment. Thus, here the Disagreement coefficient captures the impact of constituency party disagreement in the control condition. The Treatment × Disagreement coefficient estimates the difference between the impact of disagreement in the treatment condition and its impact in the control condition. While the estimated coefficient for this interaction term is positive – meaning that the impact of disagreement is less negative in the treatment condition than in the control condition – it is not significant at conventional levels. In short, we find no evidence to support H2.

In the supplementary material, we investigate three alternative operationalizations of constituency party disagreement.Footnote 13 We also produced two additional sets of model estimates: OLS models that include covariates for MPs’ party and province, and rare events models that account for the infrequency of ones in our binary-dependent variable (King and Zeng Reference King and Zeng2001).Footnote 14 These additional model estimates do not substantively change our conclusions.

Discussion and conclusions

We designed an experiment to test whether politicians seek out more information about an issue when they are more offside of constituency opinion. Our study returns null results. As we show in the supplementary material, we believe this null effect is real. MPs do not appear more likely to want or look for expert information when their party position is more distant from constituency opinion. This is so even when politicians are shown normally inaccessible local opinion data. We think the finding has important implications for understanding the conditions under which policy-makers will choose to become better informed.

Our results have three notable limitations and we anticipate further studies to address these limitations. First, questions could be raised about the attractiveness of the webinar and, perhaps its neutrality. Seeking out other instruments of information in a future experiment could be useful. Second, we may have conducted our experiment on an issue for which simply too much information already exists, making a low uptake more likely. An equally pressing but more novel or technical issue may be better for a replication. Finally, there is the possibility that effective divisions of labor and expertise mean that an experiment such as this will always lead to null findings, unless it is able to identify and test effects on the very small number of politicians tasked by their parties with being experts on any individual issue. That possibility is a serious challenge to our research design but could be addressed in a future experiment.

Each of these limitations underscore the challenge of elite experiments. The ever-present demands on politicians’ time mean that their participation in any one event or academic study is likely to be modest – a constraint also evident in other elite survey work and in more qualitative, interview-based research involving elite respondents. Scholars are well advised to take such limitations into account when designing future experiments that involve politicians as participants.

Despite these limitations, we believe our findings do present important new insights showing the lack of propensity among representatives to seek out nonpartisan expert information on a salient issue at a time when the issue was coming to a head. Moreover, this hesitance to take on information is present even when politicians are reminded their constituents may on average be far from their party on the issue – a situation that intuitions about representative democracy and electoral incentives imply ought to lead to the opposite effect. This has important implications for our understanding of the functioning of parliamentary representation.

Supplementary Material

To view supplementary material for this article, please visit https://doi.org/10.1017/XPS.2020.42.

Footnotes

1 The authors acknowledge support for this research from the Social Sciences and Humanities Research Council of Canada. There are no conflicts of interest to report. The data, code, and any additional materials required to replicate all analyses in this article are available at the Journal of Experimental Political Science Dataverse within the Harvard Dataverse Network, at: doi:10.7910/DVN/MXQ8O2. See Loewen et al. (Reference Loewen, Rubenson and McAndrews2020).

2 The positions of the parliamentary parties are made clear in a vote on federal support for the Trans Mountain Pipeline expansion held on June 6, 2017, approximately 1 year before we conducted our field experiment. Of the 301 MPs who voted, only 2 – both Liberals from the BC Lower Mainland – rebelled against their party’s position. We report the results of this parliamentary vote – disaggregated by party – in Section 1 of the supplementary material.

3 The webinar was conducted in English and the recording was provided with English audio only. The summary was written in English and translated into French.

4 We excluded the three MPs from the northern territories because the Local Parliament Project survey did not sample residents of the territories. An additional seat was vacant due to resignation.

5 Each Canadian MP’s office has multiple email addresses: typically, there is a general email inbox that is publicly advertised, a set of email addresses used by the MP’s staffers, and an email address reserved for use by the MP themselves. In order to maximize the chances that our email was received and read by someone in each office, we typically sent the email to the most common staffer email address and the other to the MP’s direct email address. Based on our previous email contact with MPs’ offices, however, we identified 15 offices for which we had reason to believe that one or both of these email addresses were either not used or not monitored. For these MPs, we substituted the general email address for the MP. The result was that all but two MP offices received our email through two different accounts.

6 The study was conducted under University of Toronto Ethics Protocol 32778.

7 Constituency means were computed using survey weights. MPs in the treatment group were provided with this mean score, as well as the percentage of constituency respondents in each of the five response categories.

8 In our experimental design, we cannot be certain whether – for a given office – it was the MP or the staffer who accessed the materials, hence our use of the term “MPs/their staff.” Based on the names and email addresses provided by those who accessed the webinar materials, participants reflected a combination of staff and actual MPs. We do not believe that the presence of some staffers in the study poses an issue. Each MP employs their own staff, who are in a good position to act on the MP’s behalf. Moreover, staffers are often delegated the task of conducting research for their MPs – meaning that the presence of staff in the study reflects real-world practice.

9 Upon landing on the Qualtrics survey, MPs/staff were also asked to provide their name and email address – providing a further validation of who accessed the webinar materials.

10 In fact, 17 different people accessed the webinar but 2 were from the same MP office. Since the MP office is the unit of analysis, this is collapsed in a single observation.

11 For the descriptive statistics of this measure of constituency party disagreement, see Section 3 of the supplementary material.

12 Although 334 MPs’ offices were initially contacted, we subsequently exclude 2 independent MPs – who, by definition, are not cross-pressured. We also drop the record of an MP who passed away shortly before the invitations were sent.

13 See Section 3 of the supplementary material for details on the construction and descriptive statistics of these alternative operationalizations.

14 The alternative model estimates are provided in Section 4 of the supplementary material.

References

Broockman, David E., and Skovron, Christopher. 2018. Bias in Perceptions of Public Opinion among Political Elites. American Political Science Review 112(3): 542–63.CrossRefGoogle Scholar
Butler, Daniel M., and Nickerson, David W.. 2011. Can Learning Constituency Opinion Affect How Legislators Vote? Results from a Field Experiment. Quarterly Journal of Political Science 6(1): 5583.CrossRefGoogle Scholar
Docherty, David C. 1997. Mr. Smith Goes to Ottawa: Life in the House of Commons. Vancouver: UBC Press.Google Scholar
Docherty, David C. 2005. Legislatures. Vancouver: UBC Press.Google Scholar
Fenno, Richard F. 1973. Congressmen in Committees. Boston: Little, Brown.Google Scholar
Franks, C. E. S. 1987. The Parliament of Canada. Toronto: University of Toronto Press.CrossRefGoogle Scholar
Grose, Christian R., Malhotra, Neil, and Van Houweling, Robert. 2015. Explaining Explanations: How Legislators Explain Their Policy Positions and How Citizens React. American Journal of Political Science 59(3): 724–43.CrossRefGoogle Scholar
King, Gary, and Zeng, Langche. 2001. Logistic Regression in Rare Events Data. Political Analysis 9(2): 137–63.CrossRefGoogle Scholar
Kingdon, John. 1997 (2003). Agendas, Alternatives, and Public Policies. (2nd ed.), New York: Pearson.Google Scholar
Lijphart, Arend. 2012. Patterns of Democracy: Government Forms and Performance in Thirty-Six Countries. New Haven: Yale University Press.Google Scholar
Loewen, Peter John, Rubenson, Daniel, and Koop, Royce. 2018. Local Parliament Project 2015 Canadian Election Survey. Harvard Dataverse. doi: 10.7910/DVN/DACHKP Google Scholar
Loewen, Peter John, Rubenson, Daniel, and McAndrews, John. 2020. Replication Data for: When do politicians pursue more policy information? doi: 10.7910/DVN/MXQ8O2, Harvard Dataverse, V1, UNF:6:sExx3iZ8nR193qYkVKR1OQ== [fileUNF]Google Scholar
Malloy, Jonathan. 1996. Reconciling Expectations and Reality in House of Commons Committees: The Case of the 1989 GST Inquiry. Canadian Public Administration 39(3): 314–35.CrossRefGoogle Scholar
McGraw, Kathleen M. 1991. Managing Blame: An Experimental Test of the Effects of Political Accounts. The American Political Science Review 85(4): 1133–57.CrossRefGoogle Scholar
Quirk, Paul J. and Bendix, William. 2011. Deliberation in Congress. In The Oxford Handbook of the American Congress, eds. Schickler, E. and Lee, F. E. Oxford: Oxford University Press, 550–74.Google Scholar
Russell, Meg, Gover, Daniel, and Wollter, Kristina. 2016. Does the Executive Dominate the Westminster Legislative Process?: Six Reasons for Doubt. Parliamentary Affairs 69(2): 286308.CrossRefGoogle Scholar
Soroka, Stuart, Penner, Erin, and Blidook, Kelly. 2009. Constituency Influence in Parliament. Canadian Journal of Political Science 42(3): 563–91.CrossRefGoogle Scholar
Figure 0

Figure 1 The distribution of constituency opinion on federal assistance for energy development, by party and overall.

Figure 1

Table 1 OLS estimates of the impact of constituency party disagreement on the probability of accessing webinar material

Supplementary material: PDF

Loewen et al. supplementary material

Loewen et al. supplementary material

Download Loewen et al. supplementary material(PDF)
PDF 1 MB
You have Access
1
Cited by

Save article to Kindle

To save this article to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

When Do Politicians Pursue More Policy Information?
Available formats
×

Save article to Dropbox

To save this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Dropbox account. Find out more about saving content to Dropbox.

When Do Politicians Pursue More Policy Information?
Available formats
×

Save article to Google Drive

To save this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you used this feature, you will be asked to authorise Cambridge Core to connect with your Google Drive account. Find out more about saving content to Google Drive.

When Do Politicians Pursue More Policy Information?
Available formats
×
×

Reply to: Submit a response

Please enter your response.

Your details

Please enter a valid email address.

Conflicting interests

Do you have any conflicting interests? *