Introduction
Today, Parliament is increasingly considered irrelevant. The decline of private member influence is a well-documented phenomenon, especially in Westminster-style parliamentary systems where party discipline is usually high. No longer able to effectively legislate, ordinary members now have to rely on parliamentary speeches to communicate with their constituents or display some level of independence (Blidook, Reference Blidook2012). Unlike recorded votes, which are usually closely monitored by party leaders, legislative speeches are often perceived as a forum where members can publicly voice their dissent from the party (Proksch and Slapin, Reference Proksch and Slapin2015). This relative “unrestrainedness” has provided researchers with several advantages to estimate party positions and individual legislative preferences from political text (Lauderdale and Herzog, Reference Lauderdale and Herzog2016; Slapin and Proksch, Reference Slapin and Proksch2008), primarily because word choices during parliamentary speeches show more variation than votes and because “parties can tolerate loose talk more than they can deviation on voting” (MacGuigan, Reference MacGuigan1978: 680).
But how reasonable is this assumption? Legislative speeches are often managed by party leaders who get to decide who can speak, in what order and on what topic (Scheer, Reference Scheer2018). Are debates, just like legislative votes, constrained by parliamentary procedures and agenda control? Should we expect the rules of Parliament to have an impact on what is said in the House? Carrubba et al. (Reference Carrubba, Gabel and Hug2008) have already demonstrated that procedural rules can influence the decision to record roll-call votes, which in turn can affect the individual ideological scaling of ideal points (see Braüninger et al., 2016). The same patterns could therefore be observed during parliamentary debates, because agenda-control measures—such as closure or time limit on speeches—influence the length and content of legislative speeches.
To date, most of the empirical work that uses parliamentary text as data has focused on electoral incentives to estimate the effects of institutional constraints on legislative speech patterns (Bäck and Debus, Reference Bäck and Debus2016; Proksch and Slapin, Reference Proksch and Slapin2015; Spirling, Reference Spirling2016). These studies have found that the type of electoral system determines who speaks more during the debates. A recent analysis by Høyland and Søyland (Reference Høyland and Søyland2019) has shown that electoral reform can also influence the content of parliamentary debates. However, we have yet to find a study that explicitly looks at how parliamentary rules affect both the length and the content of legislative speeches. In other words, we have yet to investigate the effect of procedures on the government's ability to control the debate. This question is important because several studies have already confirmed that procedural talks often dominate legislative speeches, yet this type of content is frequently ignored by researchers who analyze the topics of parliamentary debates (Boussalis et al., Reference Boussalis, McElroy and Sorace2018; Lauderdale and Herzog, Reference Lauderdale and Herzog2016; Quinn et al., Reference Quinn, Monroe, Colaresi, Crespin and Radev2010).
We argue that such an omission is problematic by showing that procedural rules affect the structure and content of legislative speeches. To do so, we use a novel dataset of parliamentary debates spanning more than a hundred years to determine who controls the agenda more and what issues are being discussed in Parliament. These individual speeches are taken from the Canadian House of Commons Hansard records between 1901 and 2015. As this study will demonstrate, the recent introduction of restrictive parliamentary rules of procedure in Canada has greatly contributed to enhancing the powers of party leaders during the debates, both in terms of volume and content. What is left is a House now dominated by partisanship to an even greater extent than before (Godbout, Reference Godbout2020; Marland, Reference Marland2016).
In this article, we explore how Parliament came to be that way. We are especially interested in understanding what factors determine the number and the length of individual legislative speeches and how this may have been affected by parliamentary procedures. The three main goals of this study are thus to determine (1) who speaks more during the debates, (2) what topics are being discussed and (3) how both of these dynamics are related to each other and to procedural rule changes. To do so, we present the results of two empirical analyses. The first looks at the determinants of agenda control over time, while the second analyzes the results of a structural topic model to study the content of parliamentary speeches. Our results confirm that the introduction of new rules modified the amount of procedural talk in the Canadian House of Commons and that this type of debate is more likely to be controlled by party leaders. We also show that more frequent discussions of parliamentary procedures are associated with an increase in partisan polarization, therefore suggesting that the results of text-scaling methods are likely to be influenced by changes to legislative rules.
1. How Parliamentary Rules Affect Voting and Speeches
Until now, most empirical studies that measure the impact of legislative rules and procedures on parliamentary behaviour have done so using roll-call data. This work has primarily focused on developing theories of legislative organization to explain the voting patterns of representatives in the US Congress. At the core of these studies lies the idea that the majority wants to control the agenda in order to promote party unity within its ranks. What Cox and McCubbins (Reference Cox and McCubbins2005) label “negative agenda control” is the ability for party leaders to use procedural rules to prevent controversial bills from reaching the floor for debate. During certain periods in the history of the US Congress, majority leaders were granted special agenda-setting powers in order to facilitate the adoption of the party's legislative program (Aldrich and Rohde, Reference Aldrich and Rohde2000). These procedural tools, such as who controls the discharge process, the rules committee or speaking time on the floor, have been linked to the recent increase in congressional partisan polarization (Lee, Reference Lee2016). By scaling roll-call votes, researchers have been able to confirm, for example, that parliamentary rules can be used to constrain the behaviour of members, and that more often than not, they have the potential to increase party unity during legislative votes.
We also know that procedural rules influence legislative behaviour in other institutional settings. For example, Huber (Reference Huber1996) has shown that restrictive parliamentary procedures such as the package vote and the “guillotine” in France have the ability to promote party unity, while Dewan and Spirling (Reference Dewan and Spirling2011) have confirmed—using legislative roll-call data—that “majoritarian control over agenda setting” dominates voting in Westminster systems. These two studies offer strong evidence that procedural rules constrain the voting behaviour of legislators outside of the US context. But what about their speeches? Should we expect the same kind of pattern to hold during parliamentary debates? After all, language is more difficult to control than recorded votes.
To date, we find relatively few studies that use text data to look at the impact of procedures on the content of legislative debates. Work done by Eggers and Spirling (Reference Eggers and Spirling2017) in the British Parliament confirmed that the nineteenth-century franchise expansion promoted the creation of shadow cabinets, whose members were given more opportunities to speak in the House of Commons. In related work, Spirling (Reference Spirling2016) also found that these same reforms altered the speaking patterns of government members by making their speeches significantly easier to understand for the newly enfranchised voting population.
Other studies have confirmed the important impact that electoral institutions have on the dynamics of debate. For instance, Proksch and Slapin (Reference Proksch and Slapin2015) have shown that parliamentary rules offer more opportunities for rebels to take the floor when the electoral system creates incentives for a strong personal vote. Similarly, Høyland and Søyland (Reference Høyland and Søyland2019) have confirmed that switching to proportional representation changes the content of parliamentary speeches in Norway by making them less personal and more in line with party positions. In two related studies, Giannetti and Pedrazzani (Reference Giannetti and Pedrazzani2016) and Pedrazzani (Reference Pedrazzani2017) found that the introduction of new procedures in the Italian Camera dei Deputati shifted the balance of power by giving more speaking time to supporters of the government (see also Proksch and Slapin [Reference Proksch and Slapin2015] for a similar argument in Germany and the UK).
Taken together, these studies suggest that procedural rules should have an impact not only on who has access to the floor but also on what members talk about. That is, parliamentary rules should affect who controls the agenda, as measured by how often members speak and on what topics. In the rest of this article, we set out to test these propositions in the Canadian context.
2. The Debates of the Canadian House of Commons
The Canadian House of Commons has undergone several important rule changes to determine who could access the floor during the debates. Historically, Parliament was often the scene of “unnatural” long speeches since the original Standing Orders placed no limit on speaking time (Dawson, Reference Dawson1962: 133). Individual members took advantage of this freedom by talking a lot, sometimes for nine consecutive hours in a single speech (Dawson, Reference Dawson1962: 134).
Almost immediately following Confederation, a conflict emerged over how to regulate this time in the House. The growing volume of government business put pressure on the cabinet to control the agenda, but the first proposals to reform the Standing Orders were strongly opposed by private members, who saw these initiatives as encroachments on their parliamentary rights. Indeed, members often used speeches as dilatory tactics to obtain concessions from the executive, to block the passage of bills or even, in some cases, to force an election. This type of obstruction was less of a problem when the country was small, but as the size of the Canadian government grew, so did the demands for the curtailment of debates.
In 1906, the Standing Orders were modified permanently to give more time to government business on the legislative agenda. Several other time-management tools have been introduced since then, mainly to limit the influence of private members. For example, following the British precedent, closure was adopted in 1913, while the first time limit was put on individual speeches in 1927, and again in 1969, when the guillotine was added to the Standing Orders. Other major rule changes reduced the speaking opportunities of members: a time limit was placed on debates related to the address in reply to the speech from the Throne in 1962 and on budget debates in 1968. The organization of the committee system was also modified in 1968 by redirecting the consideration of financial bills to standing committees, away from the floor of the House where most bills were usually debated at length. Although the objectives of these earlier reforms were to limit the opportunities of private members to speak during the debates, most have done very little to restrict the length of legislative speeches in Parliament (Dawson Reference Dawson1962: 143; Hockin Reference Hockin1966: 331).
Parliament was more successful at tackling these problems during the 1980s, when the House adopted a series of rules to improve proceedings following the recommendations of the Special Committee on the Reform of the Standing Orders and Procedure (the Lefebvre Committee). Starting in December 1982, Parliament began to function under a shorter calendar, with longer periods of adjournment, and adopted a new daily schedule more in line with regular business hours. The result of these changes was a reduction in the number of sitting days per session, which was now mostly dominated by the government's agenda. The abolition of night sittings also reduced the number of adjournment debates, which could sometimes last for several hours (Dawson, Reference Dawson1962: 175).
With the 1982 rule change, the practice of requesting unanimous consent to discuss a matter of “urgent and pressing necessity” (Standing Order 43) was also scrapped (Franks, Reference Franks1987: 120). Members had previously used this opportunity to introduce motions without notice to debate issues that were not on the agenda (March, Reference March1974: 55). Although consent was rarely given, the debates surrounding these motions tended to be very long and often abused by members (Collenette, Reference Collenette1983: 4). These motions were replaced by Standing Order 31, giving members 15 minutes each day to make 90-second speeches on any topic. One final major component of the 1982 reform package was to limit the length of speeches. The House had already adopted a 40-minute time limit for speeches in 1927, but this rule proved ineffective as House speeches were still “used by the Whips to do nothing more than kill time” (Collenette, Reference Collenette1983: 3). Under the new Standing Orders, 40-minute speeches were reduced to 20 minutes, with a 10-minute rebuttal period at the end.
The final set of rule changes that had a major impact on speaking time during this decade relates to private members’ business. Following the tabling of the McGrath Report in 1986, the procedures were modified to include a new practice of considering private members’ bills and motions for an hour each day. This has become in recent years one of the few remaining opportunities for backbenchers to introduce issues for discussion in the debates.
To sum up, the majority of time-management tools adopted in Parliament after 1900 reduced the influence of private members in the legislative process. The most important of these procedural changes occurred between 1906–1913, 1955–1969 and 1982–1986, when the House was highly polarized along partisan lines (Stewart, Reference Stewart and B1977: 203). Godbout and Høyland (Reference Godbout and Høyland2017) and Godbout (Reference Godbout2020) have shown that the first set of rules dramatically increased party voting unity in the legislature, when government business came to dominate the agenda. But what about the content of legislative speeches? Given that the second set of rule changes (1955–1969) barely influenced the capacity of members to speak in the legislature (Dawson, Reference Dawson1962; Hockin, Reference Hockin1966), we should find that changes introduced in 1982–1986 will have a much more important impact on parliamentary debates, mainly because it cut the length of individual speeches in half and added time for private member business.
3. Theoretical Considerations
The main objective of this article is to demonstrate that procedural speeches have important implications for the study of legislative behaviour and that they should not be ignored by researchers. From a comparative perspective, Canada is a useful case to understand the role of procedures on parliamentary speech patterns because the House of Commons was once dominated by members of Parliament (MPs) who could intervene at any time during the debates; but these opportunities are now controlled almost exclusively by party leaders (Cochrane et al., Reference Cochrane, Godbout, Beukel, Bäck, Debus and Fernandes2021). If we follow Proksch and Slapin's (Reference Proksch and Slapin2015: 84) classification of parliamentary rules to measure how often members can speak across different types of legislatures, we find that Canada is unique because it falls both within the Anglo-American model of legislative organization (Australia, Ireland, US, UK), with strong incentives for personal votes and easy access to the floor, but also within the party lists model (Austria, France, Finland, Germany, Hungary, Italy, Netherlands, Sweden), because its Parliament later adopted stricter control to individual floor access.
As we saw earlier, the modernization of the rules of the Canadian House of Commons closely follows the practices first introduced in the British Parliament. As in Canada, the modifications of these rules were primarily aimed at reducing the influence of ordinary MPs in the legislative process by adopting “procedures to end debates,” “time limits on speeches,” and by enhancing the role of speakers by giving them “the power to adjourn legislative debates and discipline members” (Goet, Reference Goet2021: 788). And we expect that as in the US Congress (Cox and McCubbins, Reference Cox and McCubbins2005; Aldrich and Rohde, Reference Aldrich and Rohde2000), the enforcement of these new rules will be primarily delegated to party leaders, especially majority leaders, so that they can more easily control the agenda. From this “theory of procedural choice,” we can infer a certain number of testable hypotheses that could be validated in a multitude of legislative settings, wherever rules regulate access to the floor.
To begin, we hypothesize that frontbenchers will have more opportunities to intervene during the debates as the procedures of the House of Commons become complex. We also expect backbenchers to speak more during the debates but to intervene less over time as the rules become more restrictive. Next, we hypothesize that the topics of legislative debates will be influenced by parliamentary rules and that frontbenchers will use more procedural language in their speeches. The last hypothesis follows from the previous ones and suggests that an increase in procedural content will be linked to an increase in party polarization. This expectation is based on the historical analysis of the development of parliamentary rules done by Godbout (Reference Godbout2020) in the Canadian Parliament and by Goet (Reference Goet2021) in the British case. Both studies confirm that restrictive procedures are more likely to be adopted when obstruction and polarization are high in Parliament. Of course, we cannot assume that the effects of rule changes on speeches are completely exogenous. As these two studies confirm, modifications to the rules were usually introduced deliberately, often with the clear intention of restricting access to the floor. It is our contention that this reciprocal relationship will affect any attempt to measure ideology from parliamentary speech, precisely because procedural language should be more prevalent during eras of heightened partisanship.
In order to investigate these claims, our analysis begins in the next section by determining what factors influence the length and the number of speeches made by MPs between 1901 and 2015 (hypotheses 1–2). The second part presents a structural topic model to estimate the proportion of parliamentary debates spent on rules and procedures in order to determine what effects these discussions can have on the agenda of the House (hypothesis 3). Our expectation is that debates on procedures should be more prevalent when partisan conflict is high, and thus be linked to greater polarization in the legislature (hypothesis 4). We confirm this relationship by demonstrating that Wordfish—a commonly used text-scaling algorithm to measure ideology—is affected by the procedural content of the debates.
4. Empirical Analysis
The corpus of Canadian House of Commons debates used in this analysis was taken from the Lipad website, which was constructed from the digitized records of the Canadian Hansards after 1900 (Beelen et al., Reference Beelen, Thijm, Cochrane, Halvemaan, Hirst, Kimmins, Lijbrink, Marx, Naderi and Rheault2017). The data include individual entries for every speech given in the House of Commons, beginning in the 9th Parliament. Each speech represents an observation (or document), which is dated and linked with information on the speaker's name, riding, province, party and position in Parliament (for example, prime minister, leader of the opposition, postmaster general). The corpus contains over 500 million words spread across 3.5 million individual speeches made by 3,425 Members between 1901 and 2015 (see Table A1 in Appendix A for complete statistics).
4.1 Speech length and interventions
We begin our analysis by looking at the influence of members’ positions on the length of legislative speeches and the number of interventions in Parliament. This analysis is conducted by Parliament in order to gauge the impact of procedural rule changes.Footnote 1Figure 1 reports the results of a series of regression models where the dependent variables are the total number of words in each individual parliamentary speech (left-hand side) and the sum of interventions by an individual MP in a given term (right-hand side) between the 11th–41st Parliaments.Footnote 2
The effect of MPs’ position on the total number of spoken words (left) and the total number of interventions (right) in each Parliament between 1909 and 2015 (reference category [vertical line] = government frontbenchers). The bars indicate the 95 per cent confidence intervals
We are specifically interested in the difference between groups of MPs rather than in the length of speeches in absolute terms, so that in all models, the primary independent variable of interest is MP position in the House.Footnote 3 Members can either sit on the backbench or frontbench of the opposition or be a government backbencher (the reference category is government frontbenchers). The results from these parliamentary term analyses are also confirmed in a cumulative model presented in Appendix B.
From these two plots, we can see the changing patterns of debate dynamics over time. First, we note that government and opposition backbenchers tend to make longer speeches relative to government frontbenchers (first plot). Second, we see that government and opposition frontbenchers intervene more during the debates (second plot). Third, in terms of debate dynamics, we find that opposition party members (both the frontbench and backbench) have increased the length of their speeches over time. Finally, we find a trend toward a harmonization of the distribution of speech patterns beginning in the 1980s, with all groups of members now having more opportunities to intervene during the debates.
Incidentally, this last result corresponds to a period where major parliamentary rule changes were introduced in the House. It is important to point out, however, that even though government backbenchers increased their opportunities to intervene during the debates over time, they always remained the group of MPs with the fewest opportunities to speak. This finding is important because it confirms that government backbenchers still have, to this day, the least amount of influence through legislative speech. Again, these analyses are informative in that they indicate trends in who speaks more, how often and for how long in the Canadian Parliament over time. However, they cannot tell us much about the impact of specific rule changes on agenda control. The next analysis presents the results of a topic model to provide more insight into this question.
4.2 The topics of debate
In this section, we present the results of a structural topic model (STM) to analyze the content of Canadian parliamentary debates (Roberts et al., Reference Roberts, Stewart and Tingley2019). STM is a widely used unsupervised text analysis method based on the latent Dirichlet allocation (LDA) model (Blei et al., Reference Blei, Ng and Jordan2003). Like LDA, STM assumes that the distribution of words in a corpus of text is associated to different topics represented by a set of fixed vocabularies. The model assigns a probability that each individual word belongs to a topic and that each topic belongs to a document. One advantage of using STM is that it allows metadata to affect the distribution of topics in a given document. For instance, by adding “year” as a covariate for topical prevalence in the model, we “allow the [year of speech] to affect the frequency with which a topic is discussed” (Roberts et al., Reference Roberts, Stewart and Tingley2019: 8). This means that “how much each topic contributes to a document” varies according to year, which seems logical given over-time changes in the substance of parliamentary affairs (Roberts et al., Reference Roberts, Stewart and Tingley2019: 8).
Although Hansards already provide subject headings in the transcripts of the debates, these headings tend to be overly general (for example, “question period” and “adjournment debate”). Sometimes, more precise subheadings are provided (for example, “Second reading of Bill C-22” and “Canada Pension Plan”), but this indexation suffers from the opposite problem: it tends to lack generalizability. We see topic modelling as a middle-ground approach to allow for the identification of multiple subjects inside a single speech while at the same time offering a set of broader issue categories that can be found across Parliaments. After processing the data by removing stopwords, punctuation and frequently used or uncommon words, we were left with a corpus of 2,403,611 documents (or speeches) with 1,202 unique terms. We first ran the model with 15 to 45 topics (by increments of 5) and found that the 35-topic model reached a better compromise between semantic coherence and word exclusivity (see Appendix C).Footnote 4 We thus constrained the model to 35 topics and let the topic-proportion vary by the year of the speech.
STM is a data-driven approach where the researcher has to interpret the model results. We want to infer what dimensions are represented by looking at the words associated with the topics. Figure 2 reports the five most frequently used and exclusive words attached to each topic (FREX in the package), but longer lists with 50 words by topic can be found in Appendix D. Reading through the lists of most frequent and exclusive words, we find that some of the topics identified make a lot of intuitive sense, such as justice (topic 33, associated with court, justice, law), while others are a little more difficult to interpret, such as kindness (topic 5, associated with gentleman, friend, man). Seven of the 35 topics relate explicitly to parliamentary procedures (topics 1, 3, 8, 12, 19, 23, 32), three relate to actions, idioms or terms linked to speech making (topics 5, 2, 10), while 25 relate to specific issues (imperial relations, environment, budget, supply, the economy, justice, finance, elections, navigation, ways and means, foreign policy, equalisation, immigration, agriculture, transportation, monetary policy, civil service, social policy, trade, health care, employment, military, First Nations, energy and regionalism).
The five most frequently used and exclusive words in the 35 topics identified by the STM.
Not all topics are discussed equally in the corpus. The topics related to actions, idioms and parts of speech show the most variation. For instance, kindness (topic 5) displays the greatest decline over time: it is found in around 25 per cent of all speeches in the early 1900s but is virtually absent in the 2000s. Does this mean that MPs are less kind to each other in Canada today? Part of this trend can be explained by the words found in this category, such as gentleman, chairman, or [honourable] friend—forms of address that are no longer popular in modern vernacular. Like Cochrane (Reference Cochrane2010), we also find evidence of an increase in partisan rhetoric over time: references to opposite party members appear to have become less cordial.
The same pattern is observed when we look at topics related to elections and government. They have become more prevalent recently and include words such as senate, liberal, conservative, opposition, govern, mistake, bad, problem and leader. We return to this issue in the final section of the empirical analysis.
4.2.1 Predictive validity
To get a better sense of these trends, Figure 3 shows the average yearly prevalence of six topics between 1901 and 2015 and their predictive validity. For parsimony, we focus on a sample of issues where benchmarking data are available to determine if our estimates are correlated with real-life events (see also Quinn et al., Reference Quinn, Monroe, Colaresi, Crespin and Radev2010). We selected the following indicators: the unemployment rate, the consumer price index, major trade agreements, major conflicts and peace missions, and the commodity price index.Footnote 5 The last plot of Figure 3 reports the evolution of procedural speeches in the House. This measure combines the seven procedural topics identified earlier (topics 1, 3, 8, 12, 19, 23, 32) into one category and calculates its yearly average over time with indicators of major rule changes observed in the Standing Orders.Footnote 6
Over-time mean prevalence of five issue topics and procedures with their benchmarks (1901–2015). Prevalence can range from 0 to 100 per cent. The inflation and unemployment rates have been divided by 10 to be on the same scale as their corresponding topics.
The plots confirm that the proportion of speeches related to inflation and unemployment closely follows changes in the consumer price index and unemployment rates. Similarly, the topic of agriculture seems to have been affected by fluctuations in the commodity price index, reflecting in part changes in the salience of these issues in Canadian politics. Other topics also appear to respond to specific political events. For instance, the topic of trade increased significantly right before the adoption of the 1935 and 1988 trade agreements with the United States, while the topic of military became more prevalent in conjunction with wars and peacekeeping missions. These trends give us confidence in the predictive validity of the results reached automatically by the STM model.
Turning now to the combined categories of parliamentary procedures, we find that these topics are present in approximately 20 to 30 per cent of the speeches made during the whole period. In Appendix D, we provide three examples of this type of speech in the data. Procedural topics peaked in 1961 (31.7 per cent) and maintained this level for the next three decades, until the end of the 1980s, when they fell back closer to their average value. The vertical lines indicating major parliamentary reforms also seem to have a negative effect on the proportion of procedural speeches, but only after the 1980s.
4.3 Change point analysis
Having briefly discussed the topics of Canadian parliamentary speeches, we now turn to the task of analyzing the impact of rule changes on agenda control. Recall that an important number of new procedures were adopted in the House to facilitate the consideration of government business. Our first task is to detect if the introduction of these rules modified the proportion of procedural talk in the debates. We do this by using a change point model to automatically identify structural breaks in the distribution of this topic over time (see Goet, Reference Goet2021). We use the automated segmentation algorithm of Bai and Perron (Reference Bai and Perron2003) to locate the optimal number of these breaks, but our results also perfectly match those obtained by a Bayesian change point model (Erdman and Emerson, Reference Erdman and Emerson2007).Footnote 7
Figure 4 reports the location of these change points (solid lines with dates on top) and their confidence intervals (grey areas). The model converged after segmenting the data into 12 distinct periods, each ending with the following breaks: 1916, 1925, 1932, 1935, 1945, 1950, 1957, 1983, 1988, 2000 and 2005.Footnote 8 Two other things are worth mentioning from these results. First, the change point model identifies significant breaks in the amount of procedural language when important modifications to the rules were debated in the House, such as 1912–1920 (closure), 1922–1930 (40-minute speech length), 1981–1989 (Lefebvre and McGrath), and 2000–2005 (reforms of the committee system). Second, the model also subsequently identifies a long uninterrupted period between 1945 and 1980 when procedural speeches were prominent in the debates. This last finding supports the postwar narrative constructed by Dawson (Reference Dawson1962), March (Reference March1974) and Franks (Reference Franks1987) of a dysfunctional Parliament. Incidentally, this segment ends right around 1983, when the House finally adopted the most important set of rules to streamline the debates.Footnote 9
Over-time mean prevalence of procedure topics, with vertical lines for breakpoints identified by the change point model (with 95 per cent confidence intervals).
4.4 Procedures and agenda control
The previous analysis confirmed that a significant proportion of what MPs say in the House of Commons relates to procedures and that this topic is more frequently discussed when important rule changes are adopted. One could be tempted to disregard these speeches (Quinn et al., Reference Quinn, Monroe, Colaresi, Crespin and Radev2010) or label them as “noisy” (Lauderdale and Herzog, Reference Lauderdale and Herzog2016) or “junk” (Boussalis et al., Reference Boussalis, McElroy and Sorace2018). However, we believe that procedural debates are important because they often determine who controls the agenda, thus shaping the government-opposition dynamic that is so prominent in Westminster-style parliamentary systems (Eggers and Spirling, Reference Eggers and Spirling2014; Hirst et al., Reference Hirst, Riabinin, Graham, Boizot-Roche, Morris, Kaal, Maks and Elfrinkhof2014). In order to evaluate the impact of rule changes on agenda control, we use the results of our previous STM analysis to determine who speaks more about procedures. The dependent variable in this analysis is the mean prevalence of procedural topics contained in all of the speeches made during the 1901–2015 period, and the independent variable is MP position (baseline = opposition backbenchers). We performed the same analyses with the prevalence of the other general topics (idioms and issues) as dependent variables in Appendix E but report here only the results for procedural content. In Table 1, the first model calculates the mean prevalence of procedures for each group of MPs by year, whereas the second one calculates the same variable but for each Parliament.Footnote 10 The models also include a term component variable to capture the effect of time.
Effect of MP Position on Prevalence of Procedural Topics, 1901–2015
Note: Term ranges from 1 to 115 in the first model (1901 to 2015) and from 1 to 33 in the second model (9th to 41st Parliaments). Standard errors in parentheses.
∗p < .1; ∗∗p < .05; ∗∗∗ p < .01
Unsurprisingly, the analysis confirms that the prevalence of procedural topics is much higher in speeches made by government and opposition frontbenchers, when compared to backbenchers sitting on either side of the aisle.Footnote 11 These results hold both by year (column 1) and by Parliament (column 2). It makes sense to assume that members of the opposition raise procedural questions when their goal is to obstruct the proceedings of the House. On the other hand, it also seems plausible that government members speak more about procedures when they want to move the legislative agenda forward: this should be especially true when considering speeches made by government frontbenchers. These types of debates most likely influence the position taken by legislators in their speeches, reflecting in part a government-versus-opposition dynamic, rather than left-right ideology (see also Hirst et al., Reference Hirst, Riabinin, Graham, Boizot-Roche, Morris, Kaal, Maks and Elfrinkhof2014; Lauderdale and Herzog, Reference Lauderdale and Herzog2016). As we show in the next section, this has important implications for how scaling algorithms are used to measure ideology from political texts.
4.5 Procedures and polarization
This last analysis attempts to determine if the prevalence of procedural talk affects one of the most commonly used scaling algorithms to measure ideological content from textual data: Wordfish (Slapin and Proksch, Reference Slapin and Proksch2008). Wordfish is a Poisson item response model that estimates the position of texts on a general dimension from the frequencies of words found in documents. The algorithm has so far been successfully used to uncover the ideological location of parties and legislators by analyzing party platforms and legislative speeches (for a review, see Lowe, Reference Lowe2016). One limit of this technique, however, is that it can only estimate a single dimension from the data. This becomes problematic when more than one dimension divides a corpus—for example, when there is a conflict between the government and the opposition or a division between the left and the right.
One option to get around this problem is to limit the analysis to one specific set of issues, such as speeches related to the budget, so that the model can more easily identify a clear opposition between parties in the debates (see Lowe and Benoit, Reference Lowe and Benoit2013). Another approach is to remove procedural content from the corpus to expunge any unrelated vocabulary before analyzing the data. This method has been used by Lauderdale and Herzog (Reference Lauderdale and Herzog2016) to eliminate discussions of the “meeting agenda, prayers, tributes, elections of the speaker, points of order, and any other discussions concerning the rules of parliamentary procedure” in their analysis of the US Congress and Irish Dáil. It has also been used by Goet (Reference Goet2021) to eliminate “procedural phrases” from his analysis of parliamentary debates in the UK and by Rheault and Cochrane (Reference Rheault and Cochrane2020) to remove procedural terms from their word-embedding analysis of parliamentary debates in Canada, the UK and the US.
Each of these fixes has important limitations, however, mostly because procedural words can be found across several different types of speeches, not necessarily related to the heading or subheading of a debate. For instance, it is possible to find substantive discussion of issues during “adjournment debates” or “points of order.” Likewise, debates about procedures can be related to partisanship, or even in some cases ideology, when parliamentary privileges are under attack. Using a dictionary to remove procedural words (or phrases) from the corpus can also be problematic, since existing dictionaries are not all-encompassing, and the choice of vocabulary appears somewhat arbitrary.
Given these observations, we suspect that the location of MPs as estimated by the Wordfish scaling algorithm will be influenced by how frequently parliamentary rules are discussed in the debates. As we saw earlier, the distribution of procedural topics varies over time; it is much more prevalent in periods leading up to important rule changes in the Standing Orders. Furthermore, we know that not everyone uses procedural words at the same rate; party leaders are much more likely to raise these issues in their speeches. We therefore expect that a higher prevalence of procedural content should contribute to increase polarization in the debates.
To validate this hypothesis, we used Wordfish to scale the locations of the Conservative and Liberal parties in the Canadian House of Commons between 1945 and 2015. Like Chinn et al. (Reference Chinn, Sol Hart and Soroka2020), our analysis calculates an index of polarization by taking the absolute value of the difference between the Wordfish scores obtained by the Conservative and Liberal parties. For comparison purposes, we ran the model for every year and for every Parliament during this period.Footnote 12 In order to determine if the estimates generated by Wordfish are influenced by debate types, we used a regression model to measure the relationship between our polarization index and the average prevalence of the topics identified above: procedure, issues and idioms.Footnote 13 Like before, the model includes a time component variable in the analysis.
The results presented in Table 2 confirm that the prevalence of procedural topics is positively correlated with the polarization index. This is true for both the annual (column 1) and Parliament (column 2) level data. A higher proportion of procedural topics in the debates is linked to a larger ideological distance between Liberal and Conservative caucuses as calculated by the Wordfish algorithm. Although not statistically significant in the second model, we also find confirmation that an increase in the prevalence of issue topics (for example, the military, budget, agriculture) is positively correlated with polarization. This last result supports the notion that ideological polarization is linked to more substantive debates.
Effect of Procedural Content on Polarization, Measured Using Six Indicators
Note: Term ranges from 1 to 70 in the first model (1945 to 2015; 1954 missing) and from 1 to 22 in all other models (20th to 41st Parliaments). Standard errors in parentheses.
∗p < .1; ∗∗p < .05; ∗∗∗ p < .01
The next four regression models in Table 2 serve as a robustness check by replicating the previous analysis using different “gold standard” measures of party ideology but this time without procedural content. All of these indexes are measured by Parliament and taken from the analysis of Rheault and Cochrane (Reference Rheault and Cochrane2020). The first one is obtained from their word-embedding model of parliamentary debates between 1945 and 2015. It calculates the Euclidean distance between the Conservative and Liberal parties to estimate a polarization index from legislative speeches by removing some of the most common procedural words found in the debates. As we noted earlier, this dictionary approach is not perfect, but it does manage to delete 5 per cent of the vocabulary in the corpus. Vanilla, Rile and Legacy are three other measures of Liberal-Conservative polarization based on party manifesto data. These indexes should not contain discussions of procedures since there is no “parliamentary rule” or “parliamentary reform” categories in the Comparative Manifesto coding scheme.Footnote 14 The results confirm that there is no statistically significant relationship between the prevalence of procedural topics and these four alternative measures of partisan polarization.Footnote 15 Although the average correlation between these gold standards is very high, they do not appear to be linked to procedures. In other words, an increase in procedural debates is not associated with a more polarized House when we look at these different indices. Clearly, these findings suggest that the Wordfish method of ideological scaling, when used over the entire corpus of debates, gives more weight to procedural words in discriminating between the positions of the Conservative and Liberal parties. In turn, this approach inflates the distance between parties and thus increases our perception of partisan polarization in the Canadian Parliament.
5. Discussion
In this section, we put together the most interesting findings of the previous empirical analyses to draw some inferences about agenda control and parliamentary procedures in the Canadian House of Commons. To begin, the results confirmed our first two hypotheses, which predicted that frontbenchers would intervene more during the debates and that backbenchers and opposition members would make longer speeches, especially in more recent parliamentary terms. Nevertheless, government backbenchers still have fewer opportunities to speak in the House when compared to all other groups of MPs. In terms of content, our topic model demonstrated that the proportion of speeches related to procedures gradually increased in the House from the 1950s to the 1980s and declined afterward, suggesting that these shifts were influenced by rule changes. Thus we also confirmed our third hypothesis: that procedural content would be more prevalent in speeches made by government and opposition frontbenchers. Finally, we found evidence to confirm our fourth hypothesis: that polarization between the Liberal and Conservative parties should be strongest when a larger share of parliamentary debates focused on the rules and procedures of the House.
It is possible to draw interesting links between these findings. Previous work has argued for some time now that procedural debates in Canada are more prevalent during periods of heightened partisan conflicts in the legislative arena (Franks, Reference Franks1987; Stewart, Reference Stewart and B1977). Our text-scaling analysis seems to confirm this trend. When procedural content was removed from measures of party ideology, we found no statistically significant association between the amount of procedural words in a given Parliament and the level of polarization. Although Proksch and Slapin (Reference Proksch and Slapin2010) and Lauderdale and Herzog (Reference Lauderdale and Herzog2016) have warned us that text-scaling algorithms are highly susceptible to procedural language, it is still striking to see that retaining this type of content helped us uncover stronger levels of polarization in the Canadian case. After all, the job of the opposition is to confront and to use all of the procedural mechanisms at its disposal to oppose the government. Even though the opposition in Canada has long ago lost “the right to talk out government bills” through closure (1913) or the ability to speak for more than 40 minutes on a topic (1927), it still often uses lengthy debates to slow things down (Schmitz, Reference Schmitz1988: 7). These obstruction opportunities have become less frequent over time, but especially after the Standing Orders were modified in the 1980s (Dawson, Reference Dawson1962: 127–33; Franks, Reference Franks1987: 128–32; Stewart, Reference Stewart and B1977: 239–41).
Since the overall decline in the proportion of words related to procedures was influenced by changes in the Standing Orders, our results suggest that parliamentary procedures affect the orientation of debates, especially since party leaders are more likely to employ these terms. The content of legislative speeches is not always a random sample of an underlying word distribution. As with the selection of roll-call votes (Ainsley et al., Reference Ainsley, Carrubba, Crisp, Demirkaya, Gabel and Hadzic2020), we must consider the possibility that the choice of words can suffer from significant selection bias. Parliamentary rules have the potential to alter the prevalence of certain words in the debates, notably among party leaders, but they can also limit discussions of different topics through negative agenda control. In Appendix F, we show that this is indeed what occurred in Canada, with a decline in the number of topics being discussed in Parliament over time, after the rules were changed in the 1980s.
6. Conclusion
Our analysis of the debates in the Canadian Parliament between 1901 and 2015 confirmed that the introduction of different time-management tools altered the content of the legislative agenda, with an important impact observed after the Standing Orders were modified in the 1980s. Although we found an increase in both the number and length of legislative speeches made by opposition party members since the early 1900s, our analysis showed that this change occurred gradually and that it had no discernible impact on the issues raised during the debates. Using a structural topic model, we found instead that the content of individual speeches was mostly affected by the introduction of new rules. We also saw that frontbenchers (especially from the governing party) tended to talk more about procedures and that the discussion of procedural topics could influence the scaling of party ideologies in the House. In the remainder of this conclusion, we consider three implications of these results for the study of parliamentary text data.
To begin, what can our analysis tell us about the declining influence of MPs in the legislative process? The gradual increase in party discipline in Canada has led some MPs to claim that backbenchers are “treated as puppets” (Samara Centre for Democracy, 2018). Our analysis has shown that this is the case, but mostly for the government rank and file. These members have seen their relative influence in the legislative process decline over time, primarily to the benefit of opposition party leaders. Government backbenchers clearly had more influence compared to opposition members when there were only two parties in the House. The arrival of third parties coincides, first, with an increase in the speaking time and opportunities of opposition MPs. Later, modifications to the rules, such as the establishment of standing committees to examine bills from the House or the scheduling of a permanent Question Period during the debates, accentuated this trend by transferring even more time to the opposition side of the aisle. The increase in private member business after the 1986 reform of the Standing Orders did not alter this pattern. Perhaps, the loss of influence of government backbenchers was alleviated by the increasing importance of permanent committees after 1968—where government members usually dominate the proceedings—but more work needs to be done in the future to understand this dynamic.
The second implication of our results relates to the importance of procedural language in the debates. Our comprehensive analysis of legislative speeches confirmed that at least 20 per cent—and sometimes up to 30 per cent—of what is said in the House of Commons between 1901 and 2015 relates to procedures. We already knew that parliamentary rules could prevent bills from reaching the floor of the legislature, facilitate the adoption of the government's agenda and limit speaking opportunities of members (Aldrich and Rohde, Reference Aldrich and Rohde2000; Cox and McCubbins, Reference Cox and McCubbins2005; Dewan and Spirling, Reference Dewan and Spirling2011; Huber, Reference Huber1996). Despite its importance for parliamentary dynamics, however, procedural speech is almost universally discarded in studies of legislative speech data (see for example, Boussalis et al., Reference Boussalis, McElroy and Sorace2018; Quinn et al., Reference Quinn, Monroe, Colaresi, Crespin and Radev2010). In this article, we argued that ignoring procedures is a mistake, mainly because the substantial issues debated in Parliament can be a function of who controls the agenda. Even though MPs have much more freedom when they speak compared to when they vote, they are still constrained by parliamentary rules. We confirmed that measuring a member's ideology with Wordfish (Slapin and Proksch, Reference Slapin and Proksch2008) is affected by procedures and procedural change. This conclusion aligns with the empirical work of Lauderdale and Herzog (Reference Lauderdale and Herzog2016), who show that the text scaling of legislative speeches in the US Congress and the Irish Dáil is much better at identifying political disagreement (that is, government-versus-opposition) than actual individual policy preferences.
Perhaps the fact that procedural words provide a distorted view of legislative behaviour explains why we find higher partisan polarization in periods when party manifesto data report the opposite (Cochrane, Reference Cochrane2010; Rheault and Cochrane, Reference Rheault and Cochrane2020). Our text-scaling analysis of parliamentary speeches has most likely uncovered a government/opposition dimension, highly correlated with procedural content. Since this dimension is often found in parliamentary systems, we recommend two approaches to deal with this problem. The first one is to develop a more comprehensive list of procedural words used in different types of legislatures, perhaps by following the minimally supervised dictionary method to identify specialized vocabularies, as suggested by Rice and Zorn (Reference Rice and Zorn2021) (see also Goet, Reference Goet2021). As we mentioned above, the solution for dealing with this problem right now seems arbitrary; we need a clear set of rules to detect these words and apply them systematically before analyzing parliamentary speech data.
The second and more promising approach in our view would be to develop joint word- embedding models to capture ideology from parliamentary debates by simultaneously training a neural network on both the speech content and the positions of MPs in the legislature (for example, speaker, front-backbench, opposition). This type of model could be trained to ignore certain procedural terms correlated with these positions, but not with partisanship, to estimate a more valid measure of ideology through the elimination of irrelevant text (see Chauhan, Reference Chauhan2020).
The third implication of our results concerns the structural topic modelling approach. This type of analysis requires that researchers give meaning to the topics uncovered by the model. This is an inductive exercise: there is no way to objectively interpret the output results. Although we made sure to identify topics based on different measures provided by the model (Roberts et al., Reference Roberts, Stewart and Tingley2019), our analysis remains sensitive to the value of these parameters. This limit suggests that our results should be interpreted with care, which is something we made sure to do throughout this article.
Despite these flaws, we believe that STM remains a valuable tool to analyze legislative speeches, especially if researchers pay attention to how parliamentary rules affect the content of the debates. Until we find a more systematic approach to deal with this problem, it is important that we continue to study the impact of procedures on estimating the ideology of legislators from parliamentary text data.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/S0008423921000718.
