Note. A similar version of this research appeared as a chapter in The State of the Parties: The Changing Role of Contemporary American Parties, 5th ed., by John C. Green and Daniel J. Coffey
(Rowman and Littlefield, 2007).Recent methodological advances have spurred a renewed interest in party platforms, mainly because advances in content analysis programs have reduced the time necessary to code what are often lengthy political texts. Moreover, increases in the reliability of such coding algorithms have the potential to replace labor-intensive, expert-based coding with a highly reliable and more objective approach.
This article raises a cautionary note about the virtue of such methods. In a search for more reliable and less intensive methods, researchers need to be aware of the danger of placing too much faith in computer-based algorithms, which may lead to mistaken inferences about the ideological placement of political texts. Reliability may come at the expense of validity. If research produces counter intuitive findings, it may be necessary to reevaluate the conventional wisdom, but researchers using highly automated text-coding programs need to pay close attention to the results and provide evidence that they correspond to external measures.
This article challenges a finding in this journal that American party platforms reveal only minor policy differences. I analyze the same question that Kidd explored—exactly how distinct are the policies and expressed beliefs of American parties?—by applying a more traditional coding technique to a different dataset on American state party platforms written between 2000 and 2004.
In contrast to Kidd's conclusion, this analysis produces three main findings. First, at the state level, party platforms reveal considerable ideological differences between the parties. Second, differences in state public opinion do not account for such ideological differences, which are instead more closely correlated with activist positions and increases in state party competition. As such, using a dataset with a significantly larger N (121 observations for the two major parties to the total of six national party platforms for the two major parties—or 12, including the Green and Libertarian parties—used in Kidd's analysis), I find that American state parties are highly polarized on different measures.
More important, automated techniques are generally less concerned with the specific issues that are addressed in political texts. In particular, a focus on analyzing political texts ironically misses one of the key aspects of party platforms: content. As such, these techniques may overlook important indicators of the nature and degree of party conflict. I provide evidence that not only are parties ideologically polarized, but also that qualitative analysis of their platforms confirms that American parties are quite polarized indeed.
A Lack of Difference?
Recently, Kidd (Reference Kidd2008) found that American political parties are not ideologically polarized, especially on domestic issues, by applying the “Wordscores” coding program to American national party platforms written between 1996 and 2004. Word scoring treats political texts as data; instead of operating as a single text, party platforms often contain thousands of words, with each word providing a data point that can be measured. Each word in a manifesto has a certain likelihood of appearing based on word frequencies derived from a reference text such as a previous expert-coded platform. Kidd provides a clear description of his methodology (see also Laver, Benoit, and Garry Reference Laver, Benoit and Garry2003; Martin and Vanberg Reference Martin and Vanberg2008; Benoit and Laver Reference Benoit and Laver2008). In the aggregate, the distribution of words in a platform provides a highly reliable indicator of the ideology of the platform. Kidd confirms the conventional wisdom about American parties—that they are relatively moderate. He argues that Duncan Black's (Reference Black1948) theory on policy convergence explains the absence of extremism: the parties have moderated their relative positions in response to their electoral fortunes in recent years.
The point of this study is not to challenge the reliability of the scoring technique, but to present an alternative method that demonstrates that American political parties are in fact quite polarized. A different approach for a different dataset produces a more valid result. Kidd's finding that American parties are similarly positioned on domestic issues is a puzzling contrast with most recent research, which finds—through a variety of different measurements, as Kidd himself notes—considerable ideological differences between the parties at the mass attitudinal level (Abramowitz and Saunders Reference Abramowitz and Saunders2008; Hetherington Reference Hetherington2001), in Congress (McCarty, Poole, and Rosenthal Reference McCarty, Poole and Rosenthal2006), and among party activists (Layman and Carsey Reference Layman and Carsey2002).
Of course, the orthodox line about American parties is that their nonideological nature makes them distinct from other nations' parties, since their fragmented organizational structures cause them to focus more on winning elections and maintaining social cohesion than pushing dogmatic agendas (Epstein Reference Epstein1986). Some comparative studies, however, have found evidence for partisan differences. Klingemann, Hofferbert, and Budge (Reference Klingemann, Hofferbert and Budge1994) find that American national party platforms are nearly as distinct from each other as many manifestos in European party systems. Gerring's (Reference Gerring1998) extensive study of campaign speeches and national party platforms from 1828 to 1996 finds that American parties have distinct philosophies that are clearly articulated in national party platforms across decades.
State Party Platform Polarization
Using a technique that relies on a computer program (TEXTPACK) to assist with content analysis, I coded 121 state party platforms written between 2000 and 2004. In states where more than one platform was written between 2000 and 2004 for a state party, I created an average ideology score. Sentences were coded initially on the appearance of certain key words from an ideological dictionary. Because words have multiple meanings depending on the context, it was possible for sentences to be miscoded, so I manually reviewed every sentence in each platform. The gain in validity far outweighed the loss in the reliability (Coffey Reference Coffey2005).Footnote 1
The procedure was fairly simple. Party ideology was calculated by the proportion of liberal and conservative sentences in each state of the state speech. Specifically, the calculation is:
By measuring ideology in this manner, a standardized value was assigned to each category, ranging from 1 (all liberal sentences) to −1 (all conservative sentences). The denominator was set as the total number of sentences in the platform, factoring moderate sentences into the ideological calculation and moving the score closer to 0. For example, the 2002 Texas Democratic platform is 325 sentences long, of which 252 were determined to be liberal, 44 were determined to be conservative, and 29 were not coded. The corresponding score was .62.
This coding produced both valid and reliable results. Gerring defines party ideology as messages or positions that are “internally coherent, externally differentiated (from one another) and stable through time” (Reference Gerring1998, 3). Using this framework, I coded as liberal those sentences that emphasized government intervention into economic markets for reasons such as protecting lower income citizens and spending more on social welfare programs. In addition, liberal sentences included statements of support for the protection and expansion of the rights of marginalized groups. Conservative sentences generally expressed opposition to governmental regulation, called for the protection of individual opportunity (especially through lower taxes and private property rights) in the marketplace, and advocated stricter regulation of individual behavior and a greater role for religion (often Christianity) in the public sector. If a sentence was ambiguous in its ideological position, it was not coded as ideological but was included in the denominator (total sentences). I used two basic guidelines to classify sentences: (1) Would such a sentence appear in the opposition party's platform? (2) How would a reasonable person interpret such a sentence?
The platforms clearly distinguish the parties across the states. The Democratic average score is a fairly liberal .68, while the average GOP platform is −.63. In fact, there is no overlap between the parties: Indiana's 2004 Republican platform, which is the most liberal GOP platform, has a score of −.15, while the 2002 Alabama Democratic platform, which is the most conservative Democratic platform, has a score of .42.
By comparing states in which both parties wrote platforms between 2000 and 2004, the manual coding found no overlap at all between party positions. Figure 1 provides stark evidence of the parties' polarization. The dot plot shows that rather than gravitating toward the center of the political spectrum, the state parties appear to push each other away toward the poles. To draw an analogy with physics, the ideological forces of parties in the American states are centrifugal, not centripetal. In fact, increased conservatism among state Republican platforms is associated with increased liberalism among state Democratic platforms (r = −.45).
State Party Platform Polarization
Note. Figure shows average platform ideology scores for states in which both state parties had platforms between 2000 and 2004. Platform scores range from −1 (most conservative) to 1 (most liberal).
The Median Voter Theorem and the Activists' Manifesto
Kidd argues that his findings provide evidence for Downsian moderation—that is, parties are pulled to the center by the force of public opinion in a desire to maximize their chances for winning elections. Downsian moderation is, however, an empirical hypothesis, and so I tested whether median state public opinion explains the variance in state party platform ideology.
To test this relationship empirically, I modeled state party platform ideology as a function of several state-level variables that theoretically should explain the variance in party platform ideology. As Erikson, Wright, and McIver (Reference Erikson, Wright and McIver1993; Reference Erikson, Wright and McIver2007) have shown, state median public opinion has a substantial impact on state legislators' ideology and the policy outputs of state governments. If Kidd is correct, party ideology should be related to median state opinion.
Alternatively, as Berry and Schildkraut have noted, since 1980, “regardless of motive, citizen groups work to accentuate the differences between the parties, to drive a wedge in further so that the parties do not gravitate toward a more moderate position” (Reference Berry, Schildkraut, Costain and McFarland1998, 148). Activists are the group most involved in the writing of the party platform (Jewell Reference Jewell1984; Pomper Reference Pomper, Green and Farmer2003). Even Fiorina, Abrams, and Pope (Reference Fiorina, Abrams and Pope2005), who are skeptical of the degree of polarization in American politics, argue that the observed polarization of elected officials is largely a function of party activists, who push parties away from the opinions of the median voter. If the polarization view is correct, state party platforms should be shaped by party activist opinion.Footnote 2
To test whether state party ideology is related to activist ideology or public opinion, I estimated two OLS models for platform ideology, one for each party. This approach allows for a greater sample size to include states in which only one party wrote a platform and was chosen because the factors that should affect platform ideology are measured at the state level. To measure public opinion, I used the updated Erikson, Wright, and McIver dataset that includes surveys of state public opinion from 1995 to 1999.Footnote 3 One benefit of these data is that their collection preceded the writing of the party platforms. For either party, state public opinion is not correlated with state party ideology. For the Democratic Party, a slight positive relationship exists (r = .18), but the correlation is weak at best. For the Republican Party, the correlation (r = .08) does not approach statistical significance.
To measure activist opinion, I used data from the Party Elite Studies of each party's national convention delegates between 1992 and 2004.Footnote 4 Activist opinion was measured by the pooled average ideology of each state's party delegation for the four national party conventions from 1992 to 2004, creating a score ranging from 1.83 (the most liberal delegation) to 4.42 (the most conservative delegation). Activist ideology is more strongly correlated with party platform ideology for both Democratic platforms (r = −.35, p = .03) and Republican platforms (r = −.29, p =.11) than with state public opinion (see figure 2).
State Party Activist Ideology and Democratic Platform Ideology
Note. Democratic Activist Ideology created from Party Elite Studies survey of each party's national convention delegates from 1992–2004. Activist opinion is measured by the pooled average ideology of each state's party delegation for the four national party conventions from 1992 to 2004, creating a score that runs from 1.83 (most liberal delegation) to 4.42 (most conservative delegation).
I also included a variable for state party competition. In theory, party ideology should moderate with increases in party competition as parties moderate to avoid losing votes (Downs Reference Downs1957). I measured party competition using the Ranney competition index, a measure that ranges from .50, when there is no competition, to 1, when the parties are perfectly competitive (Bibby and Holbrook Reference Bibby, Holbrook, Gray and Hanson2004, 87).Footnote 5 Finally, the model controls for state racial makeup (percent of African Americans)Footnote 6 (see table 1).
OLS Regression of State Party Platforms
Note. Dependent variable is state party platform ideology. Coefficients are unstandarized OLS estimates using robust standard errors.
* p < .10;
** p < .05;
*** p < .01.
The models show that state public opinion fares worse in explaining the variance in platform ideology than does state party activist opinion. In the Democratic model, state activist ideology is positive and significant, as expected, while public opinion is not significant. The percentage of African Americans is negative and significant, likely because most states with higher percentages of African Americans are located in the South, where the public and party activists tend to be more conservative. For the Republican platforms, activist ideology and the percentage of African Americans are significant at p < .10, with all coefficients in the expected direction and apparently leading to the same conclusions that can be drawn from the Democratic model.
Importantly, party competition is positively related to party ideology. The greater the level of party competition between the state parties, the more conservative the Republican platform becomes and the more liberal the Democratic platform becomes. In the same vein, many other scholars have attacked the median voter theorem in recent years (Lublin and Voss Reference Lublin and Voss2003; Hacker and Pierson Reference Hacker and Pierson2005). While intuitively sound, the theory ignores differences in participation and information-gathering rates among the general public. Ideologically committed citizens are more likely to participate in politics and at earlier stages in the candidate-selection and policymaking processes. Studies of legislative politics have found evidence for this counterintuitive hypothesis, which posits that an increase in competition at both the district and state legislative levels increases party-line voting in the legislature by making the party labels more salient (Aldrich and Battista Reference Aldrich and Battista2002; Gulati Reference Gulati2004). It is also possible that the presence of an ideological alternative increases incentives for activists to become involved in party politics (Layman Reference Layman2001; Carmines and Stimson Reference Carmines and Stimson1989). In sum, then, empirical tests of the causes of state party platform differences indicate little support for Downs' or Black's theories but rather support the conclusions that (a) parties are polarized and (b) polarization is increased by competition and activist ideology.
The Importance of Issues
As Gerring (Reference Gerring1998) argues, the study of ideas in both a quantitative and qualitative fashion is sorely lacking in political science. He argues that “any study of party ideologies is … a study of what politics is all about. Since political parties are one of the chief disseminators of political culture, partisan rhetoric provides a window into the values and attitudes that have guided American politics” (21). One characteristic of the American party system is its relative decentralization, and the ability of state parties to write individual platforms allows them flexibility in position-taking. More intraparty diversity exists across states in terms of the mixture of issues that each party adopts than in overall ideology.
It is possible, then, that ideological measures may provide inaccurate portraits of party polarization, either over- or understating the degree of conflict that occurs between parties. Theories of “issue-ownership” or “issue salience” predict that party platforms should show little evidence of overlap on specific issues (Petrocik Reference Petrocik1996). This hypothesis seems to be especially true in the context of European party conflict (Budge, Robertson, and Hearl Reference Budge, Robertson and Hearl1990).
The evidence suggests that direct conflict is more common between state parties. Table 2 provides a few examples of direct challenges by each party on several issues. These statements provide voters with a clear sense of the parties' stance and general principles that voters could use to distinguish between them—a key standard for party responsibility, as defined by the authors of the 1950 APSA Report (American Political Science Association 1950).
Examples of Direct Confrontation on Issues
Source. 2000 and 2002 State Party Platforms.
Evidence also exists that parties will address the same issue in the same state. For example, the correlation for environmental issue content between opposing platforms within states is significant and positive. Wyoming Republicans devote 29% of platform sentences to environmental issues, while the Democrats devote 23%. This pattern extends across issues: in New Hampshire, both parties reserve 14% of their platforms for education policy, while in Illinois, Democrats address health care in 13% of their platform compared to 9% of the Republican platform.Footnote 7
I do not intend to claim that issue variation does not occur. Indeed, for Republicans, attention to social issues appears to be a clear marker of the extent to which the party is liberal or conservative. In platforms written between 2000 and 2002, liberal Republican parties avoided divisive social issues such as abortion, school prayer, and euthanasia. Of the five platforms with the most space reserved for abortion, the average ideology is −.73; for the five platforms that devote the least attention to abortion, the average ideology is a relatively moderate −.44. Of the five with the most content devoted to economic development (23.34%), the average ideology is −.44, while those devoting the least attention to economic development (4.05%) are more conservative, as indicated by an average ideological score of −.75. Parties moderate by taking positions on different issues.
Reasons for Differences with Kidd
A primary question resulting from this study is why my results differ so strongly from Kidd's analysis. There are both substantive and methodological reasons for this discrepancy. State party platforms are significantly shorter than national party platforms, with some using less than 1,000 words. A typical national party platform contains as many as 20,000 to 40,000 words, which may affect the results. The coding technique may also be responsible for some of the variance. In this study, sentences were coded as liberal, moderate, or conservative. Although I took pains to make sure that ambiguous or highly nuanced phrasings would be coded as moderate, the coding method may have produced inflated ideological differences.
However, Weinberg's (Reference Weinberg2010) study also relied on Wordscores and found significant differences between Republican and Democratic governors in the language of state-of-the-state speeches, demonstrating that what he labels as “partisan signals” reflecting the ideological divide between the parties were driven by differences in the vocabularies of governors of different parties. It should be noted that Weinberg used national and selected state party platforms as his reference texts. In other words, my findings here are supported by additional research that also relies on the Wordscore methodology.
The use of third parties as a baseline is not clearly justified if third parties' language usage differs from that of the established parties. Weinberg (Reference Weinberg2010), among others, has found that Republicans and Democrats use simple words like pronouns very differently (e.g., Republicans tend to use “I”, while Democrats use “our” or “we”). A lack of analysis of American third parties means that we do not have clear knowledge of how or why their pronoun or adjective choices are made. Certainly, the Wordscore technique has been shown to place party texts accurately in European party systems, so the use of minor parties is not in itself a problem. However, it is possible that automated coding programs work well because they can pick up on the use of an organically developed political language that expresses the terms of social disagreement in nations with more culturally entrenched third parties. In the United States, it is not clear that the Green and Libertarian parties use language in a manner that clearly signals their ideological position relative to the language used by the two major parties. Consequently, placing them as ideological anchors may not be justified.
Alternatively, the differences between Kidd's findings and my own could be due to the unit of analysis used in each study. Many states in which party conflict has historically been nonideological did not write a party platform (Mayhew Reference Mayhew1986). Thus, when party coalitions are brought together to write a single platform, it may be that the resulting platform is less distinctive from the platform of the opposition. Differences might also be process-generated. Many state party platforms, especially in caucus states, are written at the precinct level. Past research has shown that the probabilities of groups capturing state party organizations differ based on the structure of state parties (Usher Reference Usher2000; Conger Reference Conger2010). Social groups often politically target state party platform committees, and therefore, state party platforms are more likely to present highly polarized views on divisive issues
Yet it should be noted that the substantive differences in major parties' ideological positions are striking, as illustrated in the examples of direct conflict on a single issue. In many party platforms, as few as one or two sentences might be reasonably coded as running contrary to a party's ideology. The appearance of moderate sentences was rarely due to nuanced policy positions; rather, the coding reveals that moderation resulted from platforms that read more like campaign documents (such as those of the Indiana Republican Party) or lacked specific policy commitments.
Conclusions and Discussion
The use of automated content analysis programs clearly provides a significant leap forward in scholars' ability to code party platforms and political texts. There is little doubt that these techniques will become more commonly used and sophisticated over time. Researchers can now explore questions that were once prohibitively time-consuming. Yet, as tempting as automation may be, these techniques place a heavy burden on researchers to show the accuracy of the results in capturing the meaning of political texts and not simply producing the same score every time. The findings here aim to (a) raise a cautionary note about the use of automated coding techniques and (b) highlight the importance of examining the substance of platforms as a measure of party conflict.
One might claim that the findings here are outdated, since they rely on data from 2000 and 2004. On the other hand, my results are directly comparable to Kidd's, because they cover a time period at the core of his study (1996 to 2004). Moreover, although the sample sizes are small, the data are consistent with our theoretical expectations. I find that not only does the median voter theorem not explain party behavior, but that an opposite force seems to be at work, consistent with recent research on this theory's empirical validity. This article provides more evidence that the median voter theorem is more conditional than an absolute law in a two-party system.
Finally, the findings here point to the value of analyzing state party platforms, a task often overlooked by researchers. Greater variance exists at the state level, which can allow scholars to test a variety of hypotheses that are not testable by analyzing two platforms written every four years. The influence of public opinion, economic conditions, regional subcultures, population mobility, and other demographic trends, as well as institutional and legal differences in the structures of parties can be analyzed with more frequent state-level observations (since platforms are often written every two years). The federal nature of the American party system thus provides a way to incorporate new methods and test them in a more extensive fashion.
