INTRODUCTION
Recent developments in experimental research on the theory of spatial voting have deepened our understanding of how candidates’ policy positions translate into voting behavior (Claassen Reference Claassen2007, Reference Claassen2009; Tomz and van Houweling Reference Tomz and van Houweling2008, Reference Tomz and van Houweling2009). By virtue of combining formal modeling with suitable experimental designs, these studies contribute important insights to a lively debate about the shape and form of voters’ judgment on candidates’ policy stances. One of their main findings is that proximity considerations—voters preferring candidates closer to themselves in a policy space—outweigh non-proximity considerations; the extent to which this holds true varies with demographic variables and the policy domain considered.Footnote 1
Current experimental designs typically represent candidates as numerical points on line-resembling scales.Footnote 2 We argue that these designs limit the generalizability of their conclusions in two ways. First, wide-spread evidence documents that preferences can be influenced by the mode of thinking induced by elicitation methods (Lichtenstein and Slovic Reference Lichtenstein and Slovic1971). The question format’s narrow spatial framing may thus be leading and suggestive, thereby creating an artificial inflation of (certain) spatial considerations. Second, the question format deviates from the way in which political actors typically communicate their standpoints on policy issues via public speeches or writings. Whether or not, and how voters take into account spatial considerations thus depends on their cognitive abilities to transform speech or text to numerical values on the policy scales.
To overcome these problems, we propose to augment current experimental designs with text-based, objectively verifiable issue positions that acknowledge voters’ cognitive realities. Such a design provides a critical stress test of whether proximity models provide an effective approximation of voting behavior in purely issue-oriented elections. The literature on measuring policy positions offers the appropriate tools for this endeavor (an overview is provided in Laver Reference Laver2014). In particular, text-based scaling methods exist that convert the content of political texts into numerical policy stances (Benoit et al. Reference Benoit, Laver, Lowe and Mikhaylov2012; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011). Each scaling method is accompanied by a coding scheme to classify text units. The classified text is then scaled by various means to locate candidates’ numerical policy positions on latent policy dimensions.
These methods can be used to represent candidates’ policy positions in terms of text-statements in experiments by a reverse-engineering process. The researcher first constructs a relevant spatial distribution of candidates in a given policy space and then inverts the text-scaling methods to yield political texts compatible with the original spatial distribution. The advantages of such an approach are evident. For one, it gives rise to ex-ante, theoretically justified numerical representations of candidates’ policy stances in a spatial-free context. Furthermore, it acknowledges a natural cognitive aspect of issue voting. Political text is everywhere. From social media platforms to candidates’ personal websites to voting advice applications, political actors use policy statements that are similar to those offered in the coding schemes of scaling methods to interact with voters.
We designed and carried out an internet experiment on a general US population to demonstrate the validity of our proposed approach. Our design mimics the status quo of experimental research on spatial voting with one important exception: candidates are represented by text statements that follow recommendations laid out in the existing literature on how to scale policy positions from text (Benoit et al. Reference Benoit, Laver, Lowe and Mikhaylov2012; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011). We find that between 72% and 76% of voters cast votes in accordance with proximity considerations. More precisely, these voters cast votes that minimize the distance between their own issue stance and the text-based, theoretically calculated candidate position on a left-right economic policy dimension. We further find average voters’ assessments of candidates to be in line with theoretically calculated policy positions. The mean absolute deviation between the average voters’ assessment and their theoretical stance is 0.34 measured on an 11-point scale. In other words, voters seem capable enough to accurately transform unambiguous political texts into numerical stances. We also find proximity considerations to be more prevalent in voters with political experience, proxied by political platform membership and past participation in elections. These results are compatible with recent categorization-based models of spatial voting that predict prevalence of proximity preferences as voters gain political experience (Collins Reference Collins2011).
In summary, we propose and empirically validate a fruitful interplay between experimental research on the theory of spatial voting and the literature on measuring policy positions. Our design serves as a blueprint for testing the generalizability of established experimental results concerning the theory of spatial voting. In the next section we present our experimental set-up and review the text-based scaling methods that are appropriate for designing a critical stress test of proximity voting. Finally, we discuss our results and conclude with implications for current and future research.
EXPERIMENTAL DESIGN
We designed an internet survey experiment and recruited 401 participants from a general US population via the research platform Prolific.Footnote 3 Detailed descriptive statistics of the sample can be found in Table A.1 in the online supplementary materials.Footnote 4 Our experiment presented participants with a presidential election scenario involving three candidates. To minimize the influence of non-issue considerations, the candidates were labeled neutrally and referred to as A, B, and C throughout the experiment. Each candidate was represented by five statements mainly concerning the economic policy that the candidate would implement, if elected.Footnote 5 The statements were based on examples laid out in the code-book of the Manifesto Project, which estimates policy positions derived from content analysis of electoral manifestos (Budge et al. Reference Budge, Klingemann, Volkens, Bara and Tannenbaum2001; Klingemann et al. Reference Klingemann, Volkens, Bara, Budge and McDonald2006; Volkens et al. Reference Volkens, Bara, Budge, McDonald and Klingemann2013). The code-book provides coding instructions to categorize each statement of a political text as a reference to the political-left, the political-right, or an unrelated or neutral reference (Werner, Lacewell, and Volkens Reference Werner, Lacewell and Volkens2015).
Our three candidates were designated as follows: L was left-wing, M was centrist, and R was right-wing. L made three ‘left’ statements, one ‘neutral’ statement, and one ‘right’ statement. The centrist candidate M was represented by five neutral statements. R made three right, one neutral, and one left statement. Of note, in the experiment, candidate-labels A, B, and C were randomly assigned to candidates L, M, and R to minimize any labeling effects.
Our composition of left, neutral, and right statements created the necessary conditions for a stringent test of proximity considerations. That is, participants were required to interpret a variety of text-statements in order to understand each candidate’s nuanced policy stance. The statements were chosen to represent moderate and credible stances—avoiding topics publicly debated at the time of the experiment—related to the policy dimension of state involvement in the economy, i.e., expanding versus reducing the active role of the government in the economy (Benoit and Laver Reference Benoit and Laver2007; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011). The exact statements and the corresponding coding categories are presented in Table A.2 in the supplementary online materials.
Participants were first shown the description of the three candidates and then asked to rate each on a 100-point thermometer rating scale. The question wording and display format was taken from the NES with ratings between 0 and 50° expressing unfavorable feelings toward a candidate, and ratings between 50 and 100° expressing favorable feelings. The thermometer ratings provided us with a first indication of voters’ preferences in a spatial-free context.
We used the thermometer ratings to create critical voting conditions. Specifically, following the thermometer rating questions, each voter was asked to imagine that they were are about to cast a vote in the election. For reasons not further specified in the instructions, only two of three candidates decided to run for office. The candidate who dropped out of the race was the one that the voter evaluated most favorably with her or his thermometer rating.Footnote 6 This procedure forced every participant to make a compromising choice and ensured that ‘proximity’ voters had to develop a complete spatial representation over the full candidate-set. In this sense, our critical voting conditions constituted a stringent test of proximity considerations. Participants were then presented with the aforementioned voting scenario, casting their vote in a two-candidate race.
We next asked participant to place themselves and the candidates on a left-right economic policy dimension. Using the question wording and response format from the Chapel Hill Expert Survey (CHES) 2010 (Bakker et al. Reference Bakker, de Vries, Edwards, Hooghe, Jolly, Marks, Polk, Rovny, Steenbergen and Vachudova2012), we explained that candidates on the economic left wanted government to play an active role in the economy whereas candidates on the economic right emphasized a reduced economic role for the government. Voters then placed themselves and each candidate on this 11-point scale with 0, 5, and 10 representing the far-left, the center, and the far-right, respectively. We deemed the CHES 2010 question appropriate due to its empirical validity and immediate connection to the economic policy statements we used to describe candidates.
The experiment was administered in three different treatments to stringently test the empirical validity of our proposed methodological crossover. In the baseline version (N = 204 participants), respondents went through the survey questions in one sitting, in the same order as described above. In the delayed version (N = 89), we separated the self-placement of respondents and their decision to vote by approximately seven days via a two-wave design. In wave 1, participants saw the same survey as in baseline except for the voting scenario. In wave 2, participants saw the candidate description once more and were then presented with the voting scenario. This design allowed us to test the robustness of our findings with regard the temporal ephemerality and the stability of proximity considerations.Footnote 7
Both the baseline and the delayed versions employed slider measures as the response format. Slider measures clearly resemble a one-dimensional policy space, which may induce spatial considerations on their own. Our final version, the text-input version (N = 100), therefore replaced the slider measures with conventional text-input boxes. Otherwise, the text-input version was identical to the baseline version, i.e., participant went through the survey questions in one sitting in the same order as in the baseline version. Screen-shots from the decision screens for each question type can be found in the online supplementary materials.
The experiment concluded with a set of socio-demographic questions. Data was collected in a time-frame of about three weeks, beginning at the end of October, 2017, and wrapping up mid-November, 2017. The baseline version and wave one of the delay version were launched simultaneously after data collection for the text-input version had been completed.
MEASURING POLICY DISTANCES
We borrowed from the literature on measuring policy positions to transform text statements into a numerical scale and adopted the RILE score formula—and crucial modifications to it—to scale the right-left ideological economic-policy position of our candidates on the basis of the statements they make.Footnote 8 We considered three scales currently applied in the literature (Budge et al. Reference Budge, Klingemann, Volkens, Bara and Tannenbaum2001; Kim and Fording Reference Kim and Fording2002; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011): the unconditional or raw RILE, which measures the relative frequency of right statements in relation to the relative frequency of left statements; the conditional RILE, which discards neutral references from the calculation; and the empirical Logit Scale of Position, which measures the relative balance between left and right statements. Let Ls and Rs denote the absolute number of left and right statements of a political text within a fixed multi-category policy dimension, and let S denote the total number of statements. The different measures thus take the following form:
$$RIL{E_{Raw}} = {{Rs - Ls} \over S},\;RIL{E_{Conditional}} = {{Rs - Ls} \over {Rs + Ls}},\;Logit\;Scale = \log \left( {{{Rs + 0.5} \over {Ls + 0.5}}} \right).$$The theoretically calculated candidate positions were then linearly projected onto the left-right economic policy dimension, thereby obtaining candidate and voter stances in the same policy space. Table 1 presents the theoretically calculated candidate scores. Our main measure of proximity considerations were distance-minimizing votes, i.e., votes that minimize the distance between voters’ self-placement, and the theoretically calculated candidate scores on the economic left-right dimension.
Theoretical Candidate Scores and Their Projections on the 11-Point Economic Policy Dimension
RESULTS
We begin our analysis by discussing voters’ self-placement and candidate-placements on the CHES 2010 economic left-right scale, ranging from 0 (left) to 10 (right). Figure 1 presents the corresponding results. Participants placed themselves at the center-left, with an average score of 4.53 (median 4). The designated right-wing candidate R was on average placed at 6.89 (median 7), the center candidate M received an average placement of 4.33 (median 4), and the designated left-wing candidate L was on average placed at 3.47 (median 3). Comparing the average and median candidate-placements to the theoretical ones obtained through the Logit and RILE scales reveals a high degree of congruency between voters’ perceptions of the candidates and the theoretically calculated stances in the policy space.
Box-Plots for Candidates and Self-Placements on the CHES 2010 11-Point Left–Right Economic Policy Scale
Notches represent non-parametric estimates of 95% confidence intervals for the medians. ‘X’ marks the corresponding means with 95% confidence intervals.
Using the Logit Scale as our benchmark—which has the lowest mean squared error among the three scales we consider—placement of voters differed by 0.12 points (= |6.89 − 6.77|) for R and by 0.24 (= |3.47 − 3.23|) for L. The largest difference of 0.67 (= |4.33 − 5.00|) can be observed for M. Although speculative, one possible explanation is that, in times of polarized debates surrounding the presidency of Donald Trump, centrist positions might have been perceived as anti-incumbent and, therefore, more leftist than they actually were.
These results provide strong evidence that voters are endowed with the cognitive ability to convert unambiguous political text into reasonable numeric policy stances.
We next analyze whether voters are able to use this information to vote for the candidate closest to their own position. We calculate the distance between each participant and each candidate as being the absolute distance between her or his self-reported placement and the theoretical placement based on the Logit and RILE scales. Table 2 presents the absolute and relative frequencies of participants who voted for the candidate with minimum distance.
Absolute (N) and Relative Frequency (N%) of Participants Who Cast Distance-Minimizing Votes
Distance was calculated between voters' self-reported stances on the CHES left-right scale and the theoretically calculated Logit and Rile measures for the candidates. The ‘Expected’ relative frequency was calculated under the assumption of uniform-random behavior.
*** Signify significance of exact binomial tests on equality of observed and expected relative frequencies at the 1%-level (all p-values were adjusted according to Holm–Bonferroni).
To account for the possibility of errors, we compare these values to the expected relative frequencies obtained under uniform random behavior, i.e., random voting and self-placement behavior. Across all three treatments and all three scales, at least 70% of participants (Logit Scale, Baseline) cast distance minimizing votes ranging up to almost 79% (Raw RILE, Delay). Using an exact binomial test, we reject the null hypothesis of uniform random behavior for each treatment and each scale at the 1% level (applying the Holm–Bonferroni correction to account for family-wise error rates). These results validate the empirical relevance of proximity considerations in a purely text-based framing of issue positions, and generalize previous findings in the existing literature. Instructively, but also coincidentally, our estimate of proximity considerations is in line with the combined proximity and discounted proximity estimates—between which our design cannot discriminate—of Tomz and van Houweling (Reference Tomz and van Houweling2008). We also calculated the post-hoc achieved power for each statistical test reported in Table 1. Setting alpha at the conventional level of 5%, the smallest achieved post-hoc power over all tests was 98%. We also computed the a-priori required minimum sample size to detect the significance of our observed effect-sizes. We thereby set alpha and beta to the conventional levels of 5% and 20% (= 80% power), respectively. For all tests, actual sample sizes exceeded the required minimum sample sizes by a factor of at least 2.4 (i.e., actual sample size > 2.4 * required sample size).
We conclude our analysis with an investigation of antecedent factors of proximity considerations. So far, we have used cognitive ability as a catch-all phrase but have not discussed the potential mechanism relating cognitive ability to spatial preferences. One possibility is formalized in the theory of categorization-based spatial voting (Collins Reference Collins2011). This theory posits that voters categorize candidates and have preferences over categories. As political experience grows, voters build finer and more distinct categories; the finer the categories, the closer their preferences resemble proximity considerations.
This theory draws on well-established findings that show that cognitive abilities relating to categorization are not fixed and can be improved through practice, and, hence, experience. We consider two proxies for political experience: political platform membership and participation in previous elections. We calculate the average marginal effect of platform membership and previous participation on the probability of casting distance minimizing votes, based on probit estimations. In accordance with categorization-based spatial voting, platform membership and previous participation are associated with an 11.3 and 11.7 percentage point increase in the probability of casting distance minimizing votes (p-values are 0.015 and 0.064). However, this interpretation warrants caution: The observed associations cannot be interpreted as causal effects because voters with more consistent proximity preferences could also simply self-select into higher rates of political participation.Footnote 9 Nevertheless, we posit that our results are compatible with the view that political experience affects how voters judge the policy stances of candidates.
CONCLUSION
We build a bridge between experimental research on spatial voting and the literature on measuring policy positions to increase our confidence in the conclusions drawn from the former. We demonstrate the feasibility and fruitfulness of this approach via an internet survey experiment. Our experimental results generalize previous findings, showing that proximity considerations are empirically prevalent within a purely text-based framing of issue positions in the policy domain we study. We further identify political experience as a vital mechanism underlying proximity considerations.
Beyond our specific observations, we extend a call for future research to test the generalizability of experimental results. The route should be one of systematically relaxing theoretical assumptions to create a more realistic and ecologically valid testing design. Our experimental design is portable and adaptive, and serves as a blueprint for experimental research on spatial voting. By applying this design, candidates’ issue positions can easily be constructed on the spot and tailored to the needs of specific research questions. For example, as in Tomz and van Houweling (Reference Tomz and van Houweling2008), they can be constructed on the basis of voter input to tease out different forms of spatial considerations.
Extensions to multi-dimensional issue spaces or text ambiguity and uncertainty pose no difficulty as appropriate techniques are readily available in the vast literature on measuring policy positions (Benoit, Mikhaylov, and Laver Reference Benoit, Mikhaylov and Laver2009; Lowe et al. Reference Lowe, Benoit, Mikhaylov and Laver2011; Slapin and Proksch Reference Slapin and Proksch2008). It is even conceivable to extend the idea of measuring voters’ positions in a spatial-free context by allowing them to describe their policy stance in terms of pre-defined statements. We hope that the flexibility and enormous potential of our proposed methodological cross-over are self-evident and that it will inspire future research, across a variety of contexts, to close our knowledge-gap on how voters judge candidates and how they act upon these judgments.
SUPPLEMENTARY MATERIAL
To view supplementary material for this article, please visit https://doi.org/10.1017/S0003055419000492.
Replication materials can be found on Dataverse at: https://doi.org/10.7910/DVN/Z7P7RL.
Open access
Comments
No Comments have been published for this article.