Facts, Preferences, and Doctrine: An Empirical Analysis of Proportionality Judgment

Raanan Sulitzeanu-Kenan; Mordechai Kremnitzer; Sharon Alon

doi:10.1111/lasr.12203

Facts, Preferences, and Doctrine: An Empirical Analysis of Proportionality Judgment

Published online by Cambridge University Press: 01 January 2024

Raanan Sulitzeanu-Kenan ,

Mordechai Kremnitzer and

Sharon Alon

Article contents

Abstract
Proportionality Analysis
A Behavioural Analysis of Proportionality Judgments
Proportionality and Targeted Killings of Terrorist Operatives
Research Design
Results
Discussion
Supporting Information
Footnotes
References

Rights & Permissions

Abstract

Legal proportionality is one of the most important principles for adjudicating among conflicting values. However, rather little is known about the factors that play a role in the formation of proportionality judgments. This research presents the first empirical analysis in this regard, relying on a sample of 331 legal experts (lawyers and legal academics). The policy domain addressed by the experiment is the antiterrorist military practice of targeted killings, which has been the subject of a legal debate. Our experimental findings suggest that proportionality judgments are receptive to normatively relevant facts. We also find strong correlational evidence for the effect of ideological preferences on such judgments. These results are consistent for two proportionality doctrines. We suggest that proportionality judgment is anchored jointly in the experts’ policy preferences and the facts of the case. We outline the implications of the findings for the psychological and legal literature.

Type: Articles
Information: Law & Society Review , Volume 50 , Issue 2 , June 2016 , pp. 348 - 382

DOI: https://doi.org/10.1111/lasr.12203 [Opens in a new window]
Creative Commons: This is an open access article under the terms of the Creative Commons Attribution-NonCommercial-NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non-commercial and no modifications or adaptations are made.
Copyright: © 2016 The Authors. Published by Cambridge University Press for the Law and Society Association.

In recent decades, the legal principle of proportionality has become one of the most prominent constitutional principles for adjudicating among conflicting values (Reference BarakBarak 2011; Reference BeattyBeatty 2004; Reference Cohen-Eliya and PoratCohen-Eliya and Porat 2010, Reference Cohen-Eliya and Porat2014; Reference PetersenPetersen 2013; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008). As such, the concept of proportionality has been extensively studied in the legal and philosophical literature. Yet, while the socio-legal perspective suggests that law is constituted by “the discretionary decisions that give it meaning” (Reference Mather, Whittington, Kelemen and CaldeiraMather 2008: 691), rather little empirical knowledge has been accrued about the ways in which proportionality informs and shapes the decision-making process followed by legal experts – if at all. Do legal experts rigorously adhere to the structure of proportionality analysis or does the conceptual abstractness of this legal method leave a loophole for legally irrelevant preferences? To the best of our knowledge, this research provides the first set of experimental evidence in this regard.

Three research questions concerning the nature of proportionality judgments are addressed in this research. First, in any given case, a judgment should, by definition, be informed by the circumstances in which the conflict of values arises (henceforth, factual considerations; a change thereof – factual variation). However, no systematic empirical attempt has so far been undertaken to assess the extent to which proportionality judgments conform to this stipulation. Second, a growing number of studies have recently shown that various legal decisions are susceptible to legally irrelevant preferences. Following this line of inquiry, our second research question revolves around the impact of legally irrelevant preferences on proportionality judgments. We assess the validity of these effects and distinguish among two potential mechanisms that govern them. Third, some doctrinal variation in proportionality analysis has been identified across countries (Reference Cohen-Eliya and PoratCohen-Eliya and Porat 2010; Reference PetersenPetersen 2013; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008) and across legal domains (e.g., domestic law vs. international humanitarian law (IHL), see: Reference Cohen and ShanyCohen and Shany 2007; Reference MayMay 2013). The questions that are addressed in this regard are (1) whether some doctrine versions of the judgment-formation process elicit greater receptivity to factual considerations than others; and (2) whether such versions provide a better guard against normatively unwarranted effects of legally irrelevant preferences.

To probe these three sets of questions in light of empirical evidence, the present research utilizes a distinctive and robust experimental design which draws on a sample of 331 legal experts (lawyers and legal academics). The subject matter explored is the military practice of targeted killings of terrorist operatives (henceforth, targeted killings). This tactic is implemented routinely by a number of countries, and thus, the issue extends beyond the local context of this specific investigation.Footnote ¹ Our findings suggest that proportionality judgments regarding cases of targeted killings are receptive to normatively relevant factual information. Likewise, strong evidence emerged for the effect of ideological preferences on judgments, but no indication that information processing was systematically biased. These results suggest that the vague and value-laden nature of proportionality analysis may allow legal experts to be swayed by their value preferences, in addition to being guided by the facts of the case. These findings hold true for two alternative versions of proportionality (henceforth, doctrinal versions), as elaborated in what follows.

This article comprises seven sections. In section proportionality analysis, we briefly review the legal doctrine of proportionality. Section a behavioural analysis of proportionality judgments outlines a behavioral perspective regarding proportionality judgments, which gives rise to six hypotheses that were tested in the analyses. Section proportionality and targeted killings of terrorist operatives provides a brief review of the targeted killings policy. Section research design elaborates the experimental research design and statistical estimation adopted. Section results reports the research findings, and Section discussion recapitulates the findings and discusses their implications.

Proportionality Analysis

A key aim of legal systems is to foster the peaceful resolution of conflicts. In most democracies, conflicts in the public arena often involve highly revered yet competing values. Accordingly, their resolution entails balancing constitutional rights against other public interests, as well as weighing among contending rights – a challenging task by any standard. One of the most prominent methods of achieving such balance is proportionality analysis (Reference AlexyAlexy 2010; Reference BeattyBeatty 2004; Reference Kumm and GeorgeKumm 2007; Reference PetersenPetersen 2013; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008).

The origins of proportionality analysis can be traced back to nineteenth-century German law. Following World War II, this method has spread across Europe, including post-Communist states and Israel. Proportionality analysis has been absorbed into Commonwealth legal systems – Canada, South Africa, New Zealand and the UK – and is currently gaining ground in Central and South America as well. It has also been adopted in three central treaty-based multinational regimes: the European Union (EU), the European Convention on Human Rights (ECHR) and the World Trade Organization (WTO) (Reference BeattyBeatty 2004; Reference JacksonJackson 2004; Reference Kumm and GeorgeKumm 2007; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008: 74). Moreover, proportionality has become a central principle of international law, governing such critical issues as a country's right to self-defence, the legitimacy of war (ad bellum) and the legitimate use of military force (in bellum) (Reference HurkaHurka 2005).

In these contexts, the principle of proportionality is expected to guide policy makers (both legislators and executive officeholders) in evaluating policy alternatives, and judges, in adjudicating rights. Widely recognized by judicial authorities, proportionality has become a prominent decision-making principle in adjudicating between conflicting values and interests (Reference BeattyBeatty 2004; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008: 75).

Proportionality Analysis (henceforth, PA) is structured as a series of decision-making stages, of which the first two are preliminary, while the last three, putatively, constitute an integral part of the process itself (Reference AlexyAlexy 2010; Reference Kumm and GeorgeKumm 2007).

1. First preliminary stage: Infringement of Rights. In this stage, the analyst assesses whether or not the policy in question potentially entails infringements of constitutionally protected rights.Footnote ² Only if the answer is affirmative does the analysis ensue.
2. Second preliminary stage: Legitimate/Worthy Goal. In this stage, the analyst examines the legitimacy of the goal that the policy seeks to promoteFootnote ³ (henceforth, “worthy goal”).
3. PA Stage One: Suitability. The first integral decision stage of proportionality analysis (henceforth, “suitablity”) examines whether the means are rationally related to stated policy objectives, a.k.a. the worthy goal.
4. PA Stage Two: Necessity. The second integral decision stage, termed “necessity” or “least restrictive means” (LRM), examines whether the measure does not curtail a right any more than an available alternative policy equally capable of attaining the stated goal.
5. PA Stage Three: Balancing. The final stage of PA (henceforth, “balancing” or “strict-sense balancing”) is implemented if the policy action satisfies the previous tests; it assesses whether the benefits of the action justify the costs of rights infringement.Footnote ⁴

As is evident from the above guidelines, the four final stages of proportionality analysis cover both normative and empirical aspects of the decision (Reference Kumm and GeorgeKumm 2007: 137). The “worthy goal” and “balancing” stages invoke normative judgment, in that they exclude policy goals that are not aligned with human-rights protection and weigh out the infringement of a right against the benefit of adhering to the competing goal. The “suitability” and “necessity” stages pivot on empirical assessment: respectively, whether the given policy will benefit the goal designated as worthy in the preceding stage, and whether no equally effective policy is available that would mitigate the right infringement.Footnote ⁵

The rapid spread of proportionality analysis across legal systems and policy domains attests to its wide-ranging importance and appeal as a legal principle (Reference JacksonJackson 2015; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008). Its normative centrality to constitutional law is manifested by studies that regard proportionality as “a common grammar of global constitutionalism” (Reference Cohen-Eliya and PoratCohen-Eliya and Porat 2010: 263–4), “a generic constitutional law” (Reference LawLaw 2004), and even as an “ultimate rule of law” (Reference BeattyBeatty 2004). Other scholars, while acknowledging the centrality of proportionality in many jurisdictions, point to the variations in its meaning and application (Reference BomhoffBomhoff 2013; Reference Cohen-Eliya and PoratCohen-Eliya and Porat 2014).

Yet, despite its ubiquity and centrality, the factors that shape proportionality analysis, at the level of the individual legal expert, have not yet been empirically studied. Given the prominence of proportionality, understanding the factors and processes influencing legal experts in forming proportionality judgments holds potentially important normative and political implications in many countries and for a variety of policy domains. The following section elaborates this issue, which is encapsulated in the research questions of the present study.

A Behavioural Analysis of Proportionality Judgments

Although proportionality is well established as a legal principle, not much is known about the actual cognitive processes involved in legal decisions, when proportionality analysis is ostensibly applied. While PA offers an appealing model for adjudicating among competing values, it presents a number of challenges in practice. The very conceptual abstractness of PA, which makes it so widely applicable, produces unpredictable and inconsistent rulings on the part of different adjudicators and across cases.Footnote ⁶ In part, this is because the application of PA involves empirical assessments as well as value judgments, and thus requires expertise and experience.

To diminish the lacuna in the empirical investigation of decision-making processes based on PA, our study addresses three research questions. First, proportionality judgments must, by definition, rest on factual considerations that are relevant to the conflicting values in question. The proportionality doctrine assigns positive utility to a “worthy goal.” The higher the policy's expected marginal contribution to obtaining the “worthy goal,” the higher positive utility will be attributed to that policy. Conversely, the doctrine assigns negative utility to infringement of rights. This negative utility is also attributed to the policy, and it increases with the magnitude/severity of rights-infringement potential entailed by the policy (For further details, see: Reference AlexyAlexy 2010; Reference PetersenPetersen 2013; Reference VeelVeel 2010). Thus, our first research question is whether proportionality judgments are receptive to factual considerations which are expected to guide them normatively. Based on the legal theory of proportionality, the following two hypotheses can be set forth in this regard:

H₁: The probability of judging a policy as proportionalFootnote ⁷ rises as the worth of the policy goal, and the extent to which it is expected to be attained, increases, in the sense that its implementation contributes to greater marginal public welfare, in the normative sense.Footnote ⁸

H₂: The probability of judging a policy as proportional increases as the potential for rights infringement resulting from implementing that policy diminishes.

The second research question is prompted by studies that point to the impact of legally irrelevant considerations on various legal decisions, through motivated cognition (Reference Furgeson, Babcock and ShaneFurgeson, Babcock, and Shane 2008; Reference KundaKunda 1990; Reference SoodSood 2013) or implicit biases (Reference Gazal-Ayal and Sulitzeanu-KenanGazal-Ayal and Sulitzeanu-Kenan 2010; Reference RachlinskiRachlinski et al. 2008). A tension between the drive for accuracy and belief perseverance underlies all human reasoning (Reference Lodge and TaberLodge and Taber 2013: 150).Footnote ⁹ In the context of legal reasoning it has been suggested that “[p]olicy preferences and legal reasoning may be so cognitively intertwined that lawyers and judges have difficulty fully realizing what factors have influenced their conclusions” (Reference Furgeson, Babcock and ShaneFurgeson, Babcock, and Shane 2008: 226). This argument is particularly likely to apply in vague and highly abstract judgment tasks (Reference Bertrand, Chugh and MullainathanBertrand, Chugh, and Mullainathan 2005). As proportionality analysis represents a legal standard, in the sense of being open-ended, vague, and allowing fact-specific determination (Reference SunsteinSunstein 1995; Reference KaplowKaplow 1992), its implementation is more likely to be affected by motivational goals such as policy preferences. Policy preferences may operate via one or both of the following mechanisms: (1) a direct effect of the policy preference on the final judgment; and (2) through a process known in the motivated-cognition literature as “biased processing of information” – that is, when information is processed differently depending on its consistency with existing preference, in making a judgment (Reference Ditto and LopezDitto and Lopez 1992; Reference Jain and MaheswaranJain and Maheswaran 2000; Reference SoodSood 2013; Reference Taber and LodgeTaber and Lodge 2006). Given that proportionality analysis involves two main conflicting values – “worthy goal” and “rights infringements” – we expect the adjudicator's legally irrelevant preferences to affect the respective weight assigned to these considerations. To shed light on the judgment-formation process, both the above mechanisms are tested in this study. The above discussion yields two hypotheses:

H₃: The probability of judging a policy as proportional is expected to be influenced by legally irrelevant policy preferences.Footnote ¹⁰

H₄: The effect of legally irrelevant policy preferences moderates the weight assigned by the decision-maker to the information pertaining to the “worthy goal” and potential rights infringement factors.

Proportionality analysis has been shown to vary across countries (Reference Cohen-Eliya and PoratCohen-Eliya and Porat 2010; Reference PetersenPetersen 2013; Reference Stone Sweet and MathewsStone Sweet and Mathews 2008) and legal domains (e.g., domestic law vs. IHL, see: Reference Cohen and ShanyCohen and Shany 2007; Reference MayMay 2013). This doctrinal variation pertains mainly to the set of decision stages the analysis takes into account. The question that arises in this connection is whether some versions of PA facilitate greater sensitivity to normatively relevant factual considerations than others, thus, potentially improving the balance between protected rights and public interests, which is the main goal of proportionality judgments. Specifically, is a more comprehensive PA version, that is, one that accounts for the full set of decision stages (henceforth, “full PA”), more receptive to normatively relevant factual considerations, compared with a less comprehensive version e.g., “strict-sense balancing”?

H₅: Proportionality decisions are more receptive to (i.e., more strongly affected by) factual variation under a comprehensive proportionality analysis, compared with less comprehensive versions (strict-sense balancing).

A legal doctrine is expected to provide lawyers with normative criteria to form consistent and unbiased judgments. If proportionality judgments are systematically swayed by legally irrelevant preferences, it is important to assess whether such biases vary across different doctrinal choices. While motivated-cognition processes are robust vis-à-vis variation in cognitive elaboration (Reference Taber, Cann and KucsovaTaber, Cann, and Kucsova 2009), Implicit bias has been shown to characterize cases of task ambiguity (Reference Bertrand, Chugh and MullainathanBertrand, Chugh, and Mullainathan 2005; Reference Gazal-Ayal and Sulitzeanu-KenanGazal-Ayal and Sulitzeanu-Kenan 2010). Relying on this latter mechanism, we expect policy preferences to have a greater impact under less comprehensive versions of proportionality analysis, such as “strict-sense balancing,” than under a “full PA.” Hence the following hypothesis:

H₆: The effect of legally irrelevant policy preferences is more pronounced under “strict-sense balancing,” and less so under a “full PA.”

This research assesses the effects of different doctrinal versions on the judgment outcome, with regard to its receptivity to normatively factual considerations and susceptibility to the adjudicator's legally irrelevant preferences. To this end, it capitalizes on the Israeli Court of Justice's application of a comprehensive version of the proportionality doctrine in the cases of targeted killings (Reference Cohen and ShanyCohen and Shany 2007). The following section reviews this policy and the application of proportionality in cases of targeted killings.

Proportionality and Targeted Killings of Terrorist Operatives

Targeted killings of terrorist operatives (or preemptive strikes) involve the military use of lethal force against suspected individual terrorists, as part of counter-terrorism activities performed by a state outside its territory (for a review of the various definitions of targeted killings, see: Reference BarnidgeBarnidge and Robert 2012: footnote 9). In recent years, this practice has gained considerable ground, bringing the rules of its implementation under the scrutiny of judicial and oversight bodies, both domestic and international (Reference BarnidgeBarnidge and Robert 2012; Reference BowcottBowcott 2012; Reference MayMay 2013).

Targeted killings can be viewed in light of two legal models of justified killing. This issue, which has been traditionally addressed mainly by IHL, has recently been debated within the domestic framework of law enforcement (Reference MayMay 2013). Although the term “proportionality” itself does not explicitly appear in any of the IHL treaties, it is broadly understood to constitute an important principle in the laws of war. Thus, this principle is reflected in Articles 51(5(b) and 57(2) of the First Additional Protocol of 1977 to the Geneva Conventions of 1949.Footnote ¹¹ The 1977 Protocol was ratified by many states and, in international law, is now widely regarded as customary and thus binding for nonsignatory states as well. The proportionality principle as framed in the 1977 Protocol has traditionally been understood to include only the third PA stage – that is, the “strict-sense balancing” (Reference HigginsHiggins 1994: Reference MayMay 2013).

However, in December 2006, the Israeli Supreme Court (sitting as a High Court of Justice – HCJ) delivered one of the most comprehensive judicial decisions on the international rules governing targeted killings – the Public Committee Against Torture in Israel vs. The Government of Israel (henceforth, the Targeted Killings case).Footnote ¹² The main judgment was delivered by Justice Aaron Barak, the departing President of the Supreme Court. Although Barak contended that, under IHL, the third, “strict-sense” PA stage appears to be the most relevant to assessing the lawfulness of targeting, he did not rule out the relevance of other proportionality decision stages. Barak specified that a military action must conform to all three proportionality requirements (suitability, necessity and balancing), effectively harmonizing the application of PA in both IHL and Israeli domestic Law (Reference Cohen and ShanyCohen and Shany 2007).Footnote ¹³

When dealing with targeted killings, the first PA stage (suitability) requires that this strategy rationally lead to the desired military objective. The second PA stage (necessity) is centered on whether a targeted killing constitutes the least harmful way of neutralizing the risk. Less aggressive alternatives may include capture, subsequently leading to detention, interrogation and trial (Reference Cohen and ShanyCohen and Shany 2007). The third PA stage (“strict-sense balancing”) weighs out the expected military benefit against the expected collateral humanitarian harm of the targeted killing (Targeted Killing §46). This view on the part of the Israeli HCJ also resonates with the International Committee of the Red Cross's Interpretive Guidance on Direct Participation in Hostilities (Reference MayMay 2013: 57).

The 2006 Targeted Killings decision extends the applicability of proportionality to targeted killings in Israeli law. While IHL requires only that the operation should satisfy the “strict-sense balancing” requirement, the HCJ decision requires compliance with all three decision stages as well as meeting the preliminary “worthy goal” condition. However, much like other legal decisions, the Targeted Killing decision offers but limited guidance regarding the practical application of PA in targeted killings (Reference Cohen and ShanyCohen and Shany 2007). Thus, it is not clear what weight should be assigned to each of the competing values involved, nor how should one consider the level of predictive uncertainty regarding the military and humanitarian results of the attack. This vagueness necessitates an empirical assessment as to whether and to what extent the proportionality doctrine informs judgments. The following section describes and accounts for the research design that was used for this purpose.

Research Design

We conducted a survey-embedded experiment involving 331 Israeli legal experts between June and July 2013. Sampling was performed in two stages, the first with 166 Master of Law students in five universities,Footnote ¹⁴ while the second – with 165 legal experts who work in the public and private sectorsFootnote ¹⁵ and who completed an online version of the questionnaires assigned to the students in the first stage. Following a set of analyses which confirmed the causal homogeneity of the two subsamples, these were mergedFootnote ¹⁶ into a total sample of 331 lawyers. Still, to account for any potential variation that is due to the sampling method, all the analyses include a dummy variable (“online”) that has the value 1 for respondents who completed an online questionnaire, and 0 otherwise. Table 1 presents descriptive statistics for the sample.

Table 1. Sample Descriptive Statistics

Experimental Procedure

Respondents (N = 331) were randomly assigned to one of sixteen descriptions of a proposed military plan for a targeted killing, and then were asked to evaluate the proportionality of the plan using their legal expertise. Vignette survey experiments are widely used in social sciences, and were found to have strong external validity in predicting behaviour of both citizens (Reference Hainmueller, Hangartner and YamamotoHainmueller, Hangartner, and Yamamoto 2015) and professionals (Reference Peabody, Luck, Glassman, Jain, Hansen, Spell and LeePeabody et al. 2004). The versions of the military plan differed with regard to the operational goal and to the attendant infringement of rights. Overall, we used four “goal treatments” and four “rights infringement treatments,” which yielded a four by four, between-subjects, fully crossed factorial design. Table 2 presents the two sets of vignettes that were used to construct the 16 experimental conditions. Each condition combines one of the “goal treatments” and one of the “rights infringement treatments.”

Table 2. Experimental Treatments

Note: The 16 experimental conditions are based on all the possible pairings of these variations of the “goal” and “rights infringement” parameters.

The “goal treatments” were as follows: G1 – a punitive goal; G2 – a preventive goal; G3 – prevention of a more severe harm; and G4 – prevention of a more severe harm by targeting more senior terrorist operatives. The “rights infringement treatments” were as follows: R1 – targeting terrorists who are in a car in an urban area of Gaza, without mentioning the option to apprehend them; R2 – the same plan preceded by a remark that apprehending the terrorists is not a feasible option; R3 – identical to R2, except the targeted car is in a nonurban area, and therefore, the plan presents a lesser risk to uninvolved bystanders; R4 – identical to R3, except the car is supplanted by a motorcycle, thus reducing the risk of uninvolved passengers being inside the target.Footnote ¹⁷

Measuring Proportionality Judgments

The dependent variable – the proportionality judgment – was measured using a set of five questions. Four questions covered the four decision stages: (1) Is the goal of the military plan worthy (yes/no)? (2) Is the military plan adequate and effective for achieving the goal (yes/no)? (3) Is the military plan a practical means to achieve the goal with minimal infringements of human rights (yes/no)? (4) Is the proportion between the advantage to be gained by implementing the military plan and the expected infringement of human rights adequate (yes/no)? Since this study constitutes the first attempt to measure proportionality judgments empirically, we conducted a series of validation analyses for the measurement items, which are reported in detail in the online appendix.

The fifth question was prompted by the above-discussed doctrinal ambiguity of proportionality analysis and was intended to elicit an unstructured proportionality judgment (henceforth, “free PA”). Respondents were asked to indicate their judgment regarding the proportionality of the proposed plan on a scale of 1 (very disproportional) to 6 (very proportional). Respondents were informed that the boundary between un-proportionality and proportionality lay between 3 and 4 on the scale.

Responses to these five questions yielded three binary measures of proportionality judgments for each respondent, as displayed in Figure 1. The first measure is proportionality in the “strict sense,” as prescribed by IHL. It assigns the value of 1 to the affirmative answer to the forth item and 0 otherwise. The second measure is a “full PA,” as required by the Targeted Killing decision. This measure assigns the value of 1 to affirmative answers to all four items, and 0 otherwise. The third measure is a binary version of the response to the “free PA” item; it assigns the value of 1 to a score between 4 and 6, and 0 otherwise. Thus, respondents were presented with legal questions they expected a proportionality analysis to address, while we obtained results regarding both PA versions, the comprehensive (“full PA”) and the less comprehensive one (“strict-sense balancing”).

Figure 1. Question wording, and three measures of proportionality judgments.

As the three measures of proportionality judgment yielded an internally consistent index (alpha = .866), they were treated as three repeated proportionality measures for each respondent. The analysis controlled for the type of measure, and thus provided estimates for the effects of the doctrinal version (“strict-sense balancing,” “full PA,” and “free PA”) on the judgment. Further details are provided in the Estimation strategy section.

Measuring Ideological Preference

The effect of respondents’ legally irrelevant preferences on proportionality judgments were estimated based on their self-reported political ideology. Political ideology serves as a simplifying heuristics that helps individuals interpret the world, by “making assertions or assumptions about human nature, historical events, present realities, and future possibilities” (Reference Jost, Federico and NapierJost, Federico, and Napier 2009; Reference Ball and DaggerBall and Dagger 2010; Reference Erikson and TedinErikson and Tedin 2010; Reference Ball and DaggerBall and Dagger 2010). Indeed ideology is the most important spatial dimension guiding vote choice (Reference Duch, May and Ii ArmstrongDuch, May, and Armstrong 2010) and policy preference (Reference RingeRinge 2005; Reference Sulitzeanu-Kenan and HalperinSulitzeanu-Kenan and Halperin 2013). In Israel, the central domain aligning right- and left-wing supporters is the hawk versus dove approach to the Arab-Israeli conflict (Reference Benoit and LaverBenoit and Laver 2006; Reference Shamir and ArianShamir and Arian 1999). If proportionality judgments are indeed susceptible to legally irrelevant preferences (H3), hawks are expected to be more likely than doves to judge military plans for targeted killings in the context of this conflict as proportional (and vice versa). Each respondent's political ideology relative to the hawk-dove continuum was assessed using a single-item measure, in which respondents were asked to indicate their political position on a scale of 1 (extreme right) to 5 (extreme left). For each respondent we also recorded a set of covariates which included age, gender, and religiosity.Footnote ¹⁸

Estimation Strategy

To identify the effects of factual variation in the “worthy goal” and “rights infringements” parameters, respondents were randomly allocated to 16 experimental conditions. Supporting Information Table A4 in the online appendix verifies that the randomization process was successful, by demonstrating that practically all individual-level variables are balanced across the experimental treatments. As the three proportionality measures for each respondent turned out to be internally consistent, the data were structured as a repeated measures dataset of 3 × 331 = 993 observations.

Equation (1) presents our basic method for estimating proportionality judgments; it accounts for the factual variation, the doctrinal versions, and individual-level characteristics:

(1)

L o g i t [E (Y_{i j} = 1 | {(K, D, X)}_{i j}] = {α K}_{i j} + {β D}_{i j} + {γ X}_{i j}

where Y is a binomial variable that represents a judgment that the plan is proportional [Y = 1] or not [Y = 0], by respondent i, based on proportionality type j, conditional on a vector of factual variation K, a vector of doctrine alternatives D and a vector of individual characteristics X. We estimate the effects of factual variation on proportionality judgment adjusting for multiple individual-level covariates, using generalized estimating equations (Reference Liang and ZegerLiang and Zeger 1986). Generalized estimating equation is a method developed for handling clustered data, in which the observations within each cluster are intercorrelated. Given that each set of three proportionality judgments are clustered within respondent, we used generalized estimating equations to account for the correlations among observations. To address within-subject correlations, we cluster standard errors within respondent. To obtain detailed estimates for the “goal” and “rights infringement” treatments, doctrine alternatives, and ideological differences, we estimate proportionality judgments using the following elaboration of (1):

(2)

L o g i t {(Y)}_{i j} = α_{0} + α_{1} {P r e v e n t i v e (g 2)}_{i} + {α_{2} S e v e r i t y (g 3)}_{i} + α_{3} S e n i o r {(g 4)}_{i} + α_{4} L R M {(r 2)}_{i} + α_{5} n o n u n r b a n {(r 3)}_{i} + α_{6} m o t o r c y c l e {(r 4)}_{i} + α_{7} {F u l l P A}_{j} + {α_{8} F r e e P A}_{j} + α_{9} {I d e o l o g y}_{i} + α_{10} {f e m a l e}_{i} + α_{11} {a g e}_{i} + α_{12} {r e l i g .}_{i}

where the first six independent variables comprise two sets of three indicator variables, each set representing the four conditions with regard to the operation goal (with “punitive goal” as reference) and the potential infringement of rights (with the plan with most rights-infringement potential as reference), respectively. $F u l l P A$ and $F r e e P A$ are indicator variables for the type of proportionality analysis (“strict-sense” as reference). Next are individual-level characteristics (hawk-dove ideology, gender, age, and religiosity).

To evaluate the impact of factual variation on proportionality judgments as per Hypotheses 1 and 2, we first need to define the minimal set of empirical expectations required to satisfy these hypotheses. As noted above, the proportionality doctrine assigns positive utility to a “worthy goal” and negative utility to infringement of rights. However, it does not specify the relative weight of the “goal” and “rights infringement” factors, nor does it stipulate the impact of changes in these factors on utility (Reference AleinikoffAleinikoff 1987; Reference PetersenPetersen 2013; Reference TsakyrakisTsakyrakis 2009; Reference WebberWebber 2010). This particular vagueness poses a problem in stating clear empirical expectations regarding behaviors that are congruent with this doctrine and those that are not. Despite this vagueness, we can specify certain restrictions on the pattern of judgment which would qualify the doctrinal guidance. Such a pattern must be consistent with the dominance principle (Reference Hadar and RussellHadar and Russell 1969; Reference Kariv and SilvermanSee also: Kariv and Silverman 2013), in the sense that a policy should be preferred to another if it yields unambiguously greater utility (i.e., increases the share obtained from the goal, or decreases the potential for rights infringement). This entails that, when comparing proportionality judgments across such ordered set of policies, the pattern of judgment must be monotonic.Footnote ¹⁹ Only under this condition can we conclude that it conforms to a certain proportionality doctrine. Implementing this restriction in our analysis suggests that Hypotheses 1 and 2 are supported only if coefficients $α_{1}$ – $α_{3}$ , and $α_{4}$ – $α_{6}$ , respectively, have positive, and monotonically increasing values. This means that, ceteris paribus, the probability of a plan being judged as proportional should never decrease when moving to a scenario with a more worthy goal or with a lesser potential for rights infringements.Footnote ²⁰

The effect of ideological (hawk-dove) preferences on proportionality judgments is estimated in two ways. First, in order to test Hypothesis 3, we simply estimate the size and statistical significance of coefficient $α_{9}$ in (2). If such an association is found, we proceed to determine the mechanisms through which these preferences shape proportionality judgments. One possible account is that one's prior propensity for judging targeted killings as proportional depends on one's ideology. The second is that one's ideological preference influences the weight assigned to different factual considerations, thereby affecting information processing, as suggested in the motivated-cognition literature. This indirect effect of ideology on proportionality judgments, as per Hypothesis 4, can be tested by comparing the effect of the different treatments employed on hawks versus doves. For this purpose, we add to (2) interaction terms for ideological groups (hawks/doves) and the treatments within the “goal” and “rights infringements” factors:Footnote ²¹

(3)

Logit {(Y)}_{i j} = α_{0} + α_{1} {Goal}_{i} + α_{2} {Rights_infringe .}_{i} + α_{3} {Full P A}_{j} + α_{4} {Free P A}_{j} + α_{5} {Hawk \times Goal}_{i} + α_{6} {Hawk \times Rights_infringe .}_{i} + α_{7} {Hawk}_{i} + α_{8} {female}_{i} + α_{9} {age}_{i} + α_{10} {relig .}_{i}^{21}

These interaction terms allow us to test Hypothesis 4 by estimating the differences between the receptivity of proportionality judgments to factual variation across ideological groups.Footnote ²²

To assess the implication of the doctrine version on proportionality judgments, we first assess the size and significance of coefficients $α_{7}$ and $α_{8}$ in (2). Next, to test whether the doctrine version affects the level of receptivity to factual variation (Hypothesis 5), we add to (2) interaction terms for the doctrine version and ideological preference:

(4)

L o g i t {(Y)}_{i j} = α_{0} + α_{1} {G o a l}_{i} + α_{2} {R i g h t s_i n f r i n g e .}_{i} + α_{3} {F u l l P A}_{j} + α_{4} {F r e e P A}_{j} + α_{5} {F u l l P A}_{j} \times {G o a l}_{i} + α_{6} {F u l l P A}_{j} \times {R i g h t s_i n f r i n g e .}_{i} + α_{7} {F r e e P A}_{j} \times {G o a l}_{i} + α_{8} {F r e e P A}_{j} \times {R i g h t s_i n f r i n g e .}_{i} + α_{9} {I d e o l o g y}_{i} + α_{10} {f e m a l e}_{i} + α_{11} {a g e}_{i} + α_{12} {r e l i g .}_{i}

Lastly, to test whether the doctrine version affects the level of susceptibility to legally irrelevant preferences (Hypothesis 6), we add to (2) interaction terms for the doctrine version and ideological preference:

(5)

Logit {(Y)}_{i j} = α_{0} + α_{1} {G o a l}_{i} + α_{2} {R i g h t s_i n f r i n g e .}_{i} + α_{3} {F u l l P A}_{j} + α_{4} {F r e e P A}_{j} + α_{5} {F u l l P A}_{j} \times {I d e o l o g y}_{i} + α_{6} {F r e e P A}_{j} \times {I d e o l o g y}_{i} + α_{7} {I d e o l o g y}_{i} + α_{8} {f e m a l e}_{i} + α_{9} {a g e}_{i} + α_{10} {r e l i g .}_{i}

Results

Receptivity of Proportionality Judgments to Normatively Relevant Factual Variation

We estimate the effects of factual considerations on proportionality judgments by fitting to our data. Detailed results are reported in Table 3 and presented graphically in Figure 2B below.

Figure 2. The effects of “goal” and “rights infringement” treatments on the probability of “proportional” judgment (95% CI). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Table 3. Generalized Estimating Equations predicting Proportionality Judgments

Note: The generalized estimating equations employ logit link function and unstructured correlation structure. Coefficients are reported in logit values. Statistical significance levels are represented as follows: + p < 0.1, *p < 0.05, **p < 0.01, ***p < 0.001. “Goal treatments” is a four-scale ordinal variable representing the four “goal” treatments; “Preventive attack,” “Severe attack” and “Senior terrorists” are indicator variables for the “goal” treatments, with “punitive attack” serving as a reference category; “Rights infringement treatments” is a four-scale ordinal variable representing the four “rights infringement” treatments (note that a higher value indicates less potential for rights infringements); Least restrictive means”, “Car in nonurban area” and “Motorcycle in nonurban area” are indicator variables for the “rights infringement” treatments, with “no mention of alternative” serving as a reference category; “Hawk-dove” is a five-scale ordinal variable with 1 representing “extreme hawk” and 5 representing “extreme dove”; “Hawk” is an indicator variable for respondents who identified as either “extreme hawk,” “hawk” or “center” (“dove” and “extreme dove” as reference); “Gender” is an indicator variable representing a female respondent (male as reference); “Religiosity” is a four-scale ordinal variable with 1 representing “ultraorthodox” and 4 representing “secular”; “Online sample” is an indicator representing respondents who participated in the online sample (pen and paper sample as reference); “Full PA” and “Free PA” are indicator variables, with “strict sense” as a reference category.

Figure 2A displays the average probability of judging the operation as proportional for each treatment of the “goal” factor,Footnote ²³ controlling for all other variables, which are set at their respective mean values.Footnote ²⁴ The overall effect of “goal” treatments was statistically significant (χ ² =12.51, p = 0.005) based on post estimation Wald test. As shown in Figure 2A, the probability that the operational plan would be judged as proportional increases with the rise in the worthiness of the goal (moving from left to right on the X-axis). The overall effect of the “rights infringements” treatments are likewise statistically significant (χ²= 37.53, p < 0.0001) based on post estimation Wald Test. As per Figure 2B, the probability of an operation being judged as proportional increases with the decrease in the anticipated rights infringement. Note that both of these trends are monotonic, in that none of the treatments is associated with a reverse trend.Footnote ²⁵ These finding align with the empirical expectations regarding Hypotheses 1 and 2, which are presented in the Estimation strategy section, suggesting that proportionality judgments are receptive to doctrinally relevant factual variation.

Although, overall, the susceptibility of proportionality judgments with factual variations in the “goal” and “rights infringement” factors conformed to our expectations, some specific treatments within these two factors do not appear to affect the judgment at a statistically significant level. The difference between a punitive and a preventive operation resulted in a significant increase of 20.0 percentage points (p = 0.012) in the probability of judging the operation as proportional. Conversely, the effect of implying that the expected attack (to be averted by the operation) is likely to be severe was −1.2 percentage points and statistically insignificant (p = 0.986). Likewise, the marginal effect of stipulating that the operation targets a senior and important terrorist operative resulted in an increase of 7.6 percentage points, which is statistically insignificant (p = 0.302).Footnote ²⁶ With regard to the influence of the “rights infringement” factor on the probability that the operation would be judged as proportional, a putative lack of a nonlethal alternative to the operation had a marginal effect of 13.6 percentage points, which is marginally significant (p = 0.10). The reduction in the risk to uninvolved bystanders resulting from locating the targeted car in a nonurban area had a significant effect of 23.7 percentage points (p = 0.002). Finally, supplanting a car by a motorcycle and thereby increasing the precision of the attack, yielded an insignificant marginal effect of 1.9 percentage points (p = 0.476). The specific treatments that did not result in a statistically significant effect may be accounted for by the ineffective phrasing of a particular treatmentFootnote ²⁷ and/or sample size.

Inasmuch as both factors produced a significant overall effect and clearly monotonic patters, the lack of a significant effect for some specific treatments does not detract from the validity of the results, which suggest that proportionality judgments appear to be receptive to factual variation in line with doctrinal expectations. The specific treatments that did not result in a statistically significant effect may be accounted for by ineffective phrasing of a particular treatment and/or sample size. Notwithstanding these potential methodological reasons, comparing the set of treatments that were found to affect proportionality judgements and those that did not, suggests that respondents make a discrete distinction between a punitive and preventive goals but do not appear to be affected by the size of the preventable harm in weighing the worthiness of the goal. Respondents also appear to care whether a nonlethal alternative is available, and the level of risk to bystanders. However, once risk to bystanders was reduced, no indication was found that the selectiveness of the operation (i.e., the level of certainty of targeting only the suspected terrorists and not individuals accompanying them) was consequential to respondents’ decisions. While the latter considerations might be limited to professionals versed in the tactical nuances of such operations, the former null finding regarding the facts of the operational goal – expected severity of attack to be averted and the seniority of the targeted terrorists – is more surprising, as they (especially preventable severity) pertain to a central aspect to be considered in proportionality analysis, i.e. the importance of the operation. We discuss these results further in the discussion section.

The existing literature on proportionality, and particularly those studies that formally specify the theory (Reference AlexyAlexy 2010; Reference PetersenPetersen 2013; Reference VeelVeel 2010) implicitly suggest that proportionality judgment relies on the additive effects of the two conflicting values. However, as our interest in this study is in the characteristics of the actual way in which legal experts conduct the proportionality analysis, we performed a further analysis that assesses whether the weight given to either of the two values is influenced by changes in the other conflicting value.Footnote ²⁸

To estimate these possible interactions between the two factors we fitted a model that adds all the possible interactions between the levels of the two factors (“worthy goal” and “rights infringement”) to (2). Detailed results are reported in the left column of Supporting Information Table A5 in the online appendix. The results do not support an interaction model. None of the specific interaction terms were statistically significantly. Contrasts of the marginal linear predictions based on this model resulted in two main effects – for the “worthy goal” (χ ² =12.73, p = 0.005) and the “rights infringement” (χ ² =37.76, p < 0.001) – and an insignificant interaction effect (χ ² =5.76, p = 0.764). Given that some specific treatments within the two factors were not found to affect the proportionality judgments at a statistically significant level, we fitted a further model, that compares condition G1 to the three other levels of the “worthy goal” factor, collapses conditions R3 and R4 of the “rights infringement” factor, and includes all the possible interactions between these levels of the two factors. Detailed results are reported in the column titled “Reduced Interaction Model” in Supporting Information Table A5 in the online appendix. The substantive findings for this model are similar to the full interaction model. None of the specific interaction terms are statistically significantly, and contrasts of the marginal linear predictions resulted in two main effects – for the “worthy goal” (χ ² =10.84, p = 0.001) and the “rights infringement” (χ ² =31.84, p < 0.001) – and an insignificant interaction effect (χ ² =.33, p = 0.848). These results suggest that the pattern of proportionality judgments in response to the experimental treatments is consistent with an additive conception of proportionality analysis.

The effects of the “worthy goal” and “rights infringement” factors on proportionality judgments are concurrent and integrated, as is demonstrated in Figure 3, which maps the factual variation into proportionality judgments in a three-dimensional graph. The two horizontal axes represent the “goal“ and “rights infringements” treatments, and the vertical axis represents the probability of the plan being judged as proportional. Thus, the figure provides a comprehensive visual representation of the receptivity of proportionality judgments to factual variation.

Figure 3. A three-dimensional mapping of normatively relevant factual variations into proportionality judgments. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

The Effects of Ideology on Proportionality Judgments

Figure 4 is based on (2) with a minor specification change: the ordinal ideology variable is treated as categorical rather than ordinal. It presents the mean probability of the operation being judged as proportional for respondents with different positions on the hawk-dove ideological scale, controlling for all other variables (which are set at their respective mean values). This analysis clearly shows that proportionality judgments are strongly associated with ideology (χ² = 62.26, p < 0.0001). Even if we ignore the two extreme categories on both sides of the ideological spectrum (jointly constituting 8.8% of the sample), the mean difference in the probability of judging a given operation as proportional decreases from 90.9% for “hawks,” to 78.1% for “center” (χ² = 9.90, p = 0.007), and to 52.1% for “doves” (χ² = 17.21, p < 0.001) – both differences being sizable and statistically significant.Footnote ²⁹ These findings provide support for Hypothesis 3.

Figure 4. Mean probability of “proportional” judgment across ideological positions (with 95% CI) Note: only 8 respondents occupy the “extreme hawk” category, all of whom judged their respective scenario as proportional (under the three different proportionality formulae). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

As our results indicated an association between ideology and proportionality judgment, we proceeded to explore the mechanism for this influence, as per Hypothesis 4. To this end we fitted (3) to our data. This model includes interaction terms which allow us to estimate the difference in treatment effects for hawks versus doves. Accordingly, we split our sample into two ideological groups: “hawks” (extreme hawks, hawks, center; N = 200) and “doves” (doves, extreme doves; N =127).Footnote ³⁰ For simplicity, we replaced the two sets of three dummy variables with two ordinal variables – “goal” and “rights infringements.” The full analysis is presented in Table 3. Both interaction terms turned out to be statistically insignificant (Hawk × Goal: z = −.23, p = 0.821; Hawk × Rights infringements: z = −.93, p = 0.353), providing no support to Hypothesis 4. Put differently, there is no indication that the factual variation had systematically different effects on the judgments of hawks versus doves.Footnote ³¹

Proportionality Judgments vis-à-vis Doctrinal Versions

Figure 5 is based on (2). It presents the expected probability of the operational plan being judged as proportional for the three proportionality types: “strict-sense” (the IHL rule); “full PA,” involving the “worthy goal,” “suitability,” “necessity” (LRM) and “strict-sense balancing” stages (the Targeted Killing decision); and “free PA” – the unstructured judgment. Figure 5 shows distinct differences in the probability of judging the operation as proportional under the three different proportionality formulae (χ² = 67.45, p < 0.0001). The expected probability under the “strict-sense” alternative, with all other variables set at their mean values, is 75.9%. Applying the “full PA” alternative resulted in a reduction of 24.0 percentage points (z = −8.23, p < 0.001), yielding the probability of 45.2%. The probability under the “free PA” alternative was 80.3%, significantly higher than under “full PA” (z = 9.06, p < 0.001), and 4.4 percentage points higher than under the “strict-sense” decision (z = 2.11, p = 0.035).

Figure 5. Mean probability of “proportional” judgment across proportionality formulae (with 95% CI). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Thus, although the unstructured judgment (“free PA”) was solicited last, after respondents had replied to the entire set of questions regarding all the considerations involved in the decision, it was on average more similar to the one under the “strict- sense” rather than under “full PA” alternative. The sizable effect of applying “full PA” did not come as a surprise. It is reasonable to expect that the requirement that the proposed plan should clear all the decision stages would reduce the probability of approval.

To test whether a more comprehensive proportionality analysis increases the receptivity of judgements to normatively relevant factual variation (Hypothesis 5), we fitted (4) to our data. This analysis, reported in Table 3, allowed us to estimate the differences in the effect of factual variation in the “goal” and “rights infringement” parameters under each doctrinal version (“strict-sense balancing,” “full PA,” and “free PA”). All four interaction terms are negative, suggesting that the receptivity of judgments to factual variation under “full PA” and “free PA” is lower than under the “strict-sense balancing.” However, of the four interaction terms, only the “Free PA” X Goal is statistically significant (z = −2.02, p = 0.044). These results, thus, do not allow us to reject the null hypothesis that proportionality judgments under “full PA” and under “strict-sense balancing” are equally receptive to factual variation. Figure 6 presents the predicted probability of the proposed plan being judged as proportional across the “goal” (upper panel) and “rights infringements” (lower panel) treatments, and doctrine versions. The pattern of receptivity is very similar for the “goal” and “rights infringements” factors. In both, the incline of the three slopes (for three doctrine versions) are similar, with the slope representing the “strict-sense balancing” (blue) being slightly more pronounced compared to “full PA” (red) and “free PA” (green). Thus, receptivity to factual variation appears to be somewhat higher under the less comprehensive version of PA – “strict-sense balancing”. This difference, however, is statistically insignificant.

Figure 6. The effects of “goal“ (upper panel) and “rights infringement“ (lower panel) treatments on the probability of “proportional“ judgment across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Finally, to test whether proportionality type affects the susceptibility of judgements to legally irrelevant preferences, we fitted (5) to our data. This analysis, reported in Table 3, allowed us to estimate the differences in the effect of ideological preferences on proportionality judgments for different doctrine versions. The interaction terms suggest that such an effect is stronger under “full PA” (albeit at a marginally significant level), and under “free PA,” than under “strict-sense balancing” (Hawk–Dove X “Full PA”: z = −1.89, p = 0.058; Hawk–Dove X “Free PA”: z = −2.56, p = 0.010). These results provide no support to Hypothesis 5, which predicted a lesser effect of legally irrelevant preferences under “Full PA” than under “strict-sense balancing.” The differences in the effect of ideological preferences under the three types of proportionality judgments are displayed in Figure 7, which plots the predicted probability of the proposed plan being judged as proportional across the set of ideological preferences and for the three doctrine versions. The three curves are similar, but the slopes of the “Full PA” (red) and “Free PA” (green) curves are slightly more pronounced than of the “strict-sense balancing” curve (blue). In other words, the effect of ideological preferences on proportionality judgments appears to be somewhat larger under the two former PA types than under the latter.

Figure 7. The effects of ideological preferences across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Discussion

This research is the first in the field to undertake an empirical analysis of legal proportionality judgments by experimentally testing the effects of normatively relevant factual considerations in this regard. Our experimental findings suggest that legal experts’ proportionality judgments are receptive to normatively relevant factual considerations. Respondents tended to judge the proposed plan as more proportional with the increase of the normative importance of its goal and the decrease in the infringements of human right it entailed. Additionally, in light of the strong correlational evidence obtained in the analysis, proportionality judgments appear to be influenced by adjudicators' ideological preferences. However, the results do not indicate that proportionality judgments are subject to ideologically biased information processing. Lastly, the likelihood of a plan being judged as proportional seems to be lower if the judgment results from a comprehensive proportionality analysis. Yet, contrary to our expectations, implementing a comprehensive analysis neither increased the receptivity of judgments to factual variation nor reduced the effect of ideological preferences in this regard, compared with the “strict-sense” balancing.

These results suggest that proportionality analysis appears to provide a methodological structure for legal reasoning based on factual considerations, yet the resulting decisions also reflect legally irrelevant preferences. More specifically, prior policy preferences appear to affect experts’ propensity to judge a plan as proportional; yet, factual considerations pertaining to the normative legitimacy of the goal and the potential for rights infringements quite uniformly lead them to adjust their initial judgment. The resultant legal decision is thus anchored jointly in the experts’ policy preferences and the facts of the case. As the effect of policy preferences was found to be considerable, it is reasonable to claim that proportionality judgments are dependent to a large degree on legally irrelevant preferences.

Additional research is required to assess whether the strong association between policy preferences and proportionality judgements that was found in the context of this study is generalizable to other domains in which proportionality analysis is applied. It is plausible that the existential nature of counter-terrorism policy in Israel renders greater weight to policy preferences in such decisions, while it may not be quite as influential in other more mundane policy domains (Reference Schwartz and Sulitzeanu-KenanSchwartz and Sulitzeanu-Kenan 2004). Conversely, findings in previous studies have shown that policy preferences (e.g., regarding abortions in the US) were found to affect legal judgments in different domains (determining the limits of freedom of speech: Reference Sood and DarleySood and Darley 2012; Reference KahanKahan et al. 2012), Considering our results together with these studies may suggest that the conclusion that policy preferences play a role in proportionality judgments is not necessarily limited to the particular context examined in this experiment.

These results are consistent with previous findings on motivated cognition in other legal judgments, which have been described as a “compromise between preferred outcomes and the relevant information at hand” (Reference Pyszczynski and GreenbergPyszczynski and Greenberg 1987: 333). Furthermore, as proportionality analysis is methodologically vague, legally irrelevant preferences could affect the legal decision via a broad range of mediating factors, not necessarily through biasing experts’ processing of factual information. This interpretation of our results conforms with earlier findings to the effect that “motivated decision makers do not bias their judgments more than what is needed to achieve their desired conclusions” (Reference SoodSood 2013: 310; see also: Reference Boiney, Kennedy and NyeBoiney, Kennedy, and Nye 1997).

Our results regarding judgments’ receptivity to factual variation and the influence of policy preferences under different proportionality doctrines are rather similar. Importantly, compared with the less comprehensive version of proportionality analysis (“strict-sense balancing”) or even with unstructured judgment (“free PA”), the comprehensive proportionality analysis (“full PA”) yielded a lower probability of a plan being judged as proportional. Yet, it neither enhanced the receptivity to factual considerations nor reduced the effect of legally irrelevant considerations.

As noted above, our results indicate that respondents were sensitive to only some of the specific treatments. Specifically, regarding the operational goal only the preventive (vs. punitive) nature of the operation resulted in a clear change in the mean decision, while the size of the preventable harm did not. Considering that this specific null finding may be due to the particular way this consideration was treated in the vignette, and/or sample size, along with the fact that this experiment assesses proportionality judgments in one specific legal context, these results lead us to cautiously suggest that respondents appear to be more sensitive to “rights infringements” considerations in forming proportionality judgments compared to “goal worthiness” properties. However, more research regarding the nature of these decisions is needed in order to garner sufficient certainty and a clearer picture regarding the generalizability of these results.

It should be noted that the different doctrine versions were measured for each subject based on his/her set of answers. While this methodology does not seem to present a problem for the validity of full proportionality judgments (“full PA”),Footnote ³² we cannot directly determine whether decisions under “strict-sense balancing” would have been the same had respondents been presented with the latter version alone. The main concern here is that basing the decision on a single stage (unlike our method) may cause respondents to be more conservative in their judgments, that is, less likely to judge the plan as proportional. While this concern cannot be wholly eliminated, it seems to be less prominent in the present study, insofar as responses to the items in the unstructured proportionality judgment (“free PA”), presented after the “strict-sense balancing” item, were very similar to the responses to the latter item. This may indicate that respondents treated the “strict-sense balancing” decision prong as pivotal in their decision.

Considering that this article reports the first empirical study of proportionality judgments, we should be cautious in drawing broader conclusions from its findings. However, our results do allow some tentative implications for the legal theory of proportionality. The first relates to incommensurability – which is one of the central points of criticism leveled at the proportionality method – namely, the qualitative differences between competing values which militate against placing them on a common scale (Reference FrantzFrantz 1963; Reference PetersenPetersen 2013).Footnote ³³ That said, the receptivity of experts’ proportionality judgments to factual variation was relatively uniform even though their policy preferences were not divergent.Footnote ³⁴ This indicates that, in practical terms, the problem of incommensurability here is not as serious as could have been expected. Conversely, the strong influence of legally irrelevant preferences demonstrated in the analysis highlights the highly vague nature of proportionality judgments. This finding compromises the idea of proportionality analysis as an objective and neutral method of constitutional adjudicating (Reference BeattyBeatty 2004; and for a critical view of this claim see: Reference JacksonJackson 2004), and may support the argument that balancing should be relegated to the political domain (Reference PetersenPetersen 2013: 1392–3).

Stronger conclusions would require further empirical work on the nature of this central legal principle. This study is only a first attempt in this direction, and moreover, it has a number of notable limitations. First, it focuses on one policy domain, in one national setting. Second, while the data are based on decisions of legal experts, the experimental setting involves a set of hypothetical scenarios, and the results cannot be safely extrapolated to real-life situations with tangible implications. It would be plausible to assume that, relative to a real-life situation, the obtained effects of factual variation are underestimations, while of ideological preferences – overestimations. If this is indeed the case, our conclusions regarding the effect of ideological preferences, which was found to be very strong, are likely to prove robust in realistic settings as well.

Supporting Information

Figure A1 and accompanying explanation: Proportionality judgment – validation of measurement items

Table A1: Comparative table for the two sampling methods

Table A2: Assessing the comparability of the two sampling methods

Table A3: Logistic regression estimates for the effects of factual treatments on proportionality decision stages

Table A4: Balance Test

Table A5: Interaction Analyses

Footnotes

The copyright line for this article was changed on 2nd August 2016 after original online publication

For helpful comments and suggestions we thank Aparna Chandra, Hanan Cidor, Adam Chilton, Sharon Gilad, Andrej Lang, Micha Mandel, Ilana Ritov, Anna Sledzinska-Simon, Talya Steiner, Daniel Statman, Omer Yair and seminar participants at, Haifa University, and the Hebrew University, as well as participants of conferences at Ben-Gurion University (2015), MPSA (2014), and ECPR (2014). The research leading to these results has received funding from the European Research Council under the under the European Union's Seventh Framework Programme (FP7/2007-2013)/ERC Grant no. 324182.

¹ Note that this experiment was conducted slightly more than a year before the 2014 Israel-Gaza war.

² Such government action may involve a legislative act or policy, among others. In most cases, constitutional law limits infringement of protected rights by requiring governmental actions to be backed by legislation.

³ Within the literature on rights protection, some researchers claim that the status of rights should be absolute, that is, placed above the mundane cost-benefit policy considerations. According to this view, rights always take priority over the common good (Reference DworkinDworkin 1985; Reference HabermasHabermas 1998). Several Supreme Courts (inter alia, Canada, South Africa and India), as well as several legal scholars, have adopted a different approach (Reference Kumm and GeorgeKumm 2007), positing a distinction between considerations that should not limit rights and considerations that can be weighed against the application of rights. Only in conflicts involving the former type do rights take absolute priority. In the latter case, some balancing is deemed legitimate. The “worthy goal” stage of PA provides a structure for drawing these distinctions. Considerations for limiting constitutional rights that are not regarded as legitimate (“unworthy goals”) include sectarian, sexist, homophobic, and racist preferences (Reference Kumm and GeorgeKumm 2007). More general approaches to exclusion examine whether a deliberate denial of a protected right or interest took place, as opposed to right or interest infringement as an incidental consequence of a policy implementation, and rely on the perfectionist arguments for the limitation of rights (Reference Kumm and GeorgeKumm 2007, 145).

⁴ In some legal contexts (e.g., IHL) proportionality analysis has traditionally been understood to include only strict-sense balancing (Reference HigginsHiggins 1994: Reference MayMay 2013).

⁵ Note that the necessity prong also requires a normative assessment for determining that the alternative policy indeed mitigates the extent of right infringement.

⁶ The latter problem raises a second-order legitimacy dilemma: the applicable law is revealed only through the judge's ruling (Reference Stone Sweet and MathewsStone Sweet and Mathews 2008: 82–3).

⁷ While proportionality analysis assumes a policy measure to be either proportional or not, we relax this deterministic assumption, and adopt a probabilistic approach, which is more amenable to quantitative empirical testing, and potential intersubject variance. The specific method of estimating all hypothesized effects is detailed in the Estimation Strategy section.

⁸ This contribution may stem from either adding to, or preventing loss of public welfare. The valuation of the policy is conducted by the decision maker.

⁹ The theory of motivated reasoning (Reference KundaKunda 1990) suggests that in the processes of reasoning people are guided by two types of goals: accuracy goals, which enhance the use of information and inference methods that are considered most appropriate, and directional goals, which foster the use of those that seem most likely to yield the desired conclusion.

¹⁰ The expectations in every given case depend on the context. However, it is anticipated in all cases that the legal decision taken should support the action or policy that aligns with the adjudicator's policy preferences.

¹¹ Protocol Additional to the Geneva Conventions of 12 August 1949, and Relating to the Protection of Victims of International Armed Conflicts (Protocol I), 8 June 1977, 1125 U.N.T.S. 3.

¹² The HCJ 2006 decision clearly represents the most comprehensive judicial decision regarding this military practice in the Israeli legal system. Moreover, references to this decision in the international law literature attest to its relevance and importance beyond the bounds of the Israeli–Palestinian context (Reference BarnidgeBarnidge and Robert 2012; Reference Blum and HeymannBlum and Heymann 2010; Reference Cohen and ShanyCohen and Shany 2007; Reference EichensehrEichensehr 2007; Reference MayMay 2013; Reference MilanovicMilanovic 2007).

¹³ In the particular Targeted Killings case cited, only the second and third PA stages were implemented. Note that Cohen and Shany are not convinced that the application of the second PA stage is required under the existing laws of armed conflict, as the latter “do not introduce a general obligation on the parties to the conflict to enemy combatants through nonlethal means” (Reference Cohen and ShanyCohen and Shany 2007: 314).

¹⁴ Bar-Ilan University, Haifa University, Hebrew University, Interdisciplinary Center in Herzlia, and Tel-Aviv University. This sampling method was chosen to ensure that all the respondents are Law graduates.

¹⁵ The invitation to participate in the research was sent to 25 contact individuals known to the researches as practicing legal experts. These individuals were asked to distribute the invitation in their workplace to no more than 10 potential respondents, whom they personally knew to be legal experts. By opting out of mass distribution, and relying on spatial (geographic) and institutional separation, we ensured that the probability that a respondent would receive more than one invitation was negligible. Each invitation included a link that randomly assigned the invitee to one of the 16 versions of the questionnaire.

¹⁶ These samples differ in gender ratio, average age, and ideological hawk–dove attitudes. A detailed comparison of the two samples is presented in Supporting Information Table A1 in the online appendix. To decide whether the two samples can be merged, we assessed their causal homogeneity by fitting our main logistic regression model separately to each of the samples, and compared the coefficients for all 10 independent variables (Reference PaternosterPaternoster et al. 1998), as reported in Supporting Information Table A2 in the online appendix. No significant differences were found between any of the treatment coefficients. The only difference between the two samples was in the relative propensity to judge the plan as proportional under one of the three measures of proportionality judgments (see section Measuring proportionality judgments). Importantly, the constants of the two models are extremely similar, indicating that the mean propensity for judging the military plan as proportional is similar in both samples.

¹⁷ In constructing the scenarios we consulted a former senior official in the General Security Service (GSS), who asked to remain anonymous. The GSS provides intelligence for such operations, and is central to their approval procedures.

¹⁸ Religiosity plays a particularly important role in defining collective identity and political attitudes in Israeli society (Reference Shamir and ArianShamir and Arian 1999). Respondents’ left-right economic ideology was also measured, but was not found to be associated with proportionality judgments regarding targeted killings, and was thus not included in the analyses.

¹⁹ By monotonic we mean that the pattern of judgments associated with the ordered changes in the policies is consistently increasing or decreasing, with no reverse trend.

²⁰ Note that our treatment vignettes constitute a series of scenarios with consecutively increasing goal-worthiness and decreasing potential for rights infringements.

²¹ This equation is simplified by treating the two sets of three indicator variables for “goal” and “rights infringement” as two ordinal variables. For details see the Results section.

²² Note that we do not assume to be able to observe an “original weight” given to these facts in isolation from any modified or biased weights. However, if ideological preferences take part in modifying the weights assigned to information in the decision process, we should expect the same information to have different effects on the decision across hawks and doves.

²³ Punitive goal, preventive goal, preventing a severe harm, preventing a severe harm by targeting senior terrorists.

²⁴ As other variables may affect the probability that the plan would be judged proportional, this method estimates the probability for each of the “goal” treatments, with the other variables set at their mean values. This implies that the “rights infringement” treatments are calculated as a statistical average (so that each treatment exerts 0.25 of its maximum effect); for the respondent the statistical average is 0.5 male and 0.5 female; the mean level of ideological position is 3.16 – slightly to the left of center; the mean age is 35.5; the mean level of religiosity is 3.4 – between traditional and secular; the default proportionality type is “strict-sense.”

²⁵ Orthogonal polynomial contrasts analyses, which partition the effects of each of the factors into linear, quadratic, and cubic, were conducted to assess the characteristics of the judgment trend across the two factor levels. For both factors only the linear effect was statistically significant – “goal”: χ ² = 9.69, df = 1, p = 0.002; “rights infringements”: χ ² = 34.50, df = 1, p < 0.001.

²⁶ All post hoc tests are one-tailed Dunn–Šidák corrected.

²⁷ For example, the treatment designed to frame the operation as preventing a severe terrorist attack was relayed by noting that “in a similar attack two weeks ago” a larger number of civilians were killed. It is possible that respondents who realized that rocket attacks on populated areas are rather random were not very affected by this statement.

²⁸ For example, does the sensitivity to factual variation in the extent of “rights infringement” decrease when the policy goal becomes normatively very important?

²⁹ All post hoc tests are Dunn–Šidák corrected.

³⁰ The decision to merge “center” with the “hawks” was motivated by distributional considerations. The resulting “hawk” group constituted 61.3% of the sample. Conversely, merging “center” with “doves” would have resulted in a “dove” group constituting 73.2% of the sample.

³¹ We conducted a further analysis, treating “goal” and “rights infringements” as two categorical variables, thereby fitting six interaction terms between ideology and each of the treatments. None of these specific interactions were statistically significant, and contrasts of marginal linear predictions suggest no statistically significant interaction for both “goal” (χ ² = 3.79, df = 3, p = 0.285) and “rights infringement” (χ ² = 1.74, df = 3, p = 0.628). Similar results were found when merging treatments that were not found to uniquely influence judgment (collapsing the “goal” factor into two levels: G1 & G2 through G4; and risk into three levels G1, G2, G3 & 4).

³² The set of measurement items was structured to enhance their validity. Items were presented to respondents in line with the legal doctrine and their professional expectations. Moreover, convergent and discriminant validity (Reference AdcockAdcock 2001) analyses of the responses to the proportionality judgments items indicate that the four items offer a valid set of measures of proportionality judgment (for details, see online appendix).

³³ This problem is twofold: first, the absence of a definite and common unit of measurement for each value; and second, even if such a unit could be devised (see, for example: Reference VeelVeel 2010), it would not be equally applicable to different values, either because of their qualitative difference, or in default of an agreed on ratio for the relative importance of various values.

³⁴ Receptivity to factual variation was also consistent across respondents surveyed in university classes using “paper and pencil” questionnaires and those surveyed online (see Supporting Information Table A2 in the online appendix).

References

Adcock, Robert (2001) “Measurement Validity: A Shared Standard for Qualitative and Quantitative Research,” 95 American Political Science Rev. 529–46.CrossRef Google Scholar

Aleinikoff, T. Alexander (1987) “Constitutional Law in the Age of Balancing,” 96 The Yale Law J. 943–1005.CrossRef Google Scholar

Alexy, Robert (2010) A Theory of Constitutional Rights. New York: Oxford Univ. Press.Google Scholar

Ball, Terence, & Dagger, Richard (2010). Ideals and Ideologies: A Reader. 8th ed. Boston: Pearson.Google Scholar

Barak, Aharon (2011) Proportionality. Cambridge: Cambridge Univ. Press.Google Scholar

Barnidge, Robert P Jr. (2012) “Qualified Defense of American Drone Attacks in Northwest Pakistan under International Humanitarian Law, A,” 30 Boston University International Law J. 409.Google Scholar

Beatty, David M (2004) The Ultimate Rule of Law. Oxford: Oxford Univ. Press.CrossRef Google Scholar

Benoit, Kenneth, & Laver, Michael (2006) Party Policy in Modern Democracies. London: Routledge.CrossRef Google Scholar

Bertrand, Marianne, Chugh, Dolly, & Mullainathan, Sendhil (2005) “Implicit Discrimination,” 95 American Economic Rev. 94–98.CrossRef Google Scholar

Blum, Gabriella, & Heymann, Philip (2010) “Law and Policy of Targeted Killing,” 1 Harvard National Security J. 145–70.Google Scholar

Boiney, Lindsley G, Kennedy, Jane, & Nye, Pete (1997) “Instrumental Bias in Motivated Reasoning: More When More Is Needed.” 72 Organizational Behavior and Human Decision Processes 1–24.CrossRef Google Scholar

Bomhoff, Jacco (2013) Balancing Constitutional Rights: The Origins and Meanings of Postwar Legal Discourse. Cambridge Studies in Constitutional Law. Cambridge: Cambridge Univ. Press.Google Scholar

Bowcott, Owen (2012) Drone Strikes Threaten 50 Years of International Law, Says UN Rapporteur. The Guardian. Available at: http://www.theguardian.com/world/2012/jun/21/drone-strikes-international-law-un (accessed 4 March 2016).Google Scholar

Cohen, Amichai, & Shany, Yuval (2007) “A Development of Modest Proportions The Application of the Principle of Proportionality in the Targeted Killings Case,” 5 J. of International Criminal Justice 310–21.CrossRef Google Scholar

Cohen-Eliya, Moshe, & Porat, Iddo (2010) “American Balancing and German Proportionality: The Historical Origins,” 8 International J. of Constitutional Law 263–86.CrossRef Google Scholar

Cohen-Eliya, Moshe, & Porat, Iddo (2014) Proportionality and Constitutional Culture. 1st ed. Cambridge: Cambridge Univ. Press.Google Scholar

Ditto, Peter H., & Lopez, David F. (1992) “Motivated Skepticism: Use of Differential Decision Criteria for Preferred and Nonpreferred Conclusions,” 63 J. of Personality and Social Psychology 568–84.CrossRef Google Scholar

Duch, Raymond M., May, Jeff, & Ii Armstrong, David A. (2010) “Coalition-Directed Voting in Multiparty Democracies.” 104 American Political Science Rev. 698–719.CrossRef Google Scholar

Dworkin, Ronald (1985) A Matter of Principle. Oxford: Oxford Univ. Press.Google Scholar

Eichensehr, Kristen E. 2007. “On Target? The Israeli Supreme Court and the Expansion of Targeted Killings.” 116 Yale Law J. 1873–81.CrossRef Google Scholar

Erikson, Robert S., & Tedin, Kent L. (2010) American Public Opinion: Its Origins, Content and Impact. 8th Ed. New York: Pearson Education. Longman.Google Scholar

Frantz, Laurent B (1963) “Is The First Amendment Law? A Reply to Professor Mendelson,” 51 California Law Rev. 729–54.CrossRef Google Scholar

Furgeson, Joshua R., Babcock, Linda, & Shane, Peter M. (2008). “Do a Law's Policy Implications Affect Beliefs About Its Constitutionality? An Experimental Test,” 32 Law and Human Behavior 219–27.CrossRef Google Scholar PubMed

Gazal-Ayal, Oren, & Sulitzeanu-Kenan, Raanan (2010) “Let My People Go: Ethnic In-Group Bias in Judicial Decisions—Evidence from a Randomized Natural Experiment,” 7 J. of Empirical Legal Studies 403–28.CrossRef Google Scholar

Habermas, Jürgen (1998) Between Facts and Norms: Contributions to a Discourse Theory of Law and Democracy. Cambridge, MA: MIT Press.Google Scholar

Hadar, Josef, & Russell, William R. (1969) “Rules for Ordering Uncertain Prospects.” 59 American Economic Rev. 25–34.Google Scholar

Hainmueller, Jens, Hangartner, Dominik, & Yamamoto, Teppei (2015) “Validating Vignette and Conjoint Survey Experiments against Real-World Behavior,” 112 Proceedings of the National Academy of Sciences 2395–2400.CrossRef Google Scholar PubMed

Higgins, Rosalyn (1994) Problems and Process: International Law and How We Use It. Oxford: Oxford Univ. Press.Google Scholar

Hurka, Thomas (2005) “Proportionality in the Morality of War.” 33 Philosophy & Public Affairs 34–66.CrossRef Google Scholar

Jackson, Vicki C (2004) “Being Proportional About Proportionality,” 21 Constitutional Commentary 803–59.Google Scholar

Jackson, Vicki C (2015) “Constitutional Law in an Age of Proportionality,” 124 Yale Law J. 3094–196.Google Scholar

Jain, Shailendra P., & Maheswaran, Durairaj (2000) “Motivated Reasoning: A Depth-Of-Processing Perspective,” 26 J. of Consumer Research 358–71.CrossRef Google Scholar

Jost, John T., Federico, Christopher M., & Napier, Jaime L. (2009) “Political Ideology: Its Structure, Functions, and Elective Affinities,” 60 Annual Rev. of Psychology 307–37.CrossRef Google Scholar PubMed

Kahan, Dan M., et al., (2012) “They Saw a Protest: Cognitive Illiberalism and the Speech-Conduct Distinction,” 64 Stanford Law Rev. 851.Google Scholar

Kaplow, Louis (1992) “Rules versus Standards: An Economic Analysis,” 42 Duke Law J. 557–629.CrossRef Google Scholar

Kariv, Shachar, & Silverman, Dan (2013) “An Old Measure of Decision-Making Quality Sheds New Light on Paternalism.” 169 J. of Institutional and Theoretical Economics JITE 29–44.CrossRef Google Scholar

Kumm, Mattias (2007) “Political Liberalism and the Structure of Rights: On the Place and Limits of the Proportionality Requirement.” in George, P., ed., Law, Rights and Discourse: Themes from the Legal Philosophy of Robert Alexy, Portland, OR: Hart Publishing. 131–65.Google Scholar

Kunda, Ziva (1990) “The Case for Motivated Reasoning,” 108 Psychological Bulletin 480–98.CrossRef Google Scholar PubMed

Law, David S (2004) “Generic Constitutional Law.” 89 Minnesota Law Rev. 652.Google Scholar

Liang, Kung-Yee, & Zeger, Scott L. (1986) “Longitudinal Data Analysis Using Generalized Linear Models,” 73 Biometrika 13–22.CrossRef Google Scholar

Lodge, Milton, & Taber, Charles S. (2013) The Rationalizing Voter (Cambridge Studies in Public Opinion and Political Psychology). 1 ed. New York: Cambridge Univ. Press.Google Scholar

Mather, Lynn (2008) “Law and Society.” in Whittington, Keith E., Kelemen, Daniel R., & Caldeira, Gregory A., The Oxford Handbook of Law and Politics, The Oxford Handbooks of Political Science. Oxford: Oxford Univ. Press. 681–97.Google Scholar

May, Larry (2013) “Targeted Killings and Proportionality in Law Two Models,” 11 J. of International Criminal Justice 47–63.CrossRef Google Scholar

Milanovic, Marko (2007) “Lessons for Human Rights and Humanitarian Law in the War on Terror: Comparing Hamdan and the Israeli Targeted Killings Case,” 89 International Review of the Red Cross 373–93.CrossRef Google Scholar

Paternoster, Raymond, et al., (1998) “Using the Correct Statistical Test for the Equality of Regression Coefficients,” 36 Criminology 859–66.CrossRef Google Scholar

Peabody, John W., Luck, Jeff, Glassman, Peter, Jain, Sharad, Hansen, Joyce, Spell, Maureen, & Lee, Martin (2004) “Measuring the Quality of Physician Practice by Using Clinical Vignettes: A Prospective Validation Study,” 141 Annals of Internal Medicine 771–80.CrossRef Google Scholar PubMed

Petersen, Niels (2013) “How to Compare the Length of Lines to the Weight of Stones: Balancing and the Resolution of Value Conflicts in Constitutional Law,” 14 German Law J. 1387.CrossRef Google Scholar

Pyszczynski, Tom, & Greenberg, Jeff (1987) “Toward an Integration of Cognitive and Motivational Perspectives on Social Inference: A Biased Hypothesis-Testing Model,” In Advances in Experimental Social Psychology, Vol. 20, San Diego, CA: Academic Press. 297–340.Google Scholar

Rachlinski, Jeffrey J., et al., (2008) “Does Unconscious Racial Bias Affect Trial Judges,” 84 Notre Dame Law Rev. 1195.Google Scholar

Ringe, Nils (2005) “Policy Preference Formation in Legislative Politics: Structures, Actors, and Focal Points,” 49 American J. of Political Science 731–45.CrossRef Google Scholar

Schwartz, Robert, & Sulitzeanu-Kenan, Raanan (2004) “Managerial Values and Accountability Pressures: Challenges of Crisis and Disaster,” 14 J. of Public Administration Research and Theory 79–102.CrossRef Google Scholar

Shamir, Michal, & Arian, Asher (1999) “Collective Identity and Electoral Competition in Israel,” 93 The American Political Science Review 265–77.CrossRef Google Scholar

Sood, Avani Mehta (2013) “Motivated Cognition in Legal Judgments—An Analytic Review,” 9 Annual Review of Law and Social Science 307–25.CrossRef Google Scholar

Sood, Avani Mehta, & Darley, John M. (2012) “Plasticity of Harm in the Service of Criminalization Goals,” 100 California Law Rev. 1313.Google Scholar

Stone Sweet, Alec, & Mathews, Jud (2008) “Proportionality Balancing and Global Constitutionalism,” 47 Columbia J. of Transnational Law 72.Google Scholar

Sulitzeanu-Kenan, Raanan, & Halperin, Eran (2013) “Making a Difference: Political Efficacy and Policy Preference Construction,” 43 British J. of Political Science 295–322.CrossRef Google Scholar

Sunstein, Cass R (1995) “Problems with Rules,” 83 California Law Rev. 953–1026.CrossRef Google Scholar

Taber, Charles S., Cann, Damon, & Kucsova, Simona (2009) “The Motivated Processing of Political Arguments.” 31 Political Behavior 137–55.CrossRef Google Scholar

Taber, Charles S., & Lodge, Milton (2006) “Motivated Skepticism in the Evaluation of Political Beliefs,” 50 American J. of Political Science 755–69.CrossRef Google Scholar

Tsakyrakis, Stavros (2009) “Proportionality: An Assault on Human Rights?” 7 International J. of Constitutional Law, January, 468–93.CrossRef Google Scholar

Veel, Paul-Erik N (2010) “Incommensurability, Proportionality, and Rational Legal Decision-Making,” 4 Law & Ethics of Human Rights 178–228.CrossRef Google Scholar

Webber, Gregoire C. N. (2010) “Proportionality, Balancing, and the Cult of Constitutional Rights Scholarship.” 23 Canadian J. of Law and Jurisprudence 179.CrossRef Google Scholar

Table 1. Sample Descriptive Statistics

Table 2. Experimental Treatments

Figure 1. Question wording, and three measures of proportionality judgments.

Table 3. Generalized Estimating Equations predicting Proportionality Judgments

Figure 3. A three-dimensional mapping of normatively relevant factual variations into proportionality judgments. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 4. Mean probability of “proportional” judgment across ideological positions (with 95% CI) Note: only 8 respondents occupy the “extreme hawk” category, all of whom judged their respective scenario as proportional (under the three different proportionality formulae). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 5. Mean probability of “proportional” judgment across proportionality formulae (with 95% CI). [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 6. The effects of “goal“ (upper panel) and “rights infringement“ (lower panel) treatments on the probability of “proportional“ judgment across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Figure 7. The effects of ideological preferences across proportionality types. [Color figure can be viewed in the online issue, which is available at wileyonlinelibrary.com.]

Sulitzeanu-Kenan et al. supplementary material

File 105.4 KB

Article contents

Facts, Preferences, and Doctrine: An Empirical Analysis of Proportionality Judgment

Abstract

Proportionality Analysis

A Behavioural Analysis of Proportionality Judgments

Proportionality and Targeted Killings of Terrorist Operatives

Research Design

Experimental Procedure

Measuring Proportionality Judgments

Measuring Ideological Preference

Estimation Strategy

Results

Receptivity of Proportionality Judgments to Normatively Relevant Factual Variation

The Effects of Ideology on Proportionality Judgments

Proportionality Judgments vis-à-vis Doctrinal Versions

Discussion

Supporting Information

Footnotes

References

Save article to Kindle

Save article to Dropbox

Save article to Google Drive

Reply to: Submit a response

Your details

You have entered the maximum number of contributors

Conflicting interests