How Compliance Understates Effectiveness

Customarily one begins a discussion about the effectiveness of international law by quoting Louis Henkin’s famous remark that “almost all nations obey almost all principles of international law and almost all of their obligations almost all of the time.” For some, this empirical claim supports the notion that international law is a vital tool for furthering international cooperation across a broad range of issue areas. For others, the implicit suggestion that international law’s mere existence might be driving states’ behavior is a calamity of causal inference. Even if Henkin’s claim is empirically correct, effectiveness does not follow from compliance. For a third group, Henkin’s claim may not even be empirically correct. In at least some areas of international law, noncompliance may be relatively high. Deploying the same suspect causal reasoning that the second group worries about, international law skeptics have sometimes suggested that we might infer ineffectiveness on the basis of such noncompliance.


HOW COMPLIANCE UNDERSTATES EFFECTIVENESS
Timothy Meyer* Editor's note: The following is the first contribution to an AJIL Unbound symposium 1 on "The Idea of Effective International Law," a continuation of the panel discussion 2 at the 2014 ASIL-ILA joint meeting in Washington, DC. * * * * Customarily one begins a discussion about the effectiveness of international law by quoting Louis Henkin's famous remark 3 that "almost all nations obey almost all principles of international law and almost all of their obligations almost all of the time." For some, this empirical claim supports the notion that international law is a vital tool for furthering international cooperation across a broad range of issue areas. For others, the implicit suggestion that international law's mere existence might be driving states' behavior is a calamity of causal inference. Even if Henkin's claim is empirically correct, effectiveness does not follow from compliance. For a third group, Henkin's claim may not even be empirically correct. In at least some areas of international law, noncompliance may be relatively high. Deploying the same suspect causal reasoning that the second group worries about, international law skeptics have sometimes suggested that we might infer ineffectiveness on the basis of such noncompliance.
In this essay, based on my remarks at the 2014 ASIL-ILA joint meeting panel titled "The Idea of Effective International Law," I argue that an excessive focus on compliance may understate international law's effectiveness for at least two reasons. First, much legal practice focuses on compliance because judges and arbitrators are forced to make determinations about whether a state's conduct satisfies a legal standard. International law, however, remains principally a system of negotiated lawmaking. As such, compliance disputes are often as much about defining expectations for how states should behave going forward as they are about determining how a state should have behaved in the past. Put differently, compliance is often just one part of a larger contest between states over the content of legal obligations. Indeed, noncompliance can be a negotiating tool, indicating a desire to engage with international law rather than ignore it. Second and relatedly, international law can be highly effective at changing state behavior over time, even if compliance with legal standards remains low. States may join international legal regimes expecting to be noncompliant but hoping to obtain some benefits from membership, such as foreign assistance or pressure on domestic governments to adopt certain policies. Noncompliance, in other words, has benefits in terms of encouraging changes in state behavior.

Effectiveness versus Compliance
At the outset, I should define what I mean by "effective." Effectiveness refers to whether the law has changed a state's behavior from what it would have been in the absence of the law. Basically, I would say a law is "effective" when the law is a but-for cause of the state's subsequent conduct. To determine whether law is "effective" in this sense, one compares two different behaviors-a state's behavior without the law in question and its behavior with the law in question. The difference in behavior, controlling for other factors that may have influenced the state's behavior, is attributable to the law. Effectiveness in this sense is thus really a behavioral question. It asks whether the law is an influence on a state's conduct, and to what degree.
In keeping with a great deal of scholarship (see, for example, Raustiala; 4 Neyer & Wolf; 5 Martin 6 ) I distinguish effectiveness from compliance. Compliance refers to whether or not a state's conduct meets the prescribed legal standard. For example, if a state accepted binding emissions reductions commitments under the Kyoto Protocol, we could ask at the end of 2012 whether the state had in fact reduced its emissions limits as required. Unlike effectiveness, compliance does not ask whether the law influenced the state's behavior. Compliance does require, however, a more concrete notion of what the legal rule mandates. For example, we cannot know whether a state has complied with its obligation to accord a foreign investor fair and equitable treatment unless we define more concretely what that obligation entails. We have to know whether the rule prohibits taking certain specified actions against foreign investors. Even where something as seemingly straightforward as reducing emissions by a specified percentage are concerned, assessing compliance still requires a common understanding of how emissions should be measured and what acts count as reducing them. Effectiveness, by contrast, does not require information about the content of the legal rule. Instead, all we need to know is whether the law's enactment changed the state's behavior.
Compliance and effectiveness are thus two different scales upon which legal rules can be measured. A rule can exhibit (1) high compliance and high effectiveness; (2) low compliance and low effectiveness; (3) high compliance and low effectiveness; or (4) low compliance and high effectiveness. Starting at least with Downs, Rocke, and Barsoom, 7 international relations scholars have argued that international law may reside largely in category 3. States are both the subjects of international law and its authors. The concern is therefore that states will agree to "shallow" commitments-obligations that do not require them to change their behavior significantly from what they would do in the absence of a treaty. These shallow commitments produce high levels of compliance but few changes in state behavior. In short, if we are interested in effectiveness, compliance rates may be a red herring.
The example of baselines in environmental treaties illustrates the point. When states agree to reduce their emissions of a particular pollutant, they often pick a year against which to measure future reductions. The choice of the baseline year often determines how strenuous the commitment is. For example, the Kyoto Protocol measures its various emissions reductions commitments against 1990. The 1990 baseline year allowed Russia to measure its emissions against a time prior to the collapse of the Soviet economy. The effect was that Russia had already achieved (or from its point of view suffered) huge reductions in its emissions when it ratified the Kyoto Protocol, albeit for reasons completely unrelated to its legal commitments. Alternatively, states might choose technology-based standards against which to measure compliance. During the negotiations of the Convention on Long-Range Transboundary Air Pollution (designed to address acid rain), the United States argued for a "best available technology" metric. It did so because it had already put in place substantial technological controls, meaning it would already largely be in compliance with a technology-based standard. Emissions reductions measured from the date of negotiation could have been considerably more difficult to comply with, as the United States had already achieved the least costly gains through its technological improvements. While international relations scholars worry that effectiveness cannot be inferred from compliance, other commentators seem willing to infer ineffectiveness from noncompliance. Based on concerns that compliance rates in certain areas of the law may be low, these commentators argue that international law is a weak tool for coordinating state behavior, essentially putting international law into category 2 above. At the extreme, these claims suggest that individual noncompliant acts show that international law is ineffective generally, an argument Peter Spiro has referred to as the "Perfect Compliance Fallacy. 8 " The effectiveness of international law is thus under strain from two directions. Strong reasons exist to think that even when we observe compliance we cannot necessarily infer effectiveness. At the same time, there is a temptation to view compliance as a necessary but not sufficient condition for inferring effectiveness, and therefore to infer ineffectiveness from noncompliance.

Compliance Understates Effectiveness
In contrast to these pessimistic arguments, I worry that an excessive focus on compliance understates the effect international law (including international soft law) has on state behavior. Compliance may understate effectiveness for at least two reasons. First, international legal rules are indeterminate. States are constantly contesting the meaning of legal rules. Treating this contestation as failures to comply with the law risks severely underestimating the extent to which international law influences state behavior. Second, international legal rules may exhibit low compliance but high effectiveness for a number of reasons, including the fact that governments and domestic groups may push to join international agreements in the hopes that noncompliance can be a tool to change state behavior over time. I discuss each of these reasons below.

(Non)Compliance as Lawmaking
The view that effectiveness cannot be measured from compliance rests on a claim that law's effect on behavior is inextricably linked to the process of lawmaking. Much scholarship understands this connection sequentially. First, states negotiate international rules; then, they change their behavior (or not) in response to those rules. Foreseeing the need to comply tomorrow with the rules they negotiate today, states make it easy on themselves.
I share the view that effectiveness cannot be understood without first understanding international lawmaking. But this sequential lawmaking/compliance framework fails to adequately describe the pervasive role that lawmaking plays, including in compliance decisions and assessments. International legal rules are usually indeterminate. They are closer to standards than they are to rules. 9 Simply reading a treaty may tell a state very little about what the treaty requires. Instead, most legal agreements require interpretation. States regularly use claims about noncompliance to test and advance particular interpretations of legal rules. Indeed, states may use noncompliance as a tool to renegotiate legal rules. 10 For example, Monica Hakimi 11 has recently argued that efforts to boost enforcement against Iran for possible violations of the Nuclear Nonproliferation Treaty, including enforcement actions that have at times been viewed as noncompliant, are best understood as part of an effort to make new law, rather than efforts to enforce and comply with existing law.
Lawmaking thus goes side by side with-not only before-compliance. The continuing and constant nature of lawmaking means we must be very hesitant to infer ineffectiveness based on noncompliance. Such an inference wrongly assumes that states abandon their role as lawmakers once an agreement has been concluded. It assumes that once an agreement has been concluded, noncompliance indicates the agreement's failure to change the state's behavioral incentives and thus represents an institutional failure to create cooperation. This is not necessarily true. Just as initial negotiations among different possible rules have distributive consequences, so too does interpretation. Noncompliance might well reflect a state's commitment to an international agreement but its preference for an interpretation of that agreement under which it captures more of the gains from cooperation.
At the root of this problem with the sequential lawmaking/compliance framework is what Harlan Cohen 12 has called the distinction between two archetypes of lawmaking, adjudicated versus negotiated lawmaking. In adjudicated lawmaking, tribunals play the primary interpretative role, filling gaps in legal rules. Compliance looks more like a second stage in a linear process, one in which the role of states has shifted from negotiators to litigants before a court defending specific actions (although states may of course bring strategic challenges aimed at developing the law in a particular direction). Thinking about effectiveness through the sequential lawmaking/compliance framework is based on this adjudicative model of law, in which states lose authorship over law's content to tribunals that make judgments about compliance at particular moments of time based on discrete sets of facts.
Negotiated lawmaking looks very different. There, states retain primary control over the evolution of legal rules. Compliance disputes involve efforts to reach a common expectation among states about whether a legal rule permits a particular action going forward. Assessing compliance, in other words, is merely part of an ongoing lawmaking process that can affect state behavior over time. The compliance architecture of many international agreements reflects this very different role for compliance disputes. States collectively retain jurisdiction and therefore discretion over the development of legal rules through non-judicial oversight mechanisms. Perhaps the most common of these oversight methods is a legislative one. Many treaties give the Conference of the Parties (COPs) the ability to make recommendations regarding the implementation of treaties and to devise non-judicial noncompliance procedures. COPs have used these decision-making procedures to make collective decisions 13 about how treaties should be interpreted. Moreover, party-administered noncompliance procedures have used compliance disputes to define and punish violations of treaties. For example, under the Montreal Protocol, the Secretariat may receive reports of noncompliance from parties. These reports are then forwarded to an Implementation Committee, which considers the reports and makes recommendations to the full Meeting of the Parties (MOP). The MOP then decides on the appropriate course of action, including the possibility of choosing potentially punitive actions from an "Indicative List of Measures that Might be Taken by Meeting of the Parties in Respect of Non-Compliance with the Protocol." In following this process, the MOP-the parties themselves-implicitly defines the obligation with greater The ability to negotiate a resolution through a collective decision-making process can avoid some of the tensions associated with adversarial proceedings and can also allow states to tailor compliance processes more readily to the circumstances, capabilities, and politics of a given circumstance. This tailoring, in turn, may increase the likelihood that a state will change its behavior over time in response to legal incentives. An excessively rigid focus on a more binary notion of compliance, including a focus on the role of tribunals in adjudicating the content of the law based on discrete incidents, could crowd out 14 these beneficial nonjudicial means of developing the law and improving effectiveness.

Low Compliance, High Effectiveness
Excessively focusing on compliance can also obscure the fact that much of international law may actually be in category 4-it may have low levels of compliance but high levels of effectiveness, particularly if viewed over time. Human rights treaties might exhibit high degrees of effectiveness if they push states towards improving their aggregate human rights practices, even if they exhibit low levels of compliance when measured at particular moments in time. Put differently, we have to allow for equal opportunity in our skepticism about the connection between compliance and effectiveness. If we are unwilling to infer effectiveness from compliance, we should pause before inferring ineffectiveness from noncompliance.
To be sure, scholars have made a strong case for why we should not automatically infer effectiveness from compliance. A similar case is necessary to explain why noncompliant behavior does not automatically permit an inference that international law is ineffective. Fortunately, we do not lack for such explanations. As discussed above, many international agreements, particularly environmental agreements, eschew noncompliance processes that result in binary determinations of noncompliance. Rather, they often establish mechanisms through which deliberate or egregious noncompliance is penalized, but efforts to comply that fall short (as opposed to willful noncompliance) are met with support to improve compliance. This dual approach to compliance creates a sliding scale in which states can reduce the sanction they face for noncompliance by expending effort to improve their practices. Simply looking at whether states have complied with their legal commitments misses this importance incentive effect of these innovative, non-adjudicatory noncompliance procedures.
States may also join international agreements precisely because, or at least with the knowledge that, they will be in noncompliance and therefore face pressure to change their behavior. As Andrew Moravcsik, 15 Tom Ginsburg, 16 and others have argued, some states (often new democracies) join international agreements as a commitment strategy. Under this theory, "internationalist" governments join supranational institutions to pressure their successors-who may not share their preferences-to carry out domestic policies favored by the internationalist government. In such situations, we may observe imperfect compliance by successor governments, but nevertheless see improvements relative to what the successor government's behavior would have been in the absence of the international agreement. Similarly, Beth Simmons has found that ratifying human rights treaties can improve human rights practices in the presence of mobilized domestic constituencies and independent courts. As with the "lock-in" hypothesis, this suggests that states may join international agreements in situations in which they (or perhaps more accurately the groups within the state pushing for ratification) expect some noncompliance in the near-term, but hope to use that noncompliance as part of a strategy for changing behavior over time.

Conclusion
Efforts to study international law's effects have tended to focus on how individual legal rules change state behavior. Rules are useful for third parties adjudicating disputes among states. But as we move from questions of state responsibility to questions of how law affects behavior, we quickly see that focusing on rules artificially separates those rules from the processes that create and maintain them. The larger processes of contesting and defining the meaning of law, both internationally and domestically, often use noncompliance as a tool to prod behavioral change. Legal regimes, in other words, may encourage a kind of creative destruction. Inferring ineffectiveness from noncompliance risks underestimating the value of these creative acts and how, even if noncompliant, they keep states engaged with each other and the international legal system as a whole.