Causal Inference and Legal Studies

In this essay, I focus on the relationship between Article 287 declarations and maritime disputes.Footnote ¹ The authors posit that Article 287 declarations 1) reduce the likelihood of new maritime disputes, 2) decrease the use of military force, and 3) increase the use of non-binding conflict management tools. The authors find that Article 287 declarations decrease the use of force and increase peaceful settlement attempts, while they find that Article 287 declarations are associated with the onset of more maritime disputes in contrast to their first hypothesis. I now turn to discussing the causal question in Mitchell and Owsiak's article.

Does X cause Y? While it is relatively straightforward to estimate regression models and compute the covariance between X and Y, it is much more difficult to identify the true causal effect of X on Y. To obtain the causal effect, we need to compare outcomes in Y across different values of the treatment, or key independent variable. However, estimating the causal effect of X on Y requires, at a minimum, that the treatment assignment mechanism,Footnote ² such as a state's decision to make Article 287 declarations (X), is independent from or unrelated to the outcome of interest, such as a state's decision to attempt peaceful settlements (Y). Violations of this assumption produce spurious statistical results. Unfortunately, this assumption is unlikely to hold in Mitchell and Owsiak's work, or in most empirical legal research, because governments or individuals are rational actors that make decisions to advance their strategic interests.

For example, it is likely that governments make Article 287 declarations with an eye towards managing their maritime disputes, which in turn violates the primary assumption in regression models because the decision to make Article 287 declarations is related to the decision to peacefully resolve or avoid maritime disputes. Indeed, the authors recognize this logic: “[r]ational potential litigants seek the best outcome possible for themselves and consider various dispute settlement mechanisms to achieve that goal.”Footnote ³ In contrast to this, the alternative, unrelated logic, would be that governments make Article 287 declarations randomly or without concern about managing maritime disputes, which seems unlikely.Footnote ⁴ The consequence of this, however, is that the basic independence assumption is violated.

Two Sources of Bias

Two sources of bias often lead to violations of the independence assumption, inhibiting the estimation of causal effects: selection effects and heterogeneous treatment effects.Footnote ⁵ The selection effect is whether there are systematic differences between governments that make Article 287 declarations and those that do not, while heterogeneous treatment effects are whether those in the treatment group (i.e., states that make Article 287 declarations) respond differently to the treatment compared to those in the control group (i.e., those that do not make Article 287 declarations).

These two common sources of bias prevent empirical researchers from attaining the true causal effect in empirical work. The solution is for analysts to identify strategies that can reduce or remove the sources of bias, so as to estimate the true causal effect. Mitchell and Owsiak use several control variables and seemingly unrelated regression to minimize threats to inference. The authors are to be applauded for recognizing this issue, but problems remain. To that end, I put forward some suggestions below based on insights from the causal inference literature.

First, with respect to selection effects, the concern is that there may be some differences between the treated group and the control group, other than the treatment itself, and these differences are related to the outcomes of interest. This is a problem because if these differences are not accounted for in the empirical models, then it is impossible to conclude that the treatment is causally related to the outcome, even if the model produces statistically significant results. Rather, it could be another, omitted factor that is a common cause of both the treatment and the outcome of interest. It is therefore necessary to identify the factors that cause variation in the decision to make Article 287 declarations and to control for them if they are also related to the outcomes of interest.

Again, this is necessary to address in Mitchell and Owsiak's analysis because there are good reasons to believe that governments strategically decide to make Article 287 declarations, as the authors recognize. The basic patterns in the data also appear to indicate a selection effect. According to the authors’ data, while 164 states have ratified UNCLOS, only forty-seven have made Article 287 declarations recognizing the jurisdiction of the ITLOS (thirty-nine), the ICJ (twenty-eight), or one of the two arbitration procedures (twenty-one) for compulsory dispute settlement. That a small number of ratifying states have made Article 287 declarations suggests that those that do make them might be systematically different from those that do not make such declarations.

Mitchell and Owsiak include several variables in their analysis to counter concerns related to selection bias. They further explore four possible sources of selection effects in greater detail: 1) declarers may have already resolved their maritime disputes; 2) state power; 3) regime-type; or 4) the salience of maritime disputes may shape a decision to declare. Mitchell and Owsiak reject that these factors bias the analysis, as they find few systematic differences between Article 287 declarers and non-declarers across these four variables. They conclude that, “it is not obvious what alternative, common factor would explain both their willingness to make such declarations and their bargaining behavior across our myriad analyses.”Footnote ⁶

Although this is a useful start, the failure to find a relationship between the treatment assignment mechanism and the outcome of interest also raises a puzzle because it is again likely that actors strategically choose to make Article 287 declarations. This suggests that Mitchell and Owsiak's analysis might have overlooked some key factors that might impact the decision to make Article 287 declarations in the first place. In other words, their models still suffer from selection bias because they fail to account for all the differences between treated and control groups. Thus, the question remains as to what factors might be a common, omitted cause of both treatment and control groups.

At the most basic level, we can assume that governments make Article 287 declarations for two general reasons: 1) they want to peacefully settle/avoid maritime disputes and they believe that international courts can help them achieve this goal or 2) they make Article 287 declarations for reasons unrelated to the management of maritime disputes. With respect to the first reason, it is important to recognize that most governments prefer to settle disputes bilaterally because they retain more control of the dispute outcome. The data again support this, as so few states have made Article 287 declarations. Nonetheless, some states do make Article 287 declarations because they believe that it is in their best interests to do so. While an extensive discussion of this is beyond the scope of this essay, the extant literature suggests some clues to explain this small subset of states.

The underlying legal issues in the (potential) maritime dispute likely play a role here for a couple of reasons. States with a legal advantage vis-à-vis the maritime dispute are likely to prefer Article 287 declarations because they can expect to receive a favorable decision from international courts. In addition, some governments may view the (potential) maritime dispute as costly to them and prefer to resolve it, yet they are unable to resolve the (potential) dispute bilaterally because the legal issues in the dispute are complex or ambiguous.Footnote ⁷ Applied to the current context, states with a legal advantage or with (expected) disputes that are legally ambiguous or complex are more likely to make Article 287 declarations.Footnote ⁸ Therefore, it would be useful for this type of analysis to include measures that proxy the legal claims in the dispute. Although this concept can be difficult to operationalize, one can imagine proxies that capture the underlying legal issues in the maritime dispute, such as the size of the maritime interests (e.g., fishing industries) or the number of states with overlapping exclusive economic zones and shared continental shelfs.

Second, as noted, it is also possible that some states make Article 287 declarations even though they do not intend to utilize international courts. Instead, they, for example, only make Article 287 declarations to potentially satisfy international actors.Footnote ⁹ The authors attempt to proxy this by looking at the number of past maritime settlements. While this is useful, there are other potential proxies of this, such as land-locked states without maritime borders/sovereignty, or even the depth of integration in trade and/or intergovernmental organizations networks. The inclusion of these variables in the empirical analysis, if deemed appropriate, could help reduce selection bias in the statistical analyses.

Although the authors attempt to address the selection effects issue, they overlook the other main threat to inference: heterogeneous treatment effects. Here, we can ask how might the key independent variable—shadow effects of international courts—impact states differently? As a reminder, the authors argue that shadow effects lead to more peaceful and stable outcomes because international courts serve as a focal point that help to clarify the legal issues in maritime disputes, thus pushing states to peacefully resolve or avoid maritime disputes before they seek out international courts which can be costly to them.

While the authors are correct that adjudication involves costs, it also brings benefits that can help states to peacefully resolve disputes.Footnote ¹⁰ Scholars argue that decisions from international courts can provide domestic political cover to leaders.Footnote ¹¹ This is especially valuable for governments who are required to make concessions in agreements, as many scholars find that such leaders require domestic cover to avoid a backlash at home.Footnote ¹² In Mitchell and Owsiak's framework, however, governments are able to attain peaceful outcomes and avoid disputes because of the shadow effects associated with international courts rather than the direct impact of them.

This logic suggests that governments that do not require domestic political cover to settle disputes are more likely to benefit the most from shadow effects because it seems likely that leaders that are accountable at home would pursue adjudication. In other words, shadow effects would be insufficient for governments that are especially accountable to domestic actors. In short, it would be helpful for scholars that are interested in the impact of shadow effects on dispute resolution to account for domestic factors in their empirical analyses. The extant literature again provides some suggestions regarding how to operationalize domestic political accountability, beyond the regime-type of states. For example, Allee and Huth, in their work on adjudication and political accountability, use several variables to proxy domestic costs across both democracies and non-democracies, such as strength of the domestic political opposition, among other variables.Footnote ¹³

Conditional or Universal Findings?

There are at least two key points from the above discussion. First, and more conceptually, a deeper look at the sources of bias suggest that there might be some scope conditions in the present analysis and some concerns regarding the generalizability of the results. The authors’ own argument, the empirical findings, and the basic assumption of rational, strategic states suggest that the decision to make Article 287 declarations is not random. Furthermore, while it is possible that some governments make declarations to satisfy international actors, it appears that most states make sincere Article 287 declarations. In turn, this suggests that states that want to peacefully resolve their (expected) maritime disputes are the ones that are making declarations. At the same time, it also appears that shadow effects are most likely to be meaningful for states that want to settle their (expected) maritime disputes without using force, yet are secure enough at home to have little need for domestic political cover.

Taken together, the empirical results apply to a specific subset of states and therefore there are some important scope conditions associated with this analysis. Article 287 declarations appear to be most consequential for governments with a preference for resolving complex potential maritime disputes that may require legal-based clarification, yet have no need for the legitimacy associated with pursuing adjudication. This is important because it suggests that the implications of Mitchell and Owsiak's analysis are narrower than they claim. Yes, they note some concerns regarding the generalizability of the findings due to the high degree of institutionalization in UNCLOS, but they overlook the other scope conditions that also limit the generalizability of the empirical results.

Further, Article 287 declarations appear to indicate a screening effect on states rather than a constraining effect on them. In other words, the key variation seems to be about the decision to make Article 287 declarations rather than the impact of these declarations on state behavior. This has important real-world implications for understanding the impact of international courts on state behavior. Of course, this is not to suggest that Article 287 declarations are irrelevant, but rather, if states make Article 287 declarations to resolve disputes as suggested above, then it means that the focus of international and domestic actors should be on helping governments develop preferences for peacefully settling disputes and ultimately making Article 287 declarations, rather than focusing on the impact of Article 287 declarations on state behavior.

Second, most empirical work on international law, including Mitchell and Owsiak's important analysis, contain biases that need to be addressed to provide confidence in the estimated causal relationships. The authors implicitly assume that their models are identified using the conditional independence assumption. However, the discussion here and the focus on the sources of bias suggest that the authors need to think more carefully about the variation in the decision to make Article 287 declarations, and consequently include additional control variables in their models, such as proxies for state preferences regarding (expected) disputes, the underlying legal claims, and domestic political accountability. Thinking more comprehensively about causality and the sources of bias can help scholars identify a larger set of factors that might better satisfy the conditional independence assumption and produce more credible inferences. Put differently, scholars need to think about the causal story that links together the treatment assignment mechanism and the impact of the treatment on the outcome of interest in order to attain more reliable results.Footnote ¹⁴