1. Introduction
The adoption in 2015 of the Sustainable Development Goals (SDGs)Footnote 1 heralds a significant shift in the United Nation’s approach to international sustainability governance. The embrace of this quantitative, performance-oriented approach is not surprising given the rising role and influence of such approaches. Metrics such as goals, targets, and indicators have come to play important roles in several international regime complexes,Footnote 2 notably climate change and biodiversity, and in various governance initiatives at national, international, and transnational levels. The turn to data-driven approaches to sustainability makes a good deal of sense considering the great importance of scientific and economic inputs to sustainability. Nevertheless, the increasing prominence of metrics in an ever-expanding list of regimes and issue areas is cause for concern.Footnote 3 In particular, this increasing prominence leads to reduced space for legal obligation and reasoning, as well as political debate, discussion, and decision-making, in favour of technical, data-driven inputs that are often inaccessible to non-experts.
There are many reasons for international legal scholars to pay close attention to the growing importance of metrics in international governance. The first and most obvious is the possibility that law is losing ground to metrics. Law has long been a vital dimension of international governance, but for the foreseeable future, metrics will play a significant role alongside law in an array of issue-areas. Coming to terms with the roles and contributions of metrics is essential to anyone who seeks to understand international law, politics, political economy, and so forth. Second, there is a good deal of evidence that part of the reason for the turn to metrics is a direct response to actual or perceived limitations and deficiencies of law.Footnote 4 The challenges of grounding the legitimacy and authority of international law in a heterogeneous and inegalitarian international society are significant, and metrics offer an apparent advantage in this respect. Grounded in expert disciplines such as science and economics, and presented in quantitative terms, metrics seemingly possess an objectivity, and potentially a universal basis of validity, that stand in sharp contrast to law’s normativity. However, we draw on a body of critical literature in arguing that this is a false dichotomy – indicators are social technologies dependent on human judgement, not snapshots of an objective truth. Third, there are ways in which metrics and law could interact and support one another, to the benefit of both and to the broader benefit of international governance institutions and processes. More in particular, legal norms could play a greater role in guiding the processes through which metrics are incorporated into decision-making, reasserting the crucial role of legal and political normativity in these processes.
In this article, we consider the implications of the Sustainable Development Goals (SDGs) for international environmental law and cognate bodies of law relevant to sustainability, and explore potentially fruitful interactions between law and metrics, placing emphasis on the ongoing need for a strong contribution to environmental and sustainability objectives from law. The steadily declining attention devoted to law in international sustainability governance, of which the advent of the SDGs is merely the most recent and clearest example, suggests a disenchantment with law as a driver of change, and a desire for an approach more likely to deliver measurable, communicable results. We argue that the impression that law has outlived its usefulness is the product of a misapprehension of the nature of law and the range of contributions that it can make, rather than of law’s inherent weaknesses or limitations. If we are right about law’s crucial role in environment and sustainability governance, then it falls to scholars of international law, among others, to better articulate the roles and functions of law in these fields. This article seeks to contribute to this end by concentrating on interactions, and potential complementarity, between the SDGs and the law of sustainability.
This article begins with a brief overview of the SDGs (Section 2), then analyses the turn to metrics in governance generally and sustainability governance more in particular (Section 3), followed by a review of some of the most cogent critiques of metrics (Section 4). We then address the potential for complementarity between the SDGs and international law (Section 5), and the distinct and essential contributions of law to environmental and sustainability governance (Section 6), before concluding (Section 7).
2. The Sustainable Development Goals: Entrenchment of metrics in international environmental governance?
The title of the document that presents the SDGs promises sweeping scope and ambition: Transforming our World. This ambition is maintained throughout: for example, the SDGs are described as a roadmap and blueprint for a sustainable global society.Footnote 5 This type of hyperbole is typical of UN prose, but the metaphors are particularly ill-chosen, in that they suggest that the plan for a sustainable future is already in place and need now only be implemented. The technocratic tone of the text is enhanced by heavy reliance on the passive voice, obscuring agency in progress towards the goals and targets. Nevertheless, the Goals, viewed as a framing device and communications strategy, present rhetorical advantages over the interlocking rings of environment, society, and economy through which sustainable development is typically depicted. The Goals represent a similar effort to encapsulate the challenge of sustainability in a comprehensive way, but extend beyond the trivial assertion that everything is connected to everything by plotting out thematic areas of endeavour in both substantive and conceptual terms. One potential advantage of this communicative strategy is to make the sustainability project appear manageable by rendering its elements concrete and tangible. Goals seem more affirmative and precise than policy objectives, and the establishment of targets and indicators contributes to this affirmative logic by appearing to lay out a clear path and, importantly, a means of measuring progress along it.Footnote 6
The passive voice in which the Goals are communicated strengthens the impression of a turn to technocracy, and this may be deliberate: by drawing attention away from agency, this presentation suggests that progress towards goals will depend to a much greater extent on scientific – and therefore, presumably, objective, universally valid – expertise, rather than being subjected to the vagaries of political will. Political authorities will not be able to prevaricate, or to obscure a lack of progress in rhetoric and creative interpretation of legal obligation and political commitment; the numbers will speak, if not quite for themselves, then at least in a less mediated, judgment-dependent manner.
There is evidence that the turn to metrics is driven, at least in part, by frustration and disappointment with law within international governance institutions. Law was central to discourse and action at the international level from the period when environmental quality became a priority on the international agenda, in the 1970s, to around the beginning of this century. By the time the Rio +20 sustainability summit was held in 2012, law hardly merited a mention in official action documents. The adoption in 2015 of the SDGs is consistent with a belief that law, while it retains an important role in the background of international sustainability governance, has lost its centrality.
The SDGs comprise 17 goals that bring together a broad range of sustainability objectives. They range in focus from economic development to human rights and environmental protection.Footnote 7 Associated with each goal is a series of targets and indicators. The targets are representative rather than comprehensive; they do not purport to address each facet of the goal with which they are associated. As for the indicators, they, too, are representative, providing a means to measure progress towards the targets. Targets and indicators are proxies, selected for their capacity to provide important insights into broader and more complex phenomena.Footnote 8 Relatively well-known indicators include the Gini coefficient, which uses income distribution to measure inequality,Footnote 9 and gross domestic product, which tracks the value added by production of goods and services in a given state to measure economic activity.Footnote 10 The great advantage of such indicators lies in their capacity to synthesize large amounts of data into a simple data point, which can then be used to evaluate complex states of affairs and phenomena, make comparisons across time and space, and identify correlations between distinct factors.Footnote 11 The great disadvantage lies in this same synthesising, reductive approach: nuance and complexity are lost.Footnote 12 Furthermore, any flaws in the selection of indicators or in data collection and analysis will contribute not only to a highly simplified picture but to an inaccurate one. Nevertheless, indicators are powerful instruments for research as well as planning and governance, provided those who are making use of them keep their limitations squarely in mind.Footnote 13
The SDGs build on important experience gained through a similar project of narrower scope, the Millennium Development Goals (MDGs).Footnote 14 Sustainability comprises a much broader array of issue-areas, incorporating development but extending to a host of social and environmental issues.Footnote 15 The SDGs are more ambitious in another way as well: in addition to covering more ground, each goal is more complex and multi-faceted than the MDGs, seeking to address environmental, social, and economic dimensions through a broader set of targets and indicators. This is a laudable objective: one major weakness of indicators is the tendency to measure objects because they are easily measurable, and not because they provide important insights into the broader context.Footnote 16 However, it is necessary to be thoughtful and creative when seeking to measure difficult-to-grasp phenomena, such as the subtle effects of sexism or racism on people’s capacity to benefit from economic development, the uneven distribution of costs and benefits associated with environmental protection measures, or objects that are not only hard to measure but also hard to define, such as freedom, democracy, good governance, and rule of law.Footnote 17
It is far from clear that all the SDG targets and indicators have been carefully designed or selected. As we discuss below, some goals have clearly suffered from lack of consensus and from political compromise. Furthermore, many of the indicators are acknowledged to be insufficiently developed.Footnote 18 This should lead all those who work with the data produced through the SDG project to proceed with caution, bearing in mind the weaknesses and limitations of the information they are working with. However, it is difficult for non-experts to look behind the apparently simple and clear figures and data points that the Goals produce. Therefore, there are reasons to fear that their limitations will not be kept squarely in mind as they are fed into political and legal decision-making processes.
3. Expanding influence of indicators
The use of performance indicators first became widespread in private sector auditing practices. In the corporate context, auditing responded to demands for the enhancement and transformation of corporate organizational governance after repeated corporate failures, as fiscal and budgetary constraints spurred the search for new forms of accountability.Footnote 19 Similar disillusionment with traditional bureaucracies led to reliance on indicators in several jurisdictions through the passage of a series of policy reforms now known as New Public Management (NPM). NPM is a governance approach that involves a division of labour between political agencies that set programme objectives, and usually private organizations tasked with implementing programmes and delivering services.Footnote 20 With this division of labour comes a need for means to ensure that those private organizations are carrying out their assigned responsibilities appropriately. NPM borrows from corporate practices of performance audits and evaluations to meet this need.Footnote 21 The essence of NPM is to substitute, or supplement, traditional bureaucratic structures with private sector management strategies. In so doing, governments seek to promote fiscal restraint, reduce the role of the state in the provision of services, and improve public sector accountability.Footnote 22 Reforms favoured more nimble, independent public organizations and competitive tendering processes with the private sector. To this end, managers were given wide latitude in programme design, resource allocation, service delivery, and in contracting outside of the bureaucracy. In this effort to shift the public sector to a more result-oriented model, audits and performance measures were designed to respond to criticisms that NPM resulted in accountability deficits.Footnote 23 Audits were meant to assuage concerns over departments operating at arm’s length, and performance indicators presented a seemingly objective way of assessing private sector bids for public contracts.Footnote 24
Performance evaluations are now prevalent in a range of contexts and substantive issue areas,Footnote 25 including several contexts in which they are of questionable utility due to complexity, data gaps, methodological limitations, and other problems.Footnote 26 Metrics have long been in use to measure the size of economies and economic growth,Footnote 27 country risk profiles for investment purposes,Footnote 28 levels of corruption,Footnote 29 democracy and good governance,Footnote 30 and economic inequality,Footnote 31 and are increasingly relied on by international environmental regimes.Footnote 32 In some cases, such as the ozone layer protectionFootnote 33 and climate regimes,Footnote 34 metrics are built into the architecture of the regime itself; in other cases, they are a supplemental means of promoting particular objectives within regimes.Footnote 35 The Intergovernmental Panel on Climate Change (IPCC) and, more recently, the Intergovernmental Platform on Biodiversity and Ecosystem Services (IPBES), were created to provide scientific inputs to governments and international regimes. While not all of these inputs take the form of metrics, they are a prevalent communicative conduit between science and policy. The IPBES in particular is investing heavily in the development of metrics to measure the robustness of ecosystem services.Footnote 36 The Planetary Boundaries project deserves mention in this context as well: launched by a team of natural and social scientists, this project endeavours to delineate a ‘safe operating space for humanity’ through the identification of nine ecological boundaries that cannot be surpassed without risk of ecological collapse.Footnote 37 Another similar scholarly initiative is the Environmental Performance Index, which ranks states based on a series of environmental indicators.Footnote 38
When used with skill and care by analysts and decision-makers who have a clear understanding of the limitations of metrics, indicators make significant, even essential, contributions. In fields such as environmental protection and sustainability, which are not only dependent on expert input but are also deeply complex, indicators can shed important light. The comparisons across time and space that they allow could lead to insights into initiatives or approaches that tend to work well, as well as the conditions under which they seem to be effective. They may also contribute to the identification of initiatives that are not effective at meeting their objectives and that may also be generating excessive costs in the form of unintended and undesirable consequences. However, indicators can also be deeply misleading. This is very often the case when decision-makers put too much store in indicators, failing to bear their limitations in mind and mistaking them for objectively true, unmediated information.Footnote 39
4. Shrinking the space for normativity: Key critiques of indicators
Given the growing importance and influence of metrics in sustainability governance, it is essential to scrutinize them, and the ways in which they are relied on in governance, very carefully. Scrutiny is particularly important given the propensity of indicators to obscure the crucial contributions of professional and normative judgment in their creation and use. Where political consensus is elusive and the authority of decision-makers is precarious, it can be tempting to seek to transform complex and difficult decision-making processes into essentially technical exercises.Footnote 40 Legal scholars have increasingly recognized the need to attend carefully and critically to the ways in which indicators are deployed in decision-making contexts.Footnote 41
4.1 Impacts of quantification
Some of the most trenchant criticisms of the use of metrics in governance target the various ways in which metrics obscure as much as, or more than, they reveal. Important concerns are often raised about the reductivist nature of metrics, or the flattening effects they have on the objects they address: forests become carbon; poverty becomes lack of money; wellbeing becomes GDP; access to resources becomes formal title to land.Footnote 42 As Sally Engle Merry has put it, ‘[n]umerical measures produce a world knowable without the detailed particulars of contexts and history’.Footnote 43 Particularly problematic is what is obscured: the data-driven, often numerical representation of phenomena foregrounds certain disciplines, types of expertise, and forms of knowing, further privileging those that are already highly influential.Footnote 44 Concerns are often raised at the ease with which numerical representations may be converted to dollar values, thus monetizing and commodifying the non-human world.Footnote 45 The political and other normative dimensions of issues are obscured: as complex phenomena come to be represented as indicators, the indicators themselves acquire a taken-for-granted quality,Footnote 46 such that the assumptions, ideologies, choices, and judgments that went into their construction cease to be visible.Footnote 47 Many critics further argue that reliance on metrics tends to promote minor policy initiatives and modest innovations that leave existing power dynamics, inequalities, and other deep-rooted, often structural problems intact, possibly even enhancing and reinforcing such problems.Footnote 48
While some critics’ concerns about metrics lead them to be deeply suspicious about their use in governance, others argue that their potential contributions are significant enough to warrant their use, as long as their limitations and weaknesses are kept in view and the uses to which they are put are carefully selected. The highly influential work of Theodore Porter on quantification, while critical of this phenomenon, also provides insights into its functions, highlighting important paradoxes relating to quantification and its influence on collective decision-making. For instance, he notes associations between dependence on impersonal, objective expertise and democracy, on the one hand, and the highly contextual exercise of individual judgment and the presence of a ruling class such as an aristocracy, on the other.Footnote 49 Some of the key reasons why indicators have been taken up so enthusiastically by international organizations such as the UN relate to the difficulties of generating confidence and trust in international public authorities, particularly in socio-economic domains such as sustainability that are particularly normative, addressing social policy and social justice issues. In contexts in which people know one another and trust may be built over repeated interactions anchored in a broader social and cultural context, quantification and appeals to the authority of expertise and rule-bound decision-making are less necessary. However, as Porter has pointed out, in contexts in which the authority of a decision-maker cannot be established on the basis of personal knowledge and experience, nor with reference to common social institutions or shared understandings that bestow such authority, some other means must be devised to ensure that the decision-maker is not acting in an arbitrary or self-interested manner.
In the context of collective decision-making, trust is not merely a matter of expert qualifications but also of showing one’s work, for example by making public the basis on which a conclusion was reached. Because neither raw data nor detailed analyses are accessible to laypersons, decision-makers resort to demonstrating that they followed prescribed rules or accepted practices in reaching their decisions.Footnote 50 Indicators facilitate this public accounting: definitions, categorizations, criteria for evaluating and measuring phenomena, and highly standardized structures for reporting results serve to constrain the agents whose task it is to apply indicators and produce the requisite statistics, rankings, or ratings. In commenting on the use of statistics in medical research, among other fields, Porter argues that:
they work mainly as social technologies, not as guides to private thinking [on the part of researchers]. The advances of statistics in medicine must be understood as responses to problems of trust, which have been most acute in the context of regulatory and disciplinary confrontations.Footnote 51
This observation is highly pertinent to the equally fraught world of collective sustainability decision-making at the global level, where the authority of governance institutions is weak and shared understandings thin. In this context, accountability of public authorities increases in importance.Footnote 52
This discussion highlights the fundamental importance of legal normativity, notably procedural rules, to the structure of decision-making processes. Such procedural rules serve a range of interdependent, pragmatic, and normative objectives. In addition to their potential for generating trust in processes and the decisions they produce, procedural rules can enhance fairness by promoting the like treatment of like cases and the capacity of implicated actors to participate in the process and defend their interests. They can make it more difficult for decision-makers to pursue their own interests and priorities rather than those of the society on whose behalf the decision is being made. Procedural rules can also address inputs into decision-making, for example by identifying the forms of knowledge and expertise, and the range of perspectives, that must be addressed and incorporated.
The importance of procedural rules, and the range of contributions they make to the quality of exercises of authority and conclusions among stakeholders about the legitimacy of that authority, are well captured in literature on Global Administrative Law (GAL).Footnote 53 Nico Krisch and Benedict Kingsbury describe the field of GAL as:
encompass[ing] the legal mechanisms, principles and practices, along with supporting social understandings, that promote or otherwise affect the accountability of global administrative bodies, in particular by ensuring these bodies meet adequate standards of transparency, consultation, participation, rationality and legality, and by providing effective review of the rules and decisions these bodies make.Footnote 54
Anna Huggins, applying insights from GAL to compliance systems in international environmental regimes, notes that the adoption of procedural rules in this area ‘constrains power politics and enhances regard for the interests of affected states’, and further that such rules ‘help to level the playing field for less powerful states by giving them greater voice in compliance deliberations, increase the responsiveness of decision making to states’ diverse capacities and circumstances, and justify the imposition of consequences for non-compliance’.Footnote 55
At the international level, concerns about the politicization of decision-making in particular and exercises of authority in general are particularly serious. As discussed, one of the central means of reining in the politicization of decision-making is to place the process in the hands of experts, or at least to give experts extensive roles in decision-making.Footnote 56 However, as argued above, experts do not make decisions in purely technical ways, driven by the data rather than their own analysis – even when the matter at hand is, in fact, technical. In addition, as Huggins argues, attempting to depoliticize decision-making in this manner can lead to re-politicization. This can happen, for example, when the authority of experts expands into normative areas, effectively requiring them to make judgments not strictly based on expertise. When significant weight is placed on scientific inputs to political and legal decision-making, the space for political disagreement and debate is reduced, and disagreements are thus far more likely to take place on scientific terrain: disputes about the scientific basis of different positions become proxies for political or normative disagreements.Footnote 57 Repoliticization can also happen if assumptions about the neutrality of experts are not examined; for example if their professional dependence on the states that appoint them is not attended to.Footnote 58
Quantification and measurement may make important contributions to the development of procedures and standards to ensure objectivity and rigour. However, there is a strong tendency to assume that numbers in and of themselves possess universal validity and objectivity.Footnote 59 Standard forms of measurement as well as standard procedures and methodologies for analysis and decision-making can certainly inject a degree of transparency, thus promoting trust. While the architects of the process through which the SDGs were adopted paid close attention to participation and inclusion,Footnote 60 there is little evidence that the normative dimensions of the design and application of the Goals continue to be a high priority.Footnote 61 Furthermore, there are reasons to be concerned that too much is being demanded of metrics, and in particular that assumptions about their objectivity and validity contribute to dubious conclusions regarding their capacity to replace more subjective processes which avowedly involve the exercise of professional and political judgment.
4.2 Obviating the need for judgment
Time and again, critical literature on indicators refers to one of the underlying purposes that indicators in governance contexts seems to serve: the promotion of objectivity in assessment and evaluation of phenomena of interest, and in decision-making by public authorities. Many authors have noted the multiple ways in which objectivity is understood. One common conception, often used to explain the putative superiority of scientific inquiry over other forms of knowledge, is to see objectivity (in science and elsewhere) as ‘bound up with questions about the truth and referential character’ of scientific theories and other knowledge claims.Footnote 62 But, as Helen Longino notes, another, more pertinent understanding holds that objectivity is ‘achieved by reliance upon nonarbitrary and nonsubjective criteria for developing, accepting, and rejecting the hypotheses and theories that make up the view [provided by science]’.Footnote 63 This second sense of objectivity does not demand that objective knowledge be just that knowledge that can be gained through unmediated observation of the world as it really is. It demands, rather, that observers have good reasons for believing their analyses and conclusions to be valid, and that those reasons will also be communicable and acceptable to other observers, in turn giving them grounds for accepting the observations in question. In this sense, the ends of objectivity run parallel to those of the rule of law, implying ‘the subordination of personal interests and prejudices to public standards’.Footnote 64
This approach to objectivity is highly relevant to understanding the respective contributions of metrics and professional and political judgment to collective decision-making. Metrics are very often introduced into such decision-making contexts, and reliance on them increased, deliberately to reduce the reliance on the judgment of individual experts, including civil servants, political authorities, and judges. If one assumes that objectivity must refer to the extraction of pure, unmediated information that possesses universal validity, then one would have reason to distrust the exercise of professional judgment through established procedures for similar reasons to those that provoke distrust in judgments made by individuals based on their personal preferences. However, the picture changes if one acknowledges the important role of judgment in science. Governance metrics are particularly dependent on professional judgment because of their hybrid nature: designed not only to encapsulate complex data sets for greater portability and ease of comparison, these metrics are crafted to measure explicitly normative objects, namely progress towards objectives identified in political, economic, and other realms. Each major step in the design of metrics, and in the interpretation of the evaluative data that they produce, involves the exercise of professional judgment, explicitly informed, or implicitly influenced by normative assumptions about what counts as corruption, stability, democracy, empowerment, sustainable management, and so forth.Footnote 65 This could be taken to mean that metrics are not really objective, after all. But as the above discussion suggests, objectivity may be understood in different ways; furthermore, scientific knowledge is not objective in the narrow sense of being free of judgment and therefore arbitrariness, providing unmediated access to knowledge of the world.
If the exercise of judgment cannot be purged from decision-making processes simply by making those processes excessively dependent on scientific inputs, certainly process, structure, and standards can move decision-making closer to objectivity in the sense referred to by Longino. Using this definition, we can conceive of metrics whose normativity and dependence on judgment are not denied, but that are nevertheless based on non-arbitrary and non-subjective criteria. By contrast, when metrics are used to reduce reliance not only on personal, subjective judgment but also on professional judgment, problems arise, as noted by AnnJanette Rosga and Margaret L. Satterthwaite in their analysis of indicators for measuring respect for economic, social, and cultural rights. The need for some objective means to evaluate state compliance with their obligations under human rights conventions is apparent, but the nature and structure of economic, social, and cultural rights pose difficulties. This flows in part from the content of the rights themselves. To determine whether the prohibition on torture is being respected, it suffices to determine whether people in state custody are subject to torture, whereas a state could promote, or fail to promote, the right of workers to a living wage in a variety of ways – and the reasons for which workers could fail to receive a living wage could have only indirectly to do with state behaviour. First-generation rights tend to depend in a clearer and more direct manner on state action – punishing political dissenters, prohibiting demonstrations, or censoring the press, for example. The realization of economic, social, and cultural rights may require a network of initiatives including legislated standards and requirements but also extending to information campaigns, education and awareness-raising, and modifying incentive structures, all of which may influence but will not determine progress.Footnote 66 Furthermore, the nature of states’ obligations in this context tend to be different: for obvious reasons it is not enough for a state to merely endeavour to refrain from torturing political dissidents, while a goal such as high levels of access to primary school education for girls is more a matter of degree than kind. This is explicitly recognized in the structure of states’ obligations in this context.Footnote 67 Respect for these rights is thus measured along two dimensions: from the point of view both of the state – efforts made towards realization of rights – and that of its population – enjoyment of rights.Footnote 68 While the relevance of both these dimensions to the underlying question regarding states’ respect for their human rights obligations is quite apparent, there are problematic means-ends assumptions underlying the measurement of these dimensions. Rosga and Satterthwaite note that the indicators were designed explicitly to reflect cause-effect relationships between the institutional mechanisms put in place by states, efforts invested to promote rights, and the purportedly resulting enjoyment of rights,Footnote 69 and argue that ‘understanding cause and effect requires extremely detailed, comprehensive, and context-specific analysis’.Footnote 70
The reason that judgment is viewed as potentially problematic in this context relates to the structure of international society. Obstacles to validation and legitimation of legal norms are a feature of the modern world, but are more acute at the international level. Bodies such as the United Nations High Commissioner for Human Rights (UNHCHR), the High-level Political Forum on Sustainable Development responsible for the administration of the SDGs, international courts and tribunals, and myriad other institutions and agencies struggle to articulate and affirm the bases of their own authority. Given structural inequalities and historically rooted differences in power and influence, many states and other actors do not trust international agencies to see issues from their perspectives or to understand and seek to promote their interests. In contexts such as these, the appeal of metrics lies in their apparent ability to bypass normative processes of interpretation and evaluation and get at what is really there – at tangible, graspable phenomena such as numbers of children in school, numbers of states with minimum wages, acres of forest under protection, numbers of species on red lists, and so on.Footnote 71 However, the utility and value of these numbers cannot be realised unless the context that gives them meaning, and which makes the determination of their utility and value possible, is clearly recognized and acknowledged. An unavoidable component of this context is judgment: judgments about the standards and criteria by which tools and technologies such as indicators are to be evaluated, judgments about the construction and implementation of such technologies, and, above all, judgments about what ought to be done in light of the insights that they provide.
The experience with the UNHCHR is instructive. Prior to that body’s turn to indicators, the approach taken to assessing compliance with human rights was ‘a qualitative and quasi-judicial exercise’Footnote 72; this approach was seen as flawed, and a move to what was regarded as a more objective approach was deemed desirable. Given the context of international society, and more in particular the context of second-generation human rights, it is not surprising that the authority of the OHCHR was not sufficiently anchored to admit of deep and broad consensus on its judgments.Footnote 73 If an objective, technical, even scientific process could be substituted for this quasi-judicial one, arguably the problem posed by this lack of authority could be diminished.Footnote 74 Naturally, everything turns on whether such a substitution is feasible or appropriate. Even in the presence of robust indicators and good data, there is a gap between the number produced by way of the indicator (e.g., proportion of females with professional or university qualification), on the one hand, and the two questions that require answers, namely the adequacy of state efforts to realise rights, and their enjoyment by members of the population. This gap may be filled, argue Rosga and Satterthwaite, only by judgment.Footnote 75 Thus, they note, the problem of the treaty bodies’ authority is not solved, merely, at best, relocated.
A valuable lesson to be learned from the human rights experience may be that the utility of metrics depends on a frank acknowledgement of both their contributions and their limitations. As Rosga and Satterthwaite argue:
The value of indicators as a social technology cannot be determined in advance, nor on the basis of the fact that they are largely quantitative. While it may be true that quantitative methods, in their very abstraction and stripping away of contextualizing information have particular – and especially high – risks for misuse by those with the power to mobilize them, they are tools like any other. All tools can be misused; all social actors with power can misuse that power. The key lies in knowing where – and how – human judgment and political contestation should enter.Footnote 76
Similarly, Andrew Williams and Abu Siddique, following a critical assessment of governance indicators measuring phenomena such as corruption, political stability, and quality of political institutions, conclude that their criticisms should be read ‘more as a cautionary tale of their limitations, rather than as statements describing why they should never be used. Researchers need to be aware of both the strengths and limitations of these governance measures’.Footnote 77
It goes without saying that legal analysis depends on interpretation and judgment, on the part not only of adjudicators but also lawyers, as well as domestic and international civil servants, whether legally trained or not, who are required to make assessments and evaluations of states of affairs in light of norms and standards. The effort to restrict the space for judgment based on interpretation of rules may have simply demonstrated the necessity of judgment, as well as the persistent relevance of legal normativity.
5. The SDGs and international law: Working towards complementarity
The SDGs need not be in tension with law; their action-oriented, data-driven approach could potentially serve as a complement to law’s normativity. Conversely, there are good reasons to believe that governance metrics cannot do without law and should not be understood to supplant legal normativity. In contrast to the passive voice of the SDGs, law speaks the language of right and obligation, respect and violation, responsibility and accountability. It is therefore of great interest to explore ways in which these two different logics could reinforce one another. In this section, we address the potential contributions of the SDGs to international law; in the section following, we shift perspectives and consider the persistent need for legal normativity for international sustainability governance.
The effectiveness of a legal norm may be evaluated along material and formal dimensions, and in these respects the SDGs may be of great utility. However, legal regimes and systems are much more than a collection of individual norms, a point to which we return in the following section. Material effectiveness refers to the extent to which legal norms promote the objectives for which they were adopted: for example, legal obligations to preserve the habitat of endangered species ought to produce measurable improvements in habitat preservation. Formal effectiveness refers to compliance with legal norms. Data collection and analysis activities under the auspices of the SDGs could shed important light on both dimensions of effectiveness, promoting targeted experimentation, learning, and adaptation of legal norms and regimes. The value of feedback mechanisms to provide insights into the impact of legal regimes has been championed by scholars of adaptive management such as J.B. Ruhl. Ruhl notes the significant investment in the design of regulations, policies, and regimes – what he describes as the front end of the regulatory process – compared to little or no attention to the back end, involving analysis of the impact of regulatory regimes.Footnote 78 The complexity of sustainability issues and of governance efforts designed to address them create a great need for learning and adaptation, processes that depend on insights from experiences on the ground. It is not only the impact of legal norms on the environment that is of interest; their impact on the communities in which these norms operate, and on other dimensions of environmental protection, must also be considered, particularly when these impacts are negative.Footnote 79
The challenges are daunting. Many of the SDG indicators are not robust, either because of weak methodology, lack of access to good data sources, or both.Footnote 80 Data for certain indicators will be particularly difficult to collect in some locations. Furthermore, there are problems with seeking causal linkages between legal norms and progress towards environmental and sustainability objectives. In the first place, legal norms are not merely material and instrumental; they have myriad other dimensions, which can be difficult to capture through indicators. Second, interactions between legal systems and their environment are complex, and reasons for the successes or failures of legal rules and institutions – assuming that one can reach consensus on how success or failure is to be conceived – are difficult to identify. Any insights gained into these interactions will require careful and critical interpretation and analysis of the information that indicators make available alongside other data sources, ideally conducted by interdisciplinary teams whose members can see beyond the indicator itself to the broader context. Any decisions made by political authorities should not be made on the basis of the indicators themselves but rather should be informed by a richer and more multi-faceted analysis.
5.1 Measuring the effectiveness of law: Beyond instrumentality
Certain important weaknesses of the SDGs, as well as limitations of indicators and other metrics in general, are brought into sharp focus through an examination of rule of law indicators. Legal indicators that ‘purport to measure some aspect of the performance of the legal system’ are a fast-growing category of the newest generation of global governance indicators.Footnote 81 International organizations consider the state of the rule of law in a country to be an important determinant of development.Footnote 82 International organizations and developed country governments operating under this assumption have relied heavily on rule of law indicators to help guide investment in the legal institutions of developing countries.Footnote 83 The rule of law indicators represent a fruitful case study of global governance indicators in part because of their influence, but also because they seek to standardize a fundamentally amorphous concept. Unfortunately, with such weak conceptual starting points, many rule of law indicators do not reflect the underlying phenomena they are meant to shed light on.Footnote 84
The difficulties of framing rule of law indicators are exemplified by the troubled negotiations over the place of rule of law in the SDGs. Goal 16, ‘Promote peaceful and inclusive societies for sustainable development, provide access to justice for all and build effective, accountable and inclusive institutions at all levels’, comprises rule of law and access to justice as its third of 12 targets. The decision not to articulate rule of law as a goal was driven by lack of consensus on multiple issues, including disagreement as to whether rule of law should be included in the SDGs at all, about its content and definition, and regarding how best to measure it.Footnote 85 Instrumental approaches to the rule of law treat it as a means to an array of objectives, notably the provision of a secure investment and financial environment, to the benefit of investors and creditors; clearly, such approaches are of minimal utility for the SDGs.Footnote 86 This disagreement extends well beyond participants in the SDG negotiations to political theorists, international political leaders, and of course experts in governance metrics.Footnote 87 Nor is there consensus on the purposes to be achieved by measuring rule of law.Footnote 88 Versteeg and Ginsburg have studied the most cited rule of law indicators, namely those developed by the World Bank’s World Governance Indicators project, the Heritage Foundation, Freedom House, and the World Justice Project, and have remarked on their idiosyncratic approaches to framing the concept.Footnote 89 Conceptualising the rule is difficult, because producers must contend with the elusive nature of legal norms, precedent that is open to various interpretations, and the conflicting ways that administrators enforce legal norms. In short, the conceptualization of rule of law indicators reflects the pre-existing fault lines in the theoretical literature on the rule of law literature.Footnote 90 Scholars have criticized indicator producers for failing to justify building significantly different values into their definitions of contested concepts.Footnote 91 Since indicator producers are starting from such different definitions, indicators purporting to measure the same thing are liable to differ significantly. In their study of a few rule of law indicators, Møller and Skaaning found such a lack of correlation.Footnote 92 By contrast, Versteeg and Ginsburg were surprised to find a strong correlation between the indicators they studied,Footnote 93 but argued that measurement strategies explained the convergence: even if rule of law is being defined in different ways and measured through differently constructed indicators, the scarcity of relevant data means that these different indicators are being measured through reference to many of the same data sources.
As noted above, measurement of the effectiveness of law, whether in a broader rule-of-law context or a narrower evaluation of the impact of legal regimes and instruments, is based on assumptions about the capacity of law to bring about identified ends. Measurement of the effectiveness of legal regimes is more avowedly instrumental, and risks minimizing other contributions of legal regimes, systems, and institutions, such as fairness, stability, or, even more ambitiously, the extent to which legal subjects see themselves and their aspirations reflected in legal systems – that is, law as the concept is known to legal and political theory. Gutmann and Voigt come closer than most, firmly grounding their approach in theoretically informed conceptions of the rule of law, informed in particular by Friedrich HayekFootnote 94 and Lon Fuller,Footnote 95 scholars who were concerned with the form of law. The dimension of rule of law that Gutmann and Voigt ultimately wish to study comprises a series of ‘formal traits’, including general, open, and certain rules; separation of powers; judicial independence; and a subset of human rights such as prohibition of torture and extrajudicial killings.Footnote 96 They readily acknowledge that these traits can be measured only indirectly, and therefore focus on enforcement.Footnote 97 Given these limitations, other, more expansive research approaches are clearly required, based on a range of methodologies and data sources, and proceeding through analytical and dialectical reasoning processes that necessarily entail subjective evaluation and judgment. In this manner, there is some reason to hope that metrics may be used to evaluate legal initiatives without thereby reducing law to the single dimension of instrumentality. However, this is certainly not the approach taken in the SDGs: lack of consensus made it impossible to adopt such a rich, multifaceted perspective on this concept.
6. Legal normativity and global sustainability governance
The references made within the SDGs to law suggest that two main roles have been carved out for law: first, through the provision of material norms relating, in particular, to human rights, corporate and commercial practices, and environmental protection; and second, to rule of law and good governance. The approach taken to these last two issues is very narrow and limited, standing in sharp contrast to the ambition and scope of most of the other goals. While there are important points of intersection and much potential for complementarity between international law and the SDGs, truly fruitful interaction can only be achieved if important weaknesses and limitations of the SDGs are acknowledged. Law is well suited to address many of these weaknesses and limitations, pointing to ways in which the deployment of sustainability metrics could benefit from the support, as well as the discipline, of legal normativity. In what follows, we discuss two features of the SDGs that give rise to concern. The first, the process of identifying a unit of analysis, is a perennial issue facing the use of indicators: complex phenomena must be broken out into relatively clearly identifiable categories that are assigned values for ease of comparison and analysis. This flattening of the landscape is a feature of metrics rather than a bug, but when these categories and values are mistaken for the phenomena themselves, the capacity of metrics to render visible what can only be glimpsed or guessed at through other methodological approaches may be eclipsed by a tendency to obscure other perspectives. Second, metrics are often put to uses for which they are ill-suited: their promise of objectivity and universal validity can tempt political authorities and observers to rely on them too heavily.
There is an inevitable tension between the objective of developing deep and detailed understandings of phenomena such as the impact on women of environmental degradation and of programmes to combat it, on one hand, and that of packaging data in forms that lend themselves to relatively easy analysis at the global level, on the other. We have outlined some of the key contributions of indicators: permitting relatively straightforward comparisons across time and space, which can foster the generation of hypotheses about the impacts of legal or policy initiatives or the conditions under which they succeed or fail; and identifying correlations among factors or phenomena that might otherwise pass unnoticed. As noted above, metrics, particularly those that are presented quantitatively, can readily assume a taken-for-granted quality, and the mediated, filtered nature of the insights that they produce can easily be forgotten. Communication about the robustness of metrics and of the methodologies and data on which they are built is one way of attenuating this problem somewhat, but it can be difficult to provide information that is both meaningful and easily interpreted by non-experts, that is, actors not well versed in the construction and deployment of metrics. As Porter suggests, an important dimension of trust and confidence in decisions made or conclusions reached by experts is to show one’s work – to demonstrate that the inputs and processes through which the conclusion or decision was derived meet standards of rigour, validity, or appropriateness. GAL literature helps us to understand that the way decisions are made can be as important as the substance of the decision. Law could make a potentially significant contribution to the SDGs in the form of structures and processes for decision-making regarding the construction and use of metrics, as well as standards that could be used to validate metrics used in legal and political decision-making.
While it makes sense for experts in the scientific fields in which a given metric is grounded to have significant input into their construction and guidance in their use, metrics are normative as well as technical, and therefore their validation ought to be approached from a range of perspectives. As Benoît Frydman has argued, if indicators are understood mainly as a means of conveying facts, questions about their legitimacy would not normally arise; one would rather be concerned about their relevance and validity.Footnote 98 However, metrics do not merely convey knowledge or information; they are steering devices.Footnote 99 This is explicitly acknowledged in the documentation on the SDGs. As a result, Frydman argues, ‘the validity of these indicators must be assessed on the basis not only of their accuracy, but also of their political legitimacy’.Footnote 100 Legality and rule of law frameworks are certainly not the only ones available to perform this legitimising function, but they are useful, possibly even essential.
Any measurement requires the definition of the category that is to be measured. The boundaries of that category, and decisions regarding the objects that fall within it, are always subject to a degree of arbitrariness.Footnote 101 But the process of categorization is normative through and through. Furthermore, certain normative dimensions are the result not so much of deliberate decision-making but of unexamined assumptions and prejudices. Critical literature on environment, development, and sustainability draws our attention to the influence that clichés, tropes, and stereotypes have over processes of categorization, and the resulting distortions to information gathering and analysis. This is readily apparent in the treatment of gender in the context of environment and development. Seema Arora-Jonsson notes tendencies in literature on women and environment to rely on ‘[t]he category of the victimized woman of the global South’, and to ignore women’s agency.Footnote 102 Women tend to be conceptualized as being closer to nature and therefore as being effective environmental stewards, but these assumptions have been shown to contribute to a ‘feminization of responsibility’ for addressing environmental problems, irrespective of the actual capacity of women in various contexts to assume this burden.Footnote 103 These observations provide compelling evidence for the relevance to the validity and legitimacy of metrics as sources of knowledge and insight, of the processes through which they are constructed.
A good indicator must bear some relationship to a larger phenomenon that is the real object of interest; this is clear from the very term. Indicators of gender equality clearly do not measure gender equality in anything like the way a thermometer measures the temperature of a liquid: a well-designed indicator will shed important light but leave much obscured, as would any analytical device or approach. Much vital work remains to be done at the stages of interpreting the results an indicator produces and understanding the potential implications of these results. When the absence of equivalence between indicator and object is not attended to, in particular when it is assumed that indicators not only measure objects but explain outcomes and identify causal pathways, the utility of the indicator may be sharply reduced; indeed, it would tend rather to mislead, particularly when the indicator is grounded on problematic assumptions and generalizations. Indications of ecosystem degradation may be taken as evidence of a failure of leadership or stewardship on the part of local communities and decision-making structures, while constraints imposed by national and international markets and financial flows go unnoticed.Footnote 104 Indications of degradation should rather trigger further investigation and the testing of a wide range of hypotheses. In other words, the indicator provides not a diagnosis but information that may be used to work towards a diagnosis.Footnote 105
A significant dimension of the promise, and therefore prominence, of metrics is their grounding in objective, expert knowledge; that is, their facticity, as distinguished from the normativity of law or politics. However, we have seen that this promise cannot be kept; metrics are normative and cannot be constructed or fed into analysis and decision without the use of judgment. Attending to the processes through which metrics are constructed can serve to bring this judgment out into the open, which, in the short term, may have a negative impact on the perceived validity of metrics, while in the longer term it may contribute to their legitimation, thus setting them on firmer foundations. Legal rules alone will not suffice for purposes of validation and legitimation of metrics, the processes through which they are constructed, and the decisions made in large or small part on their basis. However, standards and procedural rules, whether in the form of legally binding obligations or policy guidance, can certainly contribute to this process.Footnote 106 Frydman argues in favour of going well beyond validation of metrics by experts, subjecting metrics that are used in the exercise of political power to the rule of law.Footnote 107
In addition to contributing to the structures and processes through which metrics are designed and implemented, legal normativity also has an independent role to play alongside metrics such as the SDGs. As noted above, there has been a fairly pronounced tendency in international environmental governance towards metrics and away from legal normativity. In the early years of the development of international environmental law, by contrast, there was more emphasis on principles of public international law such as state responsibility and territorial integrity, and on the development of general principles of international environmental law such as precaution and common but differentiated responsibility. The necessity of instruction rules that create fairly clear parameters for states, such as the target-and-timetable approach taken by the Kyoto Protocol and other regimes, is clear: sharp reductions in the emission of harmful substances such as carbon or sulphuric dioxide can certainly be fostered by goals and objectives that are both legally binding and relatively easy to measure. But, as the work of GAL scholars makes patently clear, there is more to law than legal rules, and emphatically more than substantive, instruction-like rules organized around material objectives. The way substantive and instrumental rules are designed and administered, and the frameworks within which they are embedded, are ultimately more important than those individual legal rules themselves. Law creates not just obligation but structures of argumentation and means of normative evaluation that, when carefully and thoughtfully taken up, can be used to pursue multiple, interlocking objectives at once. In a society as heterogeneous and deeply unequal and inequitable as international society, and in issue-areas as closely bound up with multiple dimensions of well-being and prosperity as environmental protection and sustainable development, legal normativity must be as attuned to fairness and equity, transparency, and accessibility, as to effectiveness in pursuing material objectives.
For these and other reasons, careful and critical consideration of the role that law has played, and could potentially play, in international and global environmental governance is required. Furthermore, it is essential that this consideration extend well beyond instrumental consideration of the effectiveness of individual legal rules and regimes. Calls for a reorientation of environmental law have been made by scholars who seek to articulate a role for law in earth systems governance, and these scholars put their fingers on many of the weaknesses, limitations, and blind spots of both international environmental law and scholarship on the topic.Footnote 108 Yet a potential limitation of these approaches is a tendency to slip from law to regulation,Footnote 109 even while acknowledging the broader contributions of law.Footnote 110
One proposal made by earth system law scholars is to work across the silos and specializations that have emerged in legal fields.Footnote 111 When law is seen mainly as the regulation of individually defined issue areas or problems, this type of specialization is likely to emerge, and it will seem natural to align regulatory frameworks closely with a fairly narrow range of expert inputs, rendering law ever more technically oriented and oriented not, as Fuller would have it, to ‘subjecting human conduct to the governance of rules’,Footnote 112 but to material purposes such as environmental protection, trade regulation, or protection of intellectual property.Footnote 113 This is a strong tendency in international environmental law to which Martti Koskenniemi drew our attention as highly specialized international regimes began to emerge.Footnote 114 Specialization and dependence on expert knowledge are not, to our minds, bad outcomes; on the contrary, they promote the richness and depth of legal (and policy) responses to complex and difficult issues. However, as Koskenniemi states, a problem arises when:
fragmentation becomes a struggle for institutional hegemony. Which institution will have the authoritative voice? According to which bias will a matter be resolved? If there are no regime-independent ways of describing an issue, the door is open to the unilateral assumption of jurisdiction by experts who feel themselves powerful enough to have the last word.Footnote 115
It may be that experts sometimes do try to get the upper hand in the process of decision-making, but the discussion of the power of numbers above suggests that the foregrounding of expert knowledge is just as often a strategy of political and legal authorities to bolster, or even ground, the validity and legitimacy of their conclusions and decisions.
The SDGs are designed to provide a counterweight to the tendency to silo individual environmental and sustainability issues. It is true that the individual goals reproduce these silos, but perhaps the fact that they are brought together within the same institutional and methodological structure will promote transversal, cross-cutting inquiry and problem solving. But Koskenniemi’s concerns about the reduction of problem-solving to technical matters is apposite here: the SDGs foreground a range of expert voices through their very nature and structure. The counterbalance that, we argue, is needed here is between the managerialism of the SDGs and the very different language and logic of the law. If law is seen only as a regulatory instrument, then there would be no need to protect and promote its distinctness from metrics and ensure its place in international sustainability governance; it could indeed be folded into a larger earth system framework.Footnote 116 However, as important as the regulatory, instrumental dimensions of law are, there is much more that law accomplishes, or at least attends to. As soon as we recognized that reliance on expert discourses, particularly those forms of expertise that can present their conclusions in apparently simple, straightforward numerical form, do not obviate the need for judgment, our attention turns immediately to questions regarding the structures and processes of decision-making and the means through which the rights, interests, and priorities of those affected by these decisions may be regarded and promoted. Koskenniemi’s constitutional mindset strongly evokes these non-instrumental dimensions of law. So too does Benedict Kingsbury’s attention to the particular contributions of law to global governance.Footnote 117 To apply a legal rule is not to identify the solution that the legal rule points to; indeed, one cannot apply a single legal rule without first engaging in a process of judgment to draw together a constellation of legal rules, concepts, and principles that seems appropriate to addressing the issue or problem in question.Footnote 118 The processes of interpretation and judgment can certainly incorporate a range of expertise and knowledge drawn from beyond the strictly defined field of legal rules; for example, understanding the content of an obligation of means in a given context demands such reaching out and incorporation.
7. Concluding remarks
This article proposes neither burial nor praise of the SDGs. We have sought instead to sketch approaches for fruitful interactions between the SDGs and international law, and to draw attention to the enduring relevance and importance of law to international environmental and sustainability governance. We are acutely aware of the very cogent and serious critiques of indicators as a technique of governance. While our focus has been on the SDGs, we acknowledge the significant weaknesses and limitations of that project and recognize the need for further development of sustainability targets and indicators as well as careful reflection on its objectives and likely contributions. There is reason for optimism that weakness and limitations will be addressed, and that the quality of sustainability data and metrics will continue to improve. There may be less ground for optimism that political authorities and powerful and influential civil society actors will recognize the inherent limitations of sustainability metrics and calibrate their use of and dependence on them accordingly.
Central to the recognition of the limitations of metrics and their appropriate use in sustainability governance is a clearer and firmer articulation of the multifaceted roles and contributions of law to sustainability. Some of the more sophisticated approaches to rule-of-law and human rights and development indicators bear important lessons for the larger and more complex project of sustainability governance. But legal scholars and practitioners must also continue to articulate the contributions of law beyond instrumentality, and to promote more imaginative and expansive approaches to law and sustainability. It is our contention that engaging with rather than avoiding, ignoring, or dismissing metrics is an important component of this effort.
Open access
