2 Relationship to prior socio-legal research on indicators

Recent years have seen the publication of a large body of literature dealing with indicators from a socio-legal perspective. This includes, for example, English-language books on The Power of Global Performance Indicators (Kelley and Simmons, Reference Kelley and Simmons2020), The Palgrave Handbook of Indicators in Global Governance (Malito et al., Reference Malito, Umbach and Bhuta2018), Governance by Numbers: The Making of a Legal Model of Allegiance (Supiot, Reference Supiot2018), The Quiet Power of Indicators: Measuring Governance, Corruption, and Rule of Law (Merry et al., Reference Merry, Davis and Kingsbury2015), The World of Indicators: The Making of Government Knowledge through Quantification (Rottenburg et al., Reference Rottenburg2015), Ranking the World: Grading States as a Tool of Global Governance (Cooley and Snyder, Reference Cooley and Snyder2015), Governance by Indicators: Global Power through Quantification and Rankings (Davis et al., Reference Davis2012) and Uses and Abuses of Governance Indicators (Arndt and Oman, Reference Arndt and Oman2006). In the non-Anglophone literature, there have been, for example, the books on Numera et impera: Gli indicatori giuridici globali e il diritto comparato (Infantino, Reference Infantino2019) and Gouverner par les standards et les indicateurs: De Hume aux rankings (Frydman and Van Waeyenberge, Reference Frydman and Van Waeyenberge2013).

A number of Special Issues of journals have explored further details.Footnote ⁷ The two editors of this Special Issue (and some of its authors) have also contributed to the debate in this form, namely with a prior Special Issue of this Journal, on the ‘Global social indicators: constructing transnational legitimacy’.Footnote ⁸ In this latter publication, the term ‘global social indicators’ was used to refer to numerical measures that describe the well-being of individuals or communities (Siems and Nelken, Reference Siems and Nelken2017). It explained that such indicators have reached across many societal fields: for example, some deal with good governance and the rule of law (e.g. the Corruption Perception Index and the various indicators by the World Bank), some with personal rights and economic freedom (e.g. the World Press Freedom Index and the Global Competitiveness Report), some with human development and political stability (e.g. the Human Development Index and the Country Risk Index) and some with the performance of commercial and educational entities (e.g. the UN Global Compact and the various global university rankings).

The growing use of indicators provides various links to socio-legal research in general and research on ‘law in context’ in particular. The literature has discussed, for example, that indicators do not just provide information, but can be regarded as somehow ‘law-like’, as they are often drafted in a general fashion and have a profound influence on society due to their quantitative metric (Davis et al., Reference Davis, Kingsbury, Merry, Merry, Davis and Kingsbury2015).Footnote ⁹ In the international sphere, it may therefore be argued that they are often a form of transnational soft law (Siems and Nelken, Reference Siems, Nelken and Zumbansen2021). There are also some situations in which indicators have been explicitly included in the law itself, while, in other instances, they are of interest to socio-legal researchers, as indicators are a form of control that both affect and are affected by regulation.Footnote ¹⁰

The precise relationship between indicators and the role of context has sometimes been phrased as a paradox. For example, this line of reasoning states that indicators aim to be ‘globally commensurate’ and thus ‘cannot be rooted in local contexts, but in order to accurately reflect local situations, they need to be’ (Merry and Wood, Reference Merry and Wood2015, p. 217). Yet, it can also be argued that this ‘paradox’ simply shows that law-in-context research needs to be aware that indicators can have different functions: some of them have the explicit aim to deal with standards and solutions that have to be applied evenly even if the context is different, while others benefit from more consideration of ‘context’ in order to make them more ‘truthful’ for purposes of comparison.Footnote ¹¹

Some of the prior literature on indicators has already examined issues of public health and diseases. For example, three chapters of the aforementioned books deal with the way in which WHO and UNICEF immunisation-coverage indicators have been used (Fisher, Reference Fisher and Davis2012), the application of a scorecard in the fight against malaria in Tanzania (Gerrets, Reference Gerrets and Rottenburg2015) and the application of Millennium Development Goals (MDGs) to HIV treatment in Uganda (Park, Reference Park and Rottenburg2015). Further specialised journal publications include, for example, an analysis of the MDG's goal to improve maternal health (Yamin and Boulanger, Reference Yamin and Boulanger2014), the role of private actors for the assessment of the health-related Sustainable Development Goal indicators (Mahajan, Reference Mahajan2019) and critical historical perspectives on the evolution of global health metrics (Gorsky and Sirrs, Reference Gorsky and Sirrs2017; Shiffman and Shawar, Reference Shiffman and Shawar2020; Aue, Reference Aue2021). Finally, just as COVID-19 emerged, a monograph on The Uncounted: Politics of Data in Global Health specifically focused on the role of for health data for AIDS response (Davis, Reference Davis2020).

With respect to the COVID-19 crisis, this Special Issue is the first to discuss the role of indicators in detail.Footnote ¹² In contrast to the aforementioned previous Special Issue of this Journal, we consider all possible ‘indicators’ (i.e. not only ‘global social indicators’) within the scope of analysis. As already noted in the introduction, this includes therefore, for example, indicators of government responses, but also infection and fatality rates. A further distinction is that some of these indicators may be regarded as ‘global’ (e.g. because they aim to rank the performance of countries in the COVID crises) while others are used ‘locally’ (e.g. as they track COVID infections across time in a particular place).

The specific topics included in this Special Issue partly reflect an e-mail correspondence in which we indicated a number of questions that the authors could address, namely: How can COVID-19-related indicators be classified (descriptive, explanatory, normative, etc.)? What can the prior debates about the strengths and weaknesses of indicators add to the discussion and uses of indicators in the COVID-19-pandemic? Conversely, what can the way in which these indicators were made and used add to the academic discussion on indicators? How far do these indicators compare things that are comparable, in particular in a cross-country context? What are the advantages and disadvantages (or uses and abuses) of these indicators? How far do (and should) these indicators guide social interventions and change behaviour? What is the role of law in terms of allowing, restricting or incorporating such indicators? What is the role of technology in this field? What are the relevant ethical considerations? The following will outline the contributions to this Special Issue as they address many of these questions (while others leave scope for future research).

3 Overview of contributions to this Special Issue

The first paper of this Special Issue is by Marta Infantino on ‘Hazards and fallacies of social measurements: global indicators in the pandemic’. It presents an overview of different types of COVID-19-related indicators. It also discusses the hazards and fallacies associated with global attempts to frame the social world in numbers. Thus, this paper attempts to verify whether and to what extent these hazards and fallacies affect numerical representations of the pandemic and its effects. Its aim is therefore to gain a deeper understanding of global measurements of health and related law-and-policy measures and to suggest caution about their use as a basis for knowledge and action in the context of the pandemic.

Second, the paper by Nathan Genicot on ‘Epidemiological surveillance and performance assessment: the two roles of health indicators during the COVID-19 pandemic’ draws on the history of international public health and of the management of infectious disease. It shows that the normative power acquired by metrics during the pandemic can be understood in light of two forms of quantification that gained in importance over the last decades – namely epidemiological surveillance and performance assessment. Thus, it suggests that, on the one hand, indicators are established to evaluate and rank countries’ responses to the outbreak; on the other hand, the evolution of indicators has a direct influence on the content of public health policies, also noting the inherent partiality and precarity of the information provided by health indicators.

Third, Manjari Mahajan's paper on ‘Casualties of preparedness: a case of the Global Health Security Index’ also presents a historical and critical perspective. Specifically, it analyses the 2019 Global Health Security Index by two leading US think-tanks that ranked 195 countries on a range of metrics of preparedness. It notes that the US and the UK were assessed as the two states best prepared to address a catastrophic health event, while, in the COVID pandemic, we now know that these countries have had some of the worst public health outcomes among wealthy countries. Thus, this paper interrogates the dominant schemas from which global health security indices have emerged and the narratives and governance that they facilitated. In particular, it argues that there is a need not simply for ‘better indicators’, but also for a different framing of public health and what constitutes pandemic preparedness.

Fourth, David Nelken's paper on ‘Between comparison and commensuration: a case-study of COVID-19 rankings’ discusses the problem of indicators to compare and rank the relative performance of states or other units. Thus, it raises the question of how far indicators can, or should, be required to be faithful to the requirement to ‘compare like with like’. Specifically, using the role of indicators in documenting and responding to the COVID-19 epidemic as an example, the paper investigates in detail how the hybrid and sometimes inconsistent commitment to both comparison and commensuration may help to account for the difficulty they have had so far at establishing stable rankings of best practice.

Fifth, the paper by Mathias Siems on ‘Mapping a causal scheme of indicators in the COVID-19 crisis’ also presents a comparative (as well as empirical) perspective. It focuses on two main issues. First, it develops and maps a general causal scheme of indicators and their underlying real-world phenomena in the COVID crisis. Second, it shows how such a causal scheme has been, and can be, applied in comparative empirical legal research. Yet, the paper also notes the difficulties of proving causal relationships. Thus, it suggests that, while it is useful to conduct empirical research on the effects of COVID policies, the limitation remains that any comparative facts about a particular causal relationship do not answer the ultimate decision of what this means for the right policies in a particular place.

Sixth, John Harrington's ‘Indicators, security and sovereignty during COVID-19 in the Global South’ continues with a critical analysis specifically addressing the contest over health governance and sovereignty in countries of the Global South. It notes that indicators have been a vehicle for the government of states through the external imposition and internal self-application of standards and benchmarks. It then contextualises their use, in particular identifying a decline in Third World solidarity and its replacement by neoliberalism and global governance mechanisms in health, as in other sectors. The interaction between these modes and their relative prominence during COVID-19 is also studied through a case-study of developments in Kenya during the early months of the pandemic.

Finally, the paper by David Restrepo Amariles is entitled ‘From computational indicators to law into technologies: the Internet of Things, data analytics and encoding in COVID-19 contact-tracing apps’. It investigates the data life cycle of contact-tracing apps as well as the socio-legal implications resulting from the design and technology choices that software developers inevitably made. In particular, the paper shows, first, that technology-harvest data do not reflect an objective representation of reality, and therefore require additional context to be understood and interpreted, and, second, that the use of data analytics to extract insights from these data makes possible the production of computational indicators, which in turn enable the development of novel computational applications in the legal domain.

4 Where do we go from here?

At the moment of writing this introduction, it is possible, but by no means certain, that COVID-19 will be history in the near future.Footnote ¹³ Yet, beyond COVID-19, a pessimistic view may be that we have entered a ‘pandemic era’ given that, ‘as human societies grow in size and complexity, we create an endless variety of opportunities for genetically unstable infectious agents to emerge into the unfilled ecologic niches we continue to create’ (Morens and Fauci, Reference Morens and Fauci2020, p. 1089) with climate change as a possible ‘threat multiplier’ (Mann, Reference Mann2021). There are also further reasons why the impact of the use of indicators in the COVID-19 crisis is likely to remain relevant in the medium to long term. For example, the ongoing quantification of the social world (and its awareness by the general public),Footnote ¹⁴ the growing regulatory role of technologiesFootnote ¹⁵ and possibly also the apparent success of China in containing the pandemic will certainly have a continuing impact on domestic and international policy debates.

It is also clear that COVID-19 has many implications for research. According to Tim Besley, ‘COVID-19 will be with us for a long time as social scientists long after the crisis’, also noting the need for ‘an integrated social science’ (Besley, Reference Besley2020). Specifically for research on indicators, the contributions of this Special Issue show that such research requires a ‘law-in-context’ perspective. As some of the COVID-19 indicators relate directly to legal rules (e.g. the coding of the stringency of government responses),Footnote ¹⁶ their operation crucially depends on the context in which these indicators operate. Moreover, the contributions of this Special Issue show various overlapping in-context perspectives of indicators in the COVID-19 crisis, such as historical critiques of the origins of such indicators or perspectives focusing on the way in which such indicators interact with socio-political and technological elements of society.Footnote ¹⁷

A further key issue is what the experience from the COVID-19 crisis may tell us about the success in dealing with indicators. It may be argued that some of the countries that have fared well in the pandemic have been those that, for example, were quick in their policy reactions and were the able to contain the virus through contact-tracing and other means (e.g. Taiwan, South Korea). However, it is also clear that any country's reaction to COVID-19 did not happen independently of other country characteristics, such as the degrees of state capacity and trust.Footnote ¹⁸ Thus, it may also be argued that, despite the global nature of the pandemic, COVID-19 is also a case-study showing that the context of a country (or indeed any particular place) has often made all the difference.

A final note of caution follows from the unique nature of any emergency. For example, even a superficial of comparison of COVID-19 and HIV shows that differences related to the transmission mechanism of the virus, the age profile of the victims and the possibility of a vaccine mean that any lessons may not be transferable – and, of course, this may even be less likely for unknown viruses, or indeed any other ‘unknown unknowns’Footnote ¹⁹ that cause emergencies in the future. Finally, as regards the role of indicators, it is also best to conclude that both their use and desirability depend on the circumstances: while the COVID-19 crisis has shown that some of the indicators are clearly indispensable, other indicators are more dubious and thus require improvements in design and applicationFootnote ²⁰ or, if this is not feasible or appropriate, replacement by legal rules or other forms of social ordering.Footnote ²¹