II. Comparing comparisons

To better examine some different contexts of comparison, it can be helpful to distinguish, heuristically, three approaches to exercise in which comparison takes place. The first approach to comparison, which I shall call ‘disciplined comparison’, is one where the context of seeking to describe and understand social variation is that of developing a given intellectual discipline such as history, sociology, anthropology or political science. In one mainstream version of this type of comparison, attention is given to variables that are seen as ‘indicators’, which can be used to explain variation or which require explanation. The second approach to comparison, which I am calling ‘foil’ comparison, is one where there is a strong policy or advocacy goal that is geared to learning from what are claimed to be better (or sometimes worse) practices elsewhere. The final approach to comparison is one that finds its place in efforts to rank places according to prescriptive standards so as to influence choices between them and/or exert pressure on them to change.

I first offer a brief account of each context of comparison, drawing for illustrative purposes mainly on comparisons of crime and deviant behaviour in different societies and focusing in particular on such social ‘indicators’ as prison rates, recidivism rates and perceptions of corruption. I shall suggest that indicators play a different role in each of these different contexts (see Table 1). I shall conclude by commenting on the overlaps and feedback effects between these different types of comparisons as they are actually used in practice.Footnote ² What I seek to establish is the need to pay attention to different contexts in order to understand what, for different practical purposes, comes to be treated as ‘good’ comparisons.Footnote ³

Table 1

Three contexts of comparison

III. Disciplined comparisons: explaining neoliberalism and punishment

Why do some countries punish more than others? The rise of a ‘culture of control’ over the last thirty years was originally characterised as a general aspect of modernity, even if it was one that was most clearly exemplified by the US, and to a lesser extent the UK (Garland, Reference Garland2001). But other scholars have since gone on to emphasise the differential reach of this culture. One hypothesis to explain variation in state punitiveness is to relate it to differences in political economy. Cavadino and Dignan, for example, argue that prison rates – which they take as a rough proxy for levels of punitiveness – are highest in neoliberal polities, because these follow social and economic policies that lead to what they describe as ‘exclusionary cultural attitudes towards our deviant and marginalised fellow citizens’ (Cavadino and Dignan, Reference Cavadino and Dignan2006a, p. 447; Reference Cavadino and Dignan2006b, p. 23).

By contrast, Continental European corporatist societies (which have also been described as ‘coordinated market economies’) and, even more, Scandinavian social democratic societies are said to ‘pursue more inclusive economic and social policies that give citizens more protection from unfettered market forces’ and to ‘see offenders as needing resocialisation which is the responsibility of the community as a whole’ (Cavadino and Dignan, Reference Cavadino and Dignan2006a, p. 448; Reference Cavadino and Dignan2006b, p. 24). Table 2 updates Cavadino and Dignan's well-known classification of countries in terms of their adoption of neoliberal policies and their corresponding rates of imprisonment.

Table 2

Imprisonment rates per 100,000 in 12 countries (2002, 2008 and 2014/15)

This table is adapted from Cavadino and Dignan (Reference Cavadino and Dignan2006a). The figures in the first column are the UN figures from 2002 that are used in their book. The columns in brackets are updates from 2008. Source: http://www.kcl.ac.UK/depsta/law/research/icps/worldbrief/wpb_stats.php and from 20014/2015: seettps://en.wikipedia.org/wiki/List_of_countries_by_incarceration_rate. They show that there has been little recent change in the rank order.

The influence of neoliberalism over social and economic policy choices is certainly a plausible candidate to be part of the explanation for the recent increase in prison rates, as well as an important factor in explaining differences between places (although it also risks becoming a tautology if we define neoliberalism as including greater punitiveness). Cavadino and Dignan's approach has been highly influential even with authors who think that other factors should also be considered. Lacey (Reference Lacey2008), for example, agrees on the importance of distinguishing what she calls ‘coordinated market economies’ from neoliberal ones. But she argues that attention should also be given to the way multiparty political systems are less likely to lead to appeals to populism than two-party ones. On the other hand, it appears that their thesis does not apply so well outside the range of countries they choose to compare. There are countries, such as China, which make high use of prison without being neoliberal and others, such as Russia or South Africa, where moves towards neoliberalism have actually gone together with some reduction in the use of prison. Even for the countries chosen for comparison, given the considerable difference in the level of imprisonment, would it not be better to have an explanation that singled out the US, rather than just taking it as an illustration of the influence of neoliberalism?

What mainly concerns us here, however, is not trying to determine the best explanation of punitiveness, but rather giving some idea of the sort of debate that characterises disciplined comparisons. Some of the discussion is, inevitably, somewhat technical. Before turning to political economy as an explanation, we need to consider more obvious reasons for these differences. The comparison of punitiveness here assumes that crime levels are the same (finding higher prison levels where there are higher rates of crime can hardly be called greater punitiveness). But there are reasons to think that some of the places included in Cavadino and Dignan's table that have higher prison rates do also tend to have higher crime rates. The US certainly has more lethal violence than any of the other countries in their list and South Africa also suffers exceptional levels of homicide, violence and rape. One recent comparison of overall victimisation rates for ten types of crime places England and Wales highest, with the Scandinavian countries and Japan lowest (Van Dijk, Reference Van Dijk2008, p. 158).

It is also important to see who is in prison, for what crimes and how they arrived there. In Italy, for example, the complications of its criminal procedure offer a crucial key to its relatively low rates of incarceration. The procedural guarantees of the adversarial system (relying on the forensic contest of the trial) introduced in the 1989 reform of criminal procedure were simply added to the ones that belong to the inquisitorial tradition. Even quite minor cases thus go through a series of procedural hoops and are reviewed by a large number of judges. There are two stages of appeal (the first stage being a retrial on the facts). Uniquely, the so-called ‘prescription’ statute of limitations period, after which criminal proceedings become null and void, continues to run until the Cassation court has given its final verdict.

Another factor that needs to be examined is the correlation between prison rates and cross-national differences in public support for prison sentences. Cavadino and Dignan show that there is no one-to-one matching: Japanese public opinion, for example, would support more severe penalties for burglary than the judges impose (Cavadino and Dignan, 1996a). But, in most countries, public opinion does correlate, though, in any case, it could it always be argued that public opinion is itself shaped by neoliberalism.

These matters aside, what is of more interest for present purposes are the concerns that qualitative researchers would raise. Many of these turn on the way indicators can easily mislead. For example, the welfare system in Italy has little to do with showing inclusiveness towards deviants or those on the margin, although it offers good guarantees for those already in work. Relatively ‘tolerant’ attitudes towards law-breakers likely owe more to suspicion of the state and the influence of Catholic tradition. Indicators too easily make it look as if we are comparing like with like rather than prompting questions about what we should be comparing. Not for nothing, Cavadino and Dignan entitle their chapter on Japan ‘Iron Fist in a Velvet Glove’. In Italy, we need to think not only of the family and extended family, but also of family-like groups in maintaining social order in many sectors of public and private life. Some of those helping to maintain ‘order’ in the Southern regions (and hence keeping prison rates low) are actually organised crime groups (Scalia and Mannoia, Reference Scalia and Mannoia2008).

As this shows, whatever difficulties there may be in correctly identifying the independent variables that can explain variations in punitiveness and tolerance, it can be even more important to think about the cross-national meaning of dependent variables such as punitiveness, leniency and tolerance. Many of the countries that have lower incarceration rates: Sweden, for example, or Switzerland (or the Netherlands in its glorious period as a ‘beacon of tolerance’) used shorter prison terms but actually sent relatively more people to prison than those with higher overall rates. Does this show less punitiveness than sending fewer people for longer periods? It certainly complicates any argument we may want to make about punitiveness and inclusiveness. Can there be too little punishment? Is tolerance always good? What about tolerance of others who commit crime as a result of a lack of civicness? Can it be irrelevant that what external observers call tolerance some locals may call permissiveness, indulgence, favouritism, neglect, indifference, impunity, denial or collusion? (Nelken, Reference Nelken2006).

IV. Foil comparisons: learning from Italian juvenile justiceFootnote ⁴

The latter kind of questions become central when it comes to the role of indicators in what I am calling foil comparisons. My illustration here, again taken from the realm of criminal justice, concerns efforts to learn from the success of other jurisdictions in reducing offending amongst young people. One internationally recognised indicator of such success is the rate of recidivism or reoffending. Not surprisingly, therefore, it gets a mention in a recent glowing appreciation of the Italian juvenile court system by Frances Crook, head of the Howard League, one of the leading criminal justice campaigning organisations in the UK. In her blog entryFootnote ⁵ after a visit to Italy, she tells us:

For her, the Italian system is one that places young people first, and has even found the holy grail of an answer to recidivism – virtues that she would like the England and Wales system to display.

But can we be sure that the Italians are doing what we could and should be doing? In terms of aspiration, maybe; the Report's comment that ‘The Italian courts recognise that girls and boys often commit misdemeanours as a cry for help’ does seem on target. Melitta Cavallo, a top juvenile court judge in Rome and past president of their association, begins her recent book about juvenile justice by saying:

In practice, too, relatively few young people in Italy are brought into the juvenile system and few of these are convicted. Of special importance for the Howard League's campaign to end the imprisonment of young people, only a tiny proportion of Italian youngsters end up in custody. The official figures tell us that around 20,000 young offenders (aged fourteen to eighteen) enter the system, but that there are fewer than 400 in prison at any one time (plus around 800 in non-secure community placements) and another 19,000 under some sort of social work supervision.Footnote ⁶ Relatively speaking, especially if compared to England and Wales (or, even more, the US), these numbers are low. This is all the more remarkable as, even for youngsters, Italy has a system of obligatory rather than discretionary prosecution. There is no doubt, then, that judges do deliberately try to avoid the use of prison; indeed, the 1989 code requires that prison be used as a last resort and regular mention is also made of the UN Convention on the Rights of the Child regarding this point.

On the other hand, attributing these results to the use of provisions of the 1989 procedural code seems strange given that the overall justification of the 1989 reform was to make young people more ‘responsible’ for their actions.Footnote ⁷ (The more lax system that preceded it was one in which local government was supposed to deal with the ‘problem’ of youth offending, seen as a problem of welfare resources, but was, in practice, unwilling or under resourced for doing so.) A further problem is the way the Howard League blog refers only to what happens in Lombardy, without raising the question of how far this is representative of the rest of the country. There is reason to think that less stress is placed on rehabilitation at all costs in places such as Naples or Sicily in the south of Italy, or even Trieste in the north. We also need to ask who it is that the system is treating leniently.Footnote ⁸ Even the courts in the north and centre of Italy have major problems in knowing how to handle ‘irregular’ young immigrants, especially, but not only, those who arrive in Italy ‘unaccompanied’. They also struggle to find an answer to the problem of stealing by girls who come from Roma (gypsy) families.Footnote ⁹ The courts in the south, on the other hand, face difficulties in working constructively with young offenders because of the poor resources for welfare intervention, the high levels of unemployment and the lure of entrenched organised crime groups.

What about the supposedly greater effectiveness of the Italian response to juvenile crime? How credible is the impressive indicator of a 4 per cent recidivism rate? I followed this statistic up, first with the Howard League and then with the contacts in Italy from whom they said they had received this information. The claim turned out to be based on no more than an undocumented footnote in a paper written, with colleagues, by a psychologist who headed the Milan Youth Tribunal social work section for many years. Footnote 5 in Chessa, Gasparini and Poli (Reference Chessa, Gasparini and Poli2008, p. 109) states that ‘of 1476 youngsters, mostly not in a state of arrest, treated in the way described in this paper, from 1992 to 1996 recidivism went down from 22.83% to 3.28 %’ (my translation).Footnote ¹⁰ This figure, it turned out, referred to changes in the overall rate of recidivism in the whole period following the juvenile justice reform, for all the young people in their care, but excluding the more serious offenders who would have been in a state of arrest and temporary confinement.Footnote ¹¹ Given that the regime before 1989 is universally agreed to have broken down, the previous recidivism rate is not a safe starting point. We also know little about the sample of offenders with which such a remarkable outcome was achieved, save the fact that, by including all cases dealt with, it includes many offenders who would anyway not have been expected to reoffend given that they anyway had a low number of previous convictions.

Until recently, there have been very few serious efforts in Italy to measure the effects of juvenile justice sanctions on the reduction of recidivism, except when it comes to the outcomes of the small number receiving pre-trial probation (messa alla prova).Footnote ¹² This is where most of the resources of the juvenile justice system are expended, even though we are speaking of around 2,000 carefully chosen cases per year. The Ministry of Justice website speaks proudly of near 80 per cent success rates and hence urges extension of the measure. But two senses of success are being (deliberately?) confused (Nelken, Reference Nelken2006; Scardaccione, Reference Scardaccione1997). The one used by the Ministry relies on the judge deciding that the requirements of the ‘probation’ order have been met sufficiently for there to be no need to go to trial. When the judge comes to decide this, he/she knows that the sentence that will result if the case goes to trial is prison (or a suspended prison sentence). So the young person's performance must really be poor for judges to decide that pre-trial probation has ‘failed’ and the case is sent for trial and a potential prison sentence.Footnote ¹³ On the other hand, the Ministerial site tells us nothing about the more familiar, internationally recognised definition of success – that of not committing further crimes for a given period after the end of any measure imposed. It is very difficult to find this out because it requires collating the (confidential) records from juvenile and adult courts (but see Scivoletto, Reference Scivoletto1999; Colamussi and Mestitz, Reference Colamussi and Mestitz2012).

The Italian author of the footnote that so inspired the Howard League account of Italian juvenile justice told me that the key to preventing youth crime is clinical intervention with all young people. Her statistic was meant to demonstrate the success of a psycho-dynamic approach as compared to what happened in the previous system. In the course of our phone interview, and in the many writings she sent me, she showed herself to be a convinced and passionate believer in the efficacy of applying the psychological theories of Winnicot (an English psychologist) to all cases. But it is unrealistic to think that such a level of clinical intervention is, or could be, applied to all young people in the Italian system.Footnote ¹⁴

On a more sober assessment, the relative ‘success’ of the Italian system in keeping to a low level of punishments could even be seen as less a result of rehabilitative philosophy and more as an unintended outcome of the 1989 reform. Because the reform was a procedural one (in coincidence with the move to the accusatorial system for adults), no substantive change in punishments was introduced. Even for young people, therefore, prison remains the ‘standard’ post-trial sentence (albeit reduced by a third from that of adults). There is none of the wide range of options of non-custodial penalties found in England and Wales. Thus, the avoidance of prison sentences is, in part, a paradoxical result of the lack of alternatives that might otherwise ratchet up the severity of sanctions so that eventually prison comes to be seen as the necessary next step. Children in Italy do not have the same opportunity as in England and Wales to fail a series of social interventions and thus move up the ladder of penal severity.

It would, in any case, be rash to assume that copying the type of juvenile justice system operating in Italy would produce similar outcomes in England and Wales. An exercise in comparison more constrained by the goal of making progress in a given academic discipline would highlight the extent to which reliance on ‘the family’ in Italy may help avoid the need for more state activity to deal with young offenders. A less engaged approach would also reveal the way the same factors can have both positive and negative sides. After all, ‘family-like’ methods also underlie the embedded nepotism that can block necessary changes in public and private life, and such immobility may be linked to the 40 per cent youth unemployment rate. Similarly, although the Catholic Church helps to promote a culture of forgiveness, historically, it has also sapped legitimacy from the weak authority of the lay state; its approach to confession can also encourage a certain indulgence towards malfeasance.

The fact that judges and prosecutors try not to be swayed by what they call ‘social alarm’ (allarme sociale) does indeed provide an important bulwark against politically driven popular punitivism. But the exclusion of public participation from legal decision-making may also have its downside, such as the failure to give much heed to the victims of juvenile crime, including other youngsters. We would need to ask how far, even if we could, we really would want ‘to do it like the Italians’. Amongst the key factors that explain the better performance of Italian juvenile justice are those that have already been explicitly rejected by policy-makers in England and Wales. For example, the previous extensive use of diversion (via repeated ‘cautions’) in England and Wales was repudiated in the name of ‘no more excuses’ (Audit Commission, 1996).

Yet, foil comparisons remain seductive because they show that another way is possible. The brief account of the Italian system presented by Francis Crook was not intended to offer a dispassionate academic study but to serve as part of a campaign to reduce the number of children ending up in prison. This message still gets through even if the wider conditions facing juvenile court judges in Italy are indeed very different from those in England and Wales, and even if we are reluctant to copy all of the particular legal procedures that they use to obtain their results.

V. Standardising comparisons: global social indicators

Taking this argument further, however, raises the question of whether at some point the truth of comparisons becomes irrelevant. A good illustration of this issue is provided by what we are calling ‘standardising comparisons’: the comparison of ostensibly similar phenomena across a large numbers of countries as part of the construction of so-called ‘global social indicators’ (Merry, Reference Merry2011) used to measure levels of corruption, rule of law, friendliness to business and so on (Davis et al., Reference Davis, Fisher, Kingsbury and Merry2012b). Can we criticise comparisons for their accuracy when they deliberately set out to make very different places commensurable? If we accept that standards are being constructed so as to overcome – rather than explain or ‘appreciate’ – diversity, can this be an alibi for poor comparison?

In their recent attempt to provide a working definition of ‘global social indicators’, Davis, Kingsbury and Merry (Reference Davis, Kingsbury and Merry2012a) underline one link between indicators and comparison. ‘An indicator’, they tell us:

Such practical comparisons may be relevant to a large range of decisions made by governments, organisations or individuals that concern past or promised achievements and accountability.

It is important to note, however, that comparison is also involved in the task of gathering the information that makes up indicators, using data ranging from international and national official statistics to social surveys based on public or so-called experts’ perceptions. But those who do comparisons – including finding indicators – that are geared primarily to the requirements of a given academic discipline would find much to object to in many of these comparisons. Data are used as building blocks for comparing very different places, despite the difficulty of controlling for the varying meanings of contestable terms in local contexts. Values such as the rule of law or judicial independence are in practice given different interpretations in different places (Frydman and Twining, Reference Frydman and Twining2015) or applied differently because of different circumstances. Law itself is likely to work differently because there are likely to be a variety of other mechanisms that may substitute for it or conflict with it. But a common feature of many indicators is the assumption of what has been called ‘legal universalism’.

There are obvious problems with the information provided by those lower down in the hierarchy of the monitoring organisation, such as those charged with showing how funds are being used in refugee relief. Refugee protection field officers, we are told, are asked to collect information on 154 detailed indicators. Can they really be expected, to take just one of these, to count the number of latrines per 40,000 refugees, given the lack of access and security inside the camps? They often have to combine this task with their main responsibilities in resolving problems on the ground. Not surprisingly, they tend to give priority to doing the job of saving lives over that of monitoring compliance, especially where they consider it impossible to do both. Likewise, they may tell us about the distribution of water supplies but not the number of evictions of refugees, since this would risk alienating host states. Assessments purporting to be objective, such as those provided by World Bank officials or credit rating agencies about whether or not states are honest about repaying their debts, in fact largely rely on subjective judgments. This is certainly true of public-opinion barometers that seek to estimate levels of corruption or the quality of public services.

The uneven quality of the data available in different places increases the risk of producing what, at worst, we could call ‘junk comparisons’.Footnote ¹⁵ Many indicators are set up to measure what can most easily be measured, irrespective of whether or not this gets to the heart of the ‘problem’. It is certainly implausible to give credence to the number of conventions a state has signed up to as evidence of its political will to deal with problems of discrimination in society. These questionable choices are made worse by the way information is later processed. Fundamental errors are introduced by working with false assumptions, or confusing causes with consequences. Binary options are imposed on complex and contested materials. Questionable choices are made in adding or multiplying different sources of data that have been gathered using competing methodologies.

But what if it is only the consequences that matter? Sally Merry argues that indicators have what she calls ‘knowledge’ and ‘governance’ effects. She describes them as numerical measures that submerge local particularities and idiosyncrasies into universal categories to produce a world knowable without the detailed particulars of context and history. Such knowledge is presented as objective and often as scientific, because the interpretations that lurk behind the numbers are rarely presented explicitly (Merry, Reference Merry2011). Indicators, she tells us, are ‘performative’, naming produces knowledge by announcing categories to be measured as if they were self-evident, open to public scrutiny, simple in conception and readily accessible, in a way that private opinions are not.

‘Development indicators’, for example, send a message to the developing countries of what kind of society they ought to become and how to achieve that society. This ‘knowledge effect’ in turn determines the ‘governance effect’ produced by indicators in the development field, since the use of development indicators by the donor community for the allocation of aid creates an incentive for states in the Global South to accommodate their performance in order to receive a good ranking in the indicator. Development indicators have yet another effect on governance: they implicitly allocate responsibility for failure in the development process. Development indicators that measure state performance assign the responsibility to the developing states, because their failure to achieve the pre-established standard is seen as the prevailing reason for underdevelopment.

What many indicators purport to describe is often only a ‘constructed’ object – an artefact of the very attempt at measurement (as in the measurement of intelligence by IQ). This is easily seen with notions such as ‘state fragility’: critics allege that we do not really know how to determine the effects of political institutions across different realities so as to predict the drivers of fragility. The very idea of ‘state-ness’ is no more than an assemblage that brings into connection ideas based on prototypes of influential Western states and Weberian notions of the monopoly of violence (Bhuta, Reference Bhuta, Davis, Fisher, Kingsbury and Engle Merry2012). Or take the many popular indicators for measuring the ‘rule of law’. Procedural, substantive or institutional definitions can all be used. But each of these emphasises some political concerns over others.

In general, indicator rankings become a ‘currency’ (if everyone else thinks this is the best place to invest or to study, I can rely on this without needing to know what, if anything, lies behind such judgments). By ranking different places, indicators constitute the ‘units’ they are comparing as if they existed independently, so as to make it seem that the post-Westphalian international state system had not been affected by globalisation. This hides the way the act of monitoring countries in relation to each other is itself evidence of growing interdependence.

In relation to their role in governance, a further important feature of indicators for Merry and many other commentators is the strong link between the growth of global social indicators and neoliberalism. Going beyond the analysis of punitiveness levels as shaped by political economy – as in the analysis offered by Cavadino and Dignan – here we see neoliberalism as itself a form of social regulation: indicators seek to shape individual behaviour through governance of the soul and self-management, rather than by command and control models. Most indicators, especially, but not only, those employed by International Financial Organisations, are created in the ‘Global North’, which sets the agenda, names the indicator and assembles the criteria – while data collection typically takes place mostly in the ‘Global South’. Often, they involve an allegedly doctrinaire approach to removing state control over economic action in poorer countries. Critics of the ‘better business’ indicator, for example, argue that this helps block land reform and is implicated in the ‘land grabs’ that lead to the displacement of populations, impoverishment and the loss of livelihoods. Whilst promising to create work, what actually happens is that the best jobs go to foreigners and the dangerous, poorly paid jobs go to the locals. What is more, much of this development involves producing food for others when there is not enough locally so that dependence on foreign economic interests increases (Davis et al., Reference Davis, Fisher, Kingsbury and Merry2012b).

In general, indicators mobilise questionable categorisations of actors, actions, problems, diagnoses and solutions (Sokhi-Bulley, Reference Sokhi-Bulley2011). But those who wish to defend the use of comparison in making global social indicators are not short of arguments. They point to the vital functions which indicators try to serve, such as ensuring accountability for expenditure or the success of progressive social interventions. In terms of the so-called ‘knowledge effect’, they argue, against Merry, that the concepts used in global indicators are no more (and no less) ‘constitutive’ of culture than any other concepts. Is it epistemologically coherent to argue that concepts constitute facts whilst also criticising indicators because they use concepts that do not reflecting underlying realities? Some commentators praise the use of indicators as themselves helping to generate debate about the appropriateness and applicability of concepts such as human rights (Rosga and Satterthwaie, Reference Rosga and Satterthwaie2008; Urueña, Reference Urueña, Engle Merry, Davis and Kingsbury2015).

Is the point to try to aim for indicators that are non-political, or just to change the politics? Sen and Nussbaum's ‘human capacities’ approach to measuring development (Sen, Reference Sen2005) explicitly sets itself up against neoliberal ideology, by explicitly building into its ranking scheme recognition of the importance of enhancing human capacities. On the other hand, some would see any and every use of indicators, however benign, as part of the spread of neoliberalism because the method being used is one that is taken from business models and involves shifting and imposing responsibility.

Who guards the guardians? If indicators are used to make others accountable, can those who make and distribute them be made accountable for the way they do their comparisons? How might that work? It would certainly need to vary depending on whether we are dealing state governments, like the US, NGOs like Freedom House or Transparency International, or private organisations like financial rating agencies. What about those organisations and individuals who simply take indicators into account in making their own decisions?

How far do we want to go in making indicators contestable? We can imagine that many ‘targets’ of indicators (e.g. human rights abusers) would be all too willing to exploit such possibilities. Too much contestation could easily undermine any valuable functions performed by indicators in supporting competent and legitimate decision-making (Nelken, Reference Nelken, Engle Merry, Davis and Kingsbury2015a). The critique of global social indicators could also be seen as part of a broader methodological dispute over the use or misuse of quantitative data in social science. But quantitative researchers could reply to this that ethnography has its own problems of reliability and is not easily applied where there are a large number of places to be compared; moreover, qualitative data are no guarantee of political soundness given that anthropologists played a large role in the spread of colonialism.

VI. Conclusions: overlap and feedback effects

This paper has sought to explore three contexts of comparison and provide some sense of the ways this shapes such exercises. Table 3 identifies some ideal-type differences of comparison carried out in such contexts.

Table 3

Distinguishing ideal types of comparison

But it would obviously be wrong to see these three contexts of doing comparison as separate or separable. The various purposes of comparison that I have been discussing may be implicitly present, to a greater or lesser degree, in all of the three types of comparison. It is plausible to argue, for example, that the debate about differences in punitiveness is geared not only to advancing the explanatory agenda of the social sciences, but also to finding a way of reducing levels of imprisonment (and perhaps also finding a stick to beat neoliberalism). Conversely, foil comparisons need some level of scientific (disciplinary) plausibility, even if they are mainly ‘interested’ instrumental exercises. And they really come into their own when feeding into global indicators, just as the experience of being marked down may lead a state or organisation to seek to learn from more favoured rivals or competitors.

There is an interesting ambiguity in the passage from the second to the third sentence of the definition of indicators provided by Davis, Kingsbury and Merry that I quoted earlier. Saying that data, as they put it, are ‘capable of being used’ is not the same as actually being fashioned as a tool for this purpose. As this suggests, indicators turn from ‘facts’ into ‘standards’ and assume ever more ‘performative’ effects as they move from the supposedly neutral task of providing explanation to that of providing local guidance or overarching standards. Likewise, the ‘same’ phenomena can count as ‘facts’ for one group, whilst serving as a ‘standard’ for another group. For example, what is normal or average in terms of trial times in different countries is easily turned into a standard that all those countries should meet (Nelken, Reference Nelken, Petersen, Krunke, Kjær and Rask Madsen2008). Interestingly, Roy Walmsley, who maintains the UN statistics on incarceration rates around the world, also used it to recommend that countries should aim to keep their prison numbers down to no more than one in 1,000 (Walmsley, Reference Walmsley2003, p. 188), although this has not yet had the institutional support to become a fully fledged global social indicator.

Those who develop global social indicators draw on comparisons made for all sorts of purposes. Many indicators, however, incorporate data from other indicators in order to create their own, without paying attention to the factors and values guiding the making of the indicators that they have chosen to encompass. In terms of future work, the analysis so far should therefore also be seen as an invitation to examine the relationship between different approaches to comparison in actual practice. One obvious link is the role that ‘indicators’ play in both disciplined and standardising comparisons and the way such a role changes where the emphasis is less on explanation and more on evaluation. Just as some indicators used by inter-governmental and non-governmental organisations are prepared with the help of academics, there are many efforts at academic explanations that use concepts such as ‘fragile states’, which were originally created for practical purposes (Karstedt, Reference Karstedt and Nelken2011). The list below summarises some of the links (overlaps and feedbacks) that have been considered in this paper (but has no pretence to being exhaustive):

• • Supposedly disinterested explanations (Type 1) are often in reality shaped by foil goals (Type 2).

• • Making explanations via indicators (Type 1) overlaps with the standardising use of social indicators (Type 3).

• • Foil comparisons (Type 2) are a crucial step in making standardising indicators (Type 3) and being subjected to ranking can lead some places to make foil comparisons.

But comparative sociology of law still has much to learn, as well as to teach, about the ‘effects’ of comparison.