Skip to main content


  • David McGrogan (a1)

The field of human rights monitoring has become preoccupied with statistical methods for measuring performance, such as benchmarks and indicators. This is reflected within human rights scholarship, which has become increasingly ‘empirical’ in its approach. However, the relevant actors developing statistical approaches typically treat causality somewhat blithely, and this causes critical problems for such projects. This article suggests that resources—whether temporal or fiscal—may be better allocated towards improving methods for identifying violations rather than developing complicated, but ultimately ineffective, statistical methods for monitoring human rights performance.

  • View HTML
    • Send article to Kindle

      To send this article to your Kindle, first ensure is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about sending to your Kindle. Find out more about sending to your Kindle.

      Note you can select to send to either the or variations. ‘’ emails are free but can only be sent to your device when it is connected to wi-fi. ‘’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

      Find out more about the Kindle Personal Document Service.

      Available formats
      Send article to Dropbox

      To send this article to your Dropbox account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Dropbox.

      Available formats
      Send article to Google Drive

      To send this article to your Google Drive account, please select one or more formats and confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your <service> account. Find out more about sending content to Google Drive.

      Available formats
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (, which permits unrestricted re-use, distribution, and reproduction in any medium, provided the original work is properly cited.
Hide All

1 The UK's Fifth Periodic Report, CESCR, UN Doc E/C.12/GBR/5 (31 January 2008). The UK's most recent report, the Sixth, is available at UN Doc E/C.12/GBR/6 (25 September 2014).

2 ibid, paras 302–305. ‘Inequalities in health outcomes’ meant the difference in figures for infant mortality and life expectancy at birth between the fifth of areas with the worst health and deprivation indicators and the rest of the population.

3 ibid, para 334. The actual report states the period as ‘2006–2006’ [sic]; it is assumed this is a typographical error for ‘2005–2006’.

4 ibid, para 286.

5 See eg Barsh, R, ‘Measuring Human Rights’ (1993) 15 HumRtsQ 87.

6 See eg Rosga, A and Satterthwaite, M, ‘The Trust in Indicators: Measuring Human Rights’ (2009) 27 Berkeley Journal of International Law 253.

7 See eg N Bhuta, ‘Governmentalizing Sovereignty: Indexes of State Fragility and the Calculability of Political Order’ in K Davis et al. (eds), Governance by Indicators: Global Power through Quantification and Rankings (Oxford University Press 2012) 133–61 and T Halliday, ‘Legal Yardsticks: International Financial Institutions as Diagnosticians and Designers of the Laws of Nations’, ibid, 180–216.

8 Ho, D and Rubin, D, ‘Credible Causal Inference for Empirical Legal Studies’ (2011) 7 Annual Review of Law and Social Science 17.

9 See S Leckie and A Gallagher, Economic, Social and Cultural Rights: A Legal Resource Guide (University of Pennsylvania Press 2006) xx.

10 References to ‘Big Data’ are inescapable in the modern age. See eg ‘Data, Data Everywhere’, The Economist (25 February 2010); A Sind, Big Data Analytics (MC Press 2012).

11 See for instance the B Nozek et al., ‘Estimating the Reproducibility of Psychological Science’ (2015) Science; and the Reproducibility Project: Psychology (at <>).

12 See eg A Finkelstein et al., ‘The Oregon Health Insurance Experiment: Evidence from the First Year’ (2011) NBER Working Paper 17190, available at <>.

13 See eg J Druckman, D Green et al., ‘Experimentation in Political Science’ in Druckman, Green et al. (eds), The Cambridge Handbook of Experimental Political Science (CUP 2011) 3.

14 See eg Greiner, DJ, ‘Causal Inference in Civil Rights Litigation’ (2008) 11 HarvLRev 533.

15 See Imai, K et al. , ‘Unpacking the Black Box of Causality: Learning about Causal Mechanisms from Experimental and Observational Studies’ (2011) 105(4) American Political Science Review 765.

16 See for instance M Nussbaum, Frontiers of Justice (Belknap Press 2006) 277, arguing that what is needed is a definite account of what all the world's citizens should have, and what their dignity entitles them to.

17 Art 12(2)(c).

18 The phrase's most prominent first appearance seems to have been in the CESCR's General Comment No 12 (UN Doc E/C.12/1999/5, 12 May 1999) para 15.

19 F Mégret, ‘Nature of Obligations’ in D Moeckli et al. (eds), International Human Rights Law (OUP 2014) 102.

20 ibid.

21 See eg Fukuda-Parr, S, Lawson-Remer, T and Randolph, S, ‘An Index of Economic and Social Rights Fulfillment: Concept and Methodology’ (2009) 8(3) Journal of Human Rights 195, 197.

22 See eg R Cignarelli and D Richards, ‘Measuring Government Effort to Respect Economic Human Rights: A Peer Benchmark’ in S Hertel and L Minkler (eds), Economic Rights: Conceptual, Measurement, and Policy Issues (CUP 2007).

23 See eg H Steiner, ‘Individual Claims in a World of Massive Violations: What Role for the Human Rights Committee?’ in P Alston and J Crawford (eds), The Future of UN Human Rights Treaty Monitoring (CUP 2000) 15.

24 See eg OHCHR, Human Rights Indicators: A Guide to Measurement and Implementation (2012) 49.

25 ibid 91.

26 Committee on Economic, Social and Cultural Rights, Concluding Observations on the UK's fourth to fifth periodic report (2009) UN Doc E/C.12/GBR/CO/5, paras 18, 23 and 32.

27 ibid, paras 233–234.

28 OHCHR (n 24).

29 They include those in the UK, Brazil, Bolivia, Ecuador, Mexico, Paraguay, Portugal, the Philippines and Kenya. See the UK EHRC's Human Rights Measurement Framework, available at <>; and also OHCHR, 2014, at <>.

30 OHCHR (n 29).

31 See Cingranelli, R and Richards, D, ‘The Cingranelli and Richards (CIRI) Human Rights Data Project’ (2010) 32 HumRtsQ 395.

32 See Fukuda-Parr, Lawson-Remer and Randolph (n 21) 195; and Economic and Social Rights Fulfillment Index: Country Scores and Rankings’ (2010) 9(3) Journal of Human Rights 230.

33 See especially A Nolan et al. (eds), Human Rights and Public Finance: Budgets and the Promotion of Economic and Social Rights (Hart 2013).

34 See eg OHCHR, ‘Report of the High Commissioner for Human Rights on Implementation of Economic, Social and Cultural Rights’ (2009) UN Doc E/2009/90.

35 Most notably South Africa—see South African Human Rights Commission and Studies in Poverty and Inequality Institute, ‘How Much Are We Spending on Transforming Our Society? A Rights-Based Analysis of the 2011 Budget’ (2011).

36 See for instance a recent volume of the Leiden Journal of International Law (28(2), 2015) on the ‘new legal realism’. See also Shaffer, G and Ginsburg, T, ‘The Empirical Turn in International Legal Scholarship’ (2012) 106 American Journal of International Law 1; Ho, D and Kramer, L, ‘Introduction: The Empirical Revolution in Law’ (2013) 65 StanLRev 1195.

37 ‘Quantification is a constitutive feature of modern science and social organization’, as Espeland and Stevens put it, in Espeland, W and Stevens, M, ‘A Sociology of Quantification’ (2008) 49(3) European Journal of Sociology 401, 402.

38 OHCHR (n 24) 94.

39 ibid 92.

40 ibid 89.

41 See E Rooney and M Dutschke, ‘The Right to Adequate Housing: A Case Study of the Social Housing Budget in Northern Ireland’ in Nolan et al. (n 33) 195–217.

42 See for instance the work of the Human Rights Data Analysis Group at the American Association for the Advancement of Science, at <>.

43 S Meckled-Garcia, ‘What is the outcomes view? Contemporary consequentialist theories of human rights’ (on file with the author).

44 Koskenniemi, M, ‘Human Rights Mainstreaming as a Strategy for Institutional Power’ (2010) 1(1) Humanity: An International Journal of Human Rights, Humanitarianism and Development 47.

45 For instance, art 2 of the ICCPR requires States Parties to ‘respect and to ensure to all individuals within its territory and subject to its jurisdiction the rights recognized in the present Covenant’.

46 See eg Farrior, S, ‘State Responsibility for Human Rights Abuses by Non-State Actors’ (1988) 92 ASILProc 299.

47 As in eg Government of South Africa v Grootboom, Constitutional Court of South Africa, Case CCT 11/00, 4 October 2000.

48 Most famously the Inter-American Courts of Human Rights developed this approach in the Velasquez-Rodriguez Case (Honduras) 4 IACtHR (ser C) (1988), although the European Court of Human Rights has used similar if more restricted reasoning in eg Plattform ‘Ärzte für das Leben’ v Austria ECtHR 10126/82 (1988).

49 See the CEDAW, arts 2 and 5.

50 Only art 23(4) of the ICCPR mentions ‘appropriate steps’ (in relation to equality in marriage), but assessing appropriateness/effectiveness of measures seems implicit in the due diligence obligations generally interpreted to be in the Covenant.

51 See eg the CRC's General Comment No 5, UN Doc CRC/GC/2003/5 (2003) para 51, describing how States ought to identify the proportion of their national budgets allocated to the social sector and, within that, to children.

52 See ibid, para 45, stating the need for predicting the impacts of imposed laws and child impact evaluation in all actions concerning children.

53 See M Sepúlveda, The Nature of the Obligations under the International Covenant on Economic, Social and Cultural Rights (Intersentia 2003).

54 UNCESCR, ‘An Evaluation of the Obligation to Take Steps to the ‘‘Maximum of Available Resources’’ under an Optional Protocol to the Covenant’ UN Doc E/C.12/2007/1, paras 8–12.

55 CEDAW, arts 2(e), 5(a) and 10.

56 See eg CEDAW, General Recommendation No 28, UN Doc CEDAW/C/2010/47/GC.2, para 16.

57 Report on Mexico produced by the Committee on the Elimination of Discrimination against Women under art 8 of the Optional Protocol to the Convention, and reply from the Government of Mexico, CEDAW 27 January 2005, UN Doc CEDAW/C/2005/OP.8/MEXICO.

58 ibid, para 184.

59 CEDAW, General Comment No 19, para 24(a).

60 ibid.

61 Donabedian, A, ‘The Quality of Care: How Can it Be Assessed?Journal of the American Medical Association (23/30 September 1988) 260(12) 1743, 1746.

62 OHCHR (n 24) 89.

63 ibid.

64 I Kempf, ‘How to Measure the Right to Education: Indicators and Their Potential Use by the Committee on Economic, Social and Cultural Rights’, CESCR, 19th Session (1998), UN Doc E/C.12/1998/22.

65 ibid.

66 See also C Apodaca, ‘Measuring the Progressive Realisation of Economic and Social Rights’ in Hertel and Minkler (n 22) 176–7.

67 See eg L Pritchett, The Rebirth of Education: Schooling Ain't Learning (Center for Global Development 2013).

68 See eg J Tooley, The Beautiful Tree: A Personal Journey into How the World's Poorest People Are Educating Themselves (Cato Institute 2009).

69 See eg S Quinn, ‘Equality-Proofing the Budget: Lessons from the Experiences of Gender-Budgeting?’ in Nolan et al. (n 33) 163.

70 ibid 174.

71 Leamer, E, ‘Let's Take the Con out of Econometrics’ (1983) 73(1) American Economic Review 3143. See also eg R Berk, Regression Analysis: A Constructive Critique (Sage 2004); Donohue, J and Wolfers, J, ‘Uses and Abuses of Empirical Evidence in the Death Penalty Debate’ (2006) 58 StanLRev 791; J Pfaff, ‘A Plea for More Aggregation: The Looming Threat to Empirical Legal Scholarship’ SSRN Working Paper (2010), available at <>. For an interesting technical exposition of some of the problems with using conventional regression analysis as a tool for inferring causation, see Ho and Rubin (n 8).

72 Leamer (n 71) 31.

73 From D Hume, An Inquiry Concerning Human Understanding, section VII.

74 Of course, Karl Popper used this as a dividing line between science and non-science. See K Popper, The Logic of Scientific Discovery ([1934] 1959 Hutchinson & Co).

75 See eg D Montgomery et al., Introduction to Linear Regression Analysis (5th edn, Wiley-Blackwell 2012); J Miles and M Shevlin, Applying Regression & Correlation (Sage 2001).

76 See G King et al., Designing Social Inquiry (Princeton University Press 1994) 170–1.

77 See Pfaff (n 71) 4.

78 See Donohue, J and Levitt, S, ‘The Impact of Legalised Abortion on Crime116(2) QJEcon (2001) 379, versus Foote, C and Goetz, C, ‘The Impact of Legalised Abortion on Crime: Comment’ (2008) 123(1) QJEcon 407.

79 See D Card and A Krueger, Myth and Measurement: The New Economics of the Minimum Wage (Princeton University Press 1995) versus Burkhauser, R et al. , ‘Who Gets What from Minimum Wage Hikes: A Re-Estimation of Card and Krueger's Distributional Analysis in Myth and Measurement: The New Economics of the Minimum Wage’ (1996) 49(3) Industrial and Labor Relations Review 547.

80 See Pitt, M and Khandker, SThe impact of group-based credit programs on poor households in Bangladesh: Does the gender of participants matter?’ (1998) 106 Journal of Political Economy 958 versus Banerjee, A et al. , ‘The miracle of microfinance? Evidence from a randomized evaluation’ (2015) 7(1) American Journal of Economics: Applied Economics 22.

81 See eg Braman, D and Kahan, D, ‘Cultural Cognition and Public Policy24 YaleL&Pol'yRev (2006) 147.

82 JS Mill, A System of Logic (1843) ch X.

83 This has elsewhere been called the ‘myth of monocausality’; see Miles and Shevlin (n 75) 28.

84 J Manzi, Uncontrolled (Basic Books 2012) 136.

85 Ho and Rubin (n 8) 26.

86 See eg Epstein, L and King, G, ‘The Rules of Inference’ (2002) 69(1) UChiLRev 1, 36.

87 Other examples include unwarranted assumptions about linearity and homogeneity.

88 See eg S Morgan and C Winship, Counterfactuals and Causal Inference: Methods and Principles for Social Research (CUP 2007); C Manski, Identification Problems in the Social Sciences (Harvard University Press 1995).

89 See eg Druckman, J et al. , ‘The Growth and Development of Experimental Research in Political Science’ (2006) 100 American Political Science Review 627; Greiner, DJ, ‘Causal Inference in Civil Rights Litigation’ (2008) 11 HarvLRev 533; Chilton, A and Tingley, D, ‘Why the Study of International Law Needs Experiments52 ColumJTransnatlL (2013) 173.

90 As Manzi aptly notes, in 1998, 100, 000 people died in US hospitals due to drugs that had been approved by the Food and Drug Administration, presumably after extensive randomized trials, and which had been correctly administered. See Manzi (n 84) 68.

91 See eg Frankel, M et al. , ‘Impact of Legal Counsel in Outcomes for Poor Tenants in New York City's Housing Court: Results of a Randomized Experiment’ (2001) 35(2) Law and Society Review 419.

92 Angrist, J and Lavy, V, ‘Using Maimonides' Rule to Estimate the Effect of Class Size on Scholastic Achievement’ (1999) 114 QJEcon 533.

93 Angrist, J and Pischke, J-S, ‘The Credibility Revolution in Empirical Economics: How Better Research Design is Taking the Con out of Econometrics’ (2010) 24(2) Journal of Economic Perspectives 3.

94 See eg Leamer, E, ‘Tantalus on the Road to Asymptopia’ (2010) 24(2) Journal of Economic Perspectives 31; also CA Sims, ‘Comment on Angrist and Pischke’ (Princeton University Press 2010) (available at <>; and, for a somewhat older perspective, Heckman, J and Smith, J, ‘Assessing the Case for Social Experiments’ (1995) 9 Journal of Economic Perspectives 85.

95 For an illustration, see Apodaca: ‘[T]he real problem in collecting economic and social data is political, not methodological or even conceptual.’ See (n 66) 179.

96 See eg Imai et al. (n 15).

97 Ansah, EK and Narh-Bana, S et al. , ‘Effect of Removing Direct Payment for Health Care on Utilisation and Health Outcomes in Ghanaian Children: A Randomised Controlled Trial’ (2009) 6(1) PLoS Medicine e1000007.

98 See Manzi (n 84) 86–8.

99 See Urquiola, M and Verhoogen, E, ‘Class-Size Caps, Sorting, and the Regression-Discontinuity Design’ (2009) 99(1) American Economic Review 179.

100 OHCHR (n 24) 17.

101 Mayntz, R, ‘Mechanisms in the Analysis of Social Macro-Phenomena’ (2004) 34 Philosophy of the Social Sciences 237, 241.

102 And, indeed, extensive literature reviews have suggested that there is no evidence that class size has a systematic effect on student achievement. See Hanushek, E, ‘The Failure of Input-Based Schooling Policies’ (2003) 113 The Economic Journal 64.

103 See Pritchett (n 67).

104 The EU, for instance, provides ‘budget support’ as an ‘implicit recognition that the partner country's overall policy stance and political governance is on track’. See European Commission, ‘The Future Approach to EU Budget Support to Third Countries’ COM(2011) 638 Final (13/10/2011) para 2.1.1.

105 See M Power, The Audit Society (OUP 1997).

106 See D Michaels, Doubt is Their Product: How Industry's Assault on Science Threatens Your Health (OUP 2008); and Michaels, D and Monforton, C, ‘Scientific Evidence in the Regulatory System: Manufacturing Uncertainty and the Demise of the Formal Regulatory System’ (2005) 13 JL&Pol'y 1742.

107 See eg Beco, G de, ‘Human Rights Indicators for Assessing State Compliance with International Human Rights’ (2008) 77 Nordic Journal of International Law 23.

108 It is common to encounter statements such as that of Welling, J, in ‘International Indicators and Economic, Social and Cultural Rights’ (2008) 30 HRC 933 at 958, that without statistical indicators the international community would be ‘uninformed’ about economic, social and cultural rights performance.

109 Chapman, A, ‘A “Violations Approach” for Monitoring the International Covenant on Economic, Social and Cultural Rights’ (1996) 18 HumRtsQ 23.

110 ibid 34.

111 See eg AR Chapman, ‘The Status of Efforts to Monitor Economic, Social and Cultural Rights’ in Hertel and Minkler (n 22) 143.

112 See The Maastricht Guidelines on Violations of Economic, Social and Cultural Rights (1997) paras 14 and 15.

113 Chapman (n 109) 38.

114 Goekce v Austria, CEDAW, Communication No 5/2005, UN Doc CEDAW/C/39/D/5/2005.

115 Diergaardt et al v Namibia, HRC, Communication No 760/1997, UN Doc CCPR/C/69/D/760/1996.

116 The Jewish Community of Oslo et al v Norway, CERD, Communication No 30/2003, UN Doc CERD/C/67/D/30/2003.

117 Chapman (n 109) 38.

118 ibid.

119 ibid.

120 See CEDAW, The UK's Response to the List of Issues for Its Fifth and Sixth Periodic Reports (2008) UN Doc CEDAW/C/UK/Q/6/Add.1, para 51; Equality and Human Rights Commission, ‘Is Britain Fairer?’ (2015) available at <>.

The author wishes to thank Lee McConnell and Rhona Smith for helpful comments on an early draft of this article.

Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

International & Comparative Law Quarterly
  • ISSN: 0020-5893
  • EISSN: 1471-6895
  • URL: /core/journals/international-and-comparative-law-quarterly
Please enter your name
Please enter a valid email address
Who would you like to send this to? *



Altmetric attention score

Full text views

Total number of HTML views: 62
Total number of PDF views: 483 *
Loading metrics...

Abstract views

Total abstract views: 605 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 16th July 2018. This data will be updated every 24 hours.