Rankings of law schools and law journals are part of a trend towards more emphasis in academia on transparency and accountability with regard to the quality of research and education. Globalisation increases the need to compare law schools and law journals across borders, but this raises complicated questions due to differences in language, education systems, publishing style and so on. In this contribution, it is argued that ranking of law schools and law reviews runs the risk of driving us away from quality based on substance towards proceduralisation and quality assessment based on proxies favoured by managers of law schools, funding bodies and government agencies, instead of by the forum of legal scholars.
Rob van Gestel is professor of theory and methods of regulation at Tilburg Law School (The Netherlands), professor of methodology of legal research at Leuven Law School (Belgium) and Braudel fellow at the European University Institute. The author would like to thank Han Somsen, Mathias Siems, Jan Vranken and the faculty members of the Law Department of the EUI in Florence for their valuable input. The usual disclaimer applies.
1. Posner, R ‘Law school rankings’ (2006) 81 Ind L J 13 at 13.
2. See the World Bank Doing Business Reports, available at http://www.doingbusiness.org/rankings (accessed 17 March 2014).
3. Hart, M The 100: A Ranking of the Most Influential People in History (New York: Citadel Press, 2000).
4. The BBC, for instance, organised the 100 greatest Britons in 2002 and in Germany the ZDF organised ‘Unsere Besten’ according to a similar concept. Similar initiatives can be found in a host of other countries.
5. See http://en.wikipedia.org/wiki/William_the_Silent (accessed 17 March 2014).
6. See http://news.bbc.co.uk/2/hi/europe/4015173.stm (accessed 17 March 2014).
7. Comparative lawyers use the term ‘tertium comparationis’ to point to the fact that legal systems that one wants to compare need to be comparative by a third unit apart from the two legal comparanda. This third unit is supposed to be the common denominator. Örücü, AE ‘Methodology of comparative law’ in Smits, J (ed) Elgar Encyclopedia of Comparative Law (Cheltenham: Edward Elgar, 2nd edn, 2012) p 561.
8. Film-maker Theo van Gogh was killed by a Muslim extremist less than 2 weeks before the television show was broadcast.
10. This partly explains the criticism levelled at popular websites such as http://www.ratemyprofessors.com, which are closer to a popularity contest than a serious evaluation of professors' performance in terms of teaching qualities. See http://sites.bio.indiana.edu/~bender/resources/Assessment/fractalevals10.pdf (accessed 17 March 2014).
11. See http://www.theguardian.com/education/2011/jun/27/journals-index-angers-european-academics (accessed 17 March 2014).
12. See the special issue on globalisation and legal education: Chiesa, N, de Luca, A and Maheandiranthe, B (eds) ‘Following the call of the wild: the promises and perils of transnationalizing legal education’ (2009) 10 German L J 629–1168.
13. See the report by the Cohen Committee, Sustainable Humanities (Amsterdam: Amsterdam University Press, 2008) p 41, explaining that in the humanities there is an ‘inadequate range of tools for objectively defining and differentiating quality’.
14. See B de Witte ‘European Union law: a unified academic discipline?’ (2008) Eui Working Papers Rscas no 34, available at http://cadmus.eui.eu/dspace/handle/1814/10028 (accessed 17 March 2014).
15. See H Morris ‘University ranking shows boom in global student mobility’ International Herald Tribune, 11 September 2012, referring to data gathered by QS, World University Ranking: http://www.topuniversities.com/university-rankings/world-university-rankings/new-look-rankings-show-unstoppable-rise-student-mobili (accessed 17 March 2014).
16. No particular figures seem to be available for legal scholars. For the general trend in all fields of research, see Idea Consult ‘Study on mobility patterns and career paths of Eu researchers’, Final report (Brussels, June 2010), available at http://ec.europa.eu/euraxess/pdf/research_policies/MORE_final_report_final_version.pdf (accessed 17 March 2014).
17. Technically it is possible to generate a European law schools ranking from the QS world ranking, but the credibility of this ranking would be seriously affected by the limited data set and reliance on reputational surveys. It is also possible to use this ranking to rank schools by citation counts on Scopus, but that would also offer a very limited perspective of law school quality. See http://www.topuniversities.com/university-rankings/university-subject-rankings/2013/law-and-legal-studies (accessed 17 March 2014).
18. One could come to an informal ranking for European law journals by using Google Scholar Metrics. See http://scholar.google.com/citations?view_op=top_venues (accessed 17 March 2014). Google Scholar Metrics is a recent initiative, which currently only covers articles published between 2008 and 2012. The metrics are based on citations from all articles that were indexed in Google Scholar in July 2013. It excludes all sorts of citations that would be relevant for law, such as books and dissertations but also court opinions and so on.
19. See http://lawlib.wlu.edu/LJ/ (accessed 17 March 2014).
20. This does not imply that US student-edited journals necessarily publish lower-quality papers than European peer-reviewed journals. Some of the top law journals in the US are student-edited, although some have also introduced (un)official peer review practices for certain types of articles (eg empirical legal research). For a typical example, see http://www.professorbainbridge.com/professorbainbridgecom/2011/08/chicago-law-review-chutzpah.html (accessed 17 March 2014).
21. More generally, it is not always clear why a journal should receive a certain ranking. It can be the number of prescriptions, in which case general interest journals in bigger jurisdictions will always have an advantage over specialist journals in smaller jurisdictions; it can be done on the basis of the rejection rate, which tells little about what is rejected and why; or it can be the number of downloads or citations, which may also have different reasons than quality alone, such as accessibility. The German Law Journal, for instance, became – over a period of 10 years – the world's leading online peer-reviewed law journal in terms of the numbers of citations. See http://germanlawjournal.com/index.php?pageID=1 (accessed 17 March 2014).
22. See http://lawlib.wlu.edu/LJ/index.aspx (accessed 17 March 2014).
23. Weiler, J ‘Impact factor – the food is bad and what's more there is not enough of it’ (2012) 23 (3) Eur J Int'l L 607.
24. Koelman, J and Venniker, R ‘Public funding of academic research: the Research Assessment Exercise of the UK’ in Canton, E et al Higher Education Reform: Getting the Incentives Right (The Hague: Sdu Uitgevers, 2001).
25. See http://www.shanghairanking.com (accessed 17 March 2014).
26. See http://www.topuniversities.com/university-rankings/world-university-rankings (accessed 17 March 2014).
27. See http://www.timeshighereducation.co.uk/world-university-rankings (accessed 17 March 2014).
28. See http://www.leidenranking.com (accessed 17 March 2014).
29. For an interesting comparison of the quite different outcomes of Times and Sunday Times University League Tables 2014 (one of the UK rankings) and the Times Higher Education (THE) World University Rankings 2013–2014 on UK law schools, see Matthias Siems' legal blog of 6 October 2013, available at http://siemslegal.blogspot.nl/2013/10/the-discrepancy-between-uk-and-global.html (accessed 17 March 2014).
30. Expert Group on Assessment of University-Based Research ‘Assessing Europe's university-based research’ (Brussels: Directorate-General for Research, 2010) at 9–10.
31. Even the products of individual legal scholars are getting ranked now. For an interesting example of ranking of individual legal researchers, see Shapiro, F and Pearse, M ‘The most-cited law review articles of all time’ (2012) 110 Mich L Rev 1483.
32. Saunders, N ‘Student-edited law reviews: reflections and responses of an inmate’ (2000) 49 Duke L J 1663 at 1666 with reference to other sources.
33. Brophy, A ‘The emerging importance of law review rankings for law school rankings, 2003–2007’ (2007) 78 U Colo L Rev 35.
34. What is confusing, however, is that the concept of peer review is used in many different ways. Some reserve the use of the term for the process of subjecting an author's scholarly research to the scrutiny of independent experts in the same field, before a paper is published in a journal. Others see scrutiny by a publisher or an editorial board also as peer review, or include in the definition the refereeing of research proposals for funding.
35. See F van Vught and F Ziegele (eds) ‘U-Multirank: design and testing the feasibility of a multidimensional global university ranking’ (Consortium for Higher Education and Research Performance Assessment, June 2011) at 37.
36. See Siems, M ‘Legal originality’ (2008) 28 (1) Oxford J Legal Stud 147.
37. Robert, RM Serendipity: Accidental Discoveries in Science (New York: Wiley, 1989).
38. See Smits, J The Mind and Method of the Legal Academic (Cheltenham: Edward Elgar, 2012) p 130.
39. Numerous examples can be found in: Thornton, M Privatising the Public University: The Case of Law (New York: Routledge, 2012); Bok, D Universities in the Market Place: The Commercialization of Higher Education (Princeton, NJ: Princeton University Press, 2004); Lorenz, C (ed) If You're So Smart, Why Aren't You Rich? Universiteit, markt en management (Amsterdam: Boom, 2008).
40. Osterloh, M and Frey, B ‘Research governance in academia: are there alternatives to academic rankings?’ (2009) CESIFO Working Paper no 2797, at 19–20.
41. The quality of legal research is, for example, probably much strongly determined by the selection and training of young legal scholars than by the monitoring of their output, which of course does not mean that it cannot be meaningful to do this monitoring for other purposes.
42. Pen, M Prestatiemeting van wetenschappelijk onderzoek (Utrecht: Lemma, 2009) pp 206–207.
43. See eg http://vpf-web.harvard.edu/annualfinancial/ (accessed 17 March 2014).
44. European University Association ‘Global university rankings and their impact’, report (Brussels: European University Association, 2011) at 20.
45. Eg Thomson-Reuters Web of Science and Elsevier Scopus.
46. European University Association, above n 44, at 15.
47. See eg Newton, B ‘Preaching what they don't practice: why law faculties' preoccupation with impractical scholarship and devaluation of practical competencies obstruct reform in the legal academy’ (2010) 62 S C L Rev 105.
48. Collier, R ‘We're all socio-legal now?’ Legal education, scholarship and the ‘global knowledge economy’ – reflections on the Uk experience (2004) 26 Sydney L Rev 503; Berard, TJ ‘The relevance of the social sciences for legal education’ (2009) 19 (1/2) Legal Educ Rev 189; Hutchinson, T and Burns, K ‘Kylie, the impact of ‘empirical facts’ on legal scholarship and legal research training’ (2009) 43(2) Law Teacher 153.
49. Smits, J ‘European legal education, or: How to prepare students for global citizenship?’ (2011) 45 Law Teacher 163; Ulen, T ‘The impending train wreck in current legal education: how we might teach law as the scientific study of social governance’ (2009) 6 U St Thomas L J 302.
50. For an intelligent attempt to integrate different views: Ribstein, L ‘Practicing theory: legal education for the twenty-first century’ (2011) 96 Iowa L Rev 1649.
51. Bartie, S ‘The lingering core of legal scholarship’ (2010) 30 Legal Stud 345 at 367.
52. However, the nature of doctrinal-legal research is also changing. van Gestel, R and Micklitz, H ‘Revitalising doctrinal legal research in Europe: what about methodology?’ in Neergaard, U, Nielsen, R and Roseberry, L (eds) European Legal Method (Copenhagen: DJOF, 2011) pp 25–75 (also available at http://cadmus.eui.eu/handle/1814/16825, accessed 17 March 2014).
53. Expert Group on Assessment of University-Based Research, above n 30, at 43.
54. See Spitzer, R Saving the Constitution from Lawyers (Cambridge: Cambridge University Press, 2008) pp 22–23.
55. Thomas, DA ‘The law school rankings are harmful deceptions: a response to those who praise the rankings and suggestions for a better approach to evaluating law schools’ (2003) 40(2) Hous L Rev 419 at 456; Van Alstyne, S ‘Ranking the law schools: the reality of illusion?’ (1982) 7(3) Am Bar Found'n Res J 649 at 679.
56. Ciolli, A and Garber, J ‘How to rank law school rankings?’ (2009) 7(2) Dartmouth L J 132 at 146.
57. Newton, B ‘Law review scholarship in the eyes of the twenty-first century Supreme Court Justices: an empirical analysis’ (2012) 4 Drexel L Rev 399.
58. In the US, many students were misled by law school rankings and made expensive choices to go to study law at certain schools, so that many of them now have severe debts, which will be hard to repay. See inter alia Tamanaha, B Failing Law Schools (Chicago: University of Chicago Press, 2012).
59. Berger, M ‘Why the Us News & World Report law school rankings are both useful and important’ (2001) 51 J Legal Educ 487 at 498.
60. Korobkin, R ‘Harnessing the positive power of rankings: a response to Posner and Sunstein’ (2006) 81 Ind L J 35 at 45.
61. Posner, above n 1, at 13.
62. For an overview of the problems with the ranking of universities in general, see van Vught and Ziegele, above n 35, at 27–34.
63. Murray, J ‘Do Us law schools that report false or misleading employment statistics violate consumer protection laws?’ (2012) 15(3) J Consumer & Com L 97. See also Kuncel, NR, Credé, M and Thomas, LL ‘The validity of self-reported grade point averages, class ranks, and test scores: a meta-analysis and review of the literature’ (2005) 75(1) Rev Educ Res 63.
64. Leiter, B ‘How to rank law schools’ (2006) 81 Ind L J 47 at 51.
65. Barton, B ‘Is there a correlation between law professor publication counts, law review citations counts, and teaching evaluations? an empirical study’ (2008) 5(3) J Empirical Legal Stud 619.
66. Teaching and learning, research, knowledge transfer, international orientation and regional engagement.
67. Instead, the U-Multirank tool uses sunburst charts and field charts. See Van Vugth and Ziegele, above n 35, at 18–19.
68. See House of Lords ‘The modernisation of higher education in Europe’ (London, March 2012) at 24–25.
69. For an overview, see http://www.llm-guide.com/law-school-rankings (accessed 17 March 2014).
70. It seems more likely that student choices will be influenced by a number of issues, including customer service from the administration, tuition fees, the availability of scholarships, discussions on blogs, location, language and so on.
71. Thornton, M ‘The law school, the market and the new knowledge economy’ (2009) 10(7) German L J 641.
72. See http://siemslegal.blogspot.nl/2012/10/uk-law-journal-ranking-rae-2008-data.html (accessed 17 March 2014).
73. See http://research.unsw.edu.au/excellence-research-australia-era-outlet-ranking (accessed 17 March 2014); Ramsay, I and Stapledon, GP ‘A citation analysis of Australian law journals’ (1997) 21 Melb U L Rev 676; Warren, D ‘Australian law journals: an analysis of citation patterns’ (1996) 27 Austl Acad & Res Libr 261.
74. Perry, R ‘Ranking Hebrew law reviews: theoretical foundations and a preliminary empirical study’ (2004) 1 Haifa L Rev 401.
75. George, T and Guthrie, C ‘An empirical evaluation of specialized law reviews’ (1999) 26 Fla St U L Rev 813 at 813; Jarvis, R and Coleman, P ‘Ranking law reviews: an empirical analysis based on author prominence’ (1997) 39 Ariz L Rev 15; R Jarvis and P Coleman ‘Ranking law reviews by author prominence – ten years later’ (2007) 99(3) Law Libr J 573.
76. For an overview, see Korobkin, R ‘Ranking journals: some thoughts on theory and methodology’ (1998–1999) 26 Fla St U L Rev 851.
77. See http://www.theaustralian.com.au/higher-education/end-of-an-era-journal-rankings-dropped/story-e6frgcjx-1226065864847 (accessed 17 March 2014). One example was that research managers immediately started to set targets for publication in A and A* journals.
78. For an example, see http://siemslegal.blogspot.co.uk/2011/03/world-law-journal-ranking-2011.html (accessed 17 March 2014).
79. Campbell, K, Goodacre, A and Little, G ‘Ranking of United Kingdom law journals: an analysis of the Research Assessment Exercise 2001. Submissions and results’ (2006) 33 J L Soc'y 335.
80. An elaborate list of advantages can be found in Perry, R ‘The relative value of American law reviews: a critical appraisal of ranking methods’ (2006) 11(1) Va J L & Tech 1 at 4–6.
81. Svantesson, J ‘International ranking of law journals – can it be done and at what cost?’ (2009) 29(4) Legal Stud 678 at 685.
82. Ibid, at 683.
83. Korobkin, above n 76, at 866.
84. In the case of empirical-legal research, how will the data be double-checked?
85. See van Gestel, R and Micklitz, HW ‘Why methodology matters’ Eur L Rev, forthcoming (2014). See http://onlinelibrary.wiley.com/doi/10.1111/eulj.12049/abstract (accessed 17 March 2014). In Europe for a long period throughout history, law as a discipline was a role model for other sciences. See van Caenegem, R Judges, Legislators and Professors, Chapters in European Legal History (Cambridge: Cambridge University Press, 1987).
86. See Tamanaha, above n 58, p 71 and further. See also Lorenz, above n 39.
87. With regard to the index, see http://thomsonreuters.com/arts-humanities-citation-index/ (accessed 17 March 2014).
88. Council for the Humanities and the Social Sciences Council ‘Judging research on its merits. an advisory report by the Council for the Humanities and the Social Sciences Council’ (Amsterdam: Royal Academy of Arts and Sciences, 2005) at 17, available at https://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20051029.pdf (accessed 17 March 2014).
89. See http://pf.ujep.cz/user_files/ERIH_zakladni%20informace.pdf (accessed 17 March 2014).
90. Cook, H et al ‘Journals under threat: a joint response from history of science, technology and medicine editors’ (2009) 53(1) Med Hist 1 at 1.
91. Ochsner, M, Hug, S and Daniel, H-D ‘Indicators for research quality for evaluation of humanities research: opportunities and limitations’ (2012) 1 Bibliometrie – Praxis und Forschung at 4.4–4.5.
92. For an overview, see https://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20111024.pdf (accessed 17 March 2014).
93. See Royal Academy of Sciences ‘Quality indicators for research in the humanities’ (Amsterdam, 2011), available at http://www.knaw.nl/shared/resources/actueel/publicaties/pdf/20111024.pdf (accessed 17 March 2014).
94. Hicks, D and Wang, J ‘Towards a bibliometric database for the social sciences and humanities – a European scoping project’ (School of Public Policy, Georgia Institute of Technology, 2009).
95. See the ERiC project, which stands for Evaluating Research in Context: http://www.rathenau.nl/uploads/tx_tferathenau/ERiC_guide.pdf (accessed 17 March 2014).
96. A recent survey among 1325 law professors, 338 student-editors, 215 attorneys and 156 judges in the US shows that all of these groups agree on the fact that important reforms are desperately needed for law reviews. Surprisingly, the vast majority agreed that the introduction of some type of blind peer review would increase the quality of the papers being published and diminish the current biases in the selection process. Wise, R et al ‘Do law reviews need reform? a survey of law professors, student-editors, attorneys and judges’ (2013) 59 Loy L Rev 1.
97. McCormack, N ‘Peer review and legal publishing: what law librarians need to know about open, single-blind, and double-blind reviewing’ (2009) 101:1 Law Libr J 59.
98. Van Hoecke, M (ed) Methodologies of Legal Research: Which Kind of Method for What Kind of Discipline? (Oxford: Hart Publishing, 2011).
99. See Mertz, E The Language of Law School: Learning to Think Like a Lawyer (Oxford: Oxford University Press, 2007).
100. Spitzer, above n 54, p 23.
101. Ibid, pp 24–25.
102. See http://www.scienceguide.nl/201306/multirank-milestone-reached.aspx (accessed 17 March 2014).
103. See B Tamanaha Failing Law Schools (Chicago: University of Chicago Press, 2012). See also http://lawprofessors.typepad.com/legal_skills/2013/03/the-pernicious-influence-of-the-us-news-law-school-rankings.html (accessed 17 March 2014).
104. Crittenden, J ‘Building better ranking’ (2013) 22(5) Nat'l Jurist 24–27. See http://www.nxtbook.com/nxtbooks/cypress/nationaljurist0213/#/22 (accessed 17 March 2014).
105. Think of issues such as using a (PhD) student work in one's own publications without reference, plagiarism or selling out to sponsors of contract research.
106. In Switzerland, for example, where the Conference of the Swiss Universities (CRUS) has started a project ‘Research evaluation in law’. See http://www.rechtswissenschaft.unibe.ch/unibe/rw_neu/content/e1984/e373454/e373467/e386328/e386329/files386330/Lienhard_Amschwand_HerrmannLeGes2013_2_ger.pdf (accessed 17 March 2014). In The Netherlands, the deans of the joint law schools have established a committee on classification of law journals, chaired by the dean of Maastricht law school. See Du Perron, E ‘De kwalificatie van rechtswetenschappelijke tijdschrift, in van Dijck, G et al (eds) Cirkels: een terugblik op een vooruitziende blik (Deventer: Kluwer, 2013) pp 191–199.
* Rob van Gestel is professor of theory and methods of regulation at Tilburg Law School (The Netherlands), professor of methodology of legal research at Leuven Law School (Belgium) and Braudel fellow at the European University Institute. The author would like to thank Han Somsen, Mathias Siems, Jan Vranken and the faculty members of the Law Department of the EUI in Florence for their valuable input. The usual disclaimer applies.
Email your librarian or administrator to recommend adding this journal to your organisation's collection.