Despite being coined only in the early 1970s, ‘peer review’ has become a powerful rhetorical concept in modern academic discourse, tasked with ensuring the reliability and reputation of scholarly research. Its origins have commonly been dated to the foundation of the Philosophical Transactions in 1665, or to early learned societies more generally, with little consideration of the intervening historical development. It is clear from our analysis of the Royal Society's editorial practices from the seventeenth to the twentieth centuries that the function of refereeing, and the social and intellectual meaning associated with scholarly publication, has historically been quite different from the function and meaning now associated with peer review. Refereeing emerged as part of the social practices associated with arranging the meetings and publications of gentlemanly learned societies in the late eighteenth and nineteenth centuries. Such societies had particular needs for processes that, at various times, could create collective editorial responsibility, protect institutional finances, and guard the award of prestige. The mismatch between that context and the world of modern, professional, international science, helps to explain some of the accusations now being levelled against peer review as not being ‘fit for purpose’.
The authors wish fulsomely to acknowledge the assistance of their colleagues on the ‘Publishing the Philosophical Transactions’ project, Dr Julie McDougall-Waters and Dr Camilla Mørk Røstvik, whose archival research underpins the post-1850 parts of this article. They are grateful for the generous support and assistance of the staff of the Library and Publishing divisions of the Royal Society, London. They also thank the audiences who responded to versions of this article presented to the Cambridge HPS Twentieth-Century Think Tank, the Kent Wunderkammer, and the History of Science seminar at King's College London. They are grateful for comments received from Stuart Taylor, Didier Torny, Berris Charnley, David Teplow, Camilla Mørk Røstvik, and Stephen Curry, as well as for the comments and support of the editor and the two anonymous referees. The research for this paper was funded by the Arts & Humanities Research Council, grant AH/K001841/1.
1 Science and Technology Committee, ‘Peer review in scientific publications. Eighth Report of Session 2010–12’ (London, 2011), p. 3.
2 Since the early 1990s, the International Congress on Peer Review and Biomedical Publishing has drawn attention to the supposed failings of peer review, www.peerreviewcongress.org/index.html (accessed 12 May 2016). Lee, Carole J., Sugimoto, Cassidy R., Zhang, Guo, and Cronin, Blaise, ‘Bias in peer review’, Journal of the American Society for Information Science and Technology, 64 (2013), pp. 2–17 , is a useful review of the wide variety of studies of peer review bias.
3 Biagioli, Mario, ‘From book censorship to academic peer review’, Emergences: Journal for the Study of Media & Composite Cultures, 12 (2002), pp. 11–45 , at p. 34. The Oxford English dictionary gives 1967 for the first usage (in the context of the evaluation of hospitals), and 1971 for the academic usage.
4 See, for instance, the writings of senior publisher, Michael Mabe, and those that draw upon him, for instance Mabe, Michael, ‘Does journal publishing have a future?’, in Campbell, Robert, Pentz, Ed, and Borthwick, Ian, eds., Academic and professional publishing (Oxford, 2012), pp. 413–40, at pp. 416–17. But see also Lee, Sugimoto, Zhang, and Cronin, ‘Bias in peer review’.
5 Zuckerman, Harriet and Merton, Robert K., ‘Patterns of evaluation in science: institutionalisation, structure and functions of the referee system’, Minerva, 9 (1971), pp. 66–100 , at p. 68. For Merton's norms of science (including ‘organized scepticism’), see Merton, Robert K., ‘Science and technology in a democratic order’, Journal of Legal and Political Sociology, 1 (1942), pp. 115–26.
6 Kronick, David A., ‘Peer review in 18th-century scientific journalism’, Journal of the American Medical Association, 263 (1990), pp. 1321–2.
7 Burnham, J. C., ‘The evolution of editorial peer review’, Journal of the American Medical Association, 10 (1990), pp. 1323–9; and Baldwin, Melinda, ‘Credibility, peer review, and Nature, 1945–1990’, Notes and Records of the Royal Society, 69 (2015), pp. 337–52.
8 Despaux, Sloan Evans, ‘Fit to print? Referee reports on mathematics for the nineteenth-century journals of the Royal Society of London’, Notes and Records of the Royal Society, 65 (2011), pp. 233–52; Baldwin, Melinda, ‘Tyndall and Stokes: correspondence, referee reports and the physical sciences in Victorian Britain’, in Lightman, Bernard and Reidy, Michael S., eds., The age of scientific naturalism: Tyndall and his contemporaries (London, 2014), pp. 171–86; Clarke, Imogen, ‘The gatekeepers of modern physics: periodicals and peer review in 1920s Britain’, Isis, 106 (2015), pp. 70–93 ; Lalli, Roberto, ‘“Dirty work”, but someone has to do it: Howard P. Robertson and the refereeing practices of Physical Review in the 1930s’, Notes and Records of the Royal Society, 70 (2016), pp. 151–74.
9 Scholarly communication and peer review: the current landscape and future trends (London, 2015).
10 Ziman, John M., Public knowledge: an essay concerning the social dimension of science (Cambridge, 1968), p. 111 .
11 On the origins of the Society's motto and some of the possibilities contemplated for it, see Hunter, Michael, Establishing the new science: the experience of the early Royal Society (Woodbridge, 1989), pp. xiv and 41–2.
12 Mabe, ‘Does journal publishing have a future?’. This claim is repeated in Scholarly communication and peer review, p. 6.
13 On Oldenburg's editorial practices, see Johns, Adrian, ‘Miscellaneous methods: authors, societies and journals in early modern England’, British Journal for the History of Science, 33 (2000), pp. 159–86; Hodson, Niall, ‘Henry Oldenburg's Philosophical Transactions as a repository of translations’, paper presented to Publish or Perish conference (Royal Society, Mar. 2015); Moxham, Noah, ‘Authors, editors and newsmongers: form and genre in the Philosophical Transactions under Henry Oldenburg’, in Raymond, Joad and Moxham, Noah, eds., News networks in early modern Europe (Leiden, 2016), pp. 465–92.
14 Valle, Ellen, ‘“Reporting the doings of the curious”: authors and editors in the Philosophical Transactions of the Royal Society of London’, in Brownlees, Nicholas, ed., News discourse in early modern Britain: selected papers of CHINED 2004 (Bern, 2006), pp. 71–90 .
15 Moxham, Noah, ‘Fit for print: developing an institutional model of scientific periodical publishing in England, 1665–ca. 1714’, Notes and Records of the Royal Society, 69 (2015), pp. 241–60.
16 Johns, Adrian, The nature of the book: print and knowledge in the making (Chicago, IL, 1998), pp. 492–504 ; Biagioli, ‘From book censorship to academic peer review’. Biagioli argues that the Royal Society in London and the Académie Royale in Paris were incorporated into mechanisms of state oversight of the book trade.
17 On the context of book censorship in the Restoration, see Hinds, Peter, ‘The horrid popish plot’: Roger L'Estrange and the circulation of political discourse in late seventeenth-century London (Oxford, 2010). It is important to note that the actual mechanism of scrutiny prior to licensing rested on a decision of council of December 1663 and not on the Society's statutes; Birch, Thomas, The history of the Royal Society of London for improving of natural knowledge (4 vols., London, 1756–7), i, p. 347 .
18 Biagioli acknowledges differences in the degree of association between the respective learned bodies and the state and in the precise mechanisms of licensing used in the Académie and the Society – and in particular the Paris body's much greater willingness to assert its power to adjudicate the status of claims to knowledge by outsiders and in the formalization of the status of corresponding members – but his representation of the function of licensing as consisting in constructing an epistemic framework for peer review, and his insistence on rooting the historical origin of the ‘peer’ element of peer review in what was essentially a legal category, are broadly similar in each case. Biagioli, ‘Book censorship to academic peer review’, esp. pp. 15–18, 23–5, 30–2.
19 Rivington's, C. A. ‘Printers to the early Royal Society’, Notes and Records of the Royal Society, 39 (1984), pp. 1–27 ; and a 1986 ‘Addendum’, Notes and Records of the Royal Society, 40 (1986), pp. 219–20, supply a useful checklist of works published with the imprimatur.
20 Birch, History, i, pp. 344, 346–7.
21 For a fuller development of this argument, see Moxham, Noah, ‘The uses of licensing: publishing strategy and the imprimatur at the early Royal Society’, in Feingold, Mordechai and Giannini, Giulia, eds., Institutionalisation of sciences in early modern Europe (Leiden, 2018, forthcoming).
22 This is the instance of Moses Rusden's treatise on bees, ‘Monarchy founded in nature’, submitted to the Society with John Evelyn's encouragement in 1679 and considered for licensing at Rusden's request; following a mixed report by Thomas Croone, Rusden published without the Society's imprimatur (A further discovery of bees, 1679). See Birch, History, iii, pp. 473–4, 479. Rusden's work was exceptionally problematic because it rested its monarchist political and moral conclusions on errors of natural history.
23 For striking recent research showing how to work around the Society's mechanisms of scrutiny, Hunter, Michael, ‘John Webster, the Royal Society and The displaying of supposed witchcraft (1677)’, Notes and Records of the Royal Society, 71 (2017), pp. 7–19 . Published 12 Oct. 2016.
24 The exception was Edmond Halley who, during his first stint as editor (1686–92) was the Society's paid clerk.
25 Moxham, ‘Fit for print’.
26 There were, for instance, satirical critiques on the Transactions in 1700 and 1711: see Levine, Joseph M., Dr. Woodward's shield: history, science, and satire in Augustan England (Ithaca, NY, 1991). Some fellows thought that it was futile to keep denying responsibility; for instance, see John Harris to Sir John Hoskins, 27 Feb. 1699/1700, British Library Sloane MSS 4026, fo. 254.
27 Hill, John, A review of the works of the Royal Society (London, 1751), pp. 9–11 . The original paper was published in Philosophical Transactions, 23 (1702–3), pp. 1094–96, as ‘Observations made in the island of Ceilan…by Mr Strachan’.
28 On individual editorial responsibility, even within teams, see Kronick, David A., ‘Authorship and authority in the scientific periodicals of the seventeenth and eighteenth centuries’, Library Quarterly, 48 (1978), pp. 225–75.
29 Royal Society Council Minutes Original (RS CMO) 4.
30 Royal Society Committee Minute Book (RS CMB) 90/1.
31 McClellan, James III, ‘Specialist control: the publications committee of the Académie Royale des Sciences (Paris), 1700–1793’, Transactions of the American Philosophical Society, 93 (2003), pp. 1–134 ; Minutes of Proceedings of the Royal Society of Edinburgh, 17 July 1784, National Library of Scotland, Acc.10000/1.
32 From our examination of the minute books of the Committee of Papers, RS CMB/90/2.
33 Fyfe, Aileen, ‘Journals, learned societies and money: Philosophical Transactions, ca. 1750–1900’, Notes and Records of the Royal Society, 69 (2015), pp. 277–99.
34 Fyfe, Aileen and Moxham, Noah, ‘Making public ahead of print: meetings and publications at the Royal Society, 1752–1892’, Notes and Records of the Royal Society, 70 (2016), pp. 361–79.
36 E.g. ‘Advertisement’, Philosophical Transactions, 64 (1774), pp. iii–iv .
37 Morrell, Jack and Thackray, Arnold, Gentlemen of science: early years of the British Association for the Advancement of Science (Oxford, 1981), ch. 2; Secord, James A., Visions of science: books and readers at the dawn of the Victorian age (Oxford, 2015), ch. 2.
38 Committee for limiting the fellowship, reported to council on 11 June 1827, RS CMB/1/20/2, pp. 167–8, quoted in Babbage, Charles, Reflections on the decline of science in England (London, 1830), p. 164 .
39 [Augustus Bozzi Granville], Science without a head; or, The Royal Society dissected. By one of the 687 F.R.S. (London, 1830), p. 122.
40 Augustus, Frederick, duke of Sussex, ‘[Presidential address 1832]’, Abstracts of the papers printed in Philosophical Transactions of the Royal Society of London, 3 (1830–7), pp. 140–55, at p. 141. Although Sussex claimed this trial had been resolved by council, there is no mention of it in the council minute books.
41 Diplomata et statuta Regalis Societatis Londini (London, 1752), p. 109 .
42 [Granville], Science without a head, p. 54.
43 Sussex, ‘[Presidential address 1832]’, p. 142.
44 Ibid., p. 142.
45 Csiszar, Alex, ‘Peer review: troubled from the start’, Nature, 532 (2016), pp. 306–8. For the French model, see McClellan III, ‘Specialist control’.
46 There is an 1835 exception.
47 Some even preferred not to make a positive recommendation: see Thomas Wharton Jones's 24 June 1841 report on a paper by J. M. Ferrall, Royal Society Referees’ Reports (RS RR) 1/64.
48 RS RR/1/30 and 31.
49 Council resolved to print abstracts of the papers read at meetings on 16 Dec. 1830 (RS CMO/12 pp. 144–6); the first issue covered the meetings of 18 Nov. to 16 Dec. 1830; though the date entry on the Royal Society's account for the first issue does not appear in the printer's records until 25 Feb. 1831: Taylor and Francis Journal (St Bride's Library) 1830–40. The issues of the new periodical were titled Proceedings of the Royal Society, but the early bound volumes have a title page Abstracts of the papers printed in the Philosophical Transactions for continuity with the retrospective series of abstracts covering 1800–30.
50 Taylor and Francis Journal (St Bride's Library) 1830–40, 21 Mar. 1833. The Transactions print run at the time was 1,000.
51 From our analysis of the Register of Papers, RS MS/421.
52 Fellows could either pay annual fees, or ‘compound’ their future fees by paying a hefty £60; this ‘compounded’ fee was reduced to £40 for those with a Transactions paper. See The record of the Royal Society of London (London, 1912), p. 170 . This bias towards Transactions was discontinued in 1887, ibid., p. 275.
53 For more detailed discussion of refereeing practices in this period, see Despaux, ‘Fit to print?’; and Baldwin, ‘Tyndall and Stokes’.
54 Baldwin, ‘Tyndall and Stokes’. Julie McDougall-Waters, ‘Peer review in the nineteenth century’, paper presented to Publish or Perish conference (Royal Society, Mar. 2015).
55 Royal Society Council Minutes Printed (RS CMP) 7, 6 Dec. 1894.
56 Banks was happy for the substance of the comments to be passed on but insisted that authors should not be given the referee's exact words nor his identity. See Miller, David Philip, ‘The usefulness of natural philosophy: the Royal Society and the culture of practical utility in the later eighteenth century’, British Journal for the History of Science, 32 (1999), pp. 185–201 .
57 RS CMP/7, 6 Dec. 1894.
58 C. Piazzi Smyth, ‘Solar science at the pleasure of secret referees’, Nature, 13 Apr. 1871, pp. 468–9. Smyth was the Astronomer Royal for Scotland and had previously resigned as a fellow of the Royal Society.
59 Lister, Joseph, ‘Address of the president’, Year-book of the Royal Society, 1896–1897 (London, 1896), pp. 119–37, at p. 124. Lister made no mention of the earlier incarnation of Sectional Committees.
60 RS CMP/7, 6 Dec. 1896.
61 RS CMP/7, 26 Apr. 1894; see also Fyfe, ‘Journals, learned societies and money’.
62 RS CMP/7, 6 Dec. 1894. The limits were forty pages quarto for papers in Transactions, and no more than £35 of illustrations; but exceptions were allowed if agreed on two separate occasions by the Committee of Papers.
63 RS CMP/7, 6 Dec. 1894.
64 For example, RS RR/1/71 and 72, in which W. H. Allen and Charles Daubeney respectively suggest that a paper by the Edinburgh geologist J. D. Forbes ought to be abridged. On the grant (distinct from that for supporting scientific research), see Fyfe, ‘Journals, learned societies and money’.
65 RS CMP/7, 21 May 1896; Committee of Papers report, 24 Oct. 1907, RS CMB/90/6. The practice of sending costs appears to have lapsed during the war, but had resumed by the 1920s.
66 For a different instance of evaluating a mixture of intellectual and commercial issues, consider the publishers’ readers discussed in Nickerson, Sylvia, ‘Referees, publisher's readers and the image of mathematics in nineteenth century England’, Publishing History, 71 (2012), pp. 27–67 .
67 From our analysis of the Register of Papers, RS MS/422.
68 The rule about communication was the very first item in the explanatory notes issued from 1896, though the practice dated to at least the eighteenth century. See ‘Explanatory notes on the procedure relating to the reading and publication of papers’, Year-book of the Royal Society, 1897–1898 (London, 1898), p. 67 . For Evans's concern, see RS CMP/6, 26 Apr. 1894.
69 From our analysis of the RS Register of Papers, RS MS/421–2.
70 The expanding remit of the Society is described in Hall, Marie Boas, All scientists now: the Royal Society in the nineteenth century (Cambridge, 1984).
71 Lister, ‘Address of the president’, at p. 124.
72 From 1914, the difference was notionally nothing more than page length and number of illustrations. RS CMP/10, 21 May 1914, clauses 36 and 52. For a more detailed discussion of editorial practice at Proceedings, see Clarke, ‘Gatekeepers of modern physics’.
73 The average length of a paper in Transactions in the 1910s was 44 pages; but for 112 pages of comparative anatomy, see Fraser, Elizabeth A. and Hill, J. P., ‘The development of the thymus, epithelial bodies, and thyroid in the marsupialia. Part I. Trichosurus vulpecula’, Philosophical Transactions of the Royal Society of London B: Biological Sciences, 207 (1916), pp. 1–85 ; Elizabeth A. Fraser, ‘The development of the thymus, epithelial bodies, and thyroid in the marsupialia. Part II. phascolarctos, phascolomys, and perameles’, ibid., pp. 87–112.
74 Memorandum by H. E. Armstrong, in RS CMP/8, 6 Nov. 1902.
75 ‘Philosophical Magazine: report by research committee [of the National Union of Scientific Workers]’, Scientific Worker, 29–30 (11 Mar. 1922), p. 29, quoted in Clarke, Imogen and Mussell, James, ‘Conservative attitudes to old-established organs: Oliver Lodge and Philosophical Magazine ’, Notes and Records of the Royal Society, 69 (2015), pp. 321–36, at p. 321.
76 ‘Philosophical Magazine: report by research committee [of the National Union of Scientific Workers]’; and memorandum by H. E. Armstrong, in RS CMP/8, 6 Nov. 1902.
77 Waterston, J. J. and Rayleigh, Lord, ‘On the physics of media that are composed of free and perfectly elastic molecules in a state of motion’, Philosophical Transactions of the Royal Society of London A: Physical Sciences, 183 (1892), pp. 1–79 , at p. 3.
78 An exception was the British Medical Journal, which used refereeing from 1870; see Burnham, ‘Evolution of editorial peer review’, p. 1325.
79 For an account of this at Nature, see Baldwin, Melinda, Making ‘Nature’: the history of a scientific journal (Chicago, IL, 2015); and Baldwin, ‘Credibility, peer review, and Nature’. For the Philosophical Magazine, see Clarke and Mussell, ‘Conservative attitudes’.
80 We know of equivalent systems used at the Royal Society of Edinburgh and the American Physical Society, as well as at the (London) Geological Society and Astronomical Society. Except for Lalli's work on the journal of the American Physical Society (Lalli, ‘“Dirty work”’), and work in progress on the Royal Society of Edinburgh by Sian Burkitt and Aileen Fyfe, little is known about the common trends or the idiosyncrasies of learned society editorial practice.
81 Baldwin, Melinda, ‘“Keeping in the race”: physics, publication speed and national publishing strategies in Nature, 1895–1939’, British Journal for the History of Science, 47 (2014), pp. 257–79.
82 RS CMP/22, 15 June 1967. The proposals passed on 9 May 1968 and were implemented on 1 Jan. 1969.
83 RS CMP/22, 15 June 1967. See also ‘Notes for the guidance of associate editors’ , RS.
84 Sussex, ‘[Presidential address 1832]’, p. 142.
85 We have yet to find any such critiques of Royal Society editorial practice, but for changing sensibilities in the social sciences and humanities, see Pontille, David and Torny, Didier, ‘The blind shall see! The question of anonymity in journal peer review’, Ada: A Journal of Gender, New Media, and Technology, 4 (2014), DOI:10.7264/N3542KVW .
86 RS CMP/22, 6 May 1965.
87 See Pontille and Torny, ‘The blind shall see!’. See also Pontille, David and Torny, Didier, ‘From manuscript evaluation to article valuation: the changing technologies of journal peer review’, Human Studies, 38 (2015), pp. 57–79 .
88 The Standing Orders in use from 1899 admitted the possibility of ‘special reasons’ why it would be desirable to consult referees who were not fellows. See Year-book of the Royal Society, 1901 (London, 1901), p. 65 . Most of the known exceptions involved people who went on to become fellows shortly thereafter. For instance, the physicist Charles Galton Darwin acted as referee shortly before his 1922 election to the fellowship; and the botanist Agnes Arber was consulted in 1939, and became the third female fellow in 1946.
89 ‘Notes for the guidance of associate editors’ , RS.
90 N. K. Adam to D. C. Martin, 15 July 1950, regarding paper A128, RS Referee Reports Withdrawn 1950. See also Camilla Mørk Røstvik, ‘“I am seriously tempted to burn some of the papers which reach me for an opinion”’, Times Higher Education (2016), www.timeshighereducation.com/features/workload-survival-guide-for-academics (accessed 24 Aug. 2016).
91 [Alfred Egerton?], ‘A note on Proceedings and Transactions A’ , RS Egerton papers.
92 Martin, D. C., ‘The Royal Society's interest in scientific publications and the dissemination of information’, Aslib Proceedings, 9 (1957), pp. 127–41, at p. 134.
93 RS CMP/22, 15 June 1967.
94 Memo by L. N. G. Filon to RS Council, 9 July 1936, RS CMP14.
95 N. K. Adam to D. C. Martin, 15 July 1950, in RS RR/Withdrawn_AB_1950/A128.
96 Baldwin, ‘Credibility, peer review, and Nature’.
97 On the adoption of peer review in grant-making in the USA, see Melinda Baldwin, ‘How “real science” became peer reviewed: scientific autonomy and public accountability in the Cold War United States’ (in preparation; we are grateful for advance sight of this essay).
98 Burnham, ‘Evolution of editorial peer review’, p. 1323.
99 On an extended timeframe for professionalization, see Bowler, Peter, Science for all: the popularization of science in the early twentieth century (Chicago, IL, 2009).
100 But see Pontille and Torny, ‘From manuscript evaluation to article valuation’.
101 See Goldie, Mark, ‘Fifty years of the Historical Journal ’, Historical Journal, 51 (2008), pp. 821–55, at pp. 830, 839.
102 More has been written on the status of historical knowledge than on how academic historians actually work. See Jordanova, Ludmilla, History in practice (London, 2000), ch. 4; Bentley, Michael, Modernizing England's past: English historiography in the age of modernism, 1870–1970 (Cambridge, 2006), ch. 8; Kenyon, John P., The history men: the historical profession in England since the Renaissance (London, 1983).
103 Nickerson, ‘Referees, publisher's readers’.
* The authors wish fulsomely to acknowledge the assistance of their colleagues on the ‘Publishing the Philosophical Transactions’ project, Dr Julie McDougall-Waters and Dr Camilla Mørk Røstvik, whose archival research underpins the post-1850 parts of this article. They are grateful for the generous support and assistance of the staff of the Library and Publishing divisions of the Royal Society, London. They also thank the audiences who responded to versions of this article presented to the Cambridge HPS Twentieth-Century Think Tank, the Kent Wunderkammer, and the History of Science seminar at King's College London. They are grateful for comments received from Stuart Taylor, Didier Torny, Berris Charnley, David Teplow, Camilla Mørk Røstvik, and Stephen Curry, as well as for the comments and support of the editor and the two anonymous referees. The research for this paper was funded by the Arts & Humanities Research Council, grant AH/K001841/1.
Email your librarian or administrator to recommend adding this journal to your organisation's collection.
Full text views reflects the number of PDF downloads, PDFs sent to Google Drive, Dropbox and Kindle and HTML full text views.
* Views captured on Cambridge Core between 16th November 2017 - 25th March 2018. This data will be updated every 24 hours.