Skip to main content
×
Home

Peer-review practices of psychological journals: The fate of published articles, submitted again

  • Douglas P. Peters (a1) and Stephen J. Ceci (a2)
Abstract
Abstract

A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although questions about reliability, accountability, reviewer bias, and competence have been raised, there has been very little direct research on these variables.

The present investigation was an attempt to study the peer-review process directly, in the natural setting of actual journal referee evaluations of submitted manuscripts. As test materials we selected 12 already published research articles by investigators from prestigious and highly productive American psychology departments, one article from each of 12 highly regarded and widely read American psychology journals with high rejection rates (80%) and nonblind refereeing practices.

With fictitious names and institutions substituted for the original ones (e.g., Tri-Valley Center for Human Potential), the altered manuscripts were formally resubmitted to the journals that had originally refereed and published them 18 to 32 months earlier. Of the sample of 38 editors and reviewers, only three (8%) detected the resubmissions. This result allowed nine of the 12 articles to continue through the review process to receive an actual evaluation: eight of the nine were rejected. Sixteen of the 18 referees (89%) recommended against publication and the editors concurred. The grounds for rejection were in many cases described as “serious methodological flaws.” A number of possible interpretations of these data are reviewed and evaluated.

Copyright
References
Hide All
Abelson P. H. (1979) Problems of science faculties (editorial). Science 204:133. [CM]
Adair R. K. (1981) Anonymous refereeing. Physics Today 34:1315. [RAG]
American Psychological Association (19631969) Reports of Project on Scientific Information Exchange in Psychology. Vols. 1–3. Washington, D.C.: APA, [BCG]
American Psychological Association (1972) Eight APA journals initiate controversial blind reviewing. APA Monitor 3:5. [taDPP]
American Psychological Association (1973) Ethical principles in the conduct of research with human participants. American Psychologist 28:7980. [JLF, BM]
American Psychological Association (1980) Summary report of journal operations for 1979. American Psychologist 35:575. [taDPP]
Anastasi A. (1958) Differential psychology. 3rd ed.New York: Macmillan. [GJW]
Armstrong J. S. (1979) Advocacy and objectivity in science. Management Science 25:423–28. [JSA]
Armstrong J. S. (1980a) Advocacy as a scientific strategy: The mitroff myth. Academy of Management Review 5:509–11. [JSA]
Armstrong J. S. (1980b) Unintelligible management research and academic prestige. Interfaces 10:8086. [JSA]
Armstrong J. S. (1982, forthcoming) Research on scientific journals: Implications for editors and authors. Journal of Forecasting and Medical Hypotheses. [JSA]
Bartko J. J. (1966) The intraclass correlation coefficient as a measure of reliability. Psychological Reports 19:311. [GJW]
Bartley W. W. (1962) The retreat to commitment. New York: Alfred A. Knopf. [MJM]
Benwell R. (1979) Authors anonymous? Physics Bulletin 30:288. [RAG]
Bernard H. R. (1980) CARS: Computer assisted referee selection. Journal of Research Communication Studies 2:149–57. [HRB]
Beyer J. (1978) Editorial policies and practices among leading journals in four scientific fields. Sociological Quarterly 19:6888. [JMB]
Blissett M. (1972) Politics in science. Boston: Little, Brown. [MB]
Boffey P. M. (1975) The brain bank of America: An inquiry into the politics of science. New York: McGraw-Hill. [CM]
Bowen D. D.; Perloff R. & Jacoby J. (1972) Improving manuscript evaluation procedures. American Psychologist 27:221–25. [taDPP]
Brackbill Y. & Korton F. (1970) Journal reviewing practices: Authors' and APA members' suggestions for revision. American Psychologist 25:937–40. [taDPP]
Broad W. J. (1980a) Imbroglio at Yale. 1. Emergence of a fraud. Science 210:38–11. [DdB]
Broad W. J. (1980b) Would-be academician pirates papers. Science 208:1438–10. [DdB]
Broad W. J. (1981a) Congress told fraud issue “exaggerated.” Science 212:421. [CM]
Broad W. J. (1981b) Fraud and the structure of science. Science 212:137–41. [DdB, CM]
Broad W. J. (1981c) The publishing game: Getting more for less. Science 211:1137–39. [DdB]
Bronfenbrenner U. (1977) Toward an experimental ecology of human development. American Psychologist 32:513–31. [taDPP]
Bruner J. S. (1962) On knowing: Essays for the left hand. Cambridge, Mass.: Harvard University Press. [IIM]
California may be sued on secret files. (1978) Times Higher Education Supplement, 11 3, p. 5. [CM]
Campbell D. T. & Fiske D. W. (1959) Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin 56:81105. [rDPP]
Caplow T. & McGee R. J. (1958) The academic marketplace. New York: Basic Books. [CM]
Carey W. D. (1975) Peer review revisited. Science 189:331. [MB]
Carta D. G. (1978) Forum for rejected papers. IEEE Spectrum 15:13. [RAG]
Cherfas J. (1980) Only the names have been changed to protect … whom? New Scientist, 20 03, p. 950. [Ed.]
Chubin D. E. (1980) Competence is not enough. Contemporary Sociology 9:204–7. [DEC, CM]
Chubin D. E. & Connolly T. (1982) Research trails and science policies: Local and extra-local negotiation of scientific work. In: Scientific establishments and hierarchies, ed. Elias N., Martins H., & Whitley H.. Sociology of the Sciences, vol. 6, pp. 293311. [ALP]
Cicchetti D. V. (1980) Reliability of reviews for the American Psychologist: A biostatistical assessment of the data. American Psychologist 35:300303. [DVC, tarDPP, GJW]
Cicchetti D. V. & Conn H. O. (1976) A statistical analysis of reviewer agreement and bias in evaluating medical abstracts. Yale Journal of Biology and Medicine 49:373–83. [taDPP]
Cicchetti D. V. & Eron L. D. (1979) The reliability of manuscript reviewing for the Journal of Abnormal Psychology. 1979 Proceedings of the Social Statistics Section, pp. 596600. Washington, D.C.: American Statistics Association. [DVC, tarDPP]
Coe R. K. & Weinstock I. (1967) Editorial policies of major economic journals. Quarterly Review of Economics and Business 7:3743. [JSA]
Cohen J. (1968) Weighted kappa: Nominal scale agreement with provision for scaled disagreement or partial credit. Psychological Bulletin 70:213–20. [GJW]
Cole J. R. & Cole S. (1972) The Ortega hypothesis: Citation analysis suggests that only a few scientists contribute to scientific progress. Science 178:368–74. [CM, taDPP]
Cole J. R. & Cole S. (1973) Social stratification in science. Chicago: University of Chicago Press. [CM]
Cole J. R. & Cole S. (1981) Peer review in the National Science Foundation: Phase II of a study. Washington, D.C.: National Academy of Sciences. [Ed.]
Cole S.; Cole J. R. & Simon G. A. (1981) Chance and consensus in peer review. Science 214:881–86. [Ed., rDPP]
Cole S.; Rubin L. & Cole J. R. (1977) Peer review and the support of science. Scientific American 237:34–11. [RTL, CM]
Cole S.; Rubin L. & Cole J. R. (1978) Peer review in the National Science Foundation: Phase 1 of a study. Washington, D.C.: National Academy of Sciences. [DEC, CM]
Collins H. M. (1981) Stages in the empirical programme of relativism. Social Studies of Science 11:310. [MDG]
Colman A. M. (1979) Editorial role in author-referee disagreements. Bulletin of the British Psychological Society 32:390–91. [AMC, taDPP]
Colman A. M. (1981) What is psychology? London: Kogan Page. [AMC]
Cone J. D. & Foster S. L. (in press) Direct observation in clinical psychology. In: Handbook of research methods in clinical psychology, ed. P. C. Kendall & J. N. Butcher. New York: John Wiley & Sons. [JDC]
Cook T. D. & Campbell D. T. (1979) Quasi-experimentation: Design and analysis issues for field settings. Chicago: Rand McNally. [rDPP]
Cox W. M. & Catt V. (1977) Productivity ratings of graduate programs in psychology based on publication in the journals of the American Psychological Association. American Psychologist 32:793813. [taDPP]
Crandall R. (1977) How qualified are editors? American Psychologist 32:578–79. [RC]
Crandall R. (1978a) Interrater agreement on manuscripts is not so bad! American Psychologist 33:623–24. [RC, taDPP]
Crandall R. (1978b) The relationship between quantity and quality of publications. Personality and Social Psychology Bulletin 4:379–80. [RC]
Crandall R. & Diener E. (1978) Determining authorships of scientific papers. Drug Intelligence and Clinical Pharmacy 12:375. [RC]
Crane D. (1967) The gatekeepers of science: Some factors affecting the selection of articles for scientific journals. American Sociologist 32:195201. [MDG, MJM, taDPP]
Crane D. (1972) Invisible colleges. Chicago: University of Chicago Press. [DLE]
Davidson J. M. & Davidson R. J., eds. (1980) The psychobiology of consciousness. New York: Plenum. [MJM]
DeBakey L. (1976) Reviewing. In: The scientific journal: Editorial policies and practices, pp. 123. St. Louis: C. V. Mosby Company. [LD]
DeBakey L. (1978) Communication, biomedical: II. Scientific publishing. In: Encyclopedia of bioethics, ed. Reich W. T., vol. 1, pp. 188–94. New York: Free Press. [LD]
DeBakey L. & DeBakey S. (1976) Impartial, signed reviews. New England Journal of Medicine 294:564. [LD]
Diener E. & Crandall R. (1978) Ethics in social and behavioral research. Chicago: University of Chicago Press. [RC]
Eckberg D. & Hill L. (1979) The paradigm concept and sociology. American Sociological Review 44:925–37. [DLE]
Editorial note. (1965) Helgolander Wissenschaftliche Meeresuntersuchen 12 (07): 218. [CM]
Editoral. (1979) Physical Review Letters 43. [RKA]
Endler N. S.; Rushton J. P. & Roediger H. L. (1978) Productivity and scholarly impact (citations) of British, Canadian, and U.S. departments of psychology. American Psychologist 33:1064–82. [taDPP]
Freese L. (1979) On changing some role relationships in the editorial review process. American Sociologist 14:231–38. [DEC, rDPP]
Garfield E. (1979a) Citation indexing: Its theory and application in science, technology, and humanities. New York: John Wiley & Sons. [taDPP, MJW]
Garfield E. (1979b) Journal citation report; A bibliometric analysis of social science journals in the ISI data base. Philadelphia: Institute for Scientific Information. [taDPP, MJW]
Garvey W. D. & Griffith B. C. (1964) Scientific information exchange in psychology. Science 146:1655–59. [BCG]
Garvey W. D. & Griffith B. C. (1971) Scientific communication: Its role in the conduct of research and creation of knowledge. American Psychologist 26:349–62. [DdB, BCG, taDPP]
Garvey W.; Lin N. & Nelson C. (1970) Communication in the physical and social sciences. Science 170:1166–73. [BCG]
Gibbs J. C. (1979) The meaning of ecologically oriented inquiry in contemporary psychology. American Psychologist 34:127–40. [taDPP]
Glenn N. (1976) The journal article review process: Some proposals for change. American Sociologist 11:179–85. [DEC]
Goodstein L. D. & Brazis K. L. (1970) Credibility of psychologists: An empirical study. Psychological Reports 27:835–38. [JSA, taDPP]
Gordon M. D. (1980) The role of referees in scientific communication. In: The psychology of written communication, ed. Hartley J., pp. 263–75. London: Kogan Page. [MDG, JH, RMP, tarDPP, SP]
Gordon R. A. (1980) The advantages of a simple system of optional published refereeing. Speculations in Science and Technology 3:607–9. [RAG]
Gottfredson S. D. (1978) Evaluating psychological research reports: Dimensions, reliability, and correlates of quality judgments. American Psychologist 33:920–34. [RO, taDPP]
Gove W. R. (1979) The review process and its consequences in the major sociology journals. Contemporary Sociology 8:799804. [taDPP]
Greenberg D. S. (1980) Scams and sleaze in science. Clinical Chemistry News 6:5. [DEC]
Griffith B. & Small H. (1976) A Philadelphia study of the structure of science: The structure of the social and behavioral sciences' literature. In: Proceedings, First International Conference on Social Studies of Science. Ithaca, N.Y.: Society for the Social Studies of Science. [BCG]
Hagstrom W. O. (1974) Competition in science. American Sociological Review 39:118. [CM]
Hall J. (1979) Author review of reviewers. American Psychologist 34:798. [taDPP, GJW]
Hargens L. (1975) Patterns of scientific research: A comparative analysis of research in three scientific fields. Washington, D.C.: American Sociological Association. [JMB]
Harnad S. (1979) Creative disagreement. Sciences 19:1820. [DVC, Ed., WMH, taDPP]
Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
Hartmann D. P. & Wood D. D. (1981) Observational methods. In: International handbook of behavior modification and therapy, ed. Kazdin A. E.. New York: Plenum. [JDC]
Hawkins R. G.; Ritter L. S. & Walter I. (1973) What economists think of their journals. Journal of Political Economy 81:1017–32. [JSA]
Hendrick C. (1976) Editorial comment. Personality and Social Psychology Bulletin 2:207–8. [GJW]
Hendrick C. (1977) Editorial comment. Personality and Social Psychology Bulletin 3:12. [taDPP]
Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
Herrnstein R. J. (1977) Doing what comes naturally: A reply to Professor Skinner. American Psychologist 32:1013–16. [taDPP]
Holden C. (1979) Ethics and social science research. Science 206:357–340. [rDPP]
Holden C. (1980) Not what you know, but where you're from. Science 209:1097. [SP]
Honig W. (1980a) Concluding anti-relativity. Speculations in Science and Technology 3:361–63. [WMH]
Honig W. (1980b) Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity. Speculations in Science and Technology 3:233–43. [WMH]
Honig W. (1980c) The review process: Before, after and during. Speculations in Science and Technology 3:513–16. [WMH]
Horn R. E. (1980) Results with structured writing using the information mapping writing service standards. Paper available from the author, Information Resources, Inc., 133 Massachusetts Avenue, Lexington, MA 02173. [JH]
Horrobin D. F. (1974) Referees and research administrators: Barriers to scientific research? British Medical Journal 2:216–18. [CM]
Huxley L. (1900) Life and letters of Thomas Henry Huxley. London: Macmillan and Co. [LD]
Hagstrom W. O. (1974) Competition in science. American Sociological Review 39:118. [CM]
Hall J. (1979) Author review of reviewers. American Psychologist 34:798. [taDPP, GJW]
Hargens L. (1975) Patterns of scientific research: A comparative analysis of research in three scientific fields. Washington, D.C.: American Sociological Association. [JMB]
Harnad S. (1979) Creative disagreement. Sciences 19:1820. [DVC, Ed., WMH, taDPP]
Harnad S. (1982) Rational disagreement in peer review. (Submitted for publication.) [Ed.]
Hartmann D. P. & Wood D. D. (1981) Observational methods. In: International handbook of behavior modification and therapy, ed. Kazdin A. E.. New York: Plenum. [JDC]
Hawkins R. G.; Ritter L. S. & Walter I. (1973) What economists think of their journals. Journal of Political Economy 81:1017–32. [JSA]
Hendrick C. (1976) Editorial comment. Personality and Social Psychology Bulletin 2:207–8. [GJW]
Hendrick C. (1977) Editorial comment. Personality and Social Psychology Bulletin 3:12. [taDPP]
Hensler D. R. (1976) Perceptions of the National Science Foundation peer review process: A report on a survey of NSF reviewers and applicants. NSF publication #77–33. [RTL, CM]
Herrnstein R. J. (1977) Doing what comes naturally: A reply to Professor Skinner. American Psychologist 32:1013–16. [taDPP]
Holden C. (1979) Ethics and social science research. Science 206:357–340. [rDPP]
Holden C. (1980) Not what you know, but where you're from. Science 209:1097. [SP]
Honig W. (1980a) Concluding anti-relativity. Speculations in Science and Technology 3:361–63. [WMH]
Honig W. (1980b) Editorials on our evolving policy: Opening statements; Further statements on speculation; “They laughed at Columbus” and other author syndromes; Our first year, Modesty and age “paradigms”; Einstein centennial issue – Alternates to special relativity; Mathematics in physical science, or why the tail wags the dog; Comment on submissions; Some additional thoughts on speculation; Anti-relativity. Speculations in Science and Technology 3:233–43. [WMH]
Honig W. (1980c) The review process: Before, after and during. Speculations in Science and Technology 3:513–16. [WMH]
Horn R. E. (1980) Results with structured writing using the information mapping writing service standards. Paper available from the author, Information Resources, Inc., 133 Massachusetts Avenue, Lexington, MA 02173. [JH]
Horrobin D. F. (1974) Referees and research administrators: Barriers to scientific research? British Medical Journal 2:216–18. [CM]
Huxley L. (1900) Life and letters of Thomas Henry Huxley. London: Macmillan and Co. [LD]
Ingelfinger F. J. (1974) Peer review in biomedical publication. American Journal of Medicine 56:686–92. [taDPP]
Jones R. (1974) Rights, wrongs and referees. New Scientist 61:758–59. [taDPP]
Kelly W. (1972) Pogo: We have met the enemy and he is us. New York: Simon & Schuster. [DVC]
Kerr S.; Tolliver J. & Petree D. (1977) Manuscript characteristics which influence acceptance for management and social science journals. Academy of Management Journal 20:132–41. [JSA]
Koestler A. (1971) The case of the midwife toad. London: Hutchinson. [CM]
Korten F. & Griffith B. (ca. 1970) Editorial review in psychological journals: A survey of authors, editors, and readers. Washington, D. C.: American Psychological Association. [BCG]
Kosinski J. (1968) Steps. New York: Random House. [CR]
Koulack D. & Keselman H. J. (1975) Ratings of psychology journals by members of the American Psychological Association. American Psychologist 30:1049–53. [taDPP]
Kuhn T. S. (1970) The structure of scientific revolutions. 2d ed., enl. Chicago: University of Chicago Press. [DLE, BM]
Kuhn T. S. (1974) Second thoughts on paradigms. In: The structure of scientific theories, ed. Suppe F., pp. 459–99. Urbana: University of Illinois Press. [DLE]
Kumar K. (1979) Optional published refereeing. Physics Today 32:1314. [RAG]
Lakatos I. (1970) Falsification and the methodology of scientific research programmes. In: Criticism and the growth of knowledge, ed. Lakatos I. & Musgrave A., pp. 91196. Cambridge: Cambridge University Press. [MJM]
Lakatos I. & Musgrave A., eds. (1970) Criticism and the growth of knowledge. Cambridge: Cambridge University Press. [MJM]
Lazarus D. (1980) Changes in “The Physical Review” and “Physical Review Letters.” Physical Review Letters 45:1605–6. [RAG]
Leopold A. C. (1978) The act of creation: Creative processes in science. BioScience 28:436–40. [CM]
Levenson H.; Burford B.; Bonno B. & Davis L. (1975) Are women still prejudiced against women? A replica and extension of Goldberg's study. Journal of Psychology 89:6771. [RO]
Lewis L. S. (1972) Academic freedom cases and their disposition. Change 4:8, 77. [CM]
Lewis L. S. (1975) Scaling the ivory tower: Merit and its limits in academic careers. Baltimore: Johns Hopkins University Press. [CM]
Lindsey D. (1978) The scientific publication system in social science. San Francisco: Jossey-Bass. [DEC, rDPP, RO]
Lovas S. (1980) Higher degree examination procedures in Australian universities. Vestes (Federation of Australian University Staff Associations) 23:1020. [CM]
McCall R. B. (1977) Challenges to a science of developmental psychology. Child Development 48:333–44. [taDPP]
McCartney J. L. (1973) Manuscript reviewing. Sociological Quarterly 14:290, 444–46. [RC, taDPP]
McCutchen C. (1976) An evolved conspiracy. New Scientist 70:225. [taDPP]
McGuigan F. J. (1968) Experimental psychology: A methodological approach. 2d ed.Englewood Cliffs, N.J.: Prentice-Hall. [BM]
McGuire W. J. (1973) The yin and yang of social psychology: Seven koan. Journal of Personality and Social Psychology 26:446–56. [taDPP]
McReynolds P. (1971) Reliability of ratings of research papers. American Psychologist 26:400–101. [DP, taDPP]
Mahoney M. J. (1976) Scientist as subject: The psychological imperative. Cambridge, Mass.: Ballinger. [MJM, CM, RO]
Mahoney M. J. (1977) Publication prejudices: An experimental study of confirmatory bias in the peer review system. Cognitive. Therapy and Research 1:161–75. [JSA, MJM, RO, taDPP]
Mahoney M. J. (1979) Psychology of the scientist: An evaluative review. Social Studies of Science 9:349–75. [JSA, DEC, CM]
Mahoney M. J. (1982) Psychotherapy and human change processes. In: Psychotherapy re- search and behavior change, ed. American Psychological Association. Washington, D.C.: American Psychological Association. [MJM]
Mahoney M. J. (in press) Clinical psychology and scientific inquiry. International Journal of Psychology. [MJM]
Mahoney M. J.; Kazdin A. E. & Kenigsberg M. (1978) Getting published: The effects of self-citation and institutional affiliation. Cognitive Therapy and Research 2:6970. [JSA, MJM, RO, SP]
Mahoney M. J. & Kimper T. P. (1976) From ethics to logic: A survey of scientists. In: Scientist as subject, Mahoney M. J., ed. pp. 187–93. Cambridge, Mass.: Ballinger. [JSA]
Manwell C. (1979) Peer review: A case history from the Australian Research Grants Committee. Search (ANZAAS) 10:8186. [CM]
Manwell C. (1981) An open letter to the president of FAUSA. Australian Higher Education Supplement, ed. J. Bremer, 05 27. [CM]
Manwell C. & Baker C. M. A. (1979) The double helix: Science and myth in the act of creation. BioScience 29:742–46. [CM]
Manwell C. & Baker C. M. A. (1981) Honesty in science: A partial test of a sociobiological model of the social structure of science. Search (ANZAAS) 12:151–60. [CM]
Margulis L. (1977) Peer review attacked (letter). The Sciences 17:5, 31. [CM]
Markle A. & Rjnn R. C. (1977) Author's guide to journals in psychology, psychiatry, if social work. New York: Haworth Press. [taDPP]
Martin B. (1979) The bias of science. Society for Social Responsibility in Science, P. O. Box 48, O'Connor, A. C. T., Australia 2601. [CM]
Martin B. (1981a) The dismissal of Dr. M. E. Spautz from the University of Newcastle (New South Wales). Unpublished manuscript, availabe from Dr. Brian Martin, Dept. of Applied Mathematics, School of General Studies, Australian National University, Canberra, A. C. T. 2600. [CM]
Martin B. (1981b) The scientific straightjacket: The power structure of science and the suppression of environmental scholarship. Ecologist 11:3343. [CM]
Merton R. K. (1968a) The Matthew Effect in science. Science 159:5663. [DdB, CM]
Merton R. K. (1968b) Social theory and social structure. New York: Free Press. [MDG, taDPP]
Merton R. K. (1973) The sociology of science. Chicago: University of Chicago Press. [DLE]
Michie D. (1978) Peer review and the bureaucracy. Times Higher Education Supplement 08 4, p. 11. [CM]
Mitroff I. I. (1974) The subjective side of science. Amsterdam: Elsevier. [1IM]
Mitroff I. I. & Chubin D. E. (1979) Peer review at NSF: A dialectical policy analysis. Social Studies of Science 9:199232. [DEC, CM]
Moore M. (1978) Discrimination or favoritism? Sex bias in book reviews. American Psychologist 33:936–38. [RO]
Moravcsik M. J. (1980) How to grow science. New York: Universe Books. [Ed.]
Moyal A. (1980) The Australian Academy of Science: The anatomy of a scientific elite: parts 1 and 2. Search (ANZAAS) 11:231–39, 281–88. [CM]
Neisser U. (1976) Cognition and reality: Principles and implications of cognitive psychology. San Francisco: Freeman. [taDPP]
NIH Grants Peer Review Study Team (1978) Grants peer review. Opinions on the NIH grants peer review system. Report to the Director, NIH. [RTL]
NIH Study Committee, Dean E. Wooldridge, chairman. (1965) Biomedical science and its administration: A study of the National Institutes of Health. Washington, D.C.: The White House. [RTL]
Nisbett R. E. & Wilson T. D. (1977) The halo effect: Evidence for unconscious alterations of judgments. Journal of Personality and Social Psychology 35:250–56. [RO]
Oromaner M. (1977) Professional age and the reception of sociological publications: A test of the Zuckerman-Merton hypothesis. Social Studies of Science 7:381–88. [taDPP]
Orr R. & Kassab J. (1965) Peer group judgment on scientific merit: Editorial refereeing. Presented to the Congress of the International Federation for Documentation, Washington, D.C. [BCG]
Over R. (1981) Representation of women on the editorial boards of psychology journals. American Psychologist 36:885–91. [RO]
Patterson E. H. (1969) Evaluation of manuscripts submitted for publication. American Psychologist 24:73. [DVC]
Peters D. P., & Ceci S. J. (1980) A manuscript masquerade. How well does the review process work? Sciences 20:1619. [JJB]
Pfeffer J.; Leong A. & Strehl K. (1977) Paradigm development and particularism: Journal publication in three scientific disciplines. Social Forces 55:938–51. [JMB]
Pheterson G. I.; Kiesler S. B. & Goldberg P. A. (1971) Evaluation of the performance of women as a function of their sex, achievement, and personal history. Journal of Personality and Social Psychology 19:114–18. [RO]
Price D. (1970) Citation measures of hard science, soft science, technology and nonscience. In: Communications among scientists and engineers, ed. Nelson C. E. & Pollack D.. Lexington, Mass.: D. C. Heath and Company. [BCG]
Ravetz J. R. (1981) Avoiding fraud (correspondence). Nature 291:7. [CM]
Revusky S. (1977) Interference with progress by the scientific establishment: Examples from flavor aversion learning. In: Food aversion learning, ed. Milgram N. W., Knames L. & Alloway T. M.. London: Plenum. [JH, taDPP, CJT]
Robertson P. (1976) Towards open refereeing. New Scientist 71:410. [RAG]
Roose K. D. & Anderson C. J. (1970) A rating of graduate programs. Washington, D.C.: American Council on Education. [DP, taDPP]
Rosenblatt A. & Kirk S. A. (1980) Recognition of authors in blind review of manuscripts. Journal of Social Service Research 3:383–94. [tarDPP]
Rosenthal R. (1966) Experimenter effects in behavioral research. New York: Appleton-Century-Crofts; rev. ed., New York: Irvington, 1976. [taDPP, RR]
Rosenthal R. & Rubin D. B. (1978) Interpersonal expectancy effects: The first 345 studies. Behavioral and Brain Sciences 3:377–86. [tarDPP]
Ross C. (1979) Rejected. New West 4:3941. [JSA, CR]
Ross C. (1980) Editors choice: None of the above. Las Angeles Times, 02 17, p. 3. [JSA, CR]
Ross P. F. (1981) The sciences' self-management: Manuscript refereeing, peer review, and goals in science. Unpublished manuscript. [taDPP]
Rowney J. A. & Zenisek T. J. (1980) Manuscript characteristics influencing reviewers' decisions. Canadian Psychology 21:1721. [DP]
Roy R. (1981) An alternative funding mechanism. Science 211:1377. [DEC]
Ruderfer M. (1980) The fallacy of peer review – Judgment without science and a case history. Speculations in Science and Technology 3:533–62. [RAG, WMH, taDPP]
Rushton J. P. & Roediger H. L. (1978) An evaluation of 80 psychology journals based on the Science Citation Index. American Psychologist 33:520–23. [taDPP]
Saracevic T. (1975) Relevance: A review and framework for thinking on the notion in information science. Journal of the American Society for Information Science 26:321–13. [BCG]
Sayre A. (1975) Kosnlind Franklin and DNA. New York: Norton. [CM]
Scarr S. & Weber B. L. R. (1978) The reliability of reviews for the American Psychologist. American Psychologist 33:935. [tarDPP, GJW, JCW]
Schaeffer D. L. (1970) Do APA journals play professional favorites? American Psychologist 25:362–65. [JSA]
Scott W. A. (1974) Interreferee agreement on some characteristics of manuscripts submitted to the Journal of Personality and Social Psychology. American Psychologist 29:698702. [DVC, taDPP, WAS, GJW]
Scott W. A. (1977) Methodological consensus among journal referees: A follow-up study of JPSP manuscripts. Unpublished manuscript. [WAS]
Shaw R. & Bransford J., eds. (1977) Perceiving, acting, and knowing: Toward an ecological psychology. Hillsdale, N.J.: Lawrence Erlbaum. [MJM]
Siegfried J. J. (1970) A first lesson in econometrics. Jottrnal of Political Economy 78:1378–79. [JSA]
Slovic P. & Fischhoff B. (1977) On the psychology of experimental surprises. Journal of Experimental Psychology: Human Perception and Performance 3:544–51. [JSA]
Smigel E. D. & Ross H. L. (1970) Factors in the editorial decision. American Sociologist 5:1921. [DVC]
Snizek W. E., Fuhrmam E. R. & Wood M. R. (1981) The effect of theory group association on the evaluative content of book reviews in sociology. American Sociologist 16:185–95. [OLE]
Social Science Citation Index. (1977) Guide and journal lists. Philadelphia: Institute for Scientific Information. [DLE]
Stnrk-Adamec C. & Adamec R. (in press) Breaking into the grant proposal market. International Journal of Women's Studies. [rDPP]
Stinchcombe A. & Ofshe R. (1969) On journal editing as a probabilistic process. American Sociologist 4:116–17. [SP, rDPP]
Stumpf W. E. (1980) “Peer” review. Science 207:822–23. [taDPP]
Swindel R. F. & Perry T. O. (1975) A previously unannounced form of the Gaussian distribution: The golden rule of arts and sciences. Journal of Irreproducible Results 21:89. [DVC]
Symposium. (1979) Reviews of Landsey's The scientific publication system in social science. Contemporary Sociology 8:814–24. [DEC]
Szasz T. (1973) The second sin. London: Routledge and Kegan Paul. [JSA]
Tobach E. (1980) “… that ye be judged.” In An evaluation of the peer review system in psychological research, chairman J. Demarest, open forum presented at the American Psychological Convention, Montreal. [taDPP]
Trafford A. (1981) Behind the scandals in science labs. U.S. News and World Report, 03 2, p. 54. [JSA]
Tuckman H. P. & Leaky J. (1975) What is an article worth? Journal of Political Economy 83:951–67. [RC]
Virgo J. (1974) A statistical procedure for evaluating the importance of scientific papers. Ph.D. thesis, University of Chicago. [BCG]
Walster G. W. & deary T. A. (1970) A proposal for a new editorial policy in the social sciences. American Statistician 24:1619. [taDPP]
Watkins M. W. (1979) Chance and interrater agreement on manuscripts. American Psychologist 34:796–97. [RMP, tarDPP, JCW]
Webb W. B. (1979) Continuing education: Refereeing journal articles. Teaching Psychology 6:5960. [taDPP]
Webster E. C. (1964) Decision making in the employment interview. Montreal: Eagle. [JSA]
Weick K. E. (in press) Systematic observational methods. In: The handbook of social psychology, ed. G. Lindzey & E. Aronson. 3d ed. [JDC]
Weimer W. B. (1977) A conceptual framework for cognitive psychology: Motor theories of the mind. In: Perceiving, acting, and knowing: Toward an cological psychology, eds. Shaw R. & Bransford J., pp. 267311. Hillsdale, N.J.: Lawrence Erlbaum. [MJM]
Weimer W. B. (1979) Notes on the methodology of scientific research. Hillsdale, N.J.: Lawrence Erlbaum. [MJM]
Weimer W. B. & Palermo D. S., eds. (1981) Cognition and the symbolic processes. Vol. 2. Hillsdale, N.J.: Lawrence Erlbaum. [MJM]
Weiss R. J. (1980). The use and abuse of deception. American Journal of Public Health 70:1097–98. [JLF, rDPP]
White M. J. & White K. G. (1977) Citation analysis of psychology journals. American Psychologist 32:301–5. [tarDPP, MJW]
Winer B. J. (1971). Statistical principles in experimental design. 2d ed.New York: McGraw-Hill. [JLF]
Wolff W. M. (1973) Publication problems in psychology and an explicit evaluation schema for manuscripts. American Psychologist 28:257–61. [taDPP]
Wolin L. (1962) Responsibility for raw data. American Psychologist 17:657–58. [JSA]
Wright R. D. (1970) Truth and its keepers. New Scientist 45:402–4. [LD]
Yalow R. S. (1978) Radioimmunoassay: A probe for the fine structure of biology systems. In: Les prix nobel en 1977, pp. 243–64. Nobel Foundation. Stockholm: Almqvist & Wiksell. [RSY]
Yoels W. (1974) The structure of scientific fields and the allocation of editorship on scientific journals: Some observations on the politics of knowledge. Sociological Quarterly 15:264–76. [JMB]
Yotopoulos P. A. (1961) Institutional affiliation of the contributors to three professional journals. American Economic Review 51:665–70. [taDPP]
Ziman J. (1968) Public knowledge: The social dimension of science. Cambridge: Cambridge University Press. [MJM, CM, BM]
Ziman J. (1976) The force of knowledge: The scientific dimension of society. Cambridge: Cambridge University Press. [AMC]
Zinberg D. S. (1976) Education through science: The early stages of career development in chemistry. Social Studies of Science 6:215–46. [CM]
Zuckerman H. (1970) Stratification in American science. Sociological Inquiry 40:235–57. [taDPP]
Zuckerman H. & Merton R. (1973) Patterns of evaluation in science: Institutionalization, structure and functions of the referee system. In: The scoiology of science, ed. Storer N.. Chicago: University of Chicago Press. [BCG, taDPP, SP]
Recommend this journal

Email your librarian or administrator to recommend adding this journal to your organisation's collection.

Behavioral and Brain Sciences
  • ISSN: 0140-525X
  • EISSN: 1469-1825
  • URL: /core/journals/behavioral-and-brain-sciences
Please enter your name
Please enter a valid email address
Who would you like to send this to? *
×

Keywords:

Metrics

Altmetric attention score

Full text views

Total number of HTML views: 0
Total number of PDF views: 260 *
Loading metrics...

Abstract views

Total abstract views: 9800 *
Loading metrics...

* Views captured on Cambridge Core between September 2016 - 19th November 2017. This data will be updated every 24 hours.