You have conducted a study and analyzed the data. Now it is time to write. To publish. To tell the world what you have learned. The purpose of this book is to enhance the chances that some journal editor will let you do so.
If you are new to this enterprise, you might find it helpful to consult two additional sources of information. For detailed information on the proper format of a journal article, see the Publication manual of the American Psychological Association (2010) and recent articles in the particular journal to which you plan to submit your manuscript. The Publication manual also discusses two topics not covered in this chapter: the rules of English grammar and the appropriate terms to use when referring to gender, ethnicity, or sexual orientation. For renewing your acquaintance with the stylistic elements of English prose, you can read chapter 2 of the Publication manual or any one of several style manuals. I recommend Elements of style 2017 by Richard De A'Morelli (Reference De A'Morelli2017). It is an updated version of the classic The elements of style by Strunk and White (5th edn., 2009).
As noted in the Preface, this book focuses on reports of empirical studies, but the general writing suggestions included in this chapter apply as well to the theoretical articles, literature reviews, and methodological contributions that also appear in the professional journals.
Introducing the Problem
Many articles published in psychological journals begin with an open unanswered empirical question (e.g., Do decisions made by a group tend to be more or less risky than decisions the individual group members would have made on their own?) Other articles might present or compare explanatory hypotheses about a common observation or a previously reported empirical finding (e.g., Do groups tend to make riskier decisions than individuals because greater risk-taking is a publicly admired value in our culture or because risk-tolerant individuals dominate the group discussions?)
For Whom Should You Write?
Scientific journals are published primarily for specialized audiences who share a common background of substantive knowledge and methodological expertise. If you wish to write well, you should ignore this fact. Psychology encompasses a broader range of subjects and methodologies than do most other disciplines, and its findings are frequently of interest to a wider public. The social psychologist should be able to read an article on logistic regression in Psychometrika; the personality theorist, an article on hypothalamic function in Science; and the congressional aide with a BA in history, an article on attribution theory in Journal of Personality and Social Psychology.
Accordingly, good writing is good teaching. Direct your writing to the student in Psychology 101, your colleague in the Art History Department, and your grandmother. No matter how technical or abstruse your article is in its particulars, intelligent nonpsychologists with no expertise in statistics or experimental design should be able to comprehend the broad outlines of what you did and why. They should understand in general terms what was learned. And above all, they should appreciate why someone – anyone – should give a damn. The introduction and discussion sections in particular should be accessible to this wider audience.
The actual technical materials – those found primarily in the method and results sections – should be aimed at a reader one level of expertise less specialized than the audience for which the journal is primarily published. Assume that the reader of your article knows something about standard statistical techniques for analyzing data, but probably needs some introduction to Bayesian analyses – which are appearing with increasing frequency in the professional psychological literature. Assume that the reader of Journal of Personality and Social Psychology knows about cognitive biases in human information processing but needs some specific introduction to the phenomenon of confirmation bias.
Many of the writing techniques suggested in this chapter are thus teaching techniques designed to make your article comprehensible to the widest possible audience. They are also designed to remain invisible or transparent to your readers, thereby infusing your prose with a “subliminal pedagogy.” Good writing is good teaching.
Writing It
The primary criteria for good scientific writing are accuracy and clarity. If your article is interesting and written with style, fine. But these are subsidiary virtues. First strive for accuracy and clarity.
The first step toward clarity is to write simply and directly. A journal article tells a straightforward tale of a circumscribed problem in search of an answer. It is not a novel with subplots and flashbacks but a short story with a single, linear narrative line. Let this line stand out in bold relief. Clear any underbrush that entangles your prose by obeying Strunk and White's (2009) famous dictum, “omit needless words,” and by extending the dictum to needless concepts, topics, anecdotes, asides, and footnotes. If a point seems tangential to your basic argument, remove it. If you can't bring yourself to do this, put it in a footnote. Then, when you revise your manuscript, remove the footnote. In short, don't make your voice struggle to be heard above the ambient noise of cluttered writing. Let your 90th percentile verbal aptitude nourish your prose, not glut it. Write simply and directly.
The second step toward clarity is good organization, and the standardized format of a journal article does much of the work for you. It permits readers not only to read the report from beginning to end, as they would any coherent narrative, but also to scan it for a quick overview of the study or to locate specific information easily by turning directly to the relevant section. Within that format, however, it is still helpful to work from an outline of your own. This enables you to examine the logic of the sequence, spot important points that are omitted or misplaced, and decide how best to divide the narrative between the introduction and final discussion.
An article is written in the shape of an hourglass. It begins with broad general statements, progressively narrows down to the specifics of your study, and then broadens out again to more general considerations. Thus:
The introduction begins broadly: “Individuals differ radically from one another in the degree to which they are willing and able to express their emotions.”
It becomes more specific: “Indeed, the popular view is that such emotional expressiveness is a central difference between men and women … But the research evidence is mixed …”
And more so: “There is even some evidence that men may actually …”
Until you are ready to introduce your own study in conceptual terms: “In this study, we recorded the emotional reactions of both men and women to filmed scenes of …”
The method and results sections are the most specific, the “neck” of the hourglass:
[Method] “One hundred male and 100 female undergraduates were shown one of two films …”
[Results] “Table 1 shows that men in the father-watching condition cried significantly more …”
The discussion section begins with the implications of your study: “These results imply that sex differences in emotional expressiveness are moderated by two kinds of variables …”
It becomes broader: “Not since Charles Darwin's first observations has psychology contributed as much new …”
And more so: “If emotions can incarcerate us by hiding our complexity, at least their expression can liberate us by displaying our authenticity.”
This closing statement might be a bit grandiose for some journals – I'm not even sure what it means – but if your study is carefully executed and conservatively interpreted, most editors will permit you to indulge yourself a bit at the two broad ends of the hourglass. Being dull only appears to be a prerequisite for publishing in the professional journals.
Rewriting It
For many writers revising a manuscript is agony. Even proofreading is painful. And so they don't. So relieved to get a draft done, they send it off to a journal, thinking that they can clean up the writing after the article has been accepted. Alas, that day rarely comes. Some may find solace in the belief that the manuscript probably would have been rejected even if it had been extensively revised and polished; after all, most APA journals accept only 15–20 percent of all manuscripts submitted. But from my own experience as an editor of an APA journal, I believe that the difference between the articles accepted and the top 15–20 percent of those rejected is frequently the difference between good and less good writing. Moral: Don't expect journal reviewers to discern your brilliance through the smog of polluted writing. Revise your manuscript. Polish it. Proofread it. Then submit it.
Rewriting is difficult for several reasons. First, it is difficult to edit your own writing. You will not notice ambiguities and explanatory gaps because you know what you meant to say and you understand the omitted steps. One strategy for overcoming this difficulty is to lay your manuscript aside for a while and then return to it later when it has become less familiar. Sometimes it helps to read it aloud. But there is no substitute for practicing the art of taking the role of the nonspecialist reader, for learning to role-play grandma. As you read, ask yourself, “Have I been told yet what this concept means?” “Has the logic of this step been demonstrated?” “Would I know what the independent variable is at this point?” This is precisely the skill of the good lecturer in Psychology 101: the ability to anticipate the audience's level of understanding at each point in the presentation. Good writing is good teaching.
But because this is not easy, you should probably give a copy of a fairly polished manuscript to a friend or colleague for a critical reading. If you get critiques from several colleagues, you will have simulated the journal's review process. The best readers are those who have themselves had articles published in psychological journals but who are unfamiliar with the subject of your manuscript.
If your colleagues find something unclear, do not argue with them. They are correct: By definition, the writing is unclear. Their suggestions for correcting the unclarities may be wrongheaded; but as unclarity detectors, readers are never wrong. Also resist the temptation simply to clarify their confusion verbally. Your colleagues don't want to offend you or appear stupid, so they will simply mumble “oh yes, of course, of course” and apologize for not having read carefully enough. As a consequence, you are pacified, and your next readers, the journal's reviewers, will stumble over the same problem. But they will not apologize; they will reject.
Rewriting is difficult for a second reason: It requires a high degree of compulsiveness and attention to detail. The probability of writing a sentence perfectly the first time is vanishingly small, and good writers rewrite nearly every sentence of a manuscript in the course of polishing successive drafts. But even good writers differ from one another in their approach to the first draft. Some spend a long time carefully choosing each word and reshaping each sentence and paragraph as they go. Others pound out a rough draft quickly and then go back for extensive revision. Although I personally prefer the former method, I think it wastes time. Most writers should probably get the first draft done as quickly as possible without agonizing over stylistic niceties. Once it is done, however, compulsiveness and attention to detail become the required virtues.
Finally, rewriting is difficult because it usually means restructuring. Sometimes it is necessary to discard whole sections of a manuscript, add new ones, and then totally reorganize the manuscript just to iron out a bump in the logic of the argument. Don't get so attached to your first draft that you are unwilling to tear it apart and rebuild it. (This is why the strategy of crafting each sentence of a first draft wastes time. A beautiful turn of phrase that took me 20 minutes to shape gets discarded when I have to restructure the manuscript. Worse, I get so attached to the phrase that I resist restructuring until I can find a new home for it.) A badly constructed building cannot be salvaged by brightening up the wallpaper. A badly constructed manuscript cannot be salvaged by changing words, inverting sentences, and shuffling paragraphs.
Which brings me to the word processor. Its very virtuosity at making these cosmetic changes will tempt you to tinker endlessly, encouraging you in the illusion that you are restructuring right there in front of the monitor. Do not be fooled. You are not. For most writers, a word processor is not an adequate restructuring tool. Moreover, it can produce flawless, physically beautiful drafts of wretched writing, encouraging you in the illusion that they are finished manuscripts ready to be submitted. Do not be fooled. They are not. If you are blessed with an excellent memory (or a very large monitor) and are confident that you can get away with a purely electronic process of restructuring, do it. But don't be ashamed to print out a complete draft of your manuscript; take pencil, scissors, and Scotch tape in hand; and then, all by your low-tech self, have at it.
If after all this, your manuscript still seems interesting and you still believe the results and interpretation of your study, submit it.
Some Matters of Style
Metacomments
It is often helpful to give readers of an article an early overview of its structure and content. But beyond that, you should avoid making “metacomments” about the writing. Expository prose fails its mission if it diverts the reader's attention to itself and away from the topic; the process of writing should be invisible to the reader. In particular, the prose itself should direct the flow of the narrative without requiring you to play tour guide. Don't say, “Now that the three theories of emotion have been discussed, we can turn to the empirical work on each of them. We begin with the psychoanalytic account of affect …” Instead, move directly from your discussion of the theories into the review of the evidence with a simple transition sentence such as, “Each of these three theories has been tested empirically. For example, the psychoanalytic account of affect has received support in studies that …” In the results section, don't say: “Now that we have seen the results for negative affect, we are in a position to examine men's and women's emotional expression in the realm of positive affect. The relevant data are presented in Table 2 …” Instead use a transition sentence that simultaneously summarizes and moves the story along: “Men may thus be more expressive than women in the domain of negative emotion, but are they also more expressive in the domain of positive emotion? Table 2 shows that they are not …” Any other guideposts needed can be supplied by using informative headings and by following the advice on repetition and parallel construction given in the next section.
If you feel the need to make metacomments to keep the reader on the narrative path, then your plot line is probably already too cluttered or pretzel shaped, the writing insufficiently linear. Metacomments only oppress the prose further. Instead, copyedit. Omit needless words, don't add them.
Repetition and Parallel Construction
Inexperienced writers often substitute synonyms for recurring words and vary their sentence structure in the mistaken belief that this is more creative and interesting. Instead of using repetition and parallel construction, as in “women may be more expressive than men in the domain of positive emotion, but they are not more expressive in the domain of negative emotion,” they attempt to be more creative: “Women may be more expressive than men in the domain of positive emotion, but it is not the case that they are more prone than the opposite sex to display the less cheerful affects.”
Such creativity is hardly more interesting, but it is certainly more confusing. In scientific communication, it can be deadly. When an author uses different words to refer to the same concept in a technical article – where accuracy is paramount – readers justifiably wonder if different meanings are implied. The example in the preceding paragraph is not disastrous, and most readers will be unaware that their understanding flickered momentarily when the prose hit a bump. But consider the cognitive burden carried by readers who must hack through this “creative” jungle:
The low-dissonance participants were paid a large sum of money while not being given a free choice of whether or not to participate, whereas the individuals we randomly assigned to the small-incentive treatment (the high-dissonance condition) were offered the opportunity to refuse.
This (fictitious) writer should have written,
Low-dissonance individuals were paid a large sum of money and were required to participate; high-dissonance individuals were paid a small sum of money and were not required to participate.
The wording and grammatical structure of the two clauses are held rigidly parallel, only the variables vary. Repetition and parallel construction are among the most effective servants of clarity. Don't be creative; be clear.
Repetition and parallel construction also serve clarity at a larger level of organization. By providing the reader with distinctive guideposts to the structure of the prose, they can diminish or eliminate the need for metacomments on the writing. Here, for example, are the opening sentences from three of the paragraphs in the previous section on rewriting:
2nd paragraph: “Rewriting is difficult for several reasons. First …”
5th paragraph: “Rewriting is difficult for a second reason:”
6th paragraph: “And finally, rewriting is difficult because it …”
If I had substituted synonyms for the recurring words or varied the grammatical structure of these opening sentences, their guiding function would have been lost, the reader's sense of the section's organization blurred. (I try so hard to be helpful and I bet you didn't even notice. That, of course, is the point: You shouldn't notice!)
And finally, repetition and parallel construction can serve style and creativity as well as clarity. For example, they can establish metaphor: “A badly constructed building cannot be salvaged by brightening up the wallpaper. A badly constructed article cannot be salvaged by changing words, inverting sentences, and shuffling paragraphs.” And, they can add humor: “The word processor encourages you in the illusion that you are restructuring. Do not be fooled. You are not. The word processor encourages you in the illusion that your drafts are finished manuscripts. Do not be fooled. They are not.”
Jargon and Technical Terms
The specialized terminology of a discipline is called jargon, and it serves a number of legitimate functions in scientific communication. A specialized term may be more general, more precise, or freer of surplus meaning than any natural language equivalent (e.g., the term disposition encompasses, and hence is more general than, beliefs, attitudes, moods, and personality attributes; reinforcement is more precise and freer of surplus meaning than reward). Also, the technical vocabulary often makes an important conceptual distinction not apprehended in the layperson's lexicon (e.g., genotype versus phenotype).
But if a jargon term does not satisfy any of these criteria, opt for English. Much of our jargon has become second nature to us and serves only to muddy our prose. (As an editor, I once had to interrogate an author at length to learn that a prison program for “strengthening the executive functions of the ego” taught prisoners how to fill out job applications.) And unless the jargon term is extremely well known (e.g., reinforcement), it should be defined – explicitly, implicitly, or by context and example – the first time it is introduced.
For example, in an article on ESP, a co-author and I decided that we could not proceed beyond the opening paragraph until we had first explicitly defined and clarified the unfamiliar but central theoretical term:
The term psi denotes anomalous processes of information or energy transfer, processes such as telepathy or other forms of extrasensory perception that are currently unexplained in terms of known physical or biological mechanisms. The term is purely descriptive: It neither implies that such anomalous phenomena are paranormal nor connotes anything about their underlying mechanisms.
Here is how one might define a technical term (ego control) and identify its conceptual status (a personality variable) more implicitly:
The need to delay gratification, control impulses, and modulate emotional expression is the earliest and most ubiquitous demand that society places upon the developing child. Because success at so many of life's tasks depends critically on the individual's mastery of such ego control, evidence for life-course continuities in this central personality domain should be readily obtained.
And finally, here is a (made-up) example in which the technical terms are defined only by the context. Note, however, that the technical abbreviation, MAO, is still identified explicitly when it is first introduced.
In the continuing search for the biological correlates of psychiatric disorder, blood platelets are now a prime target of investigation. In particular, reduced monoamine oxidase (MAO) activity in the platelets is sometimes correlated with paranoid symptomatology, auditory hallucinations or delusions in chronic schizophrenia, and a tendency toward psychopathology in normal men. Unfortunately, these observations have not always replicated, casting doubt on the hypothesis that MAO activity is, in fact, a biological marker in psychiatric disorder. Even the general utility of the platelet model as a key to central nervous system abnormalities in schizophrenia remains controversial. The present study attempts to clarify the relation of MAO activity to symptomatology in chronic schizophrenia.
This kind of writing would not appear in your daily newspaper, and yet it is still accessible to a nonspecialist who may never even have heard of blood platelets, MAO activity, or biological markers. The structure of the writing itself adequately defines the relationships among these things and provides enough context to make the basic rationale behind the studies comprehensible. At the same time, this introduction is neither condescending nor boring to the technically sophisticated reader. The pedagogy that makes it accessible to the nonspecialist not only is invisible to the specialist but also enhances the clarity of the article for both readers.
Voice and Self-reference
In the past, scientific writers used the passive voice almost exclusively and referred to themselves in the third person: “This experiment was designed by the writer to test …” This practice produces lifeless prose and is no longer the norm. Use the active voice unless style or content dictates otherwise; and, in general, keep self-reference to a minimum. Remember that you are not the subject of your article. You should not refer to yourself as “the author” or “the investigator.” (You may refer to “the experimenter” in the method section, however, even if that happens to be you; the experimenter is part of the topic under discussion there.) You may use the pronoun we to refer to yourself and your co-authors, but not to yourself as sole author. In that case, you may refer to yourself as “I,” but do so sparingly. It tends to distract the reader from the topic, and it is better to remain in the background. The Publication manual also advises avoiding the use of we in broader ways that leave readers uncertain to whom it refers. For example, the sentence beginning with “We usually classify bird song …” should be changed to “Researchers usually classify bird song …”
Finally, you should leave the reader in the background, too. Don't say, “The reader will find it hard to believe that …” or “You will be surprised to learn …” (This chapter violates the rule because you and your writing are the subjects being discussed.) You may, however, occasionally refer to the reader indirectly in imperative, “you-understood” sentences: “Consider, first, the results for women.”
Beyond Publication
In this book, we have presumed that your pressing pragmatic purpose is to transform your studies into publishable – nay, published – prose. But let your grander goal be to gift that prose with the effortless grace and supple simplicity of a Mozart sonata. This guiding metaphor may not turn all your studies into publications – even Mozart died a pauper – but it will turn even your sow's ears into attractive vinyl purses.
Writing a literature review requires a somewhat different set of skills than writing an empirical research article. Indeed, some people who are very good at writing empirical research reports are not skilled at composing review papers. What are the characteristics of high-quality review papers that are likely to be published and make a difference?
As the past editor of Psychological Bulletin (the literature-review journal of the American Psychological Association) and the founding editor of Child Development Perspectives (the Society for Research in Child Development journal containing short review papers), I have frequently dealt with the issue of evaluating review papers. In the process of editing these journals, I have had to consolidate what were vague, sometimes unverbalized cognitions regarding the properties of an excellent review into criteria for guiding editorial decisions.
Before writing the first version of this chapter, published in 2000, I outlined my recommendations. As a last step before beginning to write, I read a similar paper written by Daryl Bem that was published in Psychological Bulletin in 1995. I was surprised at how similar Bem's and my ideas were; sometimes he even used the same words that I had used when talking about publishing reviews to groups at conferences and to my students. Based on this similarity, one could conclude that there is considerable interrater reliability between people who have been editors about what constitutes a high-quality review paper. In fact, in my experience, even people who occasionally review manuscripts tend to agree on the structural qualities of an acceptable review paper, although they sometimes disagree on the value or veracity of specific content in a given review paper.
In this chapter, I discuss the various types of literature reviews and provide guidelines for writing a high-quality review article. The expectations for, and criteria for acceptance of, review papers in the top journals are, of course, more stringent than those for acceptance in second- or third-tier journals. Nonetheless, many of the same recommendations apply for writing a review for a premier journal and for other journals.
The Purposes of Review Articles
A good starting place for considering the purpose of a review paper is the discussion of review papers in the Publication manual of the American Psychological Association, 6th edition (2010):
Review articles, including meta-analyses, are critical evaluations of material that has already been published. In meta-analyses, authors use quantitative procedures to statistically combine the results of studies. By organizing, integrating, and evaluating previously published material, authors of reviews consider the progress of research toward clarifying a problem. In a sense, literature reviews are tutorials, in that the authors
define and clarify the problem
summarize previous investigations to inform the reader of the state of research
identify relations, contradictions, gaps, and inconsistencies in the literature, and
suggest the next step or steps in solving the problem.
In my editorial in Psychological Bulletin (1997), I outlined three types of reviews that I expected to publish in the Bulletin. I would now add a fourth. The purposes of these types of reviews differ somewhat, although the various types of reviews are not mutually exclusive.
1. In a common type of review, an author uses existing empirical data to answer old or new questions. For example, the author might review studies on age-related changes or sex differences in a given aspect of psychological functioning or behavior and evaluate whether, to what degree, and under what conditions age changes or sex differences seem to occur (e.g., Salk, Hyde, & Abramson, Reference 30Salk, Hyde and Abramson2017). A review of this type may be based on quantitative methods such as meta-analytic procedures that combine empirical findings across studies or can be based on the author's qualitative review of the literature. In this type of review, the major contribution generally is the generation of new knowledge, although theoretical or conceptual predictions usually are examined; for example, the new knowledge is linked to conceptual frameworks already in existence or formulated in the manuscript.
2. Reviews often are a vehicle for analyzing and evaluating the specific predictions of existing theories or conceptual models. Such a paper may include solely a qualitative review of the literature (i.e., a review that does not involve any statistical computations across studies) and/or quantitative methods for combining results across studies. With meta-analytic procedures, an author can use a set of studies to empirically test theoretical claims and potential moderating variables that qualify when and if a predicted effect or relation is obtained. For example, attachment theory predicts that securely attached children should be relatively high in self-regulation; thus, one can examine if, and under what circumstances (e.g., using which measures of attachment or self-regulation), the prediction is supported (e.g., Pallini et al., Reference Pallini, Chirumbolo, Morelli, Baiocco, Laghi and Eisenberg2018). Or one could test the theoretical proposition that there is intergenerational transmission of attachment (Verhage et al., Reference Verhage, Schuengel, Madigan, Fearon, Oosterman, Cassibba and van IJzendoorn2016). The primary goal of this sort of paper is theory testing.
3. Another slightly different type of review is one in which the author integrates and/or compares conceptual frameworks and empirical findings from different subdisciplines or conceptual approaches in psychology or from psychology and other disciplines (e.g., economics, genetics, or anthropological work). For example, an author might evaluate the evidence regarding the intergenerational transmission of self-regulation using a multidisciplinary review (Bridgett, Burt, Edwards, & Deater-Deckard, Reference Bridgett, Burt, Edwards and Deater-Deckard2015). The primary purposes of this type of review are theory integration or theory testing.
4. In some reviews, a new conceptual model is the center point of the paper and an author uses this conceptual model or theoretical prediction to organize a body of literature into meaningful patterns. One example is Bjorklund's (1997) review, in which he proposed a somewhat novel prediction based on evolutionary theory (i.e., that infants’ and young children's immature behaviors and cognitions sometimes are adaptive) and organized his review around examining the merit of this proposal. Others are Dweck's (2017) recent paper proposing a theory that integrates motivation, personality, and development within one framework and Martel's (2013) article on sexual selection theory, arguing that important between-sex differences in markers, mechanisms, etiology, and developmental timing of risk and resilience are relevant to psychopathology. The primary goal of reviews of this type is to develop and evaluate new theory or conceptual arguments.
The four types of reviews listed above do not include the most common type of review published in the psychological literature. A fifth type of literature review involves the organization, presentation, and summarization of pre-existing theory and empirical findings. There is limited, if any, generation of new knowledge or theory. For example, if a field is new, an author might review what is known in an effort to organize and consolidate a sparse or scattered database so that one can more easily draw conclusions and provide direction for future research. The primary purpose of these types of reviews is the integration and presentation of existing knowledge.
There are also many other kinds of reviews, such as those in which authors evaluate the usefulness of a particular measure, trace the history of a given concept or type of research, or provide information on how to use specific clinical, intervention, or laboratory procedures. Reviews such as these generally, but not always, are published in books or journals aimed at a specific, fairly specialized audience.
Selecting a Journal
Whether a literature review is likely to be accepted by a high-quality, peer reviewed journal depends, in part, on the type of review and its appropriateness for a given journal. Literature reviews that organize and summarize an existing body of literature or spell out how to use various measures or procedures certainly have a useful place in psychology. Such reviews educate people without specific expertise in an area, are helpful to readers who want to cite a summary of the current state of knowledge in an area, and provide a jumping off point for researchers who want to move beyond the current state of empirical knowledge. However, reviews that merely summarize current knowledge generally are most appropriate for books or perhaps a special issue of a specialty journal on the specific topic. Although they sometimes may be published in some journals, they currently are not common in the premier psychological journals. Review articles that are most likely to be accepted in peer reviewed review journals are those in which authors generate new knowledge, generate or evaluate theory (broadly defined), and/or integrate ideas and findings from different disciplines in a new way. In brief, top-quality reviews are expected to provide a novel contribution of some sort and to deal with conceptual issues.
In my experience, it is relatively easy to write a review that merely summarizes existing knowledge. In contrast, writing a review that is conceptually based and provides a novel contribution is much more difficult. Doing so usually requires that your thinking about the topic of the paper has “jelled,” which generally occurs only after being immersed in a domain of study for some period of time. A review that provides an increment in knowledge or theory also requires a bit of inspiration. Of course, you can compute a meta-analysis on some issue without broaching new conceptual territory – and such a meta-analysis may generate new knowledge – but the review is likely to be a bit hollow, sterile, and uninspired if you have not reflected extensively on the topic. There is no substitute for a thoughtful and creative analysis of a domain, even if a review paper includes an empirical meta-analysis.
Not all journals that publish reviews require that articles extend the boundaries of our knowledge and thinking. As noted previously, a solid, comprehensive review, especially if carefully conducted, is acceptable in some peer reviewed journals. A good way to evaluate the fit between your review and a specific journal is to examine reviews previously published in that journal to see if their contribution is similar to that in your manuscript.
In an analysis of the degree to which ratings of criteria similar to those just discussed predicted acceptance of manuscripts in Psychological Bulletin in 1993–1996, Sternberg, Hojjat, Brigockas, and Grigorenko (1997) found that all ratings predicted final decisions, that the ratings were highly intercorrelated, and that the two strongest predictors were contribution to the field reviewed and appropriateness of manuscript for the journal. Eisenberg, Thompson, Augir, and Stanley (2002), in a similar analysis for the same journal (covering the next editorial term), obtained similar results. Thus, it is important to consider the degree and type of contribution of a review paper, as well as its appropriateness for a given journal, when evaluating if a manuscript is likely to be published in a peer reviewed journal. Moreover, the breadth of the topic should be taken into account. If your topic is of interest to a narrow audience of psychologists, it probably is more appropriate for a specialty journal in a given discipline (e.g., Personality and Social Psychology Review, Human Development, or Developmental Review) than for a journal targeted at a broad audience (e.g., Psychological Bulletin, Psychological Review). Choosing the appropriate journal for submission can save you from losing many months in the review process, time that is particularly valuable if the content of a paper is likely to become dated.
Common Problems in the Content of Review Papers
Your odds of writing a publishable review are increased if you are aware of what editors and reviewers expect in the content of reviews. One of the best ways to get a feel for this is to read reviews in top-notch journals and in the journal to which you plan to submit your manuscript (if they are not the same). Below I outline some important points to keep in mind when organizing and writing your review article.
Provide a Take-Home Message
As noted by Sternberg (Reference Sternberg1991), a past editor of Psychological Bulletin, a central ingredient of a top-notch review paper (and hopefully an ingredient in any review article) is a “take-home message.”
Literature reviews are often frustrating because they offer neither a point of view nor a take-home message. One is left with a somewhat undigested scattering of facts but little with which to put them together.
Stated somewhat differently,
Authors of literature reviews are at risk for producing mind-numbing lists of citations and findings that resemble a phone book – impressive cast, lots of numbers, but not much plot.
In brief, reviews should have a point that a reader can take away from the paper. Of course, that point needs to be clear and well justified if the paper is to be convincing. You are more likely to present a clear message if the central message is adequately introduced in the introduction and if you provide readers with a conceptual framework early in the paper that can be used to assimilate and integrate the review of empirical evidence that follows the introduction. By providing a conceptual framework and/or guideposts (i.e., advanced organizers) for readers early in the paper, you help the reader to see things from your perspective.
The Review of the Empirical Literature
There are several issues that arise regarding the quality of the section of the paper that includes the actual review of the existing empirical literature.
Breadth of the review. The breadth of desirable coverage of the empirical literature (or theory) varies with the purpose of the literature review. If you are attempting to evaluate the pattern of findings for the body of research on a particular issue, it is important that the review be inclusive. Most or much of the relevant empirical work, especially research of high quality, should be discussed. Bodies of research using diverse methods usually should be included (unless an author is focusing on a particular method), and the range of findings should be reported. Of utmost importance, the review should not be biased in coverage toward research supporting your point of view by excluding empirical work inconsistent with your perspective. A biased review is of little use to the discipline and is poor science.
There are cases in which the review of particular parts of the empirical literature would be expected to be more selective, albeit still representative. For example, if you are presenting a broad model and need to illustrate one of many points related to part of the model, an in-depth review of the literature may be unnecessary and could be distracting. Nonetheless, if you cite a small subset of studies, this subset of research should represent the best work available and you should note discrepancies across studies that are relevant to your argument.
Thus far I have been discussing breadth of coverage in terms of the range and number of studies included in the review. A related issue is the level of detail provided about particular studies or bodies of research. An inclusive literature review does not require that you discuss every study in detail. Often bodies of research can be summarized in a few sentences, especially if the methods used are similar across studies. If presentation of details is deemed important, findings and characteristics of the sample and design can be summarized for specific studies in a table that is available for the reader's perusal. A long discussion of the characteristics and findings of many individual studies is desirable only when it is necessary to make a point, such as why findings differ across studies.
Nonetheless, you do not want your paper to have a “trust me” feel to it. In general, the reader should have some idea of the methods used in the work and the nature of the empirical evidence (e.g., Are the data correlational or from an experimental design? Do the studies involve cross-sectional or longitudinal samples? What types of families tended to be involved in the research?). Often you can provide the essential information very briefly. The degree to which such detail is necessary depends, of course, on the centrality of the study or studies to the point you wish to make.
Relevance. Often authors review theory or research that is not central to their premise or focus. They get off on a mental tangent that serves only to confuse the reader. The content of your review should be on relevant literature and pertinent points; otherwise, the logic of your argument or analysis gets lost. I return to the issue of logic of the argument shortly.
Accuracy. It is a problem if the content in a review of the literature is incorrect or misleading. In addition to a review being biased based on the selection of studies, authors sometimes misrepresent the results of studies or the views of a theorist. Sometimes these misrepresentations are based on an honest difference in individuals’ interpretations or understanding of data or theory. However, often misrepresentations are due to careless reading and reporting of findings. In addition, sometimes an author knowingly slants the findings in a particular direction by omitting relevant information. It is not a problem to differ in your interpretation of findings from other researchers or authors – indeed, having a novel point of view can be the contribution of your paper. But when this is the case, it is wise to lay out carefully your reasons for interpreting the findings as you do. If an author ignores or simply dismisses others’ interpretations of the literature without sufficient cause or mis-states others’ views to make a point, he or she eventually will be seen as an advocate of a particular perspective, not as a scientist.
Another component of accuracy is being sure that the conclusions you present as stemming from your review of the literature actually do follow from the review. Often authors make assertions following their discussion of the literature that are not really substantiated by the material reviewed. Of course, it is reasonable (and even desirable) for you to go beyond the work reviewed and to speculate about relevant issues, mechanisms, processes, and the like. However, it should be clear to the reader what is speculation not grounded in the literature review and what conclusions are firmly based on your review of studies.
Inclusion of critique. A strong review paper is written from a critical point of view. The author does not simply accept what is out there in the existing literature; he or she evaluates the methods of studies, other authors’ conclusions, and other aspects of relevant theory or research that are relevant to evaluating a body of work. A critical evaluation of relevant work includes pointing out strengths as well as weaknesses in existing research and theory. Taking a critical perspective does not mean that you must tear apart every study. Criticism usually is most helpful if you provide an assessment of the strengths and weaknesses of a body of work, including the evidence for specific points or conclusions. For example, you can point out the strengths and weaknesses of various methods (e.g., self-reports, experimental manipulations), as well as note when findings converge across methods (and, thus, are likely to be more reliable).
Being critical does not mean being nasty or insulting. Your critical remarks should be focused on the work itself rather than on the researcher or author of that work. Ad hominem criticism in manuscripts (or from reviewers) usually is not welcomed by editors or readers and is deemed inappropriate and nonconstructive. Moreover, as was pragmatically noted by Bem (Reference Bem1995), the person you attack is likely to be asked to review your manuscript!
The ending. The concluding section of a review can be especially useful to readers if you go beyond merely summarizing the findings from the review. This includes not only noting gaps in the literature, unanswered questions, and future directions for research, but also drawing novel conclusions (if justified or plausible), proposing alternative explanations or processes to explain findings, and discussing the implications of your findings for theory. Authors generally are given more latitude when discussing the findings of a review of the literature than when discussing the results of an empirical paper based on one study or a series of studies. Thus, review papers provide an opportunity for you to be creative and present your own interpretations and integration of the literature, even if your thoughts go somewhat beyond the information directly provided by the review. Creativity is a valuable commodity and can enhance the contribution of your manuscript.
Quality of Writing
A good idea is not enough to get a review accepted – you must present your ideas clearly. Many an article is rejected due to poor writing rather than to lack of a good idea (or good data, for that matter, if you are submitting to an empirical journal). I agree with Bem (Reference Bem1995) that the difference between articles that are accepted and the top 15–20 percent of those rejected is frequently the quality of writing. Of course, clarity of writing will not substitute for a careful review of the literature and creative thinking. However, clarity of writing allows the reader to extract and appreciate the author's creative, analytical, and integrative contributions.
Organization
I often tell my students that, when they write a paper, they need to tell a coherent story (also see Bem, Reference Bem1995). The logic should be linear: Point A → point B → point C. As noted by Bem (Reference Bem1995), “a coherent review emerges only from a coherent conceptual structuring of the topic itself” (p. 173).
As noted by others (Bem, Reference Bem1995; Sternberg, Reference Sternberg1991), no single organization works best for all reviews. One of the best approaches is to organize a review around competing models or around evaluation of a point of view (see Bem, Reference Bem1995, for examples), but this structure may not work for all content areas. Regardless of the content, you should organize the paper around the central point and the relation of theory and research to this central issue.
I sometimes characterize specific manuscripts as “being written like an undergraduate paper.” I am referring to instances in which authors organize their reviews by studies rather than by ideas. One study after another is described, often in too much detail. Studies should not be the unit of analysis; important points to be made should be the building blocks of a review. For example, if an author wants to analyze gender differences in empathy-related responding, the review might be organized according to an analysis of bodies of work pertaining to different conceptualizations of empathy and different operationalizations of empathy-related responding (e.g., self-reports, physiological measures). Then individual studies could be discussed in the context of providing information about particular conceptualizations or operationalizations of the construct.
Take the Perspective of the Reader
Many of the common problems of communication in psychological papers could be corrected if authors simply tried to take the perspective of their readers. Often authors use pronouns (e.g., his or it) when the referent for the pronoun is unclear. Similarly, authors frequently use terms such as “more” or “less” that imply a comparison, but do not indicate what is being compared with what. Consider the sentence, “Aggressive boys were less liked by their peers whereas aggressive girls were more ignored.” In this sentence, it is unclear if aggressive boys were less liked by their peers than were less aggressive boys or aggressive girls, and if aggressive girls were more ignored than aggressive boys or nonaggressive girls. Another instance of lack of perspective taking is when authors jump from one idea to the next without an adequate transition or linkage between the two ideas.
When writing any paper, you should actively analyze how a naive reader will interpret what you are saying and where readers will be confused. Readers do not carry the same fund of knowledge in their heads as you do and are not privy to your train of thought. If reviewers or readers are frustrated by unclear references and leaps in logic, they are unlikely to take the energy to read the paper carefully, will miss important points, can get frustrated and irritated, and may even give up trying to read the paper.
Use Clear Language
There are many ways to improve the clarity of your writing. One is to learn the basics of good writing and grammar that are essential for writing at all levels, professional and otherwise. There are numerous books written on clarity of prose and grammar that are useful guides (e.g., Strunk & White, Reference Strunk and White2009; also see Sternberg & Sternberg, 2016).
One especially common problem in academic writing is that people tend to use complex phrases and word constructions. I always tell students to write in the same way that they would explain an idea or findings to an intelligent layperson who does not know the work. I also suggest that they write in a manner similar to how they talk. Most people would never say the things they write; the word constructions in their writing are too convoluted and drawn out. As advised by Bem (Reference Bem1995), “write simply and directly” (p. 173).
Bem (Reference Bem1995) provided a number of examples of concise versus nonconcise language, repetitive and parallel constructions in sentences, and concise versus nonconcise summarizing of a study (also see Sternberg & Sternberg, 2016). I would like to emphasize one particular problem – the use of jargon. Jargon is specialized terminology used in a discipline. Even in the premier journals in psychology, which are read primarily by other psychologists, the use of unnecessary jargon is confusing and undesirable. The content of a review is expected to be accessible to other people who are not working in precisely the same area as you, and jargon often undermines readers’ understanding of critical concepts.
Unless jargon is extremely well known (e.g., “reinforcement”), is more precise or freer of surplus meaning than other terms, or has a specific meaning that is not available in everyday English, it is best to use plain and simple English. If you must use jargon, you should define the terms the first time they are presented in the text.
Meta-analyses
Many reviewers use meta-analytic techniques to combine the results of multiple studies. This can be a very effective procedure for testing empirical questions when sufficient studies exist on the given topic. Meta-analytic procedures are very useful for testing main effects and moderating (i.e., statistical interaction) effects that are hypothesized in models.
The statistical procedures for computing a meta-analysis have been discussed in detail in numerous books and articles (e.g., Borenstein, Hedges, Higgins, & Rothstein, Reference Borenstein, Hedges, Higgins and Rothstein2009; Card, Reference Card2012; Cooper, Reference Cooper and Cooper2011). Here I merely point out a few of the common errors that I have seen in meta-analytic reviews.
One common error is an author assuming that the meta-analytic section of the paper eliminates the need for a thoughtful introduction section, analytic thinking, and discussion of conceptual issues. The author primarily presents the results of the analysis and assumes that says it all. The inclusion of meta-analyses does not eliminate the need for conceptual integration. Moreover, some relevant data often are not available in a form that can be used in a meta-analysis; research of this type should be reviewed qualitatively if it is pertinent.
Second, a meta-analysis is only as good as the dataset of studies used in it. Thus, if you undertake a meta-analysis, your search procedures for obtaining relevant papers should be inclusive, and usually you should include unpublished papers such as dissertations and conference papers (to avoid the problem of bias toward studies in which the results are significant). Moreover, you should describe your search procedures in detail, including what keywords were used in the computer search and what search programs were used (e.g., MedLine, PsycINFO). Typically a chart is included outlining the numbers of documents obtained at various steps in the acquisition and elimination process.
If you conduct a meta-analysis, there are many decisions you must make regarding aggregation of data from individual studies, the selection of statistics, and so forth. The procedures chosen should be clearly described and, if they are not normative, justified. Moreover, in most cases, it is important that you use the typical range of statistics suggested by experts in meta-analysis (e.g., confidence intervals, estimates of homogeneity of a group of studies, estimates of the degree of bias due to nonsignificant findings not being published [the file drawer problem]).
The important point to keep in mind is that meta-analysis is merely a statistical tool. Thus, the other requirements for a good review generally hold. In addition, the same care that is expected for reporting analyses in empirical articles is applicable to the reporting of meta-analytic procedures and results.
Revising
Once your draft of an article is finished, it is unlikely to be ready to send off for review. I usually revise and write at least four or five versions of an article before sending it off for review, and additional versions when I am revising after feedback from editors and reviewers. Virtually no papers are accepted at the premier journals without revision. However, you substantially decrease your chances of being allowed to revise if the original draft of the manuscript is poorly written and if there are many grammatical and typographical errors. The editor and reviewers have to speculate whether the authors are able to make sufficient revisions for acceptance of the paper, and poor writing undermines readers’ confidence in authors’ abilities.
In rewriting a paper, most of us need to spend considerable time and energy working on issues such as clarity of wording, clarity of presentation of ideas, and organization of ideas. Often what is needed is some distance from the paper. After you have read and reread the same paper many times, you know the paper so well that it is difficult to note missing words or links between ideas. Putting the paper aside for a few days or weeks and then coming back to it can provide you with a whole new perspective. It also is very useful to ask co-workers to read the manuscript and provide feedback prior to submitting the paper. Ideally, one reader would be familiar with your content area and could comment on content-related issues whereas another reader would be someone without expertise in your particular content area who could provide feedback on clarity of communication in the paper. I am always amazed how multiple readers each note some problems with a document that are missed by all other readers. After more than 40 years of publishing, I virtually never submit an empirical paper or a major review paper without having at least one other person read it (and usually more than one).
Extensive rewriting, if combined with high quality in terms of content, may earn you a reject-revise review. If the editor leaves the door open for a revision, this generally is cause for guarded celebration; the odds of the paper being accepted have gone up substantially. This is especially true if you think that you can address most of the important issues in the reviews and can provide good reasons for not complying with the suggestions that you do not want to incorporate into the revision. As we found in an analysis of Psychological Bulletin papers that obtained a reject-revise decision and were resubmitted, approximately 73 percent were eventually accepted (Eisenberg et al., 2002). Of course, some people did not revise and resubmit, resulting in about 55 percent of papers with a reject-revise decision being accepted eventually.
As suggested by Bem (Reference Bem1995) when discussing the revision process, “turn to the task of revising your manuscript with a dispassionate, problem-solving approach” (pp. 176–177). Do whatever seems reasonable (and perhaps slightly unreasonable) to respond to the reviewers’ and editors’ concerns. Keep in mind that reviewers generally are experts in an area and that many of their comments likely have merit.
You should, of course, feel free to argue, in a constructive way, about any suggestion by an editor or reviewer that you feel is wrong or unwarranted. However, it is wise not to simply ignore suggestions rather than discuss why the suggestions are problematic. It is important to include a letter with your resubmitted manuscript in which you outline in some detail how you handled the reviewers’ suggestions and your reasons for not making some suggested modifications.
Indeed, many editors request a letter of this sort and reviewers and editors use these letters to determine how you dealt with the various suggestions and criticisms. Thus, when writing this letter to the editor, keep in mind that reviewers as well as the editor are likely to read it. Original reviewers usually are asked to re-review revised manuscripts, especially if they had criticisms of the manuscript. Thus, it is in your own best interest to bite your tongue when writing this letter and not detail why a reviewer's comments are stupid, just plain wrong, or biased. Reviewers are people, and they often get defensive or downright mad if they are embarrassed or insulted by an author's letter. Your goal should be to have your manuscript re-reviewed by a dispassionate, rational reviewer rather than by a defensive or angered reviewer.
So, for example, when you get the silly suggestion that you reanalyze your correlational data controlling for age when age was not related to any variables in your correlations, vent with your friends or colleagues and then do the correlations. Do not say: “I did not reanalyze controlling for age because the reviewer obviously did not read my paper well enough to note that age was unrelated to any of the variables in the correlations.” Rather, you might say. “As suggested by a reviewer, I reanalyzed controlling for age and it had no effect on the results.” And perhaps, if you feel other readers might wonder about the same issue, you might add, “and I now mention this fact briefly in a footnote (or in text).” You can also mention that age was not expected to affect the pattern of findings because it was unrelated to any of the other major variables.
When a reviewer suggests that you discuss an issue that is tangential to your point, you shouldn't say, “This reviewer obviously doesn't know the area because he/she suggested I discuss work that is not relevant.” Rather, you might say something like the following: “Although the work on topic X is interesting, it is not directly relevant to the point I hope to make. Thus, I simply have briefly cited that body of work so interested readers can access that work. If this is not satisfactory, please let me know.” (See Bem, Reference Bem1995, for other examples of right and wrong responses.)
Persistence, hard work, self-regulation, and a strong ego and thick skin are important in getting you through the review process. Don't give up on revising your review simply because it is a frustrating, difficult, and time-consuming process or because you are angered by the reviews. Keep in mind the slogan of the Arizona lottery, “You can't win if you don't play.”
Summary
As I noted in my editorial outlining the types of papers I hoped to publish in Psychological Bulletin (Eisenberg, Reference Eisenberg1997):
The best reviews are those that provide readers with a well-articulated message and a framework for the integration of information. In addition, reviews are expected to be well organized, comprehensive, critical, and balanced and to communicate clearly to a broad audience. Moreover, a high-quality review identifies relations, gaps, and inconsistencies in the literature and includes ideas regarding the next steps for a solution to the problem.
This was not mere verbiage; I tried to carry through on these ideas in the review process. To assess the aforementioned qualities (and others) in papers submitted to Psychological Bulletin, I asked reviewers of manuscripts to rate on a five-point scale (from low to high) all manuscripts on the following criteria: (1) significance of paper topic, (2) interest to a broad audience of psychologists, (3) appropriateness for the Bulletin, (4) quality of analyses (e.g., meta-analysis; respond only if relevant); (5) balance and fairness in coverage of alternative views, (6) accuracy of information, (7) adequacy of literature review (was the literature adequately sampled?), (8) theoretical contribution of manuscript, (9) existence of and persuasiveness in arguing for a well-articulated point of view, (10) importance of conclusions, (11) quality of writing (clarity, coherence, and organization of the prose), (12) appropriateness of the paper's length (is it concise?), and (13) overall evaluation. Some of these criteria may not apply to some other journals or may not be explicitly assessed by other editors (for example, a review published in a specialty journal need not interest a broad audience of psychologists). Nonetheless, many of these criteria doubtlessly are used to evaluate reviews in most peer reviewed journals. Indeed, all correlations of these ratings with the reviewers’ recommendations and the final editorial action were statistically significant at the p < .001 level. Reviewers’ ratings on the persuasiveness and importance of conclusions, as well as appropriateness for the journal, had the highest correlations with the reviewers’ recommendations and the acting editor's final action. Contribution to the field is the bottom line, but that contribution is likely to be obscured if your writing is not well organized and clear.
