Titles

The title should inform the reader simply and concisely what the paper is about (Publication manual of the American Psychological Association, 6th edn., 2010; Sternberg & Sternberg, 2016). It is important that the title be self-explanatory. Readers will come across the title in other articles that refer to your own article and in PsychINFO, and they may have to decide whether to read your article solely on the basis of the title. The title should include keywords, for example the theoretical issue to which the paper is addressed, and possibly the dependent variable(s), and the independent variable(s). Keywords are important because the title will be stored in information-retrieval networks that rely on such words to determine the relevance of your study to someone else's research interests. For the same reason, it is important to avoid irrelevant and misleading words, because such words may spuriously lead an investigator uninterested in your topic to your article. The title typically should not exceed 12–15 words in length.

Everyone has his or her own style in titles, but certain titles take a form that I personally find trite. An example of such a style is “The effect of — upon —.” That may be in fact what you are studying, but the title is boring and the work sounds empirical without any driving theory. Other hackneyed forms for empirical articles are “A study of —,” “An investigation of —,” and “An experiment on —”. Such titles are also redundant, because what else is an empirical article if it is not a study, an investigation, or an experiment? If you are using a theory to drive empirical work, it helps to let this fact be known through your title.

There sometimes is a trade-off between having a catchy title and an informative one. Always err on the side of the informative title. But one possibility is to have a two-part title. Then you put the catchy part first and the informative part after a colon or a question mark. In that way, you get the best of both worlds. Just make sure the first part of the title really conveys some kind of message you really want to convey.

I recently entitled a journal symposium, “Am I famous yet? Judging scholarly merit in psychological science” (Sternberg, 2016). I thought the first part of the title was catchy, while the second part would convey the focus of the symposium. But readers got caught up in the first part of the title, and whether fame is good or bad. I learned, even as a senior investigator, that sometimes, in the desire to use a catchy title, one inadvertently can put readers off-track. A more successful use of a dual title was “What should intelligence tests test? Implications of a triarchic theory of intelligence for intelligence testing” (Sternberg, 1984). The first part of the title caught readers’ interest. The latter part said in more detail what the article was about.

The Abstract

The abstract summarizes your article (Sternberg & Sternberg, 2016). Its length typically should be 100–150 words for a report of an empirical study, and 75–100 words for a theoretical article or literature review. The abstract, like the title, should be self-explanatory and self-contained because it may be used by information-retrieval networks for indexing.

For empirical articles, the abstract should include (a) the problem being investigated, (b) the major hypotheses, (c) a summary of the method, including a description of the materials, apparatus, participants (including number, sex, and age of participants), design, and procedure, (d) a synopsis of the main results, including significance levels, and (e) the conclusions drawn from the results, as well as any implications of these conclusions. For theoretical and review articles, the abstract should include (a) the problem being studied, (b) the purpose, thesis, or organizing construct, (c) the scope of the analysis, (d) the types of sources used, and (e) the conclusions. Do not include in the abstract any information that is not included in the body of the article. Because you will not know until you are done with the outline what information you will include, you are well advised to defer writing the abstract until after you have otherwise completed the outline or even the whole article. Some journals also want a list of 3–5 keywords following the abstract, or on the title page.

The APA Publication manual lists several features of a good abstract. These features are that the abstract be (a) accurate, (b) self-contained, (c) concise and specific, (d) nonevaluative, and (e) coherent and readable. Remember that most people will read your abstract only if your title interests them, and will read your article only if your abstract interests them. It is therefore essential that the abstract be interesting. You can interest the reader by showing that the problem is an important one, that your hypotheses about the problem are insightful ones, and that you will test these hypotheses in a convincing way.

The greatest problem with abstracts, in my experience, is that they are either underspecified or overspecified. In the former case, there just is not enough detail for the reader to figure out what was done, or how it was done, or what was found. In the latter case, the abstract tries to say too much: It tries to tell the whole story of the article. Sometimes, the cost of saying too much in the abstract is that readers may believe they do not need to read the full article. So write an informative abstract, but not one that says so much that readers will not have any incentive to read the article it accompanies. Good luck!

Section I: The Opening

The opening paragraph serves two main purposes. The first is to draw the attention of your reader and garner interest in your study. The second is to set the stage for a more focused literature review by orienting the reader within a particular research framework. The initial statements are typically broad in scope, and may be directed toward an audience unfamiliar with the domain of research under study.

Several tactics are commonly employed to both engage the reader's interest and provide a general framework in the first few sentences of the opening segment (see the box, “Strategies for the opening paragraph”). Techniques often found in journal articles include rhetorical questions, analogies, striking statistics or facts, brief historical summaries of the topic, definitions, or everyday examples of a phenomenon. Consider the examples of opening statements drawn from manuscripts appearing in peer reviewed publications shown in the box.

The opening paragraph below is from a study published in the Journal of Research on Adolescence that examined adolescent decision-making (Albert & Steinberg, Reference Albert and Steinberg2011):

This study used an everyday experience to demonstrate to the reader the relevance of the research question. By opening with a scenario that most readers can relate to or imagine, the introduction attracts the attention of a wide audience. This relatable “leading scenario” also helps the authors prime the reader to consider the same question that they eventually propose – that is, what are the factors that influence adolescent decision-making?

Authors may lead the reader to ask the study's fundamental questions more directly through use of a rhetorical question. This tactic automatically forces the reader to become an active participant because he or she must consider an “answer” to the question before reading further. Consequently, the reader is compelled to form an initial opinion that may be in line with or in contradiction to previous research or the author's opinion. The use of a rhetorical question is illustrated in the introduction to an article in the Journal of Experimental Social Psychology about the effect of ovulation on female attentional and memory processes (Anderson et al., Reference Anderson, Perea, Becker, Ackerman, Shapiro, Neuberg and Kenrick2010):

This introduction draws the attention of any reader – even one who has no background in “simple social cognitive processes” – and leaves the reader curious to find out more about the study. The use of a relatable rhetorical question also helps to place the research topic in a more general framework that is readily understandable and relevant to everyday life. Notice that the question is targeted at specific constructs (attention and memory). The authors do not ask the more general question, “Why do we pay attention to certain things?” or “Why do we remember some things but not others?” Instead, the authors use the rhetorical question to focus the reader's attention on particular cognitive processes in a specific social context so that the reader will know the study's precise arena.

Another common strategy used to engage and orient the reader is to establish the need for research on a particular topic. This approach is often accomplished by opening with a striking fact or statistic that will impress upon the reader the prevalence or magnitude of a particular problem or issue. Consider, for example, the sobering effect of this introductory paragraph from an article published in Journal of Consulting and Clinical Psychology about the prediction of suicide ideation and attempts among adolescents (Nock & Banaji, Reference Nock and Banaji2007):

Although striking statistics are often very effective in catching a reader's eye, the writer should be careful not to overload the introductory segment with numbers. A laundry list of statistics may dilute the message you are trying to convey.

Another way of establishing the need for a study and garnering reader interest is by pointing to a lack of empirical research on a topic that the researcher considers to be important. When using this tactic, one argues that, although little research has been conducted on a particular area, a greater understanding of this topic or area is essential to progress in the field. For example, in an article on youth adjustment after the Boston Marathon bombing published in Pediatrics, Comer et al. (Reference Comer, Dantowitz, Chou, Edson, Elkins, Kerns and Green2014) begin by pointing to a lack of research on this important topic:

This strategy can be especially effective when combined with the previous tactic of presenting striking facts or statistics. A combination of the two approaches helps emphasize both the magnitude of and lack of attention to the problem under study, which builds a strong case for why your current study is a critical addition to the literature.

Some researchers choose to begin their introduction by briefly tracing the history of the research on a particular area. This approach is useful when the study presents a new twist or development on a topic that has had a long history of research. Field (Reference Field2006), for example, took this approach in an article published in Clinical Psychology Review revisiting a common framework for understanding the development and treatment of phobias. He began his article: “Conditioning as an explanation of phobic responding arose from Watson and Rayner's (1920) famous demonstration that aversive and avoidant responses towards a previously neutral stimulus could be learned” (p. 857). This opening sentence begins to set the historical context for his study – a research history that is detailed throughout the remainder of the introduction.

The techniques described above can attract the reader's attention and place the study in a broader context; however, they are by no means exhaustive. Although creative writing is not always appropriate in subsequent sections of a research article, some modest creativity in the introductory segment can give an article a unique appeal. Keep in mind that the more concisely one can frame the general domain of the study, the more likely the subsequent literature review will be focused and concise.

Section II: The Literature Review

The main feature of the second segment of the introduction is an illustrative, although not exhaustive, review of the literature surrounding the topic of study. The author can assume that the reader is informed about the general principles of the domain and should keep the review succinct. It is customary that the important and relevant works in the area of research be cited. Later in the paper, such as in the discussion section, the findings of the present research report will be integrated with the relevant studies reviewed in the introduction. The author should include in the initial literature review all of the studies that will be discussed later, because it is not recommended that new articles be introduced in other sections.

The literature review should be presented in a coherent, integrated fashion that links findings together around some central argument. Previous research reports can help to clarify terms, demonstrate progress in the area, or point out limitations of past research. For a longer literature review, one might consider using headings to delineate sections in which different theories, methodologies, or research traditions will be discussed. Preferences as to whether headings should be used within the introduction section vary by journal editors, so you may wish to examine several recent papers published in the journal to which you will be submitting your article before making such a decision.

When describing the relevant studies, be sure to address the features of these studies that pertain specifically to your manuscript. Recognize the aspects of your study that make it an improvement or advancement over past research, and review the literature with these features in mind. For example, if gender had been ignored in previous research and your study examined the role of gender, then the literature that is reviewed should state when male and when female participants were studied. Recognize the limits of the published literature and use the introduction to identify these weaknesses – after all, you will be addressing and seeking to remedy them in your research report. Consider the theoretical ramifications of the studies and be sure to introduce the theory(ies) that will be used when making sense of your findings in the discussion section.

Which studies and how much detail should be included in the literature review? It is essential to recognize the priority of the work of others and to credit relevant earlier works, but there is no need to cite tangential or overly general sources. Readers can be referred to other works as needed, but the text of your introduction should stay on a straight path set for introducing your research. It may be tempting to demonstrate just how much you know about your topic by squeezing in as many citations as you can – after all, you have probably read a great deal to become an expert on the topic of your study. However, a concise and relevant literature review will be more impressive and will convey a higher level of expertise than a lengthy and indiscriminate summary of studies. It is also important that the author be familiar with all of the works cited in the introduction. One surefire cue of the naiveté of an author (to a reviewer) is a mis-statement or inaccurate citation of a classic study in the field. Cite the accepted research and the established authorities, but do so accurately and only when appropriate to the research.

Typically, a researcher will cite many references in the literature review, although there is no magic number for deciding how many references to include. The number of citations found in literature reviews varies widely based on the scope of the topic and the quantity and quality of previous research. Regardless of the number of articles cited, neither journal editors nor readers will want to read the exhaustive details about each study mentioned. Instead, the writer should discuss only the most relevant details of the studies. The amount of detail in which to describe a previous study is determined by the purpose and nature of the present study. Whereas many reviews will discuss only main findings of studies, it may be relevant and important in setting the stage for your study to discuss other aspects of previous research, such as methodologies, participant characteristics, or the reliability and validity of measures. Use your judgment to present enough information to facilitate a sufficient understanding of your topic for an intelligent reader (although not necessarily a reader in your specific field) without overloading or boring the reader with extraneous information. This balance may be difficult to achieve, but it is essential in maintaining the audience's interest and understanding.

The manner in which studies are to be cited is another question often posed by novice writers. The Publication manual of the American Psychological Association (2010) describes two ways of citing research: (a) Cite the authors within the sentence itself, for example, “Smith and Smith (2015) found …” or (b) cite the authors in parentheses at the end of a sentence, for example, “It was found that … (Smith & Smith, 2015)” (consult the APA Publication manual for further details on citation formats). The first method – within-text citations – should primarily be reserved for studies that one plans to describe in specific detail or studies that are especially pivotal to one's argument. When one simply presents a brief summary of findings, it is preferable to avoid “name clutter” in the text by citing authors in parentheses. This method of citation is less distracting to readers, helps to avoid “laundry lists” of names, and allows the writer to integrate and condense the literature review by citing several related findings in the same sentence (see box, “Citing studies”).

Consider the following examples from Gar and Hudson's (2008) introduction to the role of parenting in the development and maintenance of anxiety in youth. Gar and Hudson (Reference Gar and Hudson2008) begin by citing many diverse studies in a single sentence: “Aetiological models of anxiety emphasize the importance of childrearing factors in the development and maintenance of anxiety disorders in children (Chorpita & Barlow, Reference Chorpita and Barlow1998; Ginsburg & Schlossberg, Reference Ginsburg and Schlossberg2002; Hudson & Rapee, Reference Hudson, Rapee, Heimberg, Turk and Mennin2004; Krohne & Gutenberg, Reference Krohne, Gutenberg, Hurrelmann and Loesel1990; Manassis & Bradley, Reference Manassis and Bradley1994; Rapee, Reference Rapee, Vasey and Daads2001; Rubin & Mills, Reference Rubin and Mills1991)”(p. 1266). In the following quotation, the authors demonstrate how multiple parentheses can be used within a sentence if one wishes to cite studies with different findings in a single sentence:

When one considers how much more writing would have been required to convey the same findings by individually citing each study within the text, the utility of the second citation method is evident.

One last point about the literature review: Avoid using direct quotes from articles reviewed unless referencing the author's exact words makes a meaningful contribution to the clarity or quality of your paper. Your task in writing the literature review is to read critically, synthesize, and integrate a large body of literature, and then to simplify and describe this literature for your audience in your own words.

Section III: Transition to Your Study

The third and most specific segment of an introduction provides a transition from previous research to the present study and explicitly states the purpose and rationale for the present study. The transition section is typically reserved for the closing paragraph, although it can sometimes be more than a single paragraph long. The earlier paragraphs/pages of the introduction have set the broader stage for the paper so that the reader will not be at all surprised to learn, in this brief segment, the exact purpose(s) of the present study. Just as the opening segment and the literature review lead naturally into a description of the present study, this description will naturally lead into the next section of the paper – the methods section. Although brief, this segment of the paper is an important transition – it is the link between what has been done in the past and what you will be describing in the rest of your paper.

There are three major aspects of a study that should be briefly covered in the transition segment of an introduction: (a) the potential implications of the study; (b) the general nature of the study; and (c) the researcher's hypotheses. This final segment of the introduction often begins by stating what the study will contribute to the existing research. This statement is invaluable – it is essentially your answer to the question, “Why have I spent my time conducting this study, and why should you spend your time reading it?” (a question implicit in the minds of reviewers, editors, and readers alike). You have just finished reviewing previous research; now state how the present study will add to, clarify, disprove, or otherwise advance what previous studies have reported. Demonstrate to your readers that your experiment is the next logical step in the research on your topic. Your literature review will have pointed out the flaws, issues, or theories that you are proposing to address or remedy, so this statement should flow naturally from a thoughtfully written review.

Once you have stated the question or problem your study will address, briefly lay out your plan for answering this question. This statement takes the form of a very concise summary of the nature of your experiment – more detailed information about the methods will be found in later sections. Among the information provided should be a short description of the independent and dependent variables and how they were operationalized in your study.

After laying out the research question and describing how you attempted to answer it, state your hypothesis for the outcome of the study. Hypotheses are always phrased in the past tense (e.g., “It was hypothesized,” “We expected,” or “The authors predicted”) because the study will have already been performed. A study may have only one or several hypotheses, depending on its scope. When there are multiple hypotheses, many authors choose to demarcate them with letters or numbers at the end of this final section of the introduction.

A final segment that covers all three points will provide a smooth transition to the rest of the paper and will make the rationale and hypotheses for the study clear to the reader. The following passage from the JAMA Pediatrics (Elgar et al., Reference Elgar, Napoletano, Saul, Dirks, Craig, Poteat and Koenig2014) demonstrates how each element of the transition segment – summary statement of the study, potential value to the literature, and hypotheses – can be met within one paragraph.

Beyond Structure and Organization

The three segments – opening, literature review, transition – can be used to organize an introduction and present the study to the reader with increasing focus: from a broad umbrella to specific statements of rationale. But this structure is not all there is to a quality introduction. Other matters, such as page length, writing quality, and tone also merit consideration as one moves from a penultimate draft to a final manuscript. Writers should consult the most recent edition of the APA Publication manual (currently the 6th edition) for more specific details of manuscript preparation, as journal editors will expect that all submitted manuscripts fully conform to the standards presented in this manual (e.g., page formatting, citations, headings).

Conclusion

Writing an introduction requires flexibility within a framework. Certain components are typically present (the opening paragraph, the literature review, the transition), but the actual composition of each section varies from article to article. To use an analogy germane to current psychotherapy research, the use of manuals in psychological treatment has been criticized for being rigid and stifling creativity. However, for manuals to be effective they are applied flexibly. The components of any given manual are standardized (e.g., building a relationship, identification of self talk, homework), but the specific content (e.g., test anxiety, loss of parent, interpersonal dejection) and the application of the exact techniques (e.g., playing a game, using a workbook) may vary to meet the individualized needs of the client. The same “flexibility within fidelity” (Kendall & Beidas, Reference Kendall and Beidas2007) applies when writing an introduction. The sections of an introduction can take on many different “looks” depending on the purpose of the study. The opening segment, for instance, may make use of a historical fact or rhetorical question to draw the reader's interest. The literature review may take pages and may explain a broad domain of psychology or focus on a detailed subset of research. The hypotheses are usually concise, but may be exploratory. The tone of the writing may be supportive or critical of past research. The key is communicating ideas with clarity and accuracy within a general introductory framework. How this is accomplished remains the prerogative of the author.

Why Methods Matter in a Research Report

The methods section of a research article sets forth a blueprint of operations by which one or more theory-based propositions were transformed into an empirical investigation. Methods sections usually encompass four components of a study: (1) What was the conceptual layout of constructs into conditions and variables? (2) Who were the participants and how many of them were there? (3) What procedural details were followed in the course of conducting the research? (4) What measures were collected?

Before describing these four components in some detail, let's consider a pair of more basic questions: Why do methods sections matter and what makes one effective? Neophytes to the field sometimes suppose that theory is what matters. But consider Tony Greenwald's (2012) trenchant observation that many more Nobel Prizes in the sciences were awarded for methodological than for theoretical advances. This is because of the synergy between methods and theory: Theories suggest questions that require new methods, and new methods suggest questions that could not have been answered, much less envisioned, with older methods. It is no exaggeration to say that methodological advances free the scientist's imagination.

A report of design and methods provides more than an outline of how a study was conducted; it sets boundaries for the kind of conclusions that can be drawn from the research. There are several reasons why design circumscribes one's conclusions, the most important of which is that a given design enhances the plausibility of certain interpretations while diminishing others. Because the goal in writing any research report is not just to communicate one's findings, but also to argue for the validity of particular explanations, it is imperative to be clear about how a study's design is appropriate for supporting the authors’ theories and hypotheses.

Sternberg and Gordeeva (Reference Sternberg and Gordeeva1996) asked a sample of experienced researchers to rate the importance of 45 attributes in determining the impact of an article on the field. The highest-ranked item was “Makes an obvious contribution to psychological knowledge, adding something new and substantial.” Although many factors contribute to such impact, the right methods are indispensable. Without them, alternative explanations and operational ambiguities are likely to pose significant challenges to preferred theoretical accounts. With them, no other interpretation of the data will be as plausible as the ideas that suggested your research in the first place. An effective methods section will be explicit and persuasive not only about what the researchers did, but also about why they made the choices that they made.

How to Write about Research Methods

There is, of course, a world of difference between using good research methods and writing effectively about them. Influential articles require, first, a well-crafted design that is appropriate to the topic under investigation; second, artifact-free methods that are capable of testing the researchers' hypotheses; and third, written documentation to support the first and second points. Like a good map, a write-up of methods should illuminate the many connections between one's point of departure (theory) and destination (findings), so that the reader's path is free of dead ends, detours, and misunderstandings. To do this, most researchers incorporate a brief and clear-cut overview of their design and procedure at the end of the introduction section. This description should not come as a surprise, however; the rationale for a study, as presented in the introduction, should move smoothly and informatively from concepts to operations, so that this statement is more or less self-evident by the time it appears.

In general terms, a well-written discussion of methods should follow the three cs: be clear, comprehensive, and compelling. Clarity is important because an unnecessarily complex or oblique discussion may confuse or disorient readers just when you most want them following your logic: how your conceptualization of a problem led to the specific study that you conducted. Consider the following example:

Contrast the above with:

The second statement is clearer and more concise. It makes obvious that the design was a 2 × 2 between-groups factorial, that the independent variables were anticipated outcome and ego-involvement, and that random assignment was used. Readers of the first paragraph may eventually reach the same conclusion, or they may not.

Design statements also lose clarity when they are overly eloquent or verbose rather than succinct. Compare:

with:

The best design descriptions are short and to the point. Beware of overkill. The theoretical rationale for your methods should have been made clear earlier, and there is no need to provide a primer on the general benefits of factorial designs or random assignment (unless your study presents unusual issues). If your design is very complex – but only then – an illustrative figure may be helpful. Regarding language, there may be a time and a place for metaphor and evocative adverbs within a research report, but they are distracting and annoying when presenting methods. Display your creative writing talents elsewhere. Elegance here is a function of brevity, clarity, and thoroughness. Readers may not notice the beauty of your words in a well-written methods section, but they are likely to be impressed by the lucidity of your research.

The second quality of an effective description of methods, comprehensiveness, refers to the inclusion of all relevant material that readers expect to find. Like any good organizational scheme, American Psychological Association (APA) style promotes efficiency, and there is certain information that experienced readers look for in a design statement. Although every study requires at least somewhat unique information, most articles include several common elements, so that you can get a good idea about what to include by looking at earlier publications that used similar methods. With multistudy articles having become the norm in many areas of psychological science, relying on similar methods for more than one study, it is usually expeditious to describe the first study as a template for subsequent studies, detailing only their new features.

The narrative of a methods section should be self-contained and complete, in the sense that a typical reader – that is, a student or colleague who is generally knowledgeable about psychological research but who may not be familiar with the particular area of your study – will be able to understand what you did.Footnote ¹ This requirement has gained importance as the field has come to emphasize replications and reproducibility (see Chapter 19). Nevertheless, it is almost always impossible to include all details necessary to accurately replicate a study without succumbing to manuscript-bloat (and thereby obscuring the key findings of your work). For this reason, many journals now link published articles to Supplemental Online Materials (SOM), where full and fine-grained methodological details can be presented. Some researchers even include verbatim copies of their protocols and measures in the SOM. Authors, particular young authors, sometimes have difficulty deciding what to include in the main body of their articles and what should go into SOM. The publication guidelines for Psychological Science are helpful in this regard. They suggest that “methodological minutiae … the sorts of information that only ‘insiders’ would relish and require for the purposes of replication – should be placed in SOM … not in the main text.”

If nothing else, making these methodological details crystal clear ensures that readers will not misperceive your work. As essential as it is for readers to understand what you did, however, attending to the third criterion – making your article compelling – ensures that readers appreciate why you did your research in that manner. Because there is invariably more than one way to investigate most questions in psychology, it is important to make plain how the particular design and methods that you used embody the questions that your research concerns. Researchers often choose one design or method over others for reasons that are not self-evident, in which case readers (who cannot be faulted for knowing less about the problem than you do, or for approaching it from a different vantage point) may assume that other designs or methods might have been preferable. Even when a study is not hypothesis-testing in nature – that is, when it is exploratory or hypothesis-generating – there should be a clearly articulated rationale for its method.

Try to take the perspective of an outsider. Is it clear why your design is appropriate for examining your hypotheses? Can plausible alternative explanations be discounted with these conditions, control groups, and measures? What are the specific advantages of this design or procedure over other reasonable possibilities, especially those that have been used in prior research? (But remember that modesty is called for. Science often progresses in slow steps, and it is likely that your paper will be read, if not reviewed, by the authors of those prior papers.) If there are multiple control groups, what is the function of each one? Do any conditions, groups, or assessments seem superfluous? What are the limitations, if any, of your sample and procedures? (Noting limitations, usually done at the end of the discussion section, is a useful way to avoid drawing stronger conclusions than the results can bear.) Answers to these questions should be evident even to a casual reader.

The rationale for a design is not typically reported in the design statement of a methods section, which generally does no more than provide an explicit summary. Rather, it should be developed throughout the introduction, as the text progresses from general discussion of theoretical principles to specific hypotheses. A good way to do this is to be clear about design considerations as you discuss prior research, especially your own earlier studies that used similar designs. For example, one might note that a prior study contrasted condition A with condition B in order to rule out a particular alternative explanation, but that now a different comparison is called for. The more compelling the case that a design can unequivocally support a given theoretical position while refuting plausible alternatives, the more likely it is that an article will be reviewed favorably, and the greater its likely impact.

In sum, there is nothing like an unambiguous, clearly specified methods section to set the stage for presenting your results. An appropriate, concise method lets readers anticipate the data analyses that ensue, and gives them a useful framework for following the detailed presentation of results into general conclusions. To be sure, methods do not dictate statistics – research questions do – but a clear depiction of one's design makes the results more accessible to readers.Footnote ² The more complex a study, the more necessary this account is. That is why scientific articles almost without exception describe their methods before their results.

Basic Features of a Methods Section and What to Say about Them

Psychological research is a lot more complex than it once was. Whereas in the not too distant past a few basic designs and a handful of standard methodological details sufficed for most research, the more fine-grained nature of contemporary questions and the field's growing technical complexity dictates that modern scholars become familiar with diverse designs and procedures. Thus, whereas it once might have been possible to fully describe and explain a standard research protocol in a page or two, considerably more attention is now needed. Most basic research methods textbooks explain the advantages and disadvantages of most of these designs and methods. What is not as readily apparent is how to convey their essential features clearly and effectively. Next I offer guidelines in this regard for the four main elements of a methods section.

Conclusion: Writing about Methods for Impact

From a literary standpoint, methods sections are among the easier parts of a journal article to write. Aside from making your statements clear, comprehensive, and compelling, there's little more that needs to be said. Literary simplicity should not be confused with import, however; the methods section is critical, leaving little room for error or omission. The value of elegant, creative, and timely theorizing notwithstanding, behavioral science at its core is all about the evidence, and how well it supports a given set of principles. Such support is a direct consequence of rigorous methods. Good designs and exacting procedures provide a strong foundation for the validity of conclusions by fostering particular explanations and ruling out others. Poor methods are either inappropriate to the conclusions or invite conceptual ambiguity. In short, the extent to which a study adds to knowledge depends as much on methods as on anything else.

Strategically, methods are vital for publishing one's research in top journals. My experience as an editor and reviewer suggests that the first question reviewers (and other readers) ask is whether the obtained results of a study validly and unambiguously support the conceptual conclusions that an author advocates. If the answer is no, or even maybe not, reviewers are likely to raise substantial questions about a paper's contribution to the literature, irrespective of its theoretical polish and many highly significant results. Methods are a big part of that judgment – if anything, as the field's methodological diversity grows, a persuasive, well-crafted description of methods is increasingly important – and it is therefore advantageous to prepare this material with skeptical reviewers in mind.

Using good research methods but writing poorly about them is like painting a magnificent work of art and then exhibiting it behind an opaque screen. The painting may be inherently excellent and important, but no one is likely to know about it. And unlike works of art, which are sometimes created for the artist's private enjoyment, research needs to be publicized if it is to influence future research and applications by others. Thus, although methods sections may not inspire bedtime enthusiasm, their impact is unmistakable.

Specific Strategies

The section above outlined the general mood of today's field of psychology. A data analyst today should anticipate the expectation that the data presented in the article are readily available (provided that the requirements of the funding agency or the entity, e.g., the university sponsoring the research, are met), that the raw data from which the analyzed dataset was derived are stored and recoverable, and that the analytical sequence (data preprocessing, analyses, and visualization) that led to the results section is well documented and thus reproducible by an independent researcher. In this section, some specific recommendations are presented that reflect the standards of current data analytic practices in the field of psychology.

The Data Analyses Section

The data analyses section is where the author needs to communicate how the analytic procedures were selected and implemented (i.e., the actual methods used to perform the analyses). This description should be marked by the same care and rigor as the description of the other methods (e.g., how participants were selected, what experimental paradigms were used, or what assessments were administered) used in the work. APA style does not call for a separate data analyses section; there are examples of researchers including specifics of their analytic procedures in the methods section (Hein et al., Reference Hein, Tan, Rakhlin, Doyle, Hart, Macomber and Grigorenko2017) or in the results section (Haeffel et al., Reference Haeffel, Hein, Square, Macomber, Lee, Chapman and Grigorenko2017), or placing them in their own section (Mourgues, Tan, Hein, Elliott, & Grigorenko, Reference 98Mourgues, Tan, Hein, Elliott and Grigorenko2016). As all three approaches are practiced in APA journals, all three are acceptable. The point here is that, wherever this information is placed, the degree of detail in this section should be such that a peer scientist should be able to replicate the work and arrive at the same results if given access to the data used in the published article. A paragraph in which one sentence lists a handful of analytical approaches and the other mentions a type (or types) of software utilized in the analyses is as helpful as a list of actions at the end of a recipe to “get, slice, mix, sauté, stuff, and broil.” Although the standards of different professional journals differ, a good data analyses section typically includes the following five subsections: (1) data-analytic philosophy (i.e., a subsection on the general framework in which the data analyses were carried out); (2) statistical power (i.e., a subsection including a discussion of the sample size and the likelihood that a study will detect an effect of particular size(s) when there is an effect to be detected); (3) data preprocessing (i.e., a subsection on how the raw data were handled to prepare them for the analyses); (4) analytic techniques (i.e., a subsection on what software, routines and/or macros were used); and (5) access to the data and analytic pipeline or procedures (this subsection should be included when the authors are willing to share their data and code, which is now required by some journals and granting agencies and seems to be becoming the standard in the field). All of this material is provided in the context of the ultimate task (and quality control) of the data analyses section, which is to link the analyses, all of (1)–(5) above, to the specifics of the study design.

Moreover, the data analyses section, wherever it is placed, should be guided by the following principles:

Simplicity. Given two (or more) possible expressions of the same idea, the simpler is usually preferable. The reason is that a simple idea lends itself more effectively to analysis. Data-analytic paradigms should ascend from simple to complex, not the other way around. It is recommended to start with simple techniques, such as descriptive analyses, and gradually progress to more complex models. When testing a set of models, start with a simple model and increase its complexity gradually, in a controlled way, by adding variables and effects a step at a time.

Clarity. In order to abstract the messages from data and make use of them, reports should be clearly and simply written. It is important not to be wordy, but it is also important not to be vague. Many studied phenomena are represented more effectively in words than in numbers. Numbers, however, are essential for presenting results of statistical analyses. In designing data analyses, try to balance words, figures, and tables, as different analytic techniques are differentially suited to descriptive, figural, and tabular representations.

Generality. Generality, with regard to the data analyses section, means that, hopefully, the analytic paradigm followed in your article is general enough to be used not only on the original datasets, but on other datasets that are similar in form, but might be different in content.

Applicability. Applicability assumes the presence of links between research and reality. The use of data and models has become extremely pervasive in scientific culture; mathematical results have a high credibility, but the main validity criterion is in the question of where and how the results of the study can be applied in the larger world. Therefore, when thinking about data analyses, it is important to run a reality check – do your content and analytical paradigms have relevance to real-life problems?

Analytic Philosophy

The overwhelming majority of the literature in the field utilizes classical inferential statistics, also referred to as orthodox or frequentist statistics,Footnote ² based on significance testing. Recently, however, there are calls to abandon significance testing and switch to Bayesian statistics.Footnote ³ The essence of the significance-testing approach is to yield the probability of the data given a certain hypothesis (J. Cohen, Reference Cohen1990). In this approach, there is an asymmetric relationship between the null hypothesis (H₀) and the alternative hypothesis (H_A): H₀ can be either accepted or rejected in favor of H_A, whereas H_A cannot be either accepted or rejected directly; it can only be considered when H₀ is rejected as the only alternative. In other words, every H_A has a corresponding H₀; H_A takes over (i.e., wins) only when H₀ is rejected and, therefore, no other hypothesis can be tested against this particular H₀. When multiple hypotheses (H_A(s)) are juxtaposed against the same H₀, the problem of multiple comparisons arises that requires special handling (see below).

In contrast, the essence of the Bayesian approach is to estimate the probability of a hypothesis given the data (Wagenmakers, Reference 99Wagenmakers2007). The Bayesian approach allows for more than two hypotheses, and these hypotheses are not asymmetrically related to each other. Thus, unlike in frequentist statistics, where hypothesis testing is binary, that is, accept or reject, hypothesis-testing in the Bayesian approach is comparative, where the likelihoods of two (or more) hypotheses are compared to favor one or to rank them, when appropriate. Although the Bayesian approach has many virtues (Wagenmakers, Wetzels, Borsboom, & van der Maas, Reference Wagenmakers, Wetzels, Borsboom and van der Maas2011), it is far from being widely used in the field, and both reviewers and readers are not very familiar with it.

In a nutshell, the major difference between the two approaches is how they model uncertainty. The frequentist school relies on a conditional distribution of data given specific hypotheses (H₀ and H_A). The presumption is that H₀ – the parameter specifying the conditional distribution of the data – is true; the observed data are then sampled from that distribution. The Bayesian school models uncertainty using a probability distribution over two or more hypotheses; that is, it uses probabilities for both the hypotheses and the data.

There are multiple sources, both brief and extensive, that compare and contrast these two approaches to conducting statistical analyses. Although frequentist measures (e.g., p-values and confidence intervals) continue to dominate psychological research, Bayesian measures (e.g., Bayes factor) have entered the field and their popularity is growing. Importantly, there is a growing consensus that the most effective approaches to complex problems often draw on the best insights from both approaches working in concert; there are examples when the same datasets are interrogated with multiple analytic approaches. Regardless of what is used, it is important to be very clear about one's analytic philosophy. Whatever approach is utilized, the data analyses section and any supplementary materials should be sufficiently detailed so that both reviewers and readers may understand all of the essential steps of the data analyses.

A Note on Power

Research conducted with samples with low statistical power increases the likelihood that a statistically significant finding is, in fact, a false-positive finding. Although it is well known that psychological studies, on average, tend to be underpowered, a recent study has shown just how severely underpowered they are. In this study, 3,801 recent publications in cognitive neuroscience and psychology were analyzed (Szucs & Ioannidis, Reference Szucs and Ioannidis2017), their reported effect sizes were quantified, and the corresponding power was estimated. The study demonstrated that the majority of these studies were underpowered: The median power to detect small, medium, and large effects was 12 percent, 44 percent, and 73 percent, for each of the effect sizes, respectively. It also showed that studies in cognitive neuroscience had, on average, even lower statistical power than those in psychology. Indeed, neuroimaging studies generally have low statistical power (estimated at 8 percent) due to the high cost of data collection, which results in an inflation of the number of positive results that are false (Button et al., Reference Button, Ioannidis, Mokrysz, Nosek, Flint, Robinson and Munafò2013). Yet another recent meta-analysis, conducted in the broader field of biomedical sciences, to which psychology contributes, established that approximately 50 percent of published studies have statistical power in the 0–10 percent or 11–20 percent range (Dumas-Mallet, Button, Boraud, Gonon, & Munafò, Reference Dumas-Mallet, Button, Boraud, Gonon and Munafò2017). Obviously, these estimates are well below the minimum of 80 percent that is typically considered sufficient. It has also been shown that slightly more than 50 percent of the findings reported to be significant in the analyzed studies (Szucs & Ioannidis, Reference Szucs and Ioannidis2017) were likely to be false positives, replicating similar previous findings from metascientific studies in biomedical research (Begley & Ioannidis, Reference Begley and Ioannidis2015). Appraising the statistical power of a study is very important not only for understanding the results of the data analyses, but also for projecting the study's reproducibility. The field needs to re-emphasize “the importance of well-designed studies that are run with sufficient sample sizes for drawing informative conclusions” (Patil, Peng, & Leek, Reference Patil, Peng and Leek2016, p. 540). There is a growing realization that there is a need to conduct larger studies, entailing collaboration across multiple individuals and multiple research groups (Button, Lawrence, Chambers, & Munafò, Reference Button, Lawrence, Chambers and Munafò2016). This realization might impact graduate training in psychology, where dissertations, to be adequately powered, need to be carried out within larger projects, or collaboratively, including a number of graduate students working together.

Data Preprocessing

The term “data preprocessing” is typically used to refer to the steps of data manipulation that occur after the raw data are collected and before content-driven, hypothesis-driven or exploratory analyses are performed. Note that not all types of data require data preprocessing. Typically, a dataset that has already been used in other analyses and publications does not require data preprocessing. Yet, most, if not all, “freshly” collected datasets do call for this step. Four recommendations are pertinent here.

First, there always should exist a repository where data are stored in their original form, as collected, prior to any manipulations.

Second, all steps of data preprocessing (i.e., the transformation of data from their original form into what will be analyzed) should be carefully documented and saved in a form that would allow an independent user to replicate all of the steps if the preprocessing of the raw data should need to be carried out again.

Third, some types of data (e.g., survey, questionnaire, inventory, behavioral experimental data) need only limited preprocessing (e.g., examination of distributions, selection and performance of transformations, handling outliers), whereas other types (e.g., psychophysiology, neuroimaging, genetic data) include complex steps involving decision-making. Whereas the preprocessing for survey and other such data typically takes the form of a clear description of the steps performed to prepare the data for analyses, the preprocessing for neuroimaging and other such data often results in the utilization of sophisticated data pipelines. Yet, even the preprocessing of behavioral experimental data may be only seemingly easy. For example, an examination of a small set of 30 articles published in Psychological Science featuring reaction time has exposed a range of approaches used in preprocessing these data. “Most (but not all) researchers excluded some responses for being too fast, but what constituted ‘too fast’ varied enormously: the fastest 2.5%, or faster than 2 standard deviations from the mean, or faster than 100 or 150 or 200 or 300 ms. Similarly, what constituted ‘too slow’ varied enormously: the slowest 2.5% or 10%, or 2 or 2.5 or 3 standard deviations slower than the mean, or 1.5 standard deviations slower from that condition's mean, or slower than 1,000 or 1,200 or 1,500 or 2,000 or 3,000 or 5,000 ms.” (Simmons et al., Reference Simmons, Nelson and Simonsohn2011, p. 1360). The point here is that none of these decisions is either “correct” or “incorrect,” but the lack of clear standards and specificity in the criteria for variable transformation allows room for, perhaps, self-serving justification and makes it impossible to reproduce the results of the analyses.

Fourth, an important and complex issue to consider is the issue of missing data. There is a range of approaches to handling missing data (Pampaka, Hutcheson, & Williams, Reference Pampaka, Hutcheson and Williams2016), and some of these approaches are built-in (e.g., using Full-Information Maximum Likelihood parameter estimation in MPlus), whereas others require a substantial amount of decision-making, such as the selection of the way in which missing data are imputed by the data analyst (e.g., in SPSS). Again, whatever decisions are made, they have to be justified and well documented. For example, the choice between expectation-maximization (EM), multiple imputation (MI), and direct maximum likelihood (DML) missing data imputation algorithms is not trivial (Allison, Reference Allison2001); this choice, as well as the choice of software to be selected to carry out the missing data imputation, should be done carefully, well argued, and meticulously documented. Most preprocessing steps are fairly standard and arouse little concern about biases or procedures that could affect replication. Yet, there are some techniques that might be less known; those will require more extensive referencing or even a copy of the code through a Web reference in the article or an inclusion in “Supplementary materials.”

Analytic Techniques

It is important to provide an account of what analytic techniques were used in the data analyses. If the techniques are well known (e.g., ANOVA, ANCOVA, linear regression), then just a quick mention of these techniques is sufficient. If the techniques are not well known (e.g., multiway contingency table analyses or profile analyses), they should be explained in sufficient detail. It is important to indicate whether the software used for the data analyses used standard routines from standard software (e.g., SAS or SPSS), specific toolboxes (e.g., R libraries), or custom-written code (e.g., original software written within a particular environment, such as R or SAS; it is highly recommended that custom-written code be made available to other investigators). There are advantages to standard software and toolboxes: They do not require expertise in programming, but mere conceptual understanding of the implemented technique; they are typically user friendly; they are standardized and easily documentable; they are easy to use in replications; there are examples of their usage and the interpretation of the results they generate in the literature. Yet, they are often not flexible enough to accommodate specific features of the data or particular needs in the data manipulation or analyses (e.g., SPSS does not have an option to test for interaction effects when linear regression is set up through menu-driven windows; additional steps are required to specify the interaction terms). In such cases, code is written in a hybrid mode (i.e., capitalizing on already available toolboxes, but adding code to meet the specific needs of a researcher), or is completely customized. Whether hybrid or customized, whether “beautified” (i.e., perfected to some standard accepted in the field) or simply functional (i.e., working, although not as perfect as the field standard), there is no reason not to share code (Barnes, Reference Barnes2010). There are numerous ways to share analytic code with others – through direct collaborations, via journal repositories, through websites that can be used to share code (e.g., GitHub, Google Code, figshare), and via personal or lab websites. “The code we write to analyze data is a vital part of the scientific process and, similar to data, is not only necessary to interpret and validate results but can be also used to address new research questions. Therefore, the sharing of code is as important as the sharing of data for scientific transparency and reproducibility” (Gorgolewski & Poldrack, Reference Gorgolewski and Poldrack2016, p. 5). “An article … in a scientific publication is not the scholarship itself, it is merely advertising of the scholarship. The actual scholarship is the complete software development environment and the complete set of instructions which generated the figures” (Buckheit & Donoho, Reference Buckheit, Donoho, Antoniadis and Oppenheim1995, p. 59). Moreover, articles containing usable code are known to be cited more often than their counterparts without the code (Vandewalle, Reference Vandewalle2012).

Like any good habit, quality analytic techniques require ongoing practice and commitment to exercise. Below are some recommendations that are routinely found to be helpful in producing a publishable, replicable, and citable data analyses section.

Practice what you preach. One of the fundamental principles of self-discipline is to not do what you know is not good for you. So, the first rule for doing data analyses is to promise yourself to follow best practices in the field and catch and correct yourself every time a deviation from best practices occurs.

Exercise on a regular basis. As with any skill, the skill of data analyses needs to be practiced. Most data-analytic tricks are learned and internalized not through reading about them, but through doing them. There is nothing sacred or mysterious about data analyses. Nobody possessed this aptitude at birth; it has to be learned and it is acquired through repeated practice.

Stay current. The data-analytic arm of psychology, often referred to as quantitative psychology, is a dynamic, rapidly developing field. Yet, psychologists may consult numerous data-analytic methodologists and use techniques developed in other fields and in the emerging field of data sciences. Most graduate courses in psychology, however, are focused on “traditional” techniques, mostly linear modeling, framed by frequentist statistics. So, to stay current, one needs to self-educate, primarily by reading journals in and outside psychology, and by attending different specialized courses and workshops. The quest does not end with getting through one's graduate school stats requirements; it only begins there!

Find your crowd. This is an amazing time of shared resources and common audiences. Find your expert group on the Net, connect with them, ask questions, offer your own experiences, listen, and learn. There are no stupid questions, especially about data analyses. And they are always better asked before one's article is published, not when someone cannot replicate its analyses.

Data Concealment (or Not)

There are also calls for authors to submit their manuscript together with their data (Mummendey, Reference Mummendey2012). In fact, data sharing has been mandated by some grant funding agencies, as well as journals. It is requested that data are submitted to a repository (field, agency, journal, laboratory) before the relevant paper is submitted. Placement of data in a repository allows the author to point the reviewers and readers to the location of the data in the manuscript. If done in this way, it is thought that the manuscript may benefit from increased transparency due to openly shared data, and the data itself can become a resource enabling additional future research (Gorgolewski & Poldrack, Reference Gorgolewski and Poldrack2016). There is even a new movement for so-called “data papers,” which are articles dedicated to descriptions of complex datasets (Gorgolewski, Margulies, & Milham, Reference Gorgolewski, Margulies and Milham2013). There are journals (e.g., Scientific Data, Gigascience, Data in Brief, F1000Research, Neuroinformatics, and Frontiers in Neuroscience) known to accept such data papers (Gorgolewski & Poldrack, Reference Gorgolewski and Poldrack2016). It has been observed that readiness to openly share data is associated with fewer statistical errors in analysis (Wicherts, Bakker, & Molenaar, Reference Wicherts, Bakker and Molenaar2011) and increased citations (Piwowar, Day, & Fridsma, Reference Piwowar, Day and Fridsma2007).

Overtly unethical research practices such as “p-hacking”Footnote ⁴ (Forstmeier, Wagenmakers, & Parker, Reference Forstmeier, Wagenmakers and Parker2017) or “HARKing,” hypothesizing after the results are knownFootnote ⁵ (Kerr, Reference Kerr1998), should be explicitly avoided. If one uses conventional frequentist statistics, it is important to find a way to present the p-values associated with the performed analyses to the fullest degree possible. Of course only those p-values that are meaningful and relevant to the highlights of the findings should make it into the text. Yet, exact (and all) p-values might be important for subsequent meta-analyses, if anybody ever wants to perform them; therefore, it is important, depending on the number of p-values generated for a particular analysis, to include them in a table in a publication, in a supplementary table, or via a link to a supplementary file, which most journals now gladly store and make available in their electronic resources via links in the publication. It is important to acknowledge the number of tests conducted, to demonstrate the author's awareness of the issues related to multiple testing, and to justify the threshold value chosen to indicate statistical significance. Statistical significance is directly related to Type I error, occurring when a researcher falsely concludes that an observed difference is “real,” when it is in fact not real (e.g., H_A is favored over H₀ unjustifiably). Typically, the α level or Type I error rate is set at .05 (i.e., p < .05); this means that the researcher is willing to commit a Type I error 5 percent of the time. This simple testing framework, however, is no longer sufficient when the research deals with multiple (rather than single) comparisons. There are multiple situations that may generate a multiple-comparison problem (Coppock, Reference 97Coppock2015). There are: (1) experiments that include multiple groups, in which there are a(a –1)/2 pairwise comparisons, where a is the number of groups in the study, and an arbitrary number of possible complex comparisons involving more than 2 groups; (2) treatment studies that differentiate treatment effects for multiple groups, for example men and women; (3) correlational and causational studies with multiple distinct outcomes or multiple operationalizations of a single outcome variable; and (4) studies in which multiple estimators (e.g., difference-in-means and covariate adjustments) are applied to the same dataset. In general, a single global statistical test is preferred to repeated pairwise testing. If a global test is significant, various univariate tests are in order. When a posteriori multiple tests are performed, correctional analytical procedures for multiple comparisons are recommended (McDonald, Reference McDonald2014), but the selection of the appropriate procedures should be thoughtful. The two most popular types of adjustments are the Family-Wise Error Rate (FWER) and the False Discovery Rate (FDR). The FWER is the probability of incorrectly rejecting even one (of the n tested) null hypothesis. If α is set at .05, the chance of rejecting at least one of the n tested null hypotheses is 1 – (1 – .05)ⁿ (for n independent tests); that is, given 2 independent hypotheses, the chance of rejecting at least one of them is 9.75 percent, for 3, it is 14.25 percent, for 4, 18.55 percent, for 5, 22.62 percent, and for 10, 40.12 percent, meaning that, without p-value adjustments, a Type I error will be committed about 40 percent of the time. There are multiple ways to control the FWER correction, both mathematically (e.g., the Bonferroni, Holm, Tukey, and Scheffé corrections) and by means of simulations. The FDR is the expected proportion of false discoveries among all discoveries. In general, FDR is considered to be less conservative in that it affords better power for detecting differences that do exist while still controlling the number of falsely rejected true null hypotheses; there are also multiple ways to control for FDR (e.g., the Benjamini and Hochberg, Simes, and Hochberg's step-up procedures). Whatever is done, it is important to reveal it to the reader, so the reader can appraise the likelihood of replicating the published result and be conscientious about designing the details of the replication study, adequately powering it, and knowing all the caveats there are to know while analyzing the data.

Finally, to aid an appraisal of the relevance and magnitude of results deemed statistically significant, the inclusion of effect sizes (i.e., quantitative indicators of the strength of an association) can be expressed in several ways (Nakagawa & Cuthill, Reference Nakagawa and Cuthill2007),Footnote ⁶ depending on the statistical tests used, with confidence intervals on the effect size estimates being critically important. Confidence intervals are indicators of how much trust one can put in the results. They are reflective of the interplay of two factors: the precision of the measurement and the variability of the population being studied. The inclusion of effect sizes and confidence intervals is now expected by a number of journals, which takes the field away from the binary notion of statistical significance (yes or no) and toward the discussion of the strength of effects.

Conclusion

The emergence of metascience has generated explicit expectations of how research should be conducted and reported on today; these expectations apply to the conduct and reporting of data analyses as well. The hope is that if the recommendations of metascience are followed, the degree of reproducibility of original findings in psychology will increase (Schooler, Reference Schooler2014). Data analyses include a number of steps between the collecting of data and the generating of input for the results section. Metascience advances the expectation that all of these steps (as well as the entire research process) should be open to scrutiny. The current discussion is not about whether research data and data-related code should be shared, but rather how to make such sharing a common feature of research culture (Anonymous, 2014). This means documenting every step and sharing, if not required by a funding agency or a journal, upon first demand from the field the relevant data and code. This is the new standard of the field, although still far from common. A recent analysis of 441 articles published between 2000 and 2014 identified only one that was fully reproducible (Iqbal, Wallach, Khoury, Schully, & Ioannidis, Reference Iqbal, Wallach, Khoury, Schully and Ioannidis2016). If reproducibility is the new standard, then the field should be expected to make data and the code that generated the analyses fully available in an organized and protected (e.g., by licenses) way (Gorgolewski & Poldrack, Reference Gorgolewski and Poldrack2016). To ensure reproducibility, a number of journals, for example Nature Neuroscience (Anonymous, Reference Anonymous2013), now require authors to use checklists prior to manuscript submission, and there is increasing pressure from the community to make such practices common (Maynard & Munafò, Reference Maynard and Munafò2013). The modern way of practicing science is to open every step of its practice to the community. In doing so, the field will not only reshape itself by introducing safeguards against the penetration of poorly conducted science into the public domain through publication; it will also promote good decision-making and wide consultation within the scientific community at large through its publicly available resources. This is a new challenge for both established and junior scientists, but it is well supported not only philosophically, but also by the ready availability of tools and advice. Now the task is to address this challenge so that it becomes routine.

Begin with What Is Most Important

The best organizational strategy for a manuscript reporting an experiment or set of experiments is usually not the order in which the investigator conducted the data analysis. Too often, results sections have the feel of a data analysis archive – a listing of every statistical procedure to which the data were subjected in the order in which the investigator entered them into the computer. It is a much better strategy to dispense with preliminary analyses as quickly as possible and get to the central findings. Peripheral analyses can be reported later. The most effective articles first present findings indicating that the study was properly conducted – manipulation checks for an experiment, for example – but then move quickly to the main event.

Much of my research explores how to maximize the persuasiveness of messages promoting a health behavior such as obtaining a mammogram or applying sunscreen. In a typical study, my team goes to a public beach and distributes different kinds of brochures. After sunbathers have read the brochures, a short questionnaire assessing attitudes toward sunscreen is handed out. Finally, coupons are dispensed that can be cashed in for actual bottles of sunscreen later in the day (e.g., Detweiler, Bedell, Salovey, Pronin, & Rothman, Reference Detweiler, Bedell, Salovey, Pronin and Rothman1999).

The most important potential finding in a study like this one is if different brochures actually influenced whether sunbathers obtained a bottle of sunscreen. The impact of the brochures on attitudes is interesting too, but not nearly as important as whether we observed actual behavior change. As such, it is probably a better strategy in the results section of an article reporting these findings to present the influence of the brochures on coupon redemption for sunscreen samples prior to the attitudinal data. The reader should not have to read the entire results section to know whether – bottom line – the experiment worked. If the most important question is, “So, did different brochures encourage people actually to use sunscreen?”, then the answer should be provided as early in the results section as possible.

Keep the Order of Presentation Parallel to Other Sections of the Article

While we are on the topic of the order of presenting findings, it is also generally a good idea to try to keep the order of your results section consistent with that of the introduction and method sections. So in the example above, if I plan to discuss the influence of different kinds of brochure content on actual sunscreen acquisition before I discuss its influence on attitudes toward skin cancer and sun-blocking products, my introduction and method sections should also present relevant material in the same order. In the introduction, the literature describing sunscreen use should be presented before the literature on attitudes. My hypotheses about actual behavior should be delineated before the hypotheses concerning attitudes. Likewise, in the method section, it would be preferable to describe the operationalization of sunscreen use (i.e., coupons redeemed) before the operationalization of relevant attitudes (i.e., seven-point Likert scales).

Provide Top-Down Structure

You have all read results sections, I'm sure, in which the author simply lists finding after finding, in no apparent order, with no reference to the goals of the experiment or the conclusions that should be drawn. I know that I've read many such results sections in, alas, the first drafts of doctoral dissertations. This style reflects a mistaken belief that somehow, in the results section, the investigator is just supposed to lay out the facts without editorial comment. Interpretation is saved for the discussion. This approach leads to very dry results sections and poorly integrated manuscripts.

A better approach when presenting each finding is to remind the reader why those data are proffered and then to reflect on the relationship between the reported data and the original hypotheses described, one hopes, in the introduction. So, for example, let's imagine a journal article reporting a study of the effects of mood on the recall of childhood memories. In this experiment, happy or sad moods were induced in a group of participants, who then described the first memory about their childhood that came to mind. So, a participant watched either a pleasant or unpleasant film and was then asked to recall a childhood memory. Then, the participant rated the memory on various scales. At the end of the session, the participant completed a set of mood scales as a manipulation check to make sure that the moods induced by the films lasted for the entire experimental session.

Assume that we are writing the part of the results section having to do with the manipulation check. Although it is certainly straightforward and clear to write “Participants assigned to the happy condition reported more positive moods (M = 30.04) than those assigned to the sad condition (M = 10.04), F(1, 99) = 4.60, p < .05,” it is probably better to remind the reader why you are reporting these findings and what should be concluded on the basis of them:

Although this version is a bit longer, it reminds the reader why the data were collected, and it tells the reader what to conclude on the basis of these findings. As Daryl Bem, says, “by announcing each result clearly in prose before wading into the numbers and statistics, and by summarizing frequently, you permit a reader to decide just how much detail he or she wants to pursue at each juncture and to skip ahead to the next main point whenever that seems desirable” (Bem, Reference Bem, Zanna and Darley1987, p. 185).

Don't Let the Structure of the Statistical Test Determine the Structure of Your Prose

A frequently used strategy for writing a results section is to arrange the output from your favorite statistical software package (e.g., SPSS) on your desk or tablet and then to write your prose while staring at them. This approach can lead to sentences that bear more of a resemblance to the textbook from your Introduction to Analysis of Variance course than to an article that you hope someone actually might read. Once again the theme of this chapter, “tell a story,” is your guide. And the output of an ANOVA is rarely a good story.

Consider the following example of what I call statistics-based prose:

Notice how in this passage we have no idea what was actually discovered until the very end, and we are still not really sure of the direction of the reported effects. By focusing on the terms in the ANOVA output, the author has communicated very little about what really went on in this experiment. There is also a tendency for statistics-based prose to be written in a passive voice.

Compare the previous passage to the following:

Notice how in this rewriting of the same passage, the story (not just the childhood memories of happy women) is more vivid. By side-stepping the language of analysis of variance (for the most part) and, instead, reporting what happened, the reader has a clearer sense of the empirical bottom line.

There are a few other things to note about the second passage. First, the author refers to a table where the cell means can be located (along with a measure of dispersion around each mean, such as the standard deviation). Perhaps even the ANOVA F-statistics and p-values could be moved to that table too, which could also contain an effect-size estimate, uncluttering the text further. Second, it is always good form to present descriptive statistics (e.g., cell means and standard deviations) before inferential statistics (F, t, chi-square, etc.). In other words, describe the findings – in terms with which the reader is familiar – before testing whether the trend is statistically significant or not. When presented the other way around, it is as if the author cares more about p-values meeting some criterion than the more general pattern in the data, and the results section can convey the impression of a fishing expedition rather than hypothesis-driven science.

Justify the Selection of Statistical Procedures and Tests

Often it is obvious how best to analyze your data. In the example of the experiment concerning mood and childhood memory, it seems fairly obvious that cell means will be reported by mood condition and perhaps sex, followed by some kind of analysis of variance. But sometimes one has a choice of which statistical methods to use, and it is often helpful to provide the reader with a brief justification for your selection.

Consider, once again, our work on how different kinds of persuasive brochures affect the use of sunscreen by beach-goers. Assume, for now, that we are comparing two kinds of brochures, one that describes the benefits of sunscreen use and one that describes the costs of failing to use sunscreen (see Rothman & Salovey, Reference Rothman and Salovey1997, for a discussion concerning which of these kinds of brochures should be more persuasive). To examine the differential influence of these brochures on participants’ redemption of coupons for free bottles of sunscreen, we would likely report the percentage of participants who turned in coupons by brochure condition. But we could test the significance of the difference between these percentages in a number of ways, for example using logistic regression, log-linear model fitting, or analysis of variance with arc-sine transformation. In selecting an approach, it is often helpful to tell the reader why you made the particular choice. For example:

Investigators often reduce the number of dependent variables under consideration using principal components or factor analysis. Once again, it is helpful to indicate to readers why a particular method was selected, such as principal axis or maximum likelihood, and why the factors were left unrotated or rotated according to some criterion, like varimax or promax. What assumptions were made about the data that justified the factor extraction method and rotation? Often investigators seem to select such methods by default – such as a varimax rotation – without much forethought. Similarly, the investigator should rationalize how he or she selected a certain number of factors to describe. Often “eigenvalues greater than one” is used as the criterion when other standards, such as interpretability or an elbow in a scree plot, may be more sensible.

The point is that the selection of statistical procedures should be guided by the assumptions the investigator is making about the data, and these assumptions should be made explicit to readers. Too often, the default procedures in statistical packages drive the tests reported. Early in a good results section (and sometimes at various transition points later on), the author furnishes for the reader a description of the general approach to analyzing the data, assumptions made along the way, citations to nonstandard procedures, and justification for the selection of procedures from among an array of options.

Thorough Reporting Is Good Form

Earlier, I discussed why it is good practice to provide readers with a description of what was discovered (e.g., cell means) before turning to inferential statistics, such as analyses of variance. There are similar rules of thumb that apply to other statistical procedures. I would not characterize these as hard-and-fast rules – there are many times when it is reasonable to make exceptions – but in general it is good practice to follow them.

For one, when analyses are sensitive to the correlations among your variables, a table of these correlations should be provided. For example, often investigators report multiple regression analyses in which the influence of a set of “predictors” on a “criterion” is examined. Let us say we are studying what kinds of intelligence are associated with scores on the Graduate Record Examination – Psychology Subject Test. Several hundred participants have been given an analytic intelligence test, a practical intelligence test, and an emotional intelligence test. We know that there is likely to be some overlap among the scores on these intelligence tests, and so we regress the participants’ GRE–Psychology scores on to the three intelligence test scores and discover that the best “predictor” of GRE–Psychology scores is analytic intelligence. Now it is entirely possible that the correlations among the three intelligence tests are quite high, so that entering one of them in a regression model suppresses the influence of the other two. But if the zero-order correlations among the four constructs measured in this study are not presented prior to the multiple regression model, we cannot evaluate this alternative interpretation of the findings. So, as a general rule, a table of correlations should precede the reporting of regression analyses involving the same variables. Similarly, it is good form to report the internal consistency and/or the reliability of these kinds of measures before reporting the regression. A variable may look like an especially good “predictor” simply because it is the most reliably measured variable in the regression model. Remember the rule you learned in your statistics course: reliability limits validity.

A second example concerns the reporting of some measure of dispersion, usually standard deviation, before describing analyses, such as ANOVA, that are sensitive to the variance among measures. Generally, the table in which cell means are listed should also include standard deviations, so that the reader can eyeball the table and come up with pretty good effect size estimates before even looking to see if the ANOVA produced statistically significant results. The reader might also be interested in whether there is homogeneity of variance across measures and/or whether distributions of scores are more or less normal, all assumptions of general linear models that are often honored in the breach. Even if cell means are diagrammed in figures rather than reported in tables, it is generally considered good practice to provide a sense of the distribution around these means using error bars based on standard errors or standard deviations.

In the time that has passed since the previous edition of this book, it has become common practice to add an explicit estimate of effect size to the usual reporting of statistical tests. To save space, I have not done so in the examples provided here, but, generally, you should provide information that allows the reader to assess not just the statistical significance of an effect, but also the size of that effect. You can do this in a number of ways, such as providing 95 percent confidence intervals around a difference (of means, for example) expressed in the actual unit of measurement or, as is more typical for findings in psychology, in a standardized estimate such as Cohen's d. Standardized regression weights (betas) are already effect sizes, so when reporting the results of multiple regression analyses you can report unstandardized bs with confidence intervals or, better still, standardized betas, which can be compared directly with each other. The Publication manual of the American Psychological Association recommends reading Grissom and Kim (Reference Grissom and Kim2005) for a discussion of effect size estimates, and I do too.

A Few Other Odds and Ends

Finally, I will mention a few other rules of thumb for writing good results sections. These are not hard-and-fast rules, but they will improve the clarity and impact of any article. Following them may also convey a sophistication about what you are doing with your data. The first set of tips concerns strategies for structuring the manuscript as a whole; the second set is focused more on the reporting of specific statistics.

Here are some tips concerning the organization of the manuscript:

1. 1 In order to get to your main findings as quickly as possible, consider placing many of the preliminaries in the method section, such as the demographic breakdown of the sample, evidence that the participants were randomized to experimental conditions properly, and a sense that the cover story was believable and the measures reliable.

2. 2 Put as much in tables and figures as possible. Although some editors may later ask you to move material in tables and figures back to the text in order to save space in the printed journal, results sections are usually more readable to the extent that numbers are removed from the text itself. Always refer explicitly to these tables and figures. It is not sufficient to note parenthetically at the end of a sentence “(see Table 1).” Rather, tell your readers what it is they can find in Table 1. For example: “Table 1 lists the means and standard deviations for all of the memory measures administered in this experiment. Comparing the first and second columns of Table 1 reveals that for four of the five variables, the quality and quantity of childhood memories, differed for happy as compared to sad individuals.”

3. 3 Do not confuse redundancy with clarity. If means can be found in a table, do not repeat them in the text itself or vice versa. You don't need both a figure and a table illustrating the same finding.

4. 4 Finally, consider a combined results and discussion section if your manuscript is relatively short or if the results are sufficiently complex that detailed explanations along the way would render them more intelligible.

Here are some tips about the reporting of actual statistics:

1. 1 When reporting inferential statistics, provide complete information: the name of the test, the degrees of freedom or sample size, the value of the test statistic, and whether it met the criterion you have set for statistical significance (Sternberg & Sternberg, Reference Sternberg and Sternberg2016).

2. 2 Don't make a fetish out of p-values. Generally, set an alpha level or two in advance, such as p < .05 and p < .01, and report whether a finding meets that criterion. It usually is not necessary to report exact levels of p. Moreover, reporting exact p-values conveys the impression that you think lower numbers mean larger effect sizes, when this may not be the case (e.g., if the sample sizes included in the test vary).

3. 3 Don't feel obligated to report in detail every statistical approach that you tried. For example, if you analyzed the data using several different transformations (e.g., log, reciprocal, square-root) and the analyses always produced essentially the same findings, feel free to tell the reader in a sentence or two (or even in a footnote) rather than present the same findings in different form over and over again. The same rule holds when you try different factor extraction methods and rotations in a factor analysis, or different hierarchical approaches in a regression model.

4. 4 Always try to describe your findings in the units actually measured, no matter how the data were analyzed. “Participants who first read positive words were 230 msec faster in responding to the target photograph than participants who first read negative words” is better than “Participants who first read positive words had faster reaction times (mean log RT = 2.27) to the photograph than participants who first read negative words (mean log RT = 3.89), F(1, 38) = 6.21, p < .05.”

5. 5 When statistical tests produce results that are not quite statistically significant, it is not necessary to qualify these findings with great barrages of defensive rhetoric, such as the results were “marginally significant,” “did not reach conventional levels of significance,” “just missed significance,” “trended in the right direction,” and so on. Rather, just state the claim, statistic, and p-value straightforwardly: “men smiled less often than women, F(1, 203) = 3.70, p < .06.” The readers of your journal article can see that you “just missed” and can decide for themselves whether this shakes their confidence in your conclusions. At the same time, one should be especially careful to avoid overstatement in these situations (cf. Abelson, Reference Abelson1995, especially chapter 4). Of course, some journal editors and reviewers feel that you are trying to slip something past them if you don't qualify a “nonsignificant” finding with some kind of adjective. If you believe this to be the case, then do it succinctly, such as by adding the word “somewhat” after “men smiled” in the example above, and move on without a lot of hemming and hawing.

The Bottom Line

I hope that I have persuaded you that the best results section is written as a story. And like any good story, the author needs to establish a cast of characters and a setting, justify these selections, and then take the reader through the drama of steadily rising action, climax, and denouement. A good story is rarely a chronological rendition of every idea that occurred to the investigator as he or she stared at the data. Nevertheless, it is a story that is completely (rather than selectively) told. Although the writing of the results section (indeed, the article as a whole) may deviate from the story that the investigator may have planned prior to actually conducting the study, the results section of your journal article must allow another investigator to replicate what you have done, and it cannot reflect “p-hacking,” subjecting the data to a barrage of statistical tests (even while still conducting the experiment) and reporting only those findings that are significant (or, worse, ending data collection when a peek at the findings reveals significance). Tell an accurate and good story, and let the drama of that story reveal itself through your results. If you do so with the highest ethical standards and respect for the scientists and others who will be influenced by your findings, your article – or, at least, your results section – will stand the test of time.

Summary: What Do We Know Now that We Did Not Know Before?

Your paper has jumped through all the hoops, proving itself to have theoretical grounding in the introduction, design acuity in the methods, and appropriate analyses in the results section. Now you have earned the right to frame the findings, to make a coherent explanation for the most salient discoveries, most reliable patterns, and central contributions.

Readers love the discussion's first couple of paragraphs because in a pinch they can cut to the chase, to read the authors’ favorite framing of their results. If convinced and intrigued, then the reader may go back to scrutinize the methods and results in more detail.

This is not to say that the summary should repeat the abstract, which can devote only a sentence or two to the findings. Rather, consider the discussion's summary to be a recap; the reader has already seen the material but needs to be reminded about what is most significant for the overall story: Like a video series that begins each episode “Previously …,” the summary provides highlights for the audience to retain moving forward.

The discussion structures the prior material so that the reader understands the paper's contribution. Viewing it as an argument's logic or as a story's denouement differs in emphasizing tight reasoning versus human suspense resolution. Both matter, and what they share is guiding structure. If the author cannot articulate the research's central points, the reader certainly will not. This is the time to show off the strengths of the data. Do not under-sell your findings. Be bold!

A credible summary does not over-claim, however, which merely invites reader skepticism at best, or attack at worst. An author being grandiose raises red flags, and no one needs to make readers suspicious. As an editor, I have seen authors report one new paradigm and then claim that this one study shows that “Humans do X.” Without a more programmatic approach, replication, converging methods, and varied samples, no one has any business making claims about human nature.

Nor should the summary be self-promoting and gloss over difficulties. Excessive subject selection in the methods section is still an issue in the discussion. A marginal finding in the results section is still qualified support in the discussion.

But making a principled argument for the reliable pattern is appropriate. Marshal all the evidence for a robust effect that persists across types of participants, operationalizations, measures, and alternative analyses. A mini-meta-analysis across several studies testing the same hypothesis may help readers to judge the most reliable pattern, the effect's size, and the finding's stability.

In short, the bottom line of the summary subsection is whether you would bet confidently on this new knowledge. If its authors do not believe in it, no one else will. Speaking of bets, this author is willing to bet that starting the discussion with a summary paragraph, or several, is common practice.

Limitations (to Be Sure …)

Here, modesty is in order. Admittedly, no data are perfect, so the limitations subsection anticipates the critics, acknowledges likely objections, and neutralizes them if possible. Even if the flaws cannot be fully counter-argued, merely by identifying the paper's shortcomings the author eliminates them as “gotcha!” targets. No one will read a letter-to-the-editor or a blog-post that simply repeats a concern already admitted by the authors.

Part of you may wish that if you do not point out the problems, readers will not notice them. Do not count on it. The journal-club game of kill-the-author has morphed into online forums, becoming at once faster, more widespread, and more competitive. The downside consequences of ignoring problems, in the hope that they will go away, are too damaging. They will come out. The best defense is being one step ahead, granting the limitations and defanging them where possible.

Make no mistake: This is difficult. Anticipating the critics is challenging because taking a negative perspective on the self and its hard-won achievements is unnatural. A self-respecting or at least self-sympathetic researcher may have produced data only its parent can love. Few people spontaneously recognize their own defects, and these positive illusions may be adaptive (Taylor, Reference Taylor1989), at least in Western settings. Nevertheless, train yourself to listen to the still, small voice of doubt and harness it. Do not let it paralyze you, but use its dire messages to convey limitations others might voice. It may help to imagine what specific advisors, peers, or competitors might say. Then, address them constructively.

While training a useful inner critic, an author can already draw on some of the most common limitations, besides uneven data. A frequent issue is having narrow, WEIRD samples – Western, educated, independent, rich, and democratic (Henrich, Heine, & Norenzayan, Reference Henrich, Heine and Norenzayan2010). Admitting that the sample were all small-college sophomores is not enough; say how the results might differ, and why, in other age-groups, classes, or cultures. A limitation can turn into a hypothesis about moderating conditions.

Another common limitation is restricted operationalization of the independent variables, dependent variables, or stimuli. Besides accounting for randomness, systematic factors limit any operational choices; this subsection should acknowledge both. Again, if the limits are conceptually interesting, venturing hypotheses about potential moderators advances science. The crucial limitations subsection usually should have its own heading.

In this chapter, the advice is limited by drawing on the experience of one senior social psychologist. Granted, the author has edited for several journals, and only some have been empirical (Journal of Experimental Social Psychology, PNAS, Science), but she has extensive reviewing experience for other empirical journals. And editing review/theory journals (Annual Review of Psychology, Psychological Review, Policy Insights from Behavioral and Brain Sciences) sometimes entails reading discussion sections of other kinds – not to mention writing her own and editing her collaborators’ discussion sections. Nevertheless, this chapter is limited to one author's perspective, supplemented by some of the relevant literature.

Future Directions (Mopping Up)

Limitations lead directly to future research. Gaps in the present work can be filled down the road. This subsection might seem to represent the clean-up crew, mopping up the mess left by the shattering limitations. A more elevated view is that this section resuscitates the paper's main findings, after their trauma of self-criticism.

Whatever else the discussion says here, avoid the generic clarion call: “More research is needed.” Of course. Always true. But what specific next steps follow in this research program to fill its distinctive gaps? Discovery in psychological science often entails identifiable stages: demonstrate an effect, show how it differs from past research, rule out alternative explanations, check its robustness over varied operationalizations, test its generalizability across samples and settings, and identify theory-driven moderators.

Each of these stages could provide a future direction, depending on the maturity of the current research program. This subsection should compose not a long to-do list, but instead the next stage appropriate to the current stage of development for this particular research program.

Done right, the future directions subsection can be exciting, tantalizing the reader to see what the author's lab will do next. Granted, going public with your plans can lead to being scooped, but this happens less often than people think. And one reframing is that the scooper's flattering interest shows your work having impact, and their work saves you the trouble of doing the study yourself.

Future editions of this chapter could content-analyze discussion sections in high- and low-impact empirical journals, written by authors at various career stages and degrees of eminence. Natural-language analyses might reveal what content predicts citation counts, for example. Mentions of synonyms for limitations, shortcomings, and flaws might predict higher citations – modesty might reflect an initial breakthrough, perhaps. Other content analyses might check whether this chapter's advice correlates with norms, describing what authors actually do.

Implications (Stay Tuned)

The discussion builds out from specific (summarize this research) to slightly more general (acknowledge its limitations) to hypothetical (future directions). The implications subsection can be openly speculative, although moored in the reality of evidence and the bounds of plausibility.

Conceptual implications can support or undermine existing theory, anywhere from confirming or contradicting its premises to proposing mechanisms to identifying moderator variables.

Methodological implications can supply a new method, manipulation, or measure.

Policy implications can suggest evidence-based interventions. One study does not make a policy, of course, but it can imply directions for future policy-related tests. In psychology, a research program often has implications for education, well-being, health habits, physical health, social justice, organizations, or marketing.

Ideally, one implication of this chapter, combined with the rest of the volume, would be that using it teaches first-year graduate students how to write effective articles for psychology journals.

A Note on Style

Sometimes a discussion requires a special comment that does not fit the summary–limits–implications–future framework, and this chapter illustrates this point with this note about writing style.

Good scientific writing does not call attention to itself. Instead, it uses few words, straightforward language, plain vocabulary, and simple sentences; it just highlights the science.

When I line-edit (by permission), the most common revision is omitting needless words (see Strunk & White, Reference Strunk and White2009). Wordy sentences are hard to follow, so authors can help readers by avoiding multiple words, when one word will do. Consider: Extraneous verbal padding and extra words are encouraged and facilitated through the use of passive voice by authors, as well as sentence subjects and objects made up of compound nouns that contain phrases, and also rewording that repeats a concept (40 words). Consider the alternative: Passive voice, noun phrases, and elegant restatement add unnecessary words (10 words).

Jargon is another writing sin. Unless precision requires a technical word, use a common word. Similarly, do not show off your high-verbal GRE by picking obscure words when plain ones will do: Choose everyday words with fewer syllables (but not slang or contractions).

Show, do not tell, what is important, interesting, or surprising. Readers will not take your word for it just because you say, “It is crucial that …” Not only does this phrasing waste words, it also insults the reader's own judgment. And it subordinates the main point to the author's judgment about it.

Finally, background the authors, and foreground the research. Why say “Smith and Brown (2017) conducted an experiment and found,” which emphasizes the people and subordinates the findings. Instead: “Participants reported excitement after reading the Sternberg volume, compared with controls who read the Amtrak train schedule.”

This volume's earlier chapters, as well as other authorities, offer compatible style advice. Discussion sections should avoid repetition, polemics, and triviality (APA, 2010). Hands down, the best general advice on writing is The elements of style (Strunk & White, Reference Strunk and White2009).

Conclusion

When all is said and revised, discussion sections can satisfy both writers and readers. Summaries and conclusions serve convenience; limitations provide self-correction; implications and future directions offer creativity. Conclusions wrap up.

Why Do We Provide References?

You have no doubt heard about the evils of plagiarism elsewhere. However, avoiding plagiarism is only one reason – and in our view, not even the most important reason – to document the sources for the foundation of your work. Rather, there are important reasons why you need to provide information about the source materials that comprise the foundation of your own work.

First, you have likely been told to find a “hole in the literature” when deciding how to proceed with your research. The only way to justify that your work fills a hole in the literature is to show that you have adequately and accurately reviewed the available literature. By discussing and citing key studies, you not only justify the importance of your work, but you also situate your work within a body of research. Indeed, we all have ideas for research studies that are rooted in personal experience or other relatively random insights; however, such inspirations must lead to a review of the empirical and theoretical literature to establish the importance of your research idea.

Second, you are showing respect for the people who inspired your research when you cite their work. Regardless of whether you agree with their research or not, when you cite important and relevant work, you are showing that you have taken the time to read and think about what previous scholars have accomplished. It is important to document, to the extent it is relevant to do so, the historical context of the topic that you are researching.

Third, by documenting your sources, you are providing the blueprints needed for future studies. Think about building a house. What information would you need to do this project? You would need to know what materials are needed, where those materials are to be used (e.g., to construct the kitchen, to provide roofing, to support the flooring), how to use those materials to construct each part of the house, how much the materials will cost, and, of course, how long it normally takes to build the house with those materials. In a sense, building a house is analogous to doing a research project. You do background research to see what has been done before. You use the information from previously constructed houses to decide how to build your ideal house. Even after you have the blueprints for your house, permits are required to ensure that you are not violating city codes. Often blueprints for similar constructions are used to justify why your blueprints are appropriate. Although you do not need to always credit the architect of similar house blueprints, you do need to give credit to the sources used in your research. Not only is this crediting a form of respect to those that have inspired your work, the citations become part of the blueprints of your study.

Research blueprints enable us to conduct a research study in the same manner as the original researchers conducted their study. Replicability is one of the hallmark characteristics of science. The scientific community can have more faith in repeated results from multiple research studies conducted by separate researchers. Suppose Dr. Sternberg conducted a research study at his large university. We find it interesting and decide to replicate it at our small liberal-arts college. In doing so, we must reference Dr. Sternberg's work, not simply for professional courtesy, but to show that our blueprint is situated in successful previous work.

Contents Needed in Referencing a Source

When documenting the sources that provide the foundation for your work, you need to give the “who,” “when,” “what,” and “where” for each source of information that was used to guide your work. This chapter is organized around each of these four pieces of information that, together, constitute referencing your sources.

Digital Documentation Tools and Moving Forward

Based on the discussion of the book chapter, you can see how the internet has changed how we document our references. More and more work is being conducted electronically, and APA has modified its practices accordingly to include both doi and URL as indicators of where the work was accessed. We recommend that you always check the most recent version of the APA manual if you encounter an internet citation that you are unsure how to cite.

In addition to checking the APA manual or website, increased technological capabilities have enabled the production of several reference management software programs that can be used to help you organize and document your citations. For example, EndNote is available for purchase if you would like to store the data on your desktop. Even more accessible, EndNote Basic, the web version, is available for free. BibMe, Citation Machine, and Easy Bib are more free online tools that can be used only to help you format your citations in the most recent APA format.

Finally, there are now digital programs available for you to use to make sure you do not plagiarize. Both BibMe and Turnitin.com are relatively easy-to-use tools that allow for the detection of potential plagiarism. These online tools are particularly useful because plagiarism does not always happen intentionally. To help assure you have not fallen into this trap, we recommend checking your work using a program such as Turnitin.com. If there is a problem regarding plagiarism, it is certainly better to catch it yourself before someone else does. Just ask the former crossword puzzle author at USA Today.