Graphing SCD data

Many options are available when presenting serial SCD data (e.g., cumulative graph), but contemporary standards and practicality generally dictate use of a line graph to represent change over time. A line graph allows for formative evaluation of performance across sessions, as well as summative evaluation when reviewing data collected across conditions in a study. In contrast, exclusively summative measures such as bar graphs (often used in group comparison or pre–post research paradigms) are not conducive to comprehensive visual analysis because these types of graphs only provide a quantitative summary of performance using a pre- and post-test format. Thus, when reporting data collected within a SCD it is critical to plot data from each session within each condition on a line graph, which allows for reliable analysis of data characteristics within and between conditions.

Each condition shown on a line graph is differentiated using A–B–C notation (Gast & Spriggs, Reference Gast, Spriggs, Gast and Ledford2014). Traditionally, the A condition refers to the pre-intervention or baseline condition, and the B condition the intervention condition. Each subsequent condition introduced during a study is labelled in sequential alphabetical order (e.g., C, D), with the exception of parametric variations of the independent variable, which are labelled as prime (B’), or a combination of treatments, identified by combining notation labels (e.g., BC, CD, BCD). Thus, a typical withdrawal design with two baseline conditions and two interventions conditions is referred to as an A–B–A–B design and a multitreatment design with two iterations of different intervention conditions is referred to as an A–B–C–B–C design. The notation is used less often with other designs (e.g., multiple baseline designs), but it remains critical to consider because there must be at least three potential demonstrations of effect between the same two conditions in every design. Thus, an A–B–A–C design does not meet this criterion, and neither does a multiple baseline design with one intervention introduced in two tiers and a second intervention introduced in the third tier (i.e., there must be three A–B comparisons in a multiple baseline design, with B representing the same intervention).

When constructing a line graph, the ordinate scale (y-axis) represents a previously identified metric (e.g., number of aggressive behaviours). Each data point indicates the extent to which the dependent variable occurs based on this metric (y-axis) across observations, days, or sessions (abscissa or x-axis). The intersection of the ordinate scale and the abscissa is represented by a geometric shape (marker); the markers are connected by a line, and this collection of data points is referred to as a data path. A single type of geometric shape represents each dependent variable (e.g., a single behaviour for one participant). The data points in a data path are not connected across condition changes; instead, a vertical line indicates a condition change in order to separate each data path into segments according to condition. When constructing line graphs for a study (a) include no more than three data paths on each line graph (increases the likelihood of reliable analysis) (Cooper, Heron, & Heward, Reference Cooper, Heron and Heward2007), (b) ensure ordinate scales are equal across all line graphs, and (c) ensure the proportion of the ordinate scale to the abscissa allows for discrimination between data points (e.g., ratio of 2:3 for graphs that include a few data points and 1:3 for graphs that include many data points; ensures data are not misleading to readers). When the y-value has a defined maximum (e.g., 100% correct or 100% of intervals), the entire range should be represented on the graph to avoid misleading readers (Dart & Radley, Reference Dart and Radley2017). Finally, line graphs may be constructed using software such as Microsoft Excel or Microsoft PowerPoint; graph construction is beyond the scope of this paper, but instructions are available elsewhere (Barton & Reichow, Reference Barton and Reichow2012; Vanselow & Bourret, Reference Vanselow and Bourret2012). Figure 1 shows an example of appropriate SCD data display.

FIGURE 1: Applied Example

Number of aggressive behaviours per hour for Billy. As shown in the figure, the dependent variable is the number of aggressive behaviours per hour, and the y-axis ranges from 0 to 10. The x-axis depicts the time unit, which is ‘sessions’ in this case (typically true in single case research; Ledford, Severini, Zimmerman, & Barton, Reference Ledford, Severini, Zimmerman and Barton2017). The data points are depicted by filled in circles, and the condition is labelled with ‘A’ (baseline). This graph has a ration of approximately 1:2; if the study is relatively short (e.g., 14 sessions), the graph may need to be resized to approximately 2:3 but if it is much longer (e.g., 40 sessions), it might be appropriate to resize the graph to something closer to 1:3. The importance of ratio is that data points be neither ‘stretched’ along the x-axis nor so close together that they are difficult to differentiate.

Level

In SCD research, experimenters are most often interested in changes in the amount of behaviour that occurs—that is, the level of the behaviour. In between-groups research (e.g., randomised controlled trials), differences in levels are almost always conceptualised as mean differences, but the relatively small number of data points in SCD research makes the mean particularly susceptible to outliers. Moreover, because SCD research involves repeated, and often continuous, measurement rather than pre- and post-assessments, the difference in level often occurs following a period of time in which the change is primarily characterised as a change in trend. For example, a child who performs a task with 0% accuracy in baseline may also perform that task with 0% accuracy during the first intervention session, followed by steady increases (10%, 20%, 30%, 40%, 50%) until mastery (100% accuracy) is reached. Thus, the mean level would not characterise intervention data well. For formative analysis purposes, there are two questions related to within-condition level and between-condition level:

1. (1) Is the level in the current condition sufficiently stable for a reliable prediction of value assuming the condition is not changed (i.e., within-condition level)? If it is, and you have at least three measurement occasions (data points), it is prudent to change conditions.

2. (2) Is there a level change between the current condition and the adjacent previous condition (i.e., between-condition level)? If yes, you have one demonstration of effect (i.e., change in behaviour that occurs concurrently with the condition change, in the expected direction).

For multiple baseline and multiple probe designs, changes in level between conditions are complicated because not only must data change when the intervention is applied to each tier, but it also must not change when the intervention is applied to subsequent tiers. For example, in the third tier of a multiple baseline design, data must remain at similar levels (a) during initial baseline sessions, (b) after intervention is applied to the first tier, and (c) after intervention is applied to the second tier. If data change in a later tier when intervention is applied to a different tier, this might suggest generalisation across tiers (for designs with multiple behaviours or contexts), contamination (e.g., for designs with multiple participants, the implementer may have used the intervention during baseline for participants assigned to later tiers), or history effects (e.g., something outside the study caused behaviour change). It is imperative to visually analyse data in all tiers before intervening in any tier—this is referred to as vertical analysis. If within-condition changes in level occur in any tier, continue in the current conditions until level is stable, and then intervene in the next tier.

Although SCD researchers are most often interested in the level of data in terms of change, two additional features often expected in SCD data—trend and variability—are also critical for assessing behaviour change via visual analysis.

Trend

Trend (or slope) refers to movement in the data over time, with specific attention given to the direction of a data path within and between conditions, commonly referred to as an accelerating, decelerating, or zero-celerating trend along the ordinate scale. Trend is further characterised as therapeutic or contra-therapeutic, depending on the purpose of the study (e.g., a decelerating trend is therapeutic when introducing an intervention to decrease verbal aggression, but a decelerating trend is contra-therapeutic when introducing an intervention for increasing the number of bites eaten independently). A within-condition analysis of trend is necessary to avoid premature introduction or removal of an intervention. For example, suppose under baseline conditions a participant is displaying an accelerating trend in a therapeutic direction; in this case, it is not necessary to intervene, given that improvement is likely due to maturation or influenced by factors independent of the study. Similarly, under intervention conditions, if data indicate a zero-celerating trend during initial treatment sessions, but an accelerating trend during subsequent treatment is present, it is recommended to continue collecting data to ensure such improvement continues in a therapeutic direction (i.e., behaviour change is clinically significant). Finally, when comparing adjacent conditions, a basic demonstration of effect is observed when directionality shifts across conditions. Thus, trend is often of interest because trends are present in typical learning patterns (e.g., acquisition). Moreover, trends occurring outside of intervention conditions (e.g., in baseline conditions) may be indicative of threats to internal validity (maturation) (Gast, Reference Gast, Gast and Ledford2014; Kazdin, Reference Kazdin2010).

Variability

Variability refers to the extent to which data points are similar in regards to value (ordinate scale). Generally, data points are considered stable when approximately 80% of values are within +25% of the median value in a given condition (sometimes referred to as a stability envelope) (Lane & Gast, Reference Lane and Gast2014). Data that are considerably variable in baseline are problematic because as mentioned above, it impedes your ability to accurately predict the level of the next data point, given no change in condition (see Figure 2). This, in turn, limits your conclusions about whether an observed change in level is a result of the variability of the data or the change in condition. If data are considerably variable during baseline conditions, you should continue data collection until data are stable. Alternatively, if you have a strong a priori assumption that the condition change will result in a large level change and variability in baseline was expected, you can collect at least five data points and then intervene. If data change in level, are less variable, and do not overlap with baseline, you can be confident changes occurred due to changes in condition. If data remain variable or the intervention condition includes considerable data points that overlap with the data points in baseline, your confidence is decreased. Although changes in variability alone could theoretically be of practical importance (e.g., improving consistency of checking blood sugar for a patient with diabetes) and could result in a determination of a functional relation, we are not aware of any published SCD studies in which decreased variability was the primary treatment goal.

FIGURE 2: Applied Example

Number of aggressive behaviours per hour for Billy. In Figure 1, the first three data points in the baseline (A) condition were plotted. The data were somewhat variable (see previous figure), with the patient with aggressive behaviour engaging in 7–9 aggressive behaviours per hour. Because the researcher is not convinced she could predict ‘about’ where the next data point might fall, she decides to collect at least three more data points. After those three data points are collected, as shown in this figure, she determines that the data are predictably high in level and somewhat variable, with no trend (e.g., approximately 0 slope). Thus, she decides to implement the initial intervention condition.

Formative analysis summary

Readers should note that although suggested minimums exist regarding the number of data points in each condition (e.g., 5 data points; What Works Clearinghouse, 2013), decisions about changing conditions should be made by assessing the data for level, trend, and variability. It is only appropriate to change conditions after collecting a minimum number of data points and characterising the level, trend, and variability of the data. Table 1 includes information related to common baseline data patterns and resulting decisions about condition changes. As Table 1 depicts, formative decisions about continuing baseline conditions or introducing the intervention condition should include assessment of level, trend, and variability. These data characteristics are critical regardless of design type; however, some specific considerations are notable for particular designs. These design-specific considerations are described in Table 2.

TABLE 1

Using Visual Analysis to Make Condition Change Decisions

Note: All decisions are based on data that are low in baseline and that researchers intend to increase during intervention conditions. Rules can also be applied for data intended to change in the opposite direction.

TABLE 2

Design-Specific Considerations for Visual Analysis

Consistency

Consistency refers to the extent to which data patterns are the same within like conditions (e.g., in both baseline conditions in an A–B–A–B design; in baseline conditions for all participants in a multiple baseline across participants design) and the extent to which changes (in level, trend, or variability) are the same for each potential demonstration of effect. In SCD research, the critical factor in determining a functional relation is the consistency of behaviour change between conditions; consistent but small changes in level between conditions are superior to inconsistent changes of larger magnitude. Sometimes inconsistencies are expected; for example, in A–B–A–B designs, we may expect the dependent variable to fail to fully reverse to baseline levels (for an example of this in the published literature, see Ahearn, Clark, MacDonald, & Chung, Reference Ahearn, Clark, MacDonald and Chung2007). When determining whether a functional relation occurred, the most important question is whether lack of consistency in data patterns and changes between conditions impedes confidence that differences in data between conditions occurred due to condition changes and only condition changes.

Overlap

Overlap refers to the extent to which data from one condition are at the same level as data from an adjacent condition; it may helpful to think of overlap as the proportion of data points in the intervention condition that are not improved relative to baseline. Because level is often the data change that is most important to interventionists, it is perhaps not surprising that early attempts to quantify visual analysis of change between conditions were based on the degree to which data were non-overlapping in the expected direction, since non-overlap of data often corresponds with differences in level (PND; Scruggs, Mastropieri, & Casto, Reference Scruggs, Mastropieri and Casto1987). Thus, the degree to which overlap occurs is important, since it speaks to level change, although PND and other attempts to quantify overlap are highly sensitive to procedural parameters (Pustejovsky, Reference Pustejovsky2016a); that is, the extent to which overlap-based metrics correspond with changes in level is highly dependent on study procedures in addition to outcomes. Nonetheless, overlap between conditions can be accessed via visual analysis by posing and answering the following questions:

1. (1) What is the extent of the overlap between conditions (e.g., how many data points between conditions are at about the same level)?

2. (2) Does the degree of overlap change over time?

3. (3) Is overlap consistent between comparisons?

4. (4) Was overlap expected a priori?

5. (5) Does overlap impede confidence in a functional relation?

Question (5) is critical, despite its somewhat subjective nature. Confidence should be decreased when many data points in adjacent conditions are at approximately the same level, overlap does not decrease over time, overlap was not expected, and the overall change in level is small. Note that overlap-based metrics like PND only characterise overlap based on the first consideration; all of the others are reliant on visual analysis.

Immediacy

Immediacy is the extent to which data change simultaneously with a condition change. When analysing immediacy between conditions, the following questions should be considered:

1. (1) Is there an immediate and abrupt change in the dependent variable?

2. (2) If not, is there a delayed increase in the dependent variable (gradual therapeutic change in level and trend or a change that occurs several sessions after the condition change) and

3. (3) Is this pattern of responding replicated across similar conditions? For example, if a participant displays a delayed response to the intervention and all other participants display an immediate and abrupt change in the target behaviour, researchers need to first ensure procedures and data collection occurred as intended and then assess the idiosyncrasies of that condition compared to others (e.g., implementer; pre-intervention characteristics of participants).

Some interventions might reliably lead to delayed increases in the dependent variable. Although immediate changes are preferable, non-immediate changes can still result in a functional relation determination if delayed or gradual changes were expected and these delayed/gradual changes were consistent across demonstrations (e.g., in all tiers of a multiple baseline design). Thus, immediacy considerations are dependent on visual analysis, but also on a researcher's knowledge of the participants, independent variables, and dependent variables.

Level

There are a number of statistics designed to characterise the average level of behaviour occurrence, including mean and median values. The mean value is calculated by adding all data values and dividing by the number of data points. However, a considerable weakness of using the mean is its sensitivity to outlying values, which is particularly a problem in SCD research because generally only a few data points are collected per condition. The median value is the middle value of a set of data points (i.e., the centremost value when all data values are ranked from lowest to highest). Although standard deviation is reported in most group comparison studies, SCD researchers have historically reported a range of values (i.e., the minimum and maximum values) to quantify the variability of data within a condition. We should note that none of these statistics are appropriate when quantifying the level of data within a condition if trends are present.

Trend

Traditional trend or regression lines are inappropriate for SCD research, given the relative dearth of data points within a condition. Instead, the split middle procedure is sometimes used to quantify within-condition trends. To conduct the split middle procedure, median values within each adjacent condition (A–B) are identified and used to generate trend lines. Within each condition, the data path is divided in half. For example, if there are 7 data points within the A condition, divide the number of data points by 2 and add .5 to the quotient to obtain the mid-date (or median session along the x-axis; this calculation indicates that the fourth session is the mid-date). If there are an odd number of data points, the mid-date and corresponding value along the ordinate are omitted from further calculations. For the remaining points along the data path, this calculation is repeated to identify the mid-date of each half (3 divided by 2 plus .5 indicates the second session is the mid-date for each half of the data path along the x-axis). Next, identify the mid-rate, or median value, along the ordinate (or y-axis) for each half of the data path. Continuing with the above example, if the values of the first half of the data path are 28%, 16%, and 32% for sessions 1–3, the median value is 28%; this process should be repeated for the second half of the data path. The intersections of the mid-date and mid-rate for each half of the data path are identified on the graph and a trend line is created. The expectation in the baseline condition is the data path is moving in a contra-therapeutic or zero-celerating direction and, as such, if additional data points were collected, the trend line would continue in the same direction under those conditions. Once the trend line is identified it should be moved so that half the data points fall above and below the line. The procedure is repeated for the intervention condition, with directionality compared between conditions (see Lane & Gast, Reference Lane and Gast2014 for detailed instructions).

Variability

As noted above, the range of data values within a condition (i.e., minimum and maximum values) can serve as one metric of characterising data variability; however, this is only appropriate when no trends are present. A stability envelope can be calculated for data with or without trends present. Stability is commonly defined as 80% of values within +25% of the median value within a condition. For example, suppose a baseline condition consists of 8 data points and the values along the ordinate range from 15% to 35% (ordinal rank = 9, 10, 15, 15, 25, 32, 35, 35). First, identify the median value; since there are an even number of data points, add the fourth and fifth values and divide by 2 to obtain a quotient of 20 (identify the median value on the graph and draw a horizontal line within condition). Second, multiply the quotient by .25 (i.e., 25%), which yields a value of 5. Thus, the stability envelope is 15–25% (identify each value along the ordinate and draw additional horizontal lines to denote the stability envelope), with a median value of 20%. Finally, calculate the percentage of data points that fall on or within the stability envelope (three data points [15, 15, 25] divided by 8 multiplied by 100 = 37.5%, indicating data are not stable). The same stability envelope is applied to the intervention condition. When trends are present, the stability envelope can be superimposed along the trend line obtained using the split-middle method discussed earlier, rather than the median (Gast & Spriggs, Reference Gast, Spriggs, Gast and Ledford2014; Lane & Gast, Reference Lane and Gast2014).

Overlap

Numerous overlap metrics have been developed, though none are appropriate to serve as an ‘effect size’, as described above. However, they can be used to describe the degree of overlap between conditions, which is one data characteristic that we use in visual analysis. Other sources are available that describe calculations for a number of the metrics (Pustejovsky, Reference Pustejovsky2016a; Wolery, Busick, Reichow, & Barton, Reference Wolery, Busick, Reichow and Barton2010); here, we will describe the use of the percentage of non-overlapping data (PND) and the non-overlap of all pairs (NAP). We choose these metrics because PND is widely used and easy to understand, while NAP is less sensitive to outliers (when compared to PND) and less sensitive to procedural variations (when compared to PND and other metrics; Pustejovsky, Reference Pustejovsky2016a).

When calculating PND, first identify the highest value along the ordinate for the baseline condition, assuming the goal of the intervention is to increase a socially appropriate behaviour (otherwise indicate the lowest value along the ordinate). In the intervention condition, draw a horizontal line through the condition to indicate the most extreme value of the baseline condition; this serves as a visual analysis tool for determining the number of data points that are above this line (or below this line, if the purpose is to decrease behaviour). Count the number of values in the intervention condition that are more extreme than the identified value in baseline condition and divide that sum by the total number of data points in the intervention condition and multiple by 100; this yields PND for the intervention condition when compared to baseline (e.g., if the highest value of the baseline condition is 25% and 14 of 16 data points in the intervention condition are above 25%, PND calculations would yield 87.5%) (Gast & Spriggs, Reference Gast, Spriggs, Gast and Ledford2014; Lane & Gast, Reference Lane and Gast2014).

In contrast, NAP yields a percentage obtained by conducting a series of pairwise comparisons of values from the baseline and intervention condition (Pustejovsky, Reference Pustejovsky2016a). First, identify all possible pairs of data points by multiplying the number of data points in the baseline condition by the number of data points in the intervention condition (e.g., 5 data points in baseline and 7 in the intervention condition would yield 35 comparisons). Each value in the baseline condition would be compared to each value in the intervention condition and coded as overlapping with a data point in the baseline condition (scored as 1), non-overlapping (scored as 0), or a tie (scored as .5). For example, when comparing the first data point in a baseline condition (15%) to all possible data points in an intervention condition (sequential order: 14%, 13%, 15%, 20%, 25%, 30%, 45%), the following scores would be obtained: 1, 1, .5, 0, 0, 0, 0 = 2.5. This process would be repeated for all remaining comparisons, with all coded values summed (overlap sum) and subtracted from the total number of comparisons and multiplied by 100 (Parker, Vannest, & Davis, Reference Parker, Vannest and Davis2011). Free programmes for calculating PND, NAP, and other metrics are available (Pustejovsky, Reference Pustejovsky2016b).