In the previous chapter, we discussed coding schemes and gave several examples. We emphasized that coding schemes are primarily conceptual, rooted in and reflecting theoretical assumptions; and, although we regard coding schemes as instruments that focus observers’ attention, they are not physical in the sense that microscopes are. In contrast, applying those coding schemes to the passing stream of behavior – transforming observed behavior into data – combines physical and conceptual components. Recording observational data requires physical materials ranging from simple paper and pencil to sophisticated video and computer systems. It also requires selection of an appropriate recording strategy. This chapter addresses both the strategies and materials used to record observational data.

Recording strategies provide rules that observers must follow when applying coding schemes to the stream of behavior; they serve both measurement and control. For experimental variables, control means variable manipulation or neutralization – for example, via random assignment. For observational variables, control is exerted by requiring observers to adhere to specific recording rules as detailed in this chapter.

Sometimes a book merely rests unnoticed or waits unused or stands lifeless on a shelf – although authors usually hope for more. In the hands of readers, a book becomes many books; and even in the hands of a single reader, it can be a different book each time it is picked up anew – like the same river never stepped in twice. Readers bring to a book a personal history and understanding of a topic, guided by necessity and motivation. Some, for their own edification or on a teacher’s instruction, may have read this book from start to finish. Others may have skipped around a bit, looking for parts relevant to them. And still others may have used it simply for reference.

Whatever your previous history or current motivation – whether a novice seeking an overall understanding, a curious researcher seeking to expand your repertoire, a relatively experienced investigator wanting answers to specific questions, or even a fellow author examining how and how well we have addressed various topics – we hope this book has expanded your knowledge and understanding of sequential analysis and observational methods. Used appropriately, they can be cost effective and allow you to address research questions with unique fidelity. If you choose to use them, we hope this book will speed you in your work.

Once observers have done their work – that is, once their assignment of codes to events or intervals has been committed to paper or electronic files – it is tempting to think that you can now move directly to analysis of the coded data. Almost always this is premature because it bypasses two important intervening steps. The second step involves reducing sequential data for a session into summary scores for subsequent analysis and is relatively well understood; for details see Chapters 8 and 9. The first step is equally important but often receives less attention. It is the subject of this chapter and involves representing – literally, re-presenting – the data as recorded initially into formats that facilitate subsequent data reduction and analysis.

When recording observational data, as described in the preceding chapter, observer ease and accuracy are paramount, and methods and formats for recording data appropriately accommodate these concerns. But when progressing to data preparation, reduction, and analysis, different formats may work better. In this chapter, we consider two levels of data representation. The first is a standard format – that is, a set of conventions – for sequential data that defines five basic data types and reflects the recording strategies described in the previous chapter. The second is more conceptual; it is a way of thinking about sequential data in terms of a universal code-unit grid that applies to all data types and that facilitates data analysis and data modification, as demonstrated in subsequent chapters (especially in Chapter 10).

As telescopes are for astronomy and microscopes for biology, so coding schemes are for observational methods: They bring the phenomena of interest into focus for systematic observation. However, unlike telescopes and microscopes, which are clearly physical, coding schemes are primarily conceptual. They consist of codes (i.e., names, labels, or categories) that can be applied to whatever behavior is being studied. By naming behavior, coding schemes limit the attention of our observers and state, in effect, that the aspects of behavior identified by the codes are important for our investigation and are the aspects on which we should focus.

Where Do Coding Schemes Come From?

In addition to being conceptual, coding schemes necessarily make theoretical commitments. Implicit in any coding scheme is the understanding that certain behaviors are important and certain distinctions are worth making. Necessarily, coding schemes reflect the investigator’s theory about what is important and why, even when investigators do not make the links between theories and codes explicit. Bakeman and Gottman (1986, 1997) wrote that using someone else’s coding scheme was like wearing someone else’s underwear. They used this attention-grabbing simile to make a point: Codes and underlying theories need to connect. Borrow, or more typically adapt, coding schemes from others only when you share theories, underlying theoretical orientations, and common research goals.

It makes sense to define codes and use recording procedures that work best for the observers. After all, if we expect good work, we should accommodate observers’ comfort; the data as recorded can be modified later into forms that facilitate analysis. In Chapter 4, we argued the utility of representing observational data with a few standard formats (i.e., single-code event, timed-event, interval, and multicode event data) and then conceptualizing such data as a code-unit grid – partly for the order and organization doing so brings to observational data, but also for the opportunities it presents for later data modification. In this chapter we describe, among other matters, data modification, that is, specific ways new codes can be created from existing codes – new codes that are faithful to and accurately reflect our research questions and that extend the range of our data-analytic efforts.

Given the benefits, it is a bit puzzling that data modification of observational data is not more common. Perhaps it is because data modification occupies something of a middle ground. On the one hand, there are a number of systems for computer-assisted coding that facilitate the initial recording of observational data; most produce the sorts of summary scores described in Chapter 8. On the other hand, there are a number of statistical packages that permit often quite complex data recoding and transformation of summary scores. But no coding or analysis programs we know of address the need to modify sequential data before summary scores are computed. In this respect, GSEQ, with its extensive and flexible data modification capabilities, may be uniquely helpful.

The analytic techniques presented so far have relied primarily on summary scores derived from sequential data; depending on the technique, they could be applied to one or more of the data types that we have described (i.e., single-code event, timed-event, interval, and multicode event data). In this final chapter we describe two additional techniques for detecting pattern. One is primarily graphic and can be applied to any kind of sequence; the other is a statistical approach to detecting pattern in relatively short single-code event sequences and requires few assumptions.

Recurrence Analysis

In this section we consider techniques that rely on whole sequences to display patterns graphically. Exploring a sequence as a whole can provide new insight into any patterns that may exist, where in the sequence they occur, and even whether they tend to repeat in different, but comparable, sequences. Moreover, such explorations can also be applied to two sequences to determine whether certain codes tend to repeat in both, thus revealing a possible synchronicity.

As noted in earlier chapters, measuring instruments for observational methods consist of coding schemes in the hands (and minds and eyes) of trained observers. Like all measuring instruments, they need to be calibrated; we need to be assured that the instruments are accurate. With human observers, this means demonstrating that two observers independently coding the same stream of behavior produce essentially the same results – or, perhaps better, that an observer agrees with a gold standard, a version that has been prepared by experts and is presumed accurate. It is not an exaggeration to call such demonstrations of observer agreement the sine qua non of observational methods. Without such demonstrations, we are left with individual narratives – perhaps fascinating, perhaps insightful, perhaps useful for generating research questions – but nonetheless narratives of unknown reliability.

Nothing is absolute, and a suitable level of agreement between two independent observers does not by itself guarantee accuracy: After all, two observers could share similar deviant views of the world. But almost always in the behavioral research world, observational methods require attention to and demonstration of observer agreement – either two observers with each other or one observer with a gold standard.

As noted early in Chapter 5, point-by-point agreement is often accepted as evidence that summary measures derived from sequential data will be reliable, probably because point-by-point agreement seems the more stringent approach. Kappa-based demonstrations of observer agreement are essential for observer training and for checking throughout coding; but once data collection is complete and summary measures have been derived from the sequential data, their reliability can be assessed directly with an intraclass correlation coefficient (ICC).

Relative Versus Absolute Agreement

Summary measures in observational research are typically of two kinds: first are frequencies, rates, and proportions for individual codes (see Chapter 8); and second are contingency indices and other contingency table statistics involving two or more codes (see Chapter 9). Both are continuous (i.e., measured on an interval or ratio scale), and so an ICC measure of reliability, which requires continuous or arguably ordinal measurement, is appropriate. The general question is: When values are derived from records of the same events coded independently by different observers, do the observers agree sufficiently for our purposes? Can we regard the observers as essentially interchangeable? In this regard, one important distinction concerns relative, as opposed to absolute, judgment (i.e., relative consistency vs. absolute agreement, or norm-referenced vs. criterion-referenced; see Suen, 1988). Do we want scores derived from two observers’ records to be rank-ordered the same way – that is, to be in relative agreement – or do we want them to agree regarding absolute magnitude?

The summary statistics described in the previous chapter could be called one-dimensional because they were computed for individual codes. In contrast, the statistics described in this chapter could be called two-dimensional because they are derived from two-dimensional contingency tables whose rows and columns are defined with two or more codes. Still, the overall purpose is the same: Summary statistics are computed for individual sessions, and those scores are then described and analyzed using whatever design and statistical procedures are appropriate.

Statistics derived from two-dimensional contingency tables are of three kinds. First are statistics for individual cells; these are primarily descriptive. Second are summary indices of independence and association for tables of varying dimensions (e.g., Pearson chi-square and Cohen’s kappa); these are generally well known or, in the case of kappa, already described in Chapters 5 and 6. Third, and most important for sequential analyses, are summary statistics specifically for 2×2 tables; these contingency indices often turn out to be the best way to address sequential questions as detailed in subsequent chapters.

In the previous chapter we introduced Cohen’s (1960) classic kappa and discussed a number of issues associated with it and its use. In this chapter we describe how kappa can be used with each of the data types described in Chapter 4. The basic kappa computation remains basically unchanged, but some data types require matching and tallying procedures that differ from the classic Cohen’s kappa. Where procedures differ, we regard the result as a variant of Cohen’s kappa and, to make the distinction clear, provide a name for that variant. To provide orientation (and to let you select which sections of this chapter may best fit your circumstances), Figure 6.1 links data types with their appropriate kappa variant. The remaining sections of this chapter detail appropriate kappa procedures for each data type.

Event-Based Agreement: the Alignment Problem

When events are demarcated before coding, as turns in a transcript, the classic Cohen’s kappa described in the previous chapter is appropriate. However, when observes first segment the stream of behavior into events – that is, decide where the seams between events occur – and only then code the events, the situation is more complicated.

Observing behavior – the central concern of this book – is an ancient human endeavor without which even our survival could become problematic. What will the beast we hope to stalk, kill, and bring back to the tribe do next? Is that attractive and suitable mate open to my advances? Is that child in trouble and in need of our help?

Not all questions modern researchers pose will be as dramatic as these, and as behavioral scientists search for answers, self-conscious, systematic observational methods will come to supplant raw observation. But what exactly do we mean by observational methods? A definition is in order. In an expansive vein, the eighteenth-century historian William Douglass wrote, “As an historian, every thing is in my province” (1760, p. 230). Similarly, following the nineteenth-century physiologist Claude Bernard (1865/1927), the present-day behavioral scientist could say: Everything I know and do begins with observation. I observe and describe the gait of the horse. I observe and record the infant’s weight. I observe whether my participants check strongly agree, simply agree, or some other choice on a questionnaire.

The phrase, sequential analysis – which appears in the title of this book as well as earlier ones (Bakeman & Gottman, 1997; Bakeman & Quera, 1995a) – admits to more than one meaning. In the context of microbiology, it can refer to the description of genetic material. In the context of statistics, it can mean sequential hypothesis testing – that is, evaluating data as they are collected and terminating the study in accordance with a predefined stopping rule once significant results are obtained (Siegmund, 1985).

However, in the context of observational methods generally, and in the context of this book specifically, sequential analysis refers to attempts to detect patterns and temporal associations among behaviors within observational sessions. As such, sequential analysis is more a toolbox of techniques than one particular technique. It can include any of a variety of techniques that serve its goals. Some of these techniques have already been discussed (e.g., “Contingency indices for 2×2 tables” in Chapter 9). The unifying factor is the data used; by definition, sequential analysis is based on sequential data – data for which some sort of continuity between data points can be assumed. Indeed, a common thread throughout this book has been the description and use of the four basic sequential data types we defined in Chapter 4 – single-code event, timed-event, interval, and multicode event sequential data.