We began (in Chapter 2) with arguments and proceeded (in Chapter 3) to conceptualization and measurement. In this chapter, we turn to the problem of how to analyze an argument empirically. This may be referred to variously as appraisal, assessment, corroboration, demonstration, empirics, evaluation, methods, proof, or testing. Pursued in a self-conscious fashion, this stage of research involves a research design, i.e., an explicit method of selecting and analyzing data.
We begin by introducing a set of key terms that are necessary to understand the construction of a research design. We proceed to a discussion of the general issues that all analyses encounter. This includes precision and validity, internal and external validity, sample representativeness, sample size, probability and non-probability sampling, and missing-ness. The terms and topics introduced in this chapter will enter the narrative in later chapters repeatedly. This chapter therefore plays a foundational role in the book.
Definitions
A standard empirical analysis involves a number of components, which must be clarified before we continue. Much of this vocabulary is borrowed from survey research. Nonetheless, the concepts are helpful in all styles of research, whether quantitative or qualitative, and are illustrated in Figure 4.1.
The most basic unit in any analysis is an observation. Observations are the pieces of evidence deemed relevant for demonstrating an argument. In a standard matrix (rectangular) dataset, an observation is usually represented as a row. Each row in Figure 4.1 represents a single observation.
Each observation should record values for all relevant variables. In causal analysis, this includes X (the causal factor of theoretical interest) and Y (the outcome of interest), along with any other variables deemed essential for the analysis. In a rectangular dataset, variables are usually represented with vertical lines. There are three variables in Figure 4.1: X, Z, and Y.
An observation is drawn from a unit or case – bounded entities such as individuals, organizations, communities, or nation-states, which may be observed spatially and/or temporally (through time). The terms unit and case are more or less equivalent. (While a unit is bounded spatially, a case may also have implicit or explicit temporal boundaries.)