The Little Prince articulates the invidious, de-humanizing element inherent in any attempt to measure, and thereby compare, human beings. “Treating them like statistics,” as the phrase goes. Abhorrent though it may seem (and surely, the measurement of intimate material and emotional states is an act of extreme hubris), there may also be good reasons for measuring, say, the incomes of families in a community.

Having laid out a framework of tasks, strategies, and criteria that define social science methodology, I now want to discuss how this framework maps onto the “paradigm wars” that have roiled the disciplines of social science over the past half-century.

For outsiders, as well as for many insiders, the distinctions evident across disciplines, methods, and schools are paramount. We sit at separate tables. Yet there are also many things that we share. Moreover, there is little profit in emphasizing our differences, given that the presumed objective of scientific deliberation is, ultimately, to reach consensus. Incommensurability is not conducive to a productive interchange of ideas. If taken seriously, it prohibits knowledge cumulation. Consequently, this book has emphasized the methodological coherence of the social sciences.

Preface

The field of social science methodology has been hyperactive over the past several decades. Methods, models, and paradigms have multiplied and transformed with dizzying speed, fostering a burst of interest in a heretofore moribund topic. One sign of the growing status of this field is the scholarly vituperation it inspires. Terms such as interpretivism, rational choice, poststructuralism, constructivism, randomization, positivism, and naturalism are not just labels for what we do; they are also fighting words.

Meanwhile, venerable debates over power, class, and status seem to have subsided. It is not that we no longer talk about these subjects, or care about them. Yet there appears to be greater consensus within the academy on normative political issues than there was, say, in the 1960s and 1970s. We are all social democrats now – for better, or for worse. Debates continue, especially over the role of race, gender, and identity. However, they do not seem to be accompanied by a great deal of rancor. Thus, over the past few decades methodological disagreements have largely displaced disagreements over substantive issues as points of conflict at conferences, at faculty meetings, and on editorial boards. Methodology, not ideology, seems to define the most important cleavages within the social sciences today.

The subject of this book is the set of disciplines known as the social sciences (which in earlier times would have been referred to as the moral or human sciences). By this is meant a scientific study of human action focusing on elements of thought and behavior that are in some degree social (nonbiological). “The object of the social sciences,” writes Hans Morgenthau, “is man, not as a product of nature but as both the creature and the creator of history in and through which his individuality and freedom of choice manifest themselves.” Wherever nurture matters more than nature, or where some significant decisional element is involved, we are on the turf of social science. (This does not mean that genetic dispositions are eliminated from consideration; indeed, they comprise an active research agenda in the social sciences today. However, one presumes that any outcome of interest to the social sciences is not entirely biologically determined; there must be a significant component of choice.)

Having discussed the formal (super-empirical) criteria of a good argument, we turn now to the empirical portion of social science research, the hoped-for encounter with reality. This stage may be referred to variously as analysis, assessment, corroboration, demonstration, empirics, evaluation, methods, proof, or testing. (While acknowledging the subtle differences among these terms, I shall treat them as part of the same overall enterprise.)

Of course, the distinction between theory formation and theory-testing is never clear and bright. As is the case everywhere in social science, tasks intermingle. One cannot form an argument without considering the empirical problem of how to appraise it, and vice versa. Moreover, the task of (dis)-confirming theories is intimately conjoined with the task of forming theories. As Paul Samuelson notes, “It takes a theory to kill a theory.”

I hope that my chosen approach to social science methodology strikes readers as commonsensical. Indeed, none of the tasks, strategies, and criteria were invented by the author (though I have chosen labels for things that do not have established names), and most have received extensive discussion. From this perspective, the present book qualifies as a compendium of truisms – a function shared, I might add, by any integrative work on methodology. The first, and perhaps most important, justification for the proposed framework is that it represents a formalization of what we already know.

Nevertheless, readers are bound to have qualms about some elements of the argument. They might take issue with the criterion of generality, for example. They might like what I have to say about description, but not about causation. On what grounds might one adjudicate this sort of dispute?

Chapter 9 set forth general criteria applying to research designs whose purpose is to assess causal relationships. In this chapter and Chapter 11 I lay out specific strategies of causal inference. To be sure, there is a high degree of overlap between these topics. Indeed, each strategy can be analyzed according to the criteria that it fulfills (or does not fulfill). Yet the shift from criteria to strategies is an important one. While Chapter 9 is about principles, Chapters 10 and 11 come closer to a “how-to” guide, complete with detailed discussions of specific studies.

Strategies of causal inference will be divided into three categories: (1) randomized designs; (2) nonrandomized designs; and (3) strategies that move beyond X and Y, as summarized in Table 10.1. Because this covers a lot of ground the material is divided into two chapters, with this chapter focused on the first two topics and Chapter 11 on the third.

I argued in Part II for a resuscitation of descriptive inference within the social sciences, both as a topic of methodology and as a topic of substantive research. However, I do not suppose that description will displace causation as the reigning motif of social science. We wish to know not only what happened but also, perhaps more critically, why these things happened.

Causation is the central explanatory trope by which relationships among persons and things are established – the cement of the universe, in Hume’s much quoted words. Without some understanding of who is doing what to whom we cannot make sense of the world that we live in, we cannot hold people and institutions accountable for their actions, and we cannot act efficaciously in the world. Without a causal understanding of the world it is unlikely that we could navigate even the most mundane details of our lives, much less matters of long-term policy. This is obvious in the policy world, where causal understanding undergirds any rational intervention. And it is obvious in other areas of politics, for example, in social movements, lobbying, voting, and revolutionary change. Anyone who engages in these activities must be conscious of oneself as a causal actor in the world, and accordingly, must make assumptions (implicit or explicit) about what one’s actions might achieve – whether one supports the status quo or wishes to undermine it. Lenin, like Metternich, was vitally concerned with the causes of revolution. Even where causal understanding does not lead to social change (for not all causal analysis is directly relevant to public policy, and more to the point, not all policy proposals are implemented), we are likely to be reassured when we can order events around us into cause-and-effect relationships. “When we have such understanding,” notes Judea Pearl, “we feel ‘in control’ even if we have no practical way of controlling things.”

I cannot resist inserting a few words on the stylistic properties of social science. Although not methodological in the strict sense of the term, it is nonetheless difficult to separate the desiderata of good writing from the desiderata of good argumentation. Kristin Luker comments:

In this respect, writing shares certain characteristics in all realms in which it is employed. Good writing is good thinking, to paraphrase Orwell.

A: (Knock)

B: Come in.

A: Ah, Is this the right room for an argument?

B: I told you once.

A: No you haven’t.

B: Yes I have.

A: When?

B: Just now.

A: No you didn’t.

B: Yes I did.

A: You didn’t

B: I did!

A: You didn’t!

B: I’m telling you I did!

A: You did not!!

B: Oh, I’m sorry, just one moment. Is this a five minute argument or the full half hour?

Monty Python, "The Argument Clinic"

Argumentation in contemporary social science descends from the ancient art of rhetoric and the equally ancient science of logic. A complete argument consists of a set of key concepts, testable hypotheses (aka propositions), and perhaps a formal model or larger theoretical framework. A causal argument should also contain an explication of causal mechanisms (Chapter 8). An argument is what we speculate might be true about the world; it engages the realm of theorizing.

Sometimes, it is important to distinguish among arguments lying at different levels of abstraction. The most abstract may be referred to as macro-level theories, theoretical frameworks, or paradigms. Examples would include structural functionalism, modernization theory, exchange theory, symbolic interactionism, or conflict theory. At a slightly less abstract level one finds meso-level theories or models. And at the most specific level one speaks of hypotheses, inferences, micro-level theories, or propositions, which are assumed to be directly testable. (Explanations may apply to any level.) So, for example, work on the topic of school vouchers might include a general theory about why consumer choice enhances the educational process, a formal model incorporating various elements of that theory, and a specific hypothesis or set of hypotheses regarding the impact of a voucher-based intervention on educational attainment.

We have now reviewed randomized and nonrandomized approaches to the assignment problem, and to causal analysis more generally. Our discussion has focused on the ways in which these strategies attempt to isolate the covariational pattern between a causal factor, X, and an outcome, Y. This pattern may be of many sorts: positive or negative, proximal or distal, and so forth. There are many types of causal relationships (as summarized in Table 9.2), and each presupposes a somewhat different covariational pattern. The key point is that X/Y covariation is a necessary condition of causality. If Y does not vary with X, at least some of the time and in some real or imagined sample, then X cannot be a cause of Y. Accordingly, the discovery of X/Y covariation may provide strong evidence of causality, if various assumptions hold.

However, the problem of confounders is ubiquitous when one is dealing with nonexperimental data. (Additionally, background factors lying orthogonal to the factor of theoretical interest may present an obstacle to causal inference if the signal is overwhelmed by stochastic noise.) Consequently, one is often obliged to move beyond an exclusive focus on X and Y. Other factors will need to be measured, and conditioned, if the covariation between X and Y is to be interpreted as evidence of a causal relationship.

Broadly stated, the goal of science is to discover new things about the world and to appraise the truth-value of extant propositions about the world. Consider our exemplars, democracy and vouchers, introduced in Chapter 1. We want to uncover new things about the process of democratization and the impact of vouchers on school performance. At the same time, we want to test extant theories about these two subjects. Social science may, therefore, be understood as a twin quest for discovery and for appraisal, as summarized in Table 2.1.

The chapter begins by introducing these goals, followed by a review of their implications for more specific methodological tasks. The next section approaches the goal of discovery through the concrete task of finding a research question. Since the remaining chapters of the book assume that a research question – perhaps even a specific hypothesis – has been identified, this chapter functions as a prologue to the rest of the book.

Description will be understood in this book as any empirical argument (hypothesis, theory, etc.) about the world that claims to answer a what question (e.g., how, when, whom, or in what manner). By contrast, wherever there is an implicit or explicit claim that a factor generates variation in an outcome the argument will be regarded as causal. The distinction between these two key concepts thus hinges on the nature of the truth-claim – not on the quality of the evidence at hand, which may be strong or weak. Description is the topic of Part II, while causation is the topic of Part III. Description rightly comes first; one must describe in order to explain (causally). However, the reader will find many comparisons and contrasts across the two topics interwoven throughout the book.

Because this book is focused on generalizing statements about the world (Chapter 1), I am not concerned with descriptions that reflect only on individual cases or events (without any attempt to exemplify larger patterns). Consequently, in this book description is always an inferential act. To generalize is to infer from what we know (or think we know) to what we do not know. One sort of inferential leap is from observations within a sample that are deemed secure to those that are uncertain or missing (problems of “measurement error” or “missing data”) and to dimensions that are inherently unobservable (“latent characteristics”). Another sort of inferential leap is from a studied case or sample to a larger (unstudied) population. In both respects, descriptive models offer a “theory” about the world, “a ‘formula’ through which the data can be reproduced.”