No CrossRef data available.
Article contents
Quality Meets Quantity: Case Studies, Conditional Probability, and Counterfactuals
Published online by Cambridge University Press: 01 June 2004
Abstract
In contrast to statistical methods, a number of case study methods—collectively referred to as Mill's methods, used by generations of social science researchers—only consider deterministic relationships. They do so to their detriment because heeding the basic lessons of statistical inference can prevent serious inferential errors. Of particular importance is the use of conditional probabilities to compare relevant counterfactuals. A prominent example of work using Mill's methods is Theda Skocpol's States and Social Revolutions. Barbara Geddes's widely assigned critique of Skocpol's claim of a causal relationship between foreign threat and social revolution is valid if this relationship is considered to be deterministic. If, however, we interpret Skocpol's hypothesized causal relationship to be probabilistic, Geddes's data support Skocpol's hypothesis. But Skocpol, unlike Geddes, failed to provide the data necessary to compare conditional probabilities. Also problematic for Skocpol is the fact that when one makes causal inferences, conditional probabilities are of interest only insofar as they provide information about relevant counterfactuals.Jasjeet S. Sekhon thanks Walter R. Mebane Jr., Henry Brady, Bear Braumoeller, Shigeo Hirano, Gary King, John Londregan, Bruce Rusk, Theda Skocpol, Suzanne M. Smith, Jonathan N. Wand, the editors of Perspectives on Politics, and three anonymous reviewers for valuable comments and advice.
Information
- Type
- Research Article
- Information
- Copyright
- © 2004 American Political Science Association
References
REFERENCES
Barnard, John,
Constantine E. Frangakis,
Jennifer L. Hill, and
Donald B. Rubin.
2003.
Principal stratification approach to broken randomized experiments:
A case study of school choice vouchers in New York City.
Journal of the American Statistical Association
98:462,
299–323.Google Scholar
Bartels, Larry M.
1996.
Pooling disparate observations.
American Journal of Political Science
40:3,
905–42.Google Scholar
Bennett, Andrew.
1999.
Causal inference in case studies: From Mill's methods to causal
mechanisms.
Paper presented at the annual meeting of the American Political
Science Association. Atlanta, 2–5 September.
Brady, Henry.
2002.
Models of causal inference: Going beyond the Neyman-Rubin-Holland
Theory.
Paper presented at the nineteenth annual Summer Political Methodology
Meetings, Seattle, 18–20 July.
Braumoeller, Bear F., and
Gary Goertz.
2000.
The methodology of necessary conditions.
American Journal of Political Science
44:4,
844–58.Google Scholar
Braumoeller, Bear F., and
Gary Goertz
2002.
Watching your posterior: Comments on Seawright.
Political Analysis
10:2,
198–203.Google Scholar
Burawoy, Michael.
1989.
Two methods in search of science: Skocpol versus Trotsky.
Theory and Society
18:6,
759–805.Google Scholar
Campbell, Donald T., and
Julian C. Stanley.
1966.
Experimental and Quasi-Experimental Designs for Research.
Boston:
Houghton Mifflin.
Clarke, Kevin A.
2002.
The reverend and the ravens: Comment on Seawright.
Political Analysis
10:2,
194–7.Google Scholar
Cohen, Morris R., and
Ernest Nagel.
1934.
An Introduction to Logic and Scientific Method.
New York:
Harcourt, Brace.
Collier, David.
1995.
Translating quantitative methods for qualitative researchers: The
case of selection bias.
American Political Science Review
89:2,
461–6.Google Scholar
Collier, David, and
James Mahoney.
1996.
Insights and pitfalls: Selection bias in qualitative research.
World Politics
49:1,
56–91.Google Scholar
Dawid, A. Philip.
2000.
Causal inference without counterfactuals (with discussion).
Journal of the American Statistical Association
95:450,
407–48.Google Scholar
Dion, Douglas.
1998.
Evidence and inference in the comparative case study.
Comparative Politics
30:2,
127–46.Google Scholar
Eckstein, Harry.
1975.
Case study and theory in political science. In
Handbook of Political Science. Vol. 7 of Strategies of
Inquiry, ed.
Fred I. Greenstein,
Nelson W. Polsby.
Reading, Mass.:
Addison-Wesley,
79–137.
Geddes, Barbara.
1990.
How the cases you choose affect the answers you get: Selection bias in
comparative politics.
Political Analysis
2,
131–50.Google Scholar
George, Alexander L., and
Timothy J. McKeown.
1985.
Case studies and theories of organizational decision-making. In
Advances in Information Processing in Organizations, eds.
Robert F. Coulam and
Richard A. Smith.
Greenwich, Conn.:
JAI Press,
21–58.
Gerber, Alan S., and
Donald P. Green.
2000.
The effects of canvassing, telephone calls, and direct mail on
voter turnout: A field experiment.
American Political Science Review
94:3,
653–63.Google Scholar
Goldstone, Jack A.
1991.
Revolution and Rebellion in the Early Modern World.
Berkeley:
University of California Press.
Goldstone, Jack A.
1997.
Methodological issues in comparative macrosociology.
Comparative Social Research
16,
107–20.Google Scholar
Herron, Michael C., and
Jasjeet S. Sekhon.
2003.
Overvoting and representation: An examination of overvoted
presidential ballots in Broward and Miami-Dade Counties.
Electoral Studies
22:1,
21–47.Google Scholar
Herron, Michael C., and
Jasjeet S. Sekhon.
Forthcoming.
Black candidates, and black voters: Assessing the impact of candidate
race on uncounted vote rates.
Journal of Politics.
Holland, Paul W.
1986.
Statistics and causal inference.
Journal of the American Statistical Association
81:396,
945–60.Google Scholar
Imai, Kosuke.
Forthcoming.
Do get-out-the-vote calls reduce turnout? The importance of statistical
methods for field experiments.
American Political Science Review.
Lieberson, Stanley.
1991.
Small N's and big conclusions: An examination of the reasoning
in comparative studies based on a small number of cases.
Social Forces
70:2,
307–20.Google Scholar
Lieberson, Stanley
1994.
More on the uneasy case for using Mill-type methods in small-N
comparative studies.
Social Forces
72:4,
1225–37.Google Scholar
Little, Daniel.
1998.
Microfoundations, Method, and Causation: On the Philosophy of the
Social Sciences.
New Brunswick, N.J.:
Transaction Publishers.
Mahoney, James.
1999.
Nominal, ordinal, and narrative appraisal in macrocausal analysis.
American Journal of Sociology
104:4,
1154–96.Google Scholar
McKeown, Timothy J.
1999.
Case studies and the statistical worldview: Review of King,
Keohane, and Verba's Designing Social Inquiry: Scientific
Inference in Qualitative Research.
International Organization
51:1,
161–90.Google Scholar
Mebane, Walter R., Jr., and
Jasjeet S. Sekhon.
2004.
Robust estimation and outlier detection for overdispersed
multinomial models of count data.
American Journal of Political Science
48:2,
391–410.Google Scholar
Mill, John Stuart.
1872 [1843].
A System of Logic, Ratiocinative and Inductive: Being a Connected
View of the Principles of Evidence, and the Methods of Scientific
Investigation.
8th ed.
London:
Longmans, Green.
Pledge, Humphrey Thomas.
1939.
Science since 1500: A Short History of Mathematics, Physics,
Chemistry, [and] Biology.
London:
His Majesty's Stationery Office.
Przeworski, Adam, and
Henry Teune.
1970.
The Logic of Comparative Social Inquiry.
New York:
Wiley-Interscience.
Ragin, Charles C.
2000.
Fuzzy-Set Social Science.
Chicago:
University of Chicago Press.
Ragin, Charles C.,
Dirk Berg-Schlosser, and
Gisèle de Meur.
1996.
Political methodology: Qualitative methods. In
A New Handbook of Political Science, ed.
Robert E. Goodin,
Hans-Dieter Klingemann.
New York:
Oxford University Press, and
749–68.
Robinson, W. S.
1951.
The logical structure of analytic induction.
American Sociological Review
16:6,
812–8.Google Scholar
Rubin, Donald B.
1974.
Estimating causal effects of treatments in randomized and
nonrandomized studies.
Journal of Educational Psychology
66:5,
688–701.Google Scholar
Rubin, Donald B.
1978.
Bayesian inference for causal effects: The role of randomization.
Annals of Statistics
6:1,
34–58.Google Scholar
Rubin, Donald B.
1990.
Comment: Neyman (1923) and causal inference in experiments and
observational studies.
Statistical Science
5:4,
472–80.Google Scholar
Salmon, Wesley C.
1989.
Four Decades of Scientific Explanation.
Minneapolis:
University of Minnesota Press.
Seawright, Jason.
2002a.
Testing for necessary and/or sufficient causation: Which cases
are relevant?
Political Analysis
10:2,
178–93.Google Scholar
Seawright, Jason
2002b.
What counts as evidence? Reply.
Political Analysis
10:2,
204–7.Google Scholar
Sekhon, Jasjeet S.
2003.
Making inferences from 2×2 tables: The inadequacy of the Fisher
Exact Test and a reliable Bayesian alternative.
Working paper. Available at
jsekhon.fas.harvard.edu/papers/SekhonTables.pdf. Accessed 15 December
2003.
Sewell, William H.
1996.
Three temporalities: Toward an eventful sociology. In
The Historic Turn in the Human Sciences, ed.
Terrence J. McDonald.
Ann Arbor:
University of Michigan Press,
245–80.
Skocpol, Theda.
1979.
States and Social Revolutions: A Comparative Analysis of France,
Russia, and China.
Cambridge:
Cambridge University Press.
Skocpol, Theda
1984.
Emerging agendas and recurrent strategies in historical
sociology. In
Vision and Method in Historical Sociology, ed.
Theda Skocpol.
New York:
Cambridge University Press,
356–91.
Splawa-Neyman, Jerzy.
1990 [1923].
On the application of probability theory to agricultural
experiments. Essay on principles. Section 9.
Trans. D. M. Dabrowska and T. P. Speed.
Statistical Science
5:4,
465–72.Google Scholar
Stigler, Stephen M.
1986.
The History of Statistics: The Measurement of Uncertainty Before
1900.
Cambridge:
Harvard University Press.
Vandenbroucke, Jan P.
2001.
In defense of case reports and case series.
Annals of Internal Medicine
134:4,
330–4.Google Scholar
Waldner, David.
2002.
Anti anti-determinism: Or what happens when Schrodinger's cat and
Lorenz's butterfly meet Laplace's demon in the study of political
and economic development.
Paper presented at the annual meeting of the American Political
Science Association, Boston, 29 August–1 September.
- 47
- Cited by