2015

Jens Hainmueller (Massachusetts Institute of Technology), Dan Hopkins (Georgetown University), and Teppi Yamamoto (Massachusetts Institute of Technology) Causal Inference in Conjoint Analysis,. (Vol 22: 1-30)

Citation

This excellent paper had the unanimous support of the selection committee for the Miller Prize, awarded to the best article published in Political Analysis in 2014. The paper develops a framework for defining and estimating causal effects in conjoint survey experiments. Whereas survey experimental treatments such as vignettes necessarily bundle many components, conjoint design independently manipulates distinct elements of those treatments, allowing analysts to assess and compare each elements effect on responses or choices. Rarely are our treatments unidimensional, yet it is often difficult to isolate the influence of particular dimensions or components of treatment. Conjoint analysis offers a promising tool in this regard, and it has therefore seen growing recent use in political science, in part due to the substantive work of co-authors on this paper. This method should not only improve how we study political questions, but should apply to numerous fields as well.

The paper makes several important methodological contributions that go well beyond what has been accomplished elsewhere, for example, in the marketing research literature in which conjoint analysis was initially developed. First, it formally analyzes the causal properties of conjoint analysis using the Neyman-Rubin potential outcomes framework. The authors use this framework to define a causal quantity of interest the average marginal component effect which is the marginal effect of each attribute averaged over the joint distribution of other attributes included in the experiment. They define effects for two different kinds of conjoint experimental designs, those in which respondents are forced to make a choice between two profiles (e.g. two candidates), and ratings-based outcomes in which respondents express degree of preference for one or more manipulated profiles. Second, they clearly explain how this effect can be identified under assumptions that either must hold due to the experimental design or are partially empirically testable. The paper then provides a series of diagnostic tests to assess these identification assumptions (for example, to test whether there are indeed no profile or attribute order effects). Importantly, in contrast to existing approaches in econometrics or marketing research that assume a particular behavioral model and define and estimate parameters in terms of that model approaches which have the advantage of statistical efficiency but the disadvantage that the assumed model must be true in this paper the identification results allow inferences about causal quantities without resorting to strong functional form and other assumptions. Finally, the authors illustrate the technique using two interesting examples, from studies of voting and immigration, and they offer sample R and Stata scripts as well as a graphical user interface for conjoint survey design.

In sum, the articles merit lies in (1) introducing a simple, design-based approach to analysis of an important experimental design; and (2) carefully discussing assumptions, advantages, examples, and especially limitations of the approach. We believe the paper will be widely used and widely cited, and deservingly so. We are happy to award the prize to this excellent piece of work.

Miller Prize