Economic experiments have long operated with a de facto ban on the use of deception. For example, a well-known book on experimental methodology from two decades ago (Friedman and Sunder, Reference Friedman and Sunder1994, p. 17) unambiguously states, “Do not deceive subjects or lie to them.” The ban certainly far precedes these books and likely dates back to early experiments by Vernon Smith and Charles Plott.Footnote 1 This is generally taken to encompass instructions or materials that actively mislead subjects by stating or strongly implying something that is not true. Common examples include telling subjects that they are playing games versus another subject when they are actually playing a confederate of the experimenter (or a computerized robot), paying subjects based on something other than the announced rules, or resolving random outcomes in a manner inconsistent with announced rules. Deception is generally considered a sin of commission rather than omission, so other experimental techniques that could arguably be classified as deception are considered acceptable. Examples include the use of deliberate ambiguity, where parts of the rules are not specified, not telling subjects what will happen in future portions of the experiment in cases where this information would likely affect current decisions, and using predetermined random draws. In none of these cases has the experimenter directly led the subject to believe something that is false.
The ban on deception stems primarily from the role of economic experiments in testing economic theory. The theoretical predictions in most settings are a function of the objective functions for individual decision makers. A test of these predictions is valid only if the experimenter has control over the incentives of the experimental subjects. If the subjects believe they or others will be paid according to some rule other than what they have been told, control is lost and the test is no longer valid. Deception potentially undermines the experimenter's control over payoffs. If I know that a specific experimenter has deceived me previously, why should I believe him or her now? If I know that deception is generally acceptable within a laboratory or field, why should I believe anything I am told? Once subjects doubt the veracity of the experimental instructions and materials, they can begin to form their own theories about how they will be paid. For example, there is a large literature studying individual choice under risk and uncertainty. This literature often features carefully constructed tests of prominent theories such as expected utility theory (EUT), subjective expected utility (SEU), or cumulative prospect theory (CPT). But suppose a subject choosing between lotteries believes, contrary to the instructions, that the lowest possible payoff will always be drawn to reduce the experimenter's expenditures. Any conclusions based on this subject's data are invalid. To avoid this loss of control, many experimenters not only avoid deception but go beyond this by employing techniques intended to make it clear that deception is not possible.Footnote 2 The standards are far different in psychology, where deception is generally treated as permissible, but few psychology experiments are so reliant on the subjects believing precisely what they are told by the experimenter.
If the negative effects of deception were tightly confined to the researchers using deception, concern within the experimental community might be muted. However, there are good reasons for believing that the use of deception generates negative externalities for other experimenters. No one experiment using deception destroys the reputation of a researcher, a laboratory, or the field as a whole, but the cumulative effect could be quite severe. Economic experimenters worry that any tolerance for deception will lead to a chain reaction that destroys the credibility of all economic experimenters. Research on the topic has provided mixed support for experimenters' concern. Direct experience with deception does appear to affect subjects' beliefs and decisions in future experiments (e.g., Jamison et al., Reference Jamison, Karlan and Schechter2008; Ortmann and Hertwig, Reference Ortmann and Hertwig2002), but there is little evidence of indirect effects akin to the poisoning of the well feared by many experimental economists. However, problems remain even if the problem is localized. As a referee or editor, how do I evaluate experiments without deception coming from a laboratory where deception is employed? Can I be confident that the subjects in the current experiment are truly responding to the incentives in the experiment if they are likely to have directly experienced deception in another experiment run by the same laboratory? If the primary value of economic experiments comes from having controlled environments where causal relationships can easily be established, the risk of destroying that control justifies an excess of caution by experimenters.
That said, only an extremist would claim that experimenters (or economists in general) should never use deception. Labor economics, for example, has benefited from a long series of field experiments on discrimination that use deception (for a survey, see Riach and Rich, Reference Riach and Rich2002). While I cannot claim to speak for the entire experimental community, the following four rules provide a reasonable guide for when experiments with deception should be allowable:
1. The deception does not harm subjects beyond what is typical for an economic experiment without deception.
2. The study would be prohibitively difficult to conduct without deception.
3. Subjects are adequately debriefed after the fact about the presence of deception.
4. The value of the study is sufficiently high to merit the potential costs associated with the use of deception.
I was asked to write this editorial as a comment on two papers that appeared in the JWE, Lewis and Zalan (Reference Lewis and Zalan2014) and Mastrobuoni and colleagues (Reference Mastrobuoni, Peracchi and Tetenov2014). Both contain clear examples of deception. In Lewis and Zalan, subjects were deceived into thinking that identical wines were actually different wines being sold at different prices. Mastrobuoni et al., deceive subjects about the price and origin of wines. The question is: How should a peer-reviewed journal treat experimental papers that contain deception?
I should begin by noting that, as the current editor-in-chief for Experimental Economics, I would have desk-rejected these papers due to the use of deception. Experimental Economics is the flagship journal of the Economic Science Association, the primary professional organization for experimental economists, and it would send the wrong signal to accept a paper with blatant deception. These concerns do not apply to the JWE.
Looking at the four rules I have described above, both studies clearly pass the first test. Lewis and Zalan explicitly state that a debriefing was conducted after the fact, passing the third test. Mastrobuoni and colleagues use data taken from an earlier paper (Tempesta et al., Reference Tempesta, Arboretti Cristofaro, Corain, Salmaso, Tomasi and Boatto2010). While Tempesta and colleagues say nothing one way or the other about debriefing, it seems likely that they debriefed subjects since this is a standard procedure in psychology. It can be argued that both papers fail the second test. Suppose that the goal is to manipulate tasters' perceptions of the price of the wine. You can give a price range for the bottle of wine being sampled (i.e., this bottle costs between 5 and 10 dollars). While all price ranges would contain the true price of the wine, by changing the endpoints of the range you can change subjects' perceptions of price. This would certainly lead to noisier data and make it harder to detect effects, but would avoid the use of deception. This example illustrates the seductive nature of using deception in experiments—for many issues, deception is not the only way to generate useful data, but it is often the easiest and least expensive way. It is a matter of judgment as to whether the difficulty of generating data using nondeceptive techniques is sufficient to make deception permissible, but in my opinion the default should be that deception is not acceptable without providing a strong case to justify its use. As for the fourth test, this, too, is largely a matter of opinion. Most economists would argue that the discrimination studies using deception were sufficiently important (and sufficiently unlikely to generate an externality for other experiments) that the use of deception was justified. At the same time, while Milgram's (Reference Milgram1963) famous experiments on obedience to authority were enormously informative, I doubt anyone would argue that this justified the harm caused to subjects by the deception. And Lewis and Zalan or Mastrobuoni and colleagues? The value of these studies is ultimately in the eye of the beholder. The costs are likely to be low. The subjects are not from populations that are likely to be involved in future experiments or interact with people in future experiments, although it would be reasonable to question future results from these researchers (especially if they study the same populations). The reputation of experimental economics is probably little affected by papers published in JWE, and evidence for an indirect effect is weak in any case. The correct decision is not clear, but publishing the papers is certainly a justifiable decision. I hope that this note has served to make it clear what the main issues are and why caution must be exercised.
