2 RCL’s scale variants

RCL counterbalanced (at 3-week intervals) two different administrations of the seven-item pathogen subscale of the Three-Domain Disgust Scale (TTDS-P; Reference Tybur, Lieberman and GriskeviciusTybur, Lieberman & Griskevicius, 2009). One administration asked participants to respond to each item using the standard response scale (“Please rate how disgusting you find the concepts described in the items…”; from 0 “Not at all” to 6 “Extremely”). The other administration asked participants to respond to each item using twelve different response scales (in random order). These were used to create six different scale variants: “oral inhibition” (3 items), “disapproval” (3 items), “epidermal discomfort” (2 items), “creeped out” (1 item), “run away” (1 item), and “cry” (1 item).Footnote ¹ For the multiple-item scale variants (oral inhibition, disapproval, and epidermal discomfort), RCL first averaged the items in each variant and then reported correlations of the results with other measures.

3 RCL’s scale variants are theoretically questionable

RCL’s theoretical arguments focus on two of these scale variants: the “oral inhibition” variant, which asked participants to report how much each item “makes me feel physically nauseated,” “makes me gag,” and “makes me lose my appetite”; and the “creeped out” variant, which asked participants to report how much each item “makes me feel creeped out.” RCL argue that “oral inhibition” is “a construct logically derived from the theoretical meaning of disgust as a category of food rejection” (p. 469), and that therefore the oral inhibition items are the only ones measuring “true” disgust sensitivity. The problem with this reasoning is that most theorists no longer see disgust as simply a category of food rejection, but rather as a behavioral adaptation to the threat of disease-causing pathogens and parasites (Reference Murray and SchallerMurray & Schaller, 2016; Reference Schaller and ParkSchaller & Park, 2011; Reference Tybur, Lieberman and GriskeviciusTybur, Lieberman & Griskevicius, 2009; Reference Tybur and LiebermanTybur & Lieberman, 2016). Certainly, one pathogen exposure route is oral. But pathogen exposure can also occur in other ways — for example, by skin contact with pathogen-laden substances or organisms, or by contact with infected people. Therefore, the disgust response extends beyond oral rejection to avoidance and rejection of a wide range of possible sources of contamination (Reference CurtisCurtis, 2014; Reference Curtis, Aunger and RabieCurtis, Aunger, & Rabie, 2004; Reference Oaten, Stevenson and CaseOaten, Stevenson & Case, 2009). Consequently, more recent theoretical accounts of disgust have described the behavioral response to pathogen threats as much broader than oral inhibition, including “withdrawal, distancing, stopping or dropping the object of disgust” (Reference Curtis, De Barra and AungerCurtis, de Barra & Aunger, 2011, p. 390) or “avoid[ing] physical contact with sources of infectious disease-causing organisms” (Reference Tybur, Lieberman, Kurzban and DeScioliTybur, Lieberman, Kurzban & DeScioli, 2013, p. 69).

Indeed, of the seven TDDS-P items, only one is plausibly related to oral contamination (“Seeing some mold on old leftovers in your refrigerator”). The other six items describe non-oral contamination (e.g., “touching a person’s bloody cut,” “sitting next to someone who has red sores on their arm,” “stepping on dog poop”). A priori, response scales measuring oral inhibition seem like a poor fit for these non-oral items, which seem more likely to evoke avoidance and distancing. It may be that RCL are right, and that oral inhibition is the dominant response even to non-oral pathogen threats. But this is something that RCL need to show, rather than assuming it to be true. At the least, very least, RCL need to provide evidence that that their modified scale variant has construct validity (by showing, say, that the scale predicts things it should and not things it shouldn’t).

4 What does “creeped out” mean?

Next, consider RCL’s “creeped out” scale variant, which asks participants to rate how much each TDDS-P item “makes me feel creeped out.” They claim that this scale variant measures “fear associated with uncertainty of [sic] threat” (p. 469), but their evidence for this is not strong. RCL report data from a post-test where people are asked about their understanding of this slang term in isolation, not in the context of the TDDS-P items. But there is no reason to think that this is how participants are using it when they are rating the TDDS-P items. In fact, there is good reason to think otherwise. RCL appear to have adapted many of their items, including “creeped out,” from Reference Blake, Yih, Zhao, Sung and Harmon-JonesBlake et al. (2017). This previous work finds that in response to certain stimuli such as skin lesions, cockroaches, and spiders, feeling “creeped out” is part of an avoidance response that is theoretically and empirically different from fear (Reference Blake, Yih, Zhao, Sung and Harmon-JonesBlake et al., 2017). RCL’s use of this term to (putatively) measure fear in response to pathogen threats is therefore inconsistent with the previous work on which they draw.

In sum, both the “oral inhibition” and “creeped out” scale variants are based on strong assumptions that are at odds with past theory and findings. They may measure what RCL say they do, but there is currently not sufficient reason to believe this to be the case.

5 RCL’s disgust scale variants are statistically unjustified

RCL give no evidence of construct validity for their scale variants. For the multi-item variants, they report coefficient alphas, but those are not sufficient to establish validity on their own (Reference CortinaCortina, 1993; Reference SchmittSchmitt, 1996). Because three of RCL’s six scale variants have only one item, we could not perform confirmatory factor analyses to obtain quantitative measures of how well RCL’s proposed division of items fits the data (CFA requires at least two and preferably three or more indicator variables for each latent construct; Reference BrownBrown, 2006). We did perform exploratory factor analyses, which never showed a pattern consistent with RCL’s division of items (i.e., six factors). For each of the seven TTDS-P questions, we conducted a parallel analysis test (Reference HornHorn, 1965) on the 11 response scales to determine the number of factors to retain. Parallel analysis is preferred by methodologists over the widely-used “Eigenvalue > 1” rule, which can often give inaccurate results (Reference Fabrigar, Wegener, MacCallum and StrahanFabrigar, Wegener, MacCallum & Strahan, 1999).

For five of the seven questions, parallel analysis suggested retaining three factors. For one question (“sores”), it suggested retaining two factors. For the remaining question (“mold”), it suggested retaining four. It never suggested retaining six. Of course, psychometric analyses such as the ones we report here are not conclusive evidence of (in)validity. However, they are a reason to question RCL’s division of response scales into six variants. (Interested readers may consult the Supplemental Material for the parallel analysis and factor loadings for each question.)

6 Is GM scary?

RCL’s central argument is that people’s predominant response to GM food is fear, not disgust. They never test this hypothesis directly, but, as it happens, we are able to. In one of our unpublished studies, we showed nationally-representative groups of American, German, and French respondents (N = 1,559) two short scenarios describing consumption of GM food. One scenario described someone eating a GM apple, the other described someone eating GM salmon. After reading each scenario, participants were asked to choose whether a fear, disgust, anger, or neutral face would “most closely match [their] facial expression upon viewing this situation.” For both plant and animal scenarios, disgust faces were chosen more frequently than fear faces, Fear_plant = 18.9%, Disgust_plant = 22.84%; Fear_animal = 19.82%, Disgust_animal = 31.30%. Both these differences were significant, χ²(1, N = 650) = 5.91, p = .015 for plant, χ²(1, N = 797) = 40.20, p < .001 for animal. All participants were also asked to rate (on a 1–9 Likert scale) how disgusted, angry, and afraid they were “when imagining this situation.” (We counterbalanced whether the faces or rating scales were shown first.) Here, again, participants endorsed disgust more than fear both for the plant scenario (M_fear = 4.18, M_disgust = 4.43, paired t(1,558) = 6.19, p < .001), and for the animal scenario (M_fear = 4.49, M_disgust = 4.99, paired t(1,558) = 11.47, p < .001).

One could still argue that people respond to GM food with unease, not fear (although this is not what RCL say). Or, it is possible that different emotions are evoked by GM food in the abstract vs. concrete examples of consumption.Footnote ² Nonetheless, the evidence we have seems to favor the disgust account over the fear account.

7 Next steps

RCL’s “high-granularity” disgust scales may have promise, although we believe that more theoretical and empirical work needs to be done to determine whether they are an improvement on the standard disgust measures. Better theoretical justification and better evidence of construct validity will help answer this question, but for the moment, we do not think there is good evidence for RCL’s central claim that people respond to GM food with fear more than disgust — in fact there is good evidence against it.

We are, however, intrigued by one of the other questions RCL raise: how specific is the relationship between disgust sensitivity and GM, compared to other risks/technologies? RCL’s study is not a great way to address this question, both because of their limited sample size and the small number of technologies studied. We do, however, think that the question is worth answering. RCL are correct to highlight its importance, and we are grateful to them for it.