Moral intuitions—instant feelings of approval or disapproval that come with witnessing moral actions (Haidt, Reference Haidt2001)—vary within and between cultures (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011, Reference Graham, Meindl, Beall, Johnson and Zhang2016; Haidt and Joseph, Reference Haidt and Joseph2004). To investigate morality across the globe, we need valid and reliable instruments that adopt the language and cultural context of specific regions (Atari et al., Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023). With mounting studies tailoring their moral judgment tasks to cultural idiosyncrasies (e.g., Bobbio et al., Reference Bobbio, Nencini and Sarrica2011; Kim et al., Reference Kim, Kang and Yun2012; Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020; van Leeuwen and Park, Reference van Leeuwen and Park2009), we can expand our understanding of how individuals’ moral compass is guided by regional and sociopolitical pressures (Malik et al., Reference Malik, Hopp, Chen and Weber2021).
We contribute to this line of work by adapting and validating two existing, popular moral stimulus sets for studying moral judgment among Dutch populations: the Moral Foundations Vignettes (MFV; Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015) and the Socio-Moral Image Database (SMID; Crone et al., Reference Crone, Bode, Murawski and Laham2018). Specifically, we adhere to the original validation procedures of the MFV and SMID as closely as possible by utilizing a crowd-sourced procedure based on a Dutch sample. Our focus on the Netherlands and these stimulus sets is motivated by three reasons. First, the Netherlands is a multiparty system that has recently witnessed an increase in affective polarization (Harteveld, Reference Harteveld2021), and understanding how moral intuitions diverge across partisan lines can foster mutual understanding (Puryear et al., Reference Puryear, Kubin, Schein, Bigman and Gray2022). Second, the text-based MFV have already successfully been adopted to the Portuguese language with a Brazilian sample (Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020), yet how well the MFV transfer to European populations is largely unknown (but see a pilot study by Wagemans et al., Reference Wagemans, Brandt and Zeelenberg2018, who used a small selection of 8–10 vignettes in Dutch samples, while we adopt and validate 120 vignettes). Third, although images can have diverse moral interpretations based on cultural differences, they do not require a literal translation and hence offer a more direct approach for probing cross-cultural differences in moral intuitions; however, the SMID’s applicability to European contexts remains unclear.
1. Moral foundations theory
Moral Foundations Theory (MFT; Haidt and Joseph, Reference Haidt and Joseph2004) provides a taxonomy of moral intuitions by postulating that a set of separate but interrelated moral foundations has developed over the course of cultural evolution. In its original conceptualization, MFT spanned a set of five moral foundations: care-harm, fairness-cheating, authority-subversion, loyalty-betrayal, and sanctity-degradation (Haidt and Joseph, Reference Haidt and Joseph2004). These moral intuitions were further organized into two broader moral categories: care-harm and fairness-cheating as ‘individualizing’ foundations that primarily serve to protect the rights and freedoms of individuals; and loyalty-betrayal, authority-subversion and sanctity-degradation into ‘binding’ foundations that primarily operate at the group level (Haidt, Reference Haidt2008). Since its inception, additional candidate moral foundations were discussed, including liberty (Iyer et al., Reference Iyer, Koleva, Graham, Ditto and Haidt2012), honor (Atari et al., Reference Atari, Graham and Dehghani2020), and ownership (Atari and Haidt, Reference Atari and Haidt2023). Analogously, in a recent update of MFT, Atari et al. (Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023) also split the fairness-cheating foundation into distinct and new foundations of equality and proportionality. This split aimed to capture the distinct moral concerns of fairness in procedure (proportionality) and equality of outcome (equality).
Extant studies show robust support for cultural and ideological differences in the endorsement of moral foundations (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011; Kivikangas et al., Reference Kivikangas, Fernández-Castilla, Järvelä, Ravaja and Lönnqvist2021). This line of research has primarily relied on the Moral Foundations Questionnaire (MFQ; Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011). Briefly, the MFQ includes two sets of questions that either probe (a) the relevance of moral foundations for deciding whether something is morally right or wrong (e.g., ‘Whether someone suffered emotionally’) or (b) the (dis)agreement with statements concerning the upholding of moral foundations in society (e.g., ‘It can never be right to kill a human being’). The MFQ has proven very useful for the evaluation of variations in the endorsement of moral values, particularly as they pertain to differences in political orientation (Graham et al., Reference Graham, Haidt and Nosek2009, Reference Graham, Nosek and Haidt2012) and sex (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011). Moreover, a plethora of studies tapping into cross-cultural differences in moral concerns have relied on the MFQ (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011) and its recent successor MFQ-2 (Atari et al., Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023), demonstrating that world region is a significant and reliable predictor for describing cultural variation in moral principles. Although the factor structure of the MFQ remains a topic of ongoing debate (Curry et al., Reference Curry, Jones Chesters and Van Lissa2019; De Buck and Pauwels, Reference De Buck and Pauwels2023; Harper and Rhodes, Reference Harper and Rhodes2021; Zakharin and Bates, Reference Zakharin and Bates2021), a more practical limitation of the MFQ is concerned with the types of questions that its design permits to address (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015).
In summary, the MFQ largely captures respondents’ rating of abstract principles, rather than moral judgments of concrete scenarios. As Graham et al. (Reference Graham, Haidt and Nosek2009, p. 1031) have stated, moral relevance ‘does not necessarily measure how people actually make moral judgments’, but these ratings are ‘best understood as self-theories about moral judgment’. Yet, individuals’ theories of morality (i.e., endorsement of moral principles) might diverge from their specific moral judgments (Haidt, Reference Haidt2001). As Clifford et al. (Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015, p. 1179) have argued, ‘one might view harm or loyalty as highly relevant to morality, yet refrain from making harsh judgments about others’ harmful or disloyal behavior’. Relatedly, Graham et al. (Reference Graham, Haidt, Koleva, Motyl, Iyer, Wojcik and Ditto2013) argue that the existence of a moral foundation can be doubted if there is a lack of response to third-party transgressions of that foundation. Because the MFQ does not probe respondents’ moral judgment of third-party transgressions, it may not be an ideal instrument for testing the existence of moral foundations.
Analogously, the Social Intuitionism Model (Haidt, Reference Haidt2001) postulates that people may not possess reliable introspective access to the causes of their moral judgments, and thus their reports about abstract moral concerns may not perfectly reflect how they actually form moral judgments. Moreover, many of these items include an ‘unstated, and ambiguous referent’, such as an authority figure, yet people may ‘judge MFT issues differently depending on the referents’ (Frimer et al., Reference Frimer, Biesanz, Walker and MacKinlay2013, p. 1053; see also Eriksson et al., Reference Eriksson, Simpson and Strimling2019). Finally, the brevity of the MFQ—a total of 30 items—makes it unsuitable for use in neuroimaging studies of moral judgment, partially due to insufficient statistical power as well as the lack of control for question length and complexity that introduce neurological confounds (e.g., Baciu et al., Reference Baciu, Ans and Carbonnel2002; Church et al., Reference Church, Balota, Petersen and Schlaggar2011).
As an alternative to probing individuals’ endorsement of moral foundations, some studies have used the Moral Foundations Sacredness Scale (MFSS), which was designed to examine respondents’ willingness to engage in taboo trade-offs (Tetlock et al., Reference Tetlock, Kristel, Elson, Green and Lerner2000), such as kicking a dog in the head (Care) or renouncing one’s citizenship (Loyalty) for money (Graham et al., Reference Graham, Haidt and Nosek2009). However, Clifford et al. (Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015, p. 1180) point out that ‘the MFSS is designed to measure an individual’s willingness to violate moral norms in exchange for money, as opposed to judgments of others’ behaviors’.
To better probe individuals’ moral judgment of concrete situations and behaviors, various studies have developed moral vignettes on an ad hoc basis, with some devising scenarios corresponding to the harm and purity moral foundations (Heekeren et al., Reference Heekeren, Wartenburger, Schmidt, Prehn, Schwintowski and Villringer2005; Parkinson et al., Reference Parkinson, Sinnott-Armstrong, Koralus, Mendelovici, McGeer and Wheatley2011; Schaich Borg et al., Reference Schaich Borg, Lieberman and Kiehl2008, Reference Schaich Borg, Sinnott-Armstrong, Calhoun and Kiehl2011). Others used photographic images to depict certain moral violations but collapsed across all forms of violations in their analyses, making it difficult to examine potential differences between specific moral foundations (Harenski et al., Reference Harenski, Antonenko, Shane and Kiehl2008; Harenski and Hamann, Reference Harenski and Hamann2006; Moll et al., Reference Moll, de Oliveira-Souza, Eslinger, Bramati, Mourão-Miranda, Andreiuolo and Pessoa2002).
In view of these limitations, researchers have started to develop standardized and normed moral foundation vignettes and image databases for studying moral judgment. Popular databases for morally relevant scenarios include the Moral Foundations Vignettes (MFV; Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015; cited 363 times to date on Google Scholar). The MFV span 120, one-sentence descriptions detailing the violation of one (and only one) of seven moral foundations: physical care, emotional care, fairness, liberty, loyalty, authority, and sanctity. The vignettes also contain nonmoral, social norm transgressions. Notably, the creators of the MFV split the care-harm foundation into three subcomponents to reflect the diversity of the original conception of care-harm: emotional harm to a human, physical harm to a human, and physical harm to a nonhuman animal. This division is also aligned with neural evidence showing that the introduction of bodily harm into either a moral or nonmoral scenario can influence the levels of observed neural activity in certain brain regions (Heekeren et al., Reference Heekeren, Wartenburger, Schmidt, Prehn, Schwintowski and Villringer2005).
The MFV have been employed in both behavioral (Clifford, Reference Clifford2017; Dehghani et al., Reference Dehghani, Johnson, Hoover, Sagi, Garten, Parmar, Vaisey, Iliev and Graham2016; Wagemans et al., Reference Wagemans, Brandt and Zeelenberg2018) and functional magnetic resonance imaging (fMRI) studies (Hopp et al., Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023; Khoudary et al., Reference Khoudary, Hanna, O’Neill, Iyengar, Clifford, Cabeza, De Brigard and Sinnott-Armstrong2022). While these studies have administered the MFV solely in US samples, recent work by Marques et al. (Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020) introduced a Portuguese adaptation of the MFV. Using a Brazilian sample (N = 494), they demonstrated that the Portuguese version of the MFV performed similarly to the original English version in terms of its factor structure. Aside from this Portuguese case study, there have been no attempts to adapt and validate the MFV to other contexts, although MFT’s theoretical postulations demand cross-cultural research.
Adapting the MFV for non-English countries necessitates translating and adjusting specific vignettes for cultural comprehension (Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020). A remedy for this issue may be offered by recent studies that have developed photographic and even audiovisual moral stimulus databases (Crone et al., Reference Crone, Bode, Murawski and Laham2018; McCurrie et al., Reference McCurrie, Crone, Bigelow and Laham2018). The SMID (Crone et al., Reference Crone, Bode, Murawski and Laham2018; cited 49 times to date on Google Scholar) offers a large resource for examining differences in moral judgment, both within and across cultures. The SMID contains 2,941 images, each annotated for moral and affective qualities using crowd-sourced samples from the United States and Australia. Each image was rated on how much it depicts each moral foundation as well as for general valence, arousal, and (im)morality. Notably, images in the SMID also display morally good actions, extending previous stimulus sets which solely contain moral transgressions. Moreover, images may offer increased ecological validity over text-based vignettes, which have been criticized for creating an artificial moral psychology of ‘raceless, genderless strangers’ (Hester and Gray, Reference Hester and Gray2020). Subsets of SMID images were already used in previous studies in Japan (Chunyu et al., Reference Chunyu, Zommara, Ounjai, Ju and Lauwereyns2021, 160 images; Sudo et al., Reference Sudo, Nakashima, Ukezono, Takano and Lauwereyns2021; 60 images) and China (Tao et al., Reference Tao, Leng, Huo, Peng, Xu and Deng2022a, 66 images; Reference Tao, Leng, Peng, Xu, Ge and Deng2022b, 192 images), but validations of SMID images in a European context are absent. Furthermore, prior research has predominantly utilized the SMID to gather general moral and immoral images, often relying on student samples for image evaluations. To advance MFT, it is essential to procure images that consistently elicit perceptions of distinct moral foundations in more diverse cultural populations. Given that SMID’s moral foundation ratings originate from crowd-workers in the United States and Australia, validation is required to examine the applicability of visual representations of moral foundations in other cultures.
1.1. Current work
In view of demands for culturally tailored moral stimulus sets, we adapt and validate the MFV and SMID for studying moral judgment among Dutch populations. We first translated and adapted the MFV into Dutch. Second, we selected images from the SMID that primarily display moral and immoral exemplars of each moral foundation as well as neutral images that do not display moral information. In turn, we validated these stimuli sets using a large crowd-sourced sample from the Netherlands. Crowd-sourced validations of moral stimuli are increasingly becoming the gold standard in moral psychology, particularly because they capture a more diverse moral signal and are less prone to introducing annotator biases (Crone et al., Reference Crone, Bode, Murawski and Laham2018; Hopp et al., Reference Hopp, Fisher, Cornell, Huskey and Weber2021; Hopp and Weber, Reference Hopp and Weber2021; McCurrie et al., Reference McCurrie, Crone, Bigelow and Laham2018).
2. Method
We report how we determined our sample size and all data exclusions in the study. All materials, data, analysis code, and supplementary information (SI) are accessible at https://osf.io/9gnza. This study’s design and its analysis were not preregistered. All procedures were approved by the ethics board of the host institution.
2.1. Moral foundations vignettes
The full MFV database by Clifford et al. (Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015) contains 132 moral transgressions. Previous work had adapted selections of 8–10 (Wagemans et al., Reference Wagemans, Brandt and Zeelenberg2018) and 90 (Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020) vignettes, but we aimed to adapt a larger selection of 120 vignettes that have been employed in past experimental research (Hopp et al., Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023; Khoudary et al., Reference Khoudary, Hanna, O’Neill, Iyengar, Clifford, Cabeza, De Brigard and Sinnott-Armstrong2022). Each vignette consists of a one-sentence description (14–17 words) detailing the violation of one (and only one) of seven moral foundations (see Table 1 for examples): physical care, emotional care, fairness, liberty, loyalty, authority, and sanctity. The vignettes also contain a nonmoral, social norm transgression category. Each of the eight conditions featured 15 vignettes. One of the authors—a Dutch native—translated each vignette from English into Dutch. After translating all vignettes, the translator met with the remaining authors of the paper to ensure that minor adjustments of the vignettes fit the context of the Netherlands (all edits are reported in SI Table 1).
Example moral foundations vignettes and Dutch translations
Note: All translated vignettes are available in SI Table 1.
2.2. Socio-Moral Image Database
The SMID (Crone et al., Reference Crone, Bode, Murawski and Laham2018) contains 2,941 images, all annotated for moral and affective qualities using crowd-sourced samples located in the United States and Australia. Each image was rated on how much it depicts Care, Fairness, Loyalty, Authority, and Sanctity, using a five-point Likert-type scale from 1 (not at all) to 5 (very much). Similarly, each image was also rated using five-point Likert-type scales for valence (1 = unpleasant or negative; 5 = pleasant or positive), arousal (1 = calming; 5 = exciting), and morality (1 = immoral/blameworthy; 5 = moral/praiseworthy). Because valence and morality ratings correlated at r = .87 (Crone et al., Reference Crone, Bode, Murawski and Laham2018), we only retrieved the morality ratings. Based on these ratings, we organized all images into a circumplex model typically used for stimulus sampling in emotion research (Russell, Reference Russell1980), with one axis describing morality and the other axis capturing arousal, thereby creating four image quadrants (Figure 1A): moral-high arousal (N = 340); moral-low arousal (N = 1,247); immoral-high arousal (N = 767); and immoral-low arousal (N = 500). Next, within each quadrant, we selected 20 images rated highest on a single foundation and lowest on all other foundations (Figure 1B–F). In a similar fashion, we also sampled five ‘neutral’ images in each quadrant that received high and low arousal ratings, but clustered close to a morally neutral rating of ‘3’ (i.e., ≥2.9; ≤3.1) and were rated lowest across all moral foundations. This resulted in a final sample of 120 images, with 10 moral and 10 immoral images per moral foundation category as well as 10 high arousal, morally neutral and 10 low-arousal, morally neutral images (for example images, please see Figure 2).
SMID image sampling procedure. (A) The 2,941 images were first organized into a circumplex model according to the midpoint (3) of the Arousal and Morality rating axes. (B–F) The selection of foundation-specific images proceeded as follows: From each quadrant of the original circumplex model, five images were selected that received the highest rating for a given foundation and the lowest ratings for all other foundations. Dot sizes in B–F reflect the average degree to which images in each category were perceived to display that moral foundation, with greater sizes indicating a higher average foundation-specific rating.
Examples of selected SMID images for each moral-arousal quadrant. Image border color denotes the moral foundation that received the highest rating in the original study (Crone et al., Reference Crone, Bode, Murawski and Laham2018). Neutral images rated low on all moral foundations are not shown.
2.3. Participants
We used the Prolific academic (PA) platform (https://www.prolific.co/) for recruiting participants. Eligibility criteria included speaking Dutch as a first language, holding Dutch nationality, and being located in the Netherlands. Our sample size was determined by following previous moral stimulus validation studies (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015; Crone et al., Reference Crone, Bode, Murawski and Laham2018; McCurrie et al., Reference McCurrie, Crone, Bigelow and Laham2018) and thus aimed to obtain at least 20 ratings for each stimulus on each dimension. In total, 648 survey responses were collected, of which 62 were excluded who provided incomplete responses or finished the survey in under 6 minutes (<5% quantile), leaving us with a total sample size of 586. Complete demographic information for 572 participants could be retrieved and indicated that we had a diverse sample of the Dutch population: participants had a mean age of 28.39 years (SD = 8.89) of which 326 (57%) identified as male (244 female; 1 nondisclosed). We assessed political orientation using a slider ranging from ‘very left’ (0) to ‘very right’ (100) (Dodd et al., Reference Dodd, Balzer, Jacobs, Gruszczynski, Smith and Hibbing2012). Our sample was politically diverse, with a slight skew toward the political left (M = 38.95, SD = 22.12). The majority reported a White ethnicity (497; 87%), followed by mixed (42; 8%), Asian (14; 3%), Black (8, 1%), and ‘other’ (8; 1%). 253 (44%) participants indicated to not hold a student status, 239 (41%) held a student status, and student status data for 80 participants had expired.
2.4. Procedure
Data were collected through an online survey using Qualtrics. After signing the informed consent, the survey started with a brief overview of MFT—adapted from Crone et al. (Reference Crone, Bode, Murawski and Laham2018) and translated by us into Dutch—to familiarize participants with the basic contents of moral foundations. Next, participants provided ratings of vignettes, images, or news clips (not reported here), in which the order of stimuli blocks varied randomly across participants. Each participant was assigned to a random selection of five vignettes and five images, respectively. For each vignette, participants used a five-point Likert scale to rate the vignette’s moral wrongness (‘How morally wrong is the displayed behavior?’; 1: not at all wrong – 5: extremely wrong), comprehensibility (‘How easy is it for you to understand what is described in the scenario?’; 1: not at all easy to understand – 5: extremely easy to understand), imaginability (‘How easy is it for you to clearly imagine what is happening in the scenario?’; 1: not at all easy to imagine – 5: extremely easy to imagine), frequency (‘How often do you see or hear about actions like the one described in this scenario in the media or your daily life?’; 1: not at all often – 5: extremely often), and emotional response (‘How strong was your emotional response to the behavior depicted in this scenario?’; 1: not at all strong – 5: extremely strong). Participants were also asked why the action is morally wrong and could choose one out of seven response options reflecting each vignette category (all vignette-related item prompts and response options are provided in their original English and translated Dutch version in the SI). Every participant was prompted to indicate why a vignette was morally wrong, independent of their answer to the moral wrongness rating prompt. Yet, participants did have the option to indicate that ‘It is not morally wrong and does not apply to any of the provided choices’. Hence, if a participant judged a vignette ‘not at all wrong’ they could still indicate that this vignette did not violate any moral norms.
Similarly, for each image, participants used a five-point Likert scale to rate the image’s general valence, arousal, and morality as well as the degree (henceforth: moral foundation relevance) to which the image makes them think about each moral foundation (all image-related item prompts and response options are provided in their original English and translated Dutch version in the SI).
Ratings across MFV categories
Note: Classification rate reflects the percentage of categorization into the intended foundation.
Moral foundations vignettes ratings. (A) Moral wrongness. (B) Classification rate in percent. (C) Comprehensibility. (D) Imaginability. (E) Frequency. (F) Emotional response. Each dot reflects the mean response of all participants to a single vignette item. Box plots for each condition display median (center line), upper and lower quartiles (box limits), whiskers connotate 1.5 × interquartile range (IQR) and points that fall outside the whiskers are outliers.
3. Results
3.1. Moral foundations vignettes
All vignettes were rated an average of 21.97 times (min: 14; max 30).Footnote 1 We first tested whether vignettes displaying a moral violation were rated as more morally wrong than vignettes describing a social norm transgression (Table 2). Indeed, every moral vignette item was rated as more morally wrong than every social norm vignette item (Figure 3A), except for one Authority item (MFV 61 ‘You see a teaching assistant talking back to the teacher in front of the classroom’; moral wrongness ratings for each vignette item are summarized in SI Table 1). Replicating previous work (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015; Hopp et al., Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023; Khoudary et al., Reference Khoudary, Hanna, O’Neill, Iyengar, Clifford, Cabeza, De Brigard and Sinnott-Armstrong2022; Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020), moral vignettes violating physical care received the highest moral wrongness rating, whereas loyalty violations received the lowest moral wrongness ratings among moral vignettes (Table 2). We also tested whether each moral foundation category was rated more morally wrong than social norms. Using the Tukey–Kramer Method for multiple comparisons of groups with unequal sample sizes (Kramer, Reference Kramer1956), we found that violations of each moral foundation were rated as significantly more morally wrong than social norm transgressions (Table 3).
Next, we tested whether each vignette was classified into its originally intended category (Table 2 and Figure 3B). To this end, we calculated the classification rate (%)—the percentage of times a vignette was classified into their intended category. We observed that the majority of vignettes (97%) were classified into their intended category, with average classification rates ranging from 85.63% for Fairness vignettes to 60.2% for Loyalty vignettes (classification rates for each vignette item are reported in SI Table 1). Only four vignettes were mostly classified into a nonintended category: 1) The above-mentioned Authority item MFV 61 (73.33% ‘Not Wrong’); 2) Loyalty item MFV 1: ‘You see a former Secretary of State publicly giving up his citizenship to the Netherlands’ (53.33% ‘Not Wrong’); 3) Loyalty item MFV 72: ‘You see a Dutch swimmer cheering as a Chinese foe beats his teammate to win the gold’ (56.52% ‘Not Wrong’); and 4) Emotional Care item MFV 35: ‘You see a man laughing at a disabled co-worker while at an office softball game’ (45.00% ‘Liberty’). Curiously, both Loyalty items received higher average moral wrongness than any social norm vignette, suggesting that participants may indeed have intuitively perceived them as moral violations. In addition, all vignettes were rated as highly comprehensible and imaginable, and ratings of frequency, as well as emotional response, were comparable to those reported in the original MFV (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015) study (Table 2 and Figure 3C–F).
Furthermore, we explored the correlation between moral wrongness ratings across vignette categories and participants’ political orientation (Table 4). Consistent with MFT, authority and loyalty, which both belong to MFT’s binding moral foundations, were significantly positively correlated (r = .25, p = .008). Analogously, we found that a more right-leaning political attitude correlated significantly and positively with wrongness ratings of the binding moral foundations loyalty (r = .16, p = .011), authority (r = .21, p = .001), and sanctity (r = .14, p = .022). Surprisingly, Fairness, which belongs to MFT’s individualizing foundations, significantly positively correlated with Loyalty (r = .24, p = .009) and Sanctity (r = .27, p = .004); both belonging to the binding moral foundations. As demonstrated by Hopp et al. (Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023), more right-leaning individuals also rated Fairness (r = .18, p = .004) and Social Norms (r = .21, p < .001) as more morally wrong. In view of these empirical results and how they compare to previous studies, we consider our translation and adaptation of the MFV to the Dutch context successful.
Difference of moral wrongness ratings between each moral foundation and social norms
Note: Results of Tukey’s honest significance test on the difference between moral wrongness ratings of each moral foundation and social norms.
Correlations of moral wrongness ratings between MFV categories and political orientation
Note: A positive correlation between Political Orientation and Moral Wrongness rating implies that more conservative participants made higher ratings. Bold cells indicate significant correlations at *p < .05. **p < .01.
3.2. Socio-Moral Image Database
All images were rated an average of 21.88 times (min: 10; max 36). First, we examined whether images originally rated as moral (immoral) were also judged as moral (immoral) by our Dutch sample (Table 5). Collapsing all images across their foundation-specific categories, moral images were rated as more moral (M = 3.66, SD = 0.49) and immoral images were judged to be more immoral (M = 2.64, SD = 0.56). This difference was large in terms of effect size and statistically significant t(118) = 10.63, p < .001, d = 1.94, 95% CI = [0.83, 1.21], indicating that moral images were indeed perceived to display something morally praiseworthy compared to immoral images judged to depict immoral and blameworthy content. Critically, these moral versus immoral differences were also statistically significant within each foundation-specific image category (Table 5 and Figure 4A). Likewise, images within the neutral category did not differ significantly in their moral valence ratings t(18) = 1.2, p = .122, d = 0.54, 95% CI = [−0.13, 0.48]. Yet, we also observed that eight images originally placed into the ‘immoral’ category and associated with a moral foundation were rated as moral (<3), and four supposedly moral images were rated as immoral (>3; moral valence ratings for each image item are summarized in SI Table 2).
Arousal and moral valence ratings across image categories
Note: t and p values are the results of independent, one-sided t-tests comparing moral > immoral and high arousal > low arousal for each image category separately.
Sociomoral image ratings. (A) Morality ratings for moral versus immoral images. (B) Arousal ratings for low versus high arousal images. (C) Foundation ratings for each moral foundation category. Each dot reflects the mean response of all participants to a single image. Box plots display median (center line), upper and lower quartiles (box limits), whiskers connotate 1.5 × interquartile range (IQR) and points that fall outside the whiskers are outliers.
Moral foundation ratings across intended image categories
Mean differences in foundation ratings across image categories
Note: Results of five independent, one-sided t-tests. For each directional test, the average foundation rating of images within one foundation was compared against the average foundation rating across all other image categories (i.e., foundation ratings for foundation images > foundation ratings for all images not within the foundation).
Confusion matrix comparing intended and rated image categories
Note: We assigned each image the foundation which received the maximum rating and assigned ‘Neutral’ to the 20 images with the lowest mean ratings. For three images, sanctity and authority both received the highest mean rating, therefore we added them to both, authority and sanctity, when we calculated our measures. The numbers in bold are weighted averages for the respective measure.
Moreover, we investigated differences in arousal (Table 5 and Figure 4B). Similar to ratings on moral valence, high-arousal images received a higher arousal rating (M = 3.42, SD = 0.39) than low-arousal images (M = 2.70, SD = 0.61). This difference was again large in terms of effect size and statistically significant t(118) = 7.67, p < .001, d = 1.40, 95% CI = [0.53, 0.90]. Compellingly, these mean differences were statistically significant within each foundation-specific as well as neutral image category. Despite these averaged categorical differences, there were images whose arousal rating differed from the intended arousal category. Nine high-arousal items were rated with lower arousal (<3), and 23 low-arousal items were rated with higher arousal (>3; arousal ratings for each image item are summarized in SI Table 2). As morality ratings and valence ratings were again highly correlated (r = 0.73, p < .001), we provide no further analysis of valence ratings.
Thereafter, we tested whether participants rated the presence of moral foundations according to their intended foundation-specific image category (Tables 6 and 7 and Figure 4C). To this end, we conducted a series of independent, one-sided t-tests comparing the mean foundation rating for images of the intended foundation with the mean foundation rating of images across all other categories (e.g., mean rating of care-harm in images classified as care-harm compared to mean rating of care-harm for all other images). As expected, we observed that for all foundations, the corresponding images received significantly higher ratings on their foundation compared to images from all other categories (Table 7).
We also determined whether individual images received the intended foundation-specific ratings. To this end, we computed the mean foundation rating for each image and assigned each image to the foundation that received the highest mean rating. Likewise, the 20 images with the lowest mean foundation ratings were classified into the ‘neutral’ category. The resulting confusion matrix crossing intended and rated foundation is displayed in Table 8. Notably, 19 (95%) of the intended care images indeed received the highest care ratings across images, followed by 18Footnote 2 (90%) authority images and 16 (80%) ‘neutral’ images. In contrast, discrepancies were larger for fairness images (10 images; 50%), sanctity (82 images; 35%), and loyalty (7 images, 35%). Across all images, 63% were rated according to their intended category, with an average accuracy of 87.92%, suggesting that even on the individual image level, the majority of images were correctly categorized into their intended foundation.
Next, we compared the image ratings from our Dutch respondents to those from US/Australian samples studied by Crone et al. (Reference Crone, Bode, Murawski and Laham2018) during the creation of the SMID (Figure 5). We observed that image judgments were highly similar between the samples, with Pearson correlations ranging from .3 to .7 (all p < .001) across image categories. However, we did find that ratings of moral foundations showed nuanced differences between cultures. With the exception of authority-subversion images, ratings for the main moral foundation of each moral image class were higher in the US/Australian samples compared to the Netherlands. We return to an interpretation of these differences in the discussion section.
Comparison of SMID ratings between US/Australian (US/AUS) samples (Crone et al., Reference Crone, Bode, Murawski and Laham2018) and respondents from the Netherlands (NL). The blue line denotes ratings from US/AUS samples, whereas the orange line reflects ratings from Dutch respondents. Error bars connotate 95% confidence intervals based on 1,000 bootstrap samples.
Correlation table for image ratings
Note: A positive correlation between political orientation and other ratings implies that more conservative participants made higher ratings. Moral polarity refers to how distant the rating was from the scale midpoint. *p < .05. **p < .01.
Lastly, we examined the correlation across all image rating categories and participants’ political orientation (Table 9). Higher ratings on each of the moral foundations correlated with distance of morality ratings from the midpoint of the scale, a metric that we termed ‘moral polarity’. In line with exemplification theory (Zillmann, Reference Zillmann1999), this could imply that individuals who perceive an image to be more exemplary for a moral foundation also deem this image to be more moral or immoral. Interestingly, more morally polarized ratings did not correlate with arousal ratings (r = .03, p = .519). Rather, the more arousing an image, the less it was perceived to display something moral/praiseworthy (r = −.31, p < .001). Replicating findings from Crone et al. (Reference Crone, Bode, Murawski and Laham2018), all five foundation ratings were moderately correlated with each other (all r’s > .4, p < .001), although all our pairwise foundation correlations were lower than those in the original study (Figure 4 in Crone et al., Reference Crone, Bode, Murawski and Laham2018). Again, fairness was strongly related to binding foundations. In particular, the highest foundation correlations were between fairness and loyalty (r = .71, p < .001), fairness and authority (r = .60, p < .001) and loyalty and authority (r = .71, p < .001). Moreover, we found that ratings for all foundations were positively associated with more conservative political orientations—but the association between care and ideology was close to zero and not statistically significant. Note that in contrast to the MFV, foundation image ratings reflect how strongly participants perceived those foundations in the images and not how morally wrong they found those images to be. While conservatives tended to provide more polarized morality ratings overall (r = .10, p = .025), this is likely driven by the fact that conservatives rated images as more moral compared to progressives (r = .15, p < .001).
4. Discussion
We adapted and validated two widely used moral stimulus sets for examining moral judgment in a Dutch sample. We translated the MFV (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015) into the Dutch language and selected a wide range of morally salient images from the SMID (Crone et al., Reference Crone, Bode, Murawski and Laham2018), which we then validated in a crowd-sourced sample from the Netherlands. These instruments offer advantages over alternatives by allowing participants to make moral judgments about specific situations (Crone et al., Reference Crone, Bode, Murawski and Laham2018; Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020; Schein, Reference Schein2020).
The results of our MFV analysis suggest that we successfully adapted them to the Dutch context. Participants rated scenarios violating a moral foundation as more morally wrong than those describing social norm transgressions. Additionally, trends in moral wrongness ratings across MFV categories were similar to those reported in the original MFV study (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015). Furthermore, participants predominantly accurately identified the intended type of moral or social norm violation in the vignettes. These results suggest that we have successfully provided a valid and reliable MFV for the Dutch population.
Our demonstrated relationship between MFV moral wrongness ratings and political orientation only partially replicates prior findings. As Haidt and Graham (Reference Haidt and Graham2007) argued and a meta-analysis by Kivikangas et al. (Reference Kivikangas, Fernández-Castilla, Järvelä, Ravaja and Lönnqvist2021) confirmed, conservatives in the US usually judge the binding moral foundations as more morally relevant than progressives. Compellingly, this pattern also emerged in our study, and even social norms were rated more morally wrong by more right-leaning individuals. Yet, extant literature suggests that left-leaning (progressive) individuals in the US judge transgressions of individualizing foundations as more morally wrong than conservatives do (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011; Kivikangas et al., Reference Kivikangas, Fernández-Castilla, Järvelä, Ravaja and Lönnqvist2021). However, we found no statistically significant associations between progressiveness and moral wrongness ratings of care violations, and wrongness ratings of fairness transgressions even showed a small to mid-sized association with conservatism. We reason that these discrepancies might be more driven by instruments than translational artifacts or genuine cultural differences.
On the one hand, the previously mentioned studies used the MFQ (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011; Kivikangas et al., Reference Kivikangas, Fernández-Castilla, Järvelä, Ravaja and Lönnqvist2021), whereas we used the MFV. In the original MFV paper, fairness was unrelated to political orientation (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015). Analogously, Hopp et al. (Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023), also using the MFV in a US college sample, found the same pattern as we did. On the other hand, Van Leeuwen and Park (Reference van Leeuwen and Park2009) used the MFQ with Dutch participants and reported that fairness was associated with a more progressive political orientation. Why is fairness sometimes related to progressive political orientation and sometimes not? According to Janoff-Bulman (Reference Janoff-Bulman2023) and Atari et al. (Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023), MFT omits the distinction between two kinds of distributional justice: equality and proportionality. Participants may interpret fairness items in the MFQ as questions of equality, which aligns with progressive concerns (Jost, Reference Jost2017). In contrast, proportionality may be more associated with conservative political orientation (Lee et al., Reference Lee, Yoon, Lee and Royne2018). To clarify these relationships, future versions of the MFV may incorporate scenarios related to both equality and proportionality.
The SMID ratings analysis validated our image selection for studying Dutch moral judgment. We identified images that consistently evoked moral or immoral ratings across various moral foundations, while neutrally classified images were consistently rated as having neutral moral content and low appeal to all moral foundations. Furthermore, we offer evidence that foundation-specific images can be identified. Images primarily showcasing one moral foundation reliably elicited stronger perceptions of that foundation compared to images emphasizing other moral foundations. Mirroring our MFV results, when rating photographic images, Dutch conservatives also perceived a greater degree of loyalty, authority, sanctity, and fairness than progressives. This may indicate conservatives’ greater recognition of these foundations and suggests that higher moral wrongness ratings for MFV and similar stimuli might result not only from greater relevance assigned to these foundations but also from more frequent recognition of morality in various contexts. Future studies should dissect these influences and interaction of perception and evaluation in moral judgment.
Importantly, although general trends in image ratings were highly similar between US/Australian and Dutch samples, notable differences in the perception of particular moral foundations did emerge between the samples. There are at least three explanations for this: First, these differences were to be expected as we sampled images from the SMID that received the highest ratings from US/Australian samples for each of the moral foundations. Second, they nevertheless demonstrate that moral judgments of photographic images are modulated by cultural differences, even across nations within the Western, Educated, Industrialized, Rich, and Democratic (WEIRD, Henrich et al., Reference Henrich, Heine and Norenzayan2010) context. Third, this finding also invites future research to determine images that are more moralized within Dutch culture. For instance, images rated as high in Sanctity-Degradation by US/Australian respondents primarily displayed prostitution and recreational drug use. Yet, Dutch individuals are generally tolerant toward prostitution (Jonsson and Jakobsson, Reference Jonsson and Jakobsson2017) and illicit drug use (van der Sar et al., Reference Van der Sar, Ødegård, Rise, Brouwers, Van de Goor and Garretsen2012). In turn, being confronted with these concepts may not have triggered moral emotions (e.g., disgust) in Dutch citizens that precede moralization (Clifford, Reference Clifford2019). Interestingly, although the Sanctity-Degradation ratings were lower in Dutch than in US/Australian samples for all image categories—with the exception of Neutral images—our study does not suggest that the moral foundation of Sanctity-Degradation is unknown to Dutch individuals: Respondents in the Netherlands did rate images that primarily display Sanctity-Degradation as higher in that foundation compared to images from other categories. Analogously, vignettes that described violations of Sanctity-Degradation were also accurately classified into the Sanctity-Degradation category by Dutch respondents.
4.1. Limitations
This study has limitations. We used a crowd-sourced approach common for affective datasets (Crone et al., Reference Crone, Bode, Murawski and Laham2018; Hopp et al., Reference Hopp, Fisher, Cornell, Huskey and Weber2021) and had each participant rate only a fraction of stimuli. This enabled us to simultaneously investigate two large stimuli sets, yet it also came at the cost of only around 20 ratings per stimulus. These subsamples are smaller than in classic scale development or validation studies (e.g., Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015; Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011) and do not allow for factor analysis. Hence, we provide a fertile ground for future studies employing the full range of our adapted MFV and SMID within a repeated-measured design. Furthermore, we employed the same response options as the original MFV study in the United States (Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015) and ensuing MFV validations in Brazil (Marques et al., Reference Marques, Clifford, Iyengar, Bonato, Cabral, Dos Santos, Cabeza, Sinnott-Armstrong and Boggio2020). However, behavioral data from these studies have not been made openly accessible, hindering a direct comparison. We therefore invite our fellow researchers to openly share their data in order to enable a broader comparison of MFV responses across the globe.
Moreover, both stimulus sets feature scenarios that were derived from MFT’s original moral taxonomy. In view of MFT’s recent advancements that split the fairness foundation into equality and proportionality (Atari et al., Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023), future studies should design and validate vignettes that distinctly feature these new foundations. For instance, the recently introduced Moral Foundations Questionnaire 2 (MFQ-2; Atari et al., Reference Atari, Haidt, Graham, Koleva, Stevens and Dehghani2023) features this updated set of moral foundations and may serve as a template to construct concrete scenarios pertaining to equality and proportionality. At the same time, discrepancies in the inclusion, exclusion, and adaptations of moral foundations across tasks and studies hinders reproducibility. For example, the authors of the SMID did not include the liberty foundation (Iyer et al., Reference Iyer, Koleva, Graham, Ditto and Haidt2012) due to ongoing debates concerning its status as a distinct moral foundation (Crone et al., Reference Crone, Bode, Murawski and Laham2018). Because we aimed to reproduce the original MFV and SMID ratings as closely as possible, we likewise did not collect Liberty ratings for the SMID. Yet, moral psychology should strive for a more standardized and unified approach in its employment of moral judgment scales (Malle, Reference Malle2021).
Furthermore, there is some general critique of MFT arguing that all moral transgressions boil down to a single essence, such as harm (Schein and Gray, Reference Schein and Gray2017). Yet, mounting arguments from philosophy (Sinnott-Armstrong and Wheatley, Reference Sinnott-Armstrong and Wheatley2012, Reference Sinnott-Armstrong and Wheatley2014), behavioral (Sackris and Larsen, Reference Sackris and Larsen2022), and neuroimaging studies (Hopp et al., Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023; Parkinson et al., Reference Parkinson, Sinnott-Armstrong, Koralus, Mendelovici, McGeer and Wheatley2011) refute morality’s unity on any level of explanation (e.g., content, function, neurobiological, etc.). We agree with such arguments and a pluralistic approach to morality. That said, there exist other candidate frameworks that explain (cross-cultural) variability in moral judgments, including the recently introduced Morality as Cooperation (MaC) theory (Curry et al., Reference Curry, Jones Chesters and Van Lissa2019). Exploring how moral judgments of scenarios derived from competing moral theories converge and diverge across the globe presents a promising avenue for future research.
Analogously, the relationship between stimuli and political orientation rather reflects a starting point for future research than a mere limitation. For both stimuli sets, our results do not exactly replicate the expected left-right pattern, and the stimuli as such—and not only culture—present themselves as a possible explanation for our findings. This may be because of the translations we made, the sample we used, or the context that we studied (the Netherlands). We can only speculate, but think the best answer to this question lies in future research. Future studies should determine whether the differences in fairness rating for the MFV is due to wording or actual difference between general principles and actions (as this is the purported difference between MFV and MFQ; Clifford et al., Reference Clifford, Iyengar, Cabeza and Sinnott-Armstrong2015). For the SMID images, there are two possible ways to investigate the origin of our results. If we assume that method drives our findings, the variance in morality ratings should be increased. Thus, future studies should use our or alternative sampling procedures (Crone et al., Reference Crone, Bode, Murawski and Laham2018) to choose pictures which best discriminate between political orientation. In addition, if we assume that modality modulates moral judgment, neuroimaging studies may dissociate which processes are independent from modality and which are shared.
4.2. Future outlook and conclusion
Notwithstanding these limitations, several promising options exist for researchers interested in using our adapted version of the MFV and SMID for studying moral decision-making in Dutch populations. First, we suggest that future studies employ the full range of our adapted MFV and SMID, either jointly or separately, in a repeated-measures, within-subject moral judgment paradigm. Doing so will not only prove beneficial for examining the factor structure of our adapted MFV and SMID stimuli, but can also reveal novel insights into the moral decision-making process of Dutch individuals. Moral wrongness ratings, aggregated by moral foundations in either the MFV or SMID, can be taken as proxies for respondents’ sensitivity toward concrete moral transgressions. In turn, individual differences in these moral judgments may reveal which concrete moral scenarios contribute to Dutch citizen’s affective polarization (Harteveld, Reference Harteveld2021). Second, the highly controlled structure and norming of these datasets renders them an ideal measurement tool for probing the cognitive neuroscience of moral judgment (Hopp et al., Reference Hopp, Amir, Fisher, Grafton, Sinnott-Armstrong and Weber2023; Khoudary et al., Reference Khoudary, Hanna, O’Neill, Iyengar, Clifford, Cabeza, De Brigard and Sinnott-Armstrong2022). Because our knowledge concerning the neuroscience of moral cognition remains heavily US-centered, presenting our adapted MFV and SMID to Dutch individuals undergoing neuroimaging may prove useful for studying cross-cultural variation found for judgments of moral actions (Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011). Given that cultural differences may influence both the location and levels of observed neural activity (for a review see Han and Northoff, Reference Han and Northoff2008), cross-cultural investigations into the neuroscience of morality hold immense future potential. Lastly, future work should continue to probe the relative (dis)advantages of different moral measurement instruments for advancing the central tenets of MFT. We argue that MFQs remain cost-effective tools for researchers aiming to advance predictions concerning individual, sociopolitical, and cross-cultural variations in the abstract and general endorsement of moral foundations. At the same time, probing the existence of moral foundations via responses to vicarious (i.e., third-party) transgressions of moral foundations calls for moral judgment paradigms (e.g., MFV and SMID) that feature stimuli displaying concrete violations of moral foundations. Alternatively, we reason that combining both abstract and concrete moral paradigms may prepare us well to advance MFT by illuminating for whom and in which contexts general moral priorities converge and diverge with contextualized realizations of moral actions (Eriksson et al., Reference Eriksson, Simpson and Strimling2019; Hull et al., Reference Hull, Warren and Smith2024).
Taken together, our results show successful adaptation of the MFV and SMID for studying moral judgment in Dutch populations. Our herein developed work underscores the importance of exploring cultural differences, especially for nonverbal moral stimuli, and emphasizes the need for more stimulus validation in social psychology and personality science.
Data availability statement
The behavioural data that support the findings of this study, the analysis code, and the experimental stimuli are available at The Open Science Framework platform (https://osf.io/9gnza/).
Author contributions
Conceptualization: F.R.H., B.N.B.; Data curation: B.J.; Formal analysis: F.R.H., B.J.; Funding acquisition: F.R.H., B.N.B.; Investigation: F.R.H., E.K.; Methodology: F.R.H., E.K.; Project administration: F.R.H.; Writing—original draft: F.R.H., B.J., E.K.; Writing—review and editing: B.N.B.
Funding statement
F.R.H. and B.N.B. acquired funding from the Amsterdam School of Communication Research (Grant No. ASCoR-u-2022-Hopp). The funders had no role in study design, data collection and analysis, decision to publish or preparation of the manuscript.
Competing interest
The authors declare no competing interests.
Open access
