1. Introduction
A widely accepted notion in behavioral sciences posits that human beings possess 2 distinct cognitive processes, referred to as Type 1 and Type 2 (Evans, Reference Evans2003, Reference Evans2008; Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009; Kahneman, Reference Kahneman2011; Stanovich and West, Reference Stanovich and West2000). Type 1 involves autonomously operating subcognitive systems, which are evolutionarily prevalent in many animals, and operate through fast, automatic, effortless, and unconscious cognitive processes (Evans, Reference Evans2008). On the other hand, Type 2, representing a more human skill, is characterized by slow, deliberative, effortful, conscious, and controlled processes (Evans, Reference Evans2003) that rely on working memory, intelligence, and cultural factors (Barrett et al., Reference Barrett, Tugade and Engle2004; Evans, Reference Evans2003; Nisbett et al., Reference Nisbett, Peng, Choi and Norenzayan2001). Based on these distinct cognitive processes, thinking styles are commonly categorized into intuitive and analytic thinking. Examining these thinking styles and their related outcomes falls under the purview of the dual process model (DPM; Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009). Despite numerous studies exploring the effects of thinking styles across various domains (see Pennycook et al., Reference Pennycook, Fugelsang and Koehler2015 for a review) and the numerous DPM theories proposed to date (e.g., Budner, Reference Budner1962; Cacioppo and Petty, Reference Cacioppo and Petty1982; Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996; Kruglanski et al., Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013; Ståhl et al., Reference Ståhl, Zaal and Skitka2016), a theoretical consensus regarding the nature of cognitive process and thinking style differences remains ambiguous. It is still unclear whether people differ in their thinking styles on a single dimension, from intuitive to analytic, and whether there are other thinking styles (Newton et al., Reference Newton, Feeney and Pennycook2023). In addition, many different measures (e.g., cognitive reflection test, faith in intuition, need for closure, need for cognition, preference for intuition and deliberation, and importance of rationality) have been proposed so far within the framework of DPM, but the findings raise a question over the necessity of this complex variety concerning predicting very similar psychological factors and conceptual overlapping, albeit using different terminology. Consequently, there is a pressing need for clarifying structures of thinking style differences and an easy-to-use measure that comprehensively captures various aspects of thinking styles.
In this context, Newton et al. (Reference Newton, Feeney and Pennycook2023) asserted that individuals differ along multiple dimensions of intuitive–analytical thinking styles rather than simply varying between 2 ends of a single dimension. They delineated thinking styles into 4 components: Actively Open-minded Thinking, Close-minded Thinking, Preference for Intuitive Thinking, and Preference for Effortful Thinking. Introducing the 4-Component Thinking Styles Questionnaire (4-CTSQ) as a novel measure of cognitive style, Newton et al. (Reference Newton, Feeney and Pennycook2023) demonstrated its efficacy in elucidating diverse psychological outcomes associated with thinking styles, such as moral judgments, conspiracy beliefs, vaccination intentions, misperceptions about COVID-19, and truth discernment. Nevertheless, several aspects remain unexplored: (1) Newton’s 4-dimensional intuitive–analytic thinking styles proposition has not yet been tested beyond the original study, (2) certain findings contradict previous studies in the literature, and (3) the validity of the proposed measure in non-WEIRD cultures remains unvalidated. Thus, in 2 pre-registered studies, we aim to test the assertion that individuals differ along multiple dimensions of intuitive–analytic thinking styles, with 4 factors as proposed by Newton et al. (Reference Newton, Feeney and Pennycook2023). Additionally, we seek to replicate the original findings of Newton et al. (Reference Newton, Feeney and Pennycook2023) and assess the validity of the 4-CTSQ in a non-WEIRD culture, specifically Türkiye. Furthermore, we examine whether 4-CTSQ predicts various psychological outcomes associated with thinking styles, such as morality as cooperation, paranormal and religious beliefs, generic and COVID-19 conspiracy beliefs, religiosity, resistance to change, and ideology in a non-WEIRD culture. Furthermore, we compare the predictive power of the 4-CTSQ with existing performance-based measures of thinking styles in the literature, such as cognitive reflection tests and base-rate conflict problems.
1.1. Multiple DPM approaches and measures on thinking styles
Although the idea of DPM, albeit in a different form, was clearly proposed in the late 19th century (e.g., Bryan and Harter, Reference Bryan and Harter1899), recent interest in DPM approaches began with the cognitive revolution in the 1970s (Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009) and has since been extensively studied across diverse contexts, capturing the attention of scholars from various fields, including social cognition (e.g., Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996), psychology of learning (e.g., Yonelinas, Reference Yonelinas1994), developmental psychology (e.g., Klaczynski, Reference Klaczynski2004), and behavioral economics (e.g., Kahneman, Reference Kahneman2011). Within the framework of the DPM, multiple theoretical approaches have been introduced, each focusing on distinct aspects of thinking style differences and accompanied by their own conceptualizations and measures (see Evans, Reference Evans2003; Evans and Stanovich, Reference Evans and Stanovich2013; Osman, Reference Osman2004 for reviews).
Numerous scholars have developed conceptualizations and measurement scales that align with the disposition for reflective thinking. For instance, Cacioppo and Petty (Reference Cacioppo and Petty1982) introduced the Need for Cognition Scale, assessing one’s inclination for cognitive activities that require effort and their enjoyment of such activities. Going beyond the thinking more/less and reflection/impulsivity dualism, several scholars focused on the idea of what a general theory of good thinking should include (Baron et al., Reference Baron, Isler, Yılmaz, Ottati and Stern2023). In this regard, Baron (Reference Baron1993) elucidated a reasoning style marked by the proclivity to scrutinize new evidence in juxtaposition with one’s preexisting convictions, to allocate sufficient time to problem-solving endeavors before conceding abandoning, and to judiciously integrate the viewpoints of others in the process of shaping one’s own perspectives (Haran et al., Reference Haran, Ritov and Mellers2013). Then, different measurement tools have been proposed to measure this construct. For example, Stanovich and West (Reference Stanovich and West1997) emphasized the tendency to act with an open mind and revise existing beliefs upon new evidence, reflecting one of the implications of effortful thinking, and proposed the Actively Open-Minded Thinking Scale. In contrast, others sought to identify intuitive thinking dispositions using analogous conceptualizations but employing different measures, with measures like Epstein et al.’s (Reference Epstein, Pacini, Denes-Raj and Heier1996) Faith in Intuition Scale or Kruglanski et al.’s (Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013) Need for Closure Scale. Some scholars combined both dimensions, resulting in tools like Scott and Bruce’s (Reference Scott and Bruce1995) General Decision-Making Style Questionnaire, which assesses thinking style across 5 factors, and Betsch’s (Reference Betsch2004) Preference for Intuition and Deliberation Scale. Various comparable approaches have been proposed to date, with the primary objective of capturing thinking style differences based on the intuitive–analytic distinction (see Newton et al., Reference Newton, Feeney and Pennycook2023 for a comprehensive summary).
Drawing from this distinction, extensive literature shows that differences in intuitive–analytic thinking styles are associated with various outcomes, including beliefs, religiosity, ideology, and morality. For example, thinking style differences serve as a robust predictor of epistemically suspect beliefs: an increase in analytic thinking is linked to a decrease in conspiracy beliefs and paranormal beliefs (Alper et al., Reference Alper, Bayrak and Yilmaz2021; Bouvet and Bonnefon, Reference Bouvet and Bonnefon2015; Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012; Šrol, Reference Šrol2022; van Prooijen, Reference van Prooijen2017; Yelbuz et al., Reference Yelbuz, Madan and Alper2022). Studies also show that intuitive thinking is associated with religiosity (Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b; Yilmaz, Reference Yilmaz2021). Intuition positively correlates with religious belief (Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012), resulting in a stronger belief in God (Saribay et al., Reference Saribay, Yilmaz and Körpe2020; Shenhav et al., Reference Shenhav, Rand and Greene2012). A similar pattern emerges concerning ideology, which is closely related to religiosity (Caprara et al., Reference Caprara, Vecchione, Schwartz, Schoen, Bain, Silvester, Cieciuch, Pavlopoulos, Bianchi, Kirmanoglu, Baslevent, Mamali, Manzi, Katayama, Posnova, Tabernero, Torres, Verkasalo, Lönnqvist and Caprara2018; Piurko et al., Reference Piurko, Schwartz and Davidov2011). Conservatives tend to rely more on their intuition, whereas liberals tend to depend more on analytic thinking (Deppe et al., Reference Deppe, Gonzalez, Neiman, Jacobs, Pahlke, Smith and Hibbing2015; Eidelman et al., Reference Eidelman, Crandall, Goodman and Blanchar2012; Haidt, Reference Haidt2012; Jost, Reference Jost2017). In addition, some findings suggest that inducing intuitive thinking can diminish liberal tendencies and cause a shift toward conservatism (e.g., Skitka et al., Reference Skitka, Mullen, Griffin, Hutchinson and Chamberlin2002; Van de Vyver et al., Reference Van de Vyver, Houston, Abrams and Vasiljevic2016; Yilmaz and Saribay, Reference Yilmaz and Saribay2017a). The association between thinking style differences and moral values is a widely studied phenomenon in the literature: Analytic thinking strengthens individualizing moral foundations (e.g., Yilmaz and Saribay, Reference Yilmaz and Saribay2017b), while intuitive thinking exhibits a positive relationship with endorsing binding moral foundations (e.g., Pennycook et al., Reference Pennycook, Cheyne, Barr, Koehler and Fugelsang2014). Thus, existing literature indicates that the intuitive–analytic thinking dichotomy serves as an essential paradigm that may elucidate individual differences across various domains.
Nonetheless, there is an ongoing debate regarding the cognitive processes that determine an individual’s position along this single dimension (Newton et al., Reference Newton, Feeney and Pennycook2023). DPM theories converge on similar underlying constructs, namely intuitive and analytic thinking, albeit through different cognitive mechanisms. For example, some models emphasize the tendency to revise preexisting beliefs or engage in effortful thinking as indicative of reflective thinking (Cacioppo and Petty, Reference Cacioppo and Petty1982; Jarvis and Petty, Reference Jarvis and Petty1996; Krumrei-Mancuso and Rouse, Reference Krumrei-Mancuso and Rouse2016; Stanovich and West, Reference Stanovich and West2007), while others prioritize reliance on gut feelings or the need for certainty as indicative of intuitive thinking (e.g., Budner, Reference Budner1962; Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996; Kruglanski et al., Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013). Thus, approaches that address thinking style differences within a singular dimension based on analytical–intuitive thinking may commit a Jingle-jangle fallacy by oversimplifying the comprehensive nature of cognitive styles using identical denominations. In addition, the question of whether individuals vary in their thinking styles along a single dimension, ranging from intuitive to analytic, or whether other genuine dimensions exist remains unresolved (Newton et al., Reference Newton, Feeney and Pennycook2023). Furthermore, various DPM theories propose their own measurement tools, conceptualizing similar phenomena with different terms. However, these theories have remarkable similarities (Evans, Reference Evans2008), and the factors highlighted by these distinct approaches predominantly predict similar outcomes despite being labeled differently. Consequently, it is unclear how distinct their emphasized structures truly are from one another. Thus, despite the vast accumulation of empirical studies within the DPM literature providing insights into explaining individual differences, theoretical and measurement complexities still require further clarification. In response to this, Newton et al. (Reference Newton, Feeney and Pennycook2023) recently proposed a theoretical framework that addresses intuitive–analytic thinking disposition within a multidimensional structure and introduced a new measure of thinking style differences.
1.2. Four distinct styles of intuitive–analytic thinking and its comprehensive measure
To address the highlighted limitations, Newton et al. (Reference Newton, Feeney and Pennycook2023) conducted a study aimed at developing a comprehensive scale for assessing thinking style differences within the framework of DPM. They initially gathered 265 items from 12 different scales on thinking style and selected 50 candidate items for the new scale based on their correlation with the Cognitive Reflection Test (CRT; Frederick, Reference Frederick2005). Through this process, they established a 4-factor consisting of 24 items. As the CRT does not encompass all aspects of thinking styles (Hertzog et al., Reference Hertzog, Smith and Ariel2018) and multiple routes can lead to correct responses on the test (Bago and De Neys, Reference Bago and De Neys2019; Boissin et al., Reference Boissin, Caparos, Raoelison and De Neys2021; Raoelison et al., Reference Raoelison, Boissin, Borst and De Neys2021; Stupple et al., Reference Stupple, Pitchford, Ball, Hunt and Steel2017), Newton et al. (Reference Newton, Feeney and Pennycook2023) systematically examined factor structure and validity of the scale across 2 samples, considering various outcomes such as cognitive ability, cognitive style, performance measures of intuitive–analytic thinking, subjective happiness, empathizing, moral judgments, religious belief, conspiratorial belief, paranormal belief, and bullshit receptivity. Consequently, they established that the 24-item scale, featuring 4 correlated factors, namely actively open-minded thinking (AOT), close-minded thinking (CMT), preference for effortful thinking (PET), and preference for intuitive thinking (PIT), is a suitable instrument for measuring thinking style differences.
The AOTFootnote 1 focuses on profound questioning of preexisting beliefs and intuitions, along with a disposition toward reflectivity, leading to the consideration of alternative viewpoints based on evidence (Stanovich and Toplak, Reference Stanovich and Toplak2019). In contrast, CMT represents a thinking style that perceives truth in clear and distinct terms as black and white (Newton et al., Reference Newton, Feeney and Pennycook2023). PET refers to the inclination to engage in effortful cognitive activities, prioritizing deliberative thinking processes over automatic thoughts (Cacioppo and Petty, Reference Cacioppo and Petty1982). On the other hand, PIT reflects a preference for thinking based on intuitions and gut feelings (Betsch, Reference Betsch2004). Both AOT and PET can be classified as 2 subscales that stem from a foundation of analytic thinking, while the other 2 dimensions, CMT and PIT, correspond to 2 distinct manifestations of intuitive thinking. However, Newton et al. (Reference Newton, Feeney and Pennycook2023) emphasized that AOT and PET (or CMT and PIT) should not be evaluated as parallel factors operating on a continuum within a single dimension; rather, these factors represent distinct cognitive features that characterize different thinking styles.
Newton et al. (Reference Newton, Feeney and Pennycook2023) reported that the subscales exhibited varying strengths and directions in predicting the outcomes. AOT emerged as a negative predictor for all outcomes except for empathizing quotient and disgust-related moral dilemmas. CMT was a positive predictor for religious belief, subjective happiness, and moral dilemmas while serving as a negative predictor for paranormal belief. Conversely, PIT positively predicted all outcomes except for subjective happiness and religious belief. Lastly, PET positively predicted the empathizing quotient and subjective happiness.
Although the correlated 4-factor model fits the data better than the 2-factor model, paired factors are expected to be predictive in similar directions. Given that AOT and PET represent factors of analytic thinking disposition, while CMT and PIT indicate intuitive thinking, some of these results require clarification and replication. For example, Newton et al. (Reference Newton, Feeney and Pennycook2023) found that AOT was negatively associated with religious belief, whereas PIT and PET did not show any significant associations. However, prior research has identified the predictive role of PIT and PET on religious beliefs (e.g., Aarnio and Lindeman, Reference Aarnio and Lindeman2007; Dennin et al., Reference Dennin, Furman, Pretz and Roy2022). Moreover, CMT and PIT predicted paranormal belief in opposite directions. However, it is important to note that the prevailing expectation in the literature is for such relationships to generally align in a similar direction, even though the magnitudes of their effects may differ (e.g., Svedholm and Lindeman, Reference Svedholm and Lindeman2013). Furthermore, CMT positively predicted religious beliefs but negatively predicted paranormal beliefs. Nonetheless, it is important to acknowledge that the existing literature includes findings suggesting a close relationship between these 2 beliefs and significant overlap in their predictors (Aarnio and Lindeman, Reference Aarnio and Lindeman2007; Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012; Willard and Norenzayan, Reference Willard and Norenzayan2013). Hence, the anticipation was that CMT would predict both religious beliefs and paranormal beliefs in a consistent direction. Newton et al. (Reference Newton, Feeney and Pennycook2023) interpreted these findings as an indication that various thinking styles are associated with different outcomes and emphasized that categorizing individuals solely as intuitive or reflective is an oversimplification. However, as stated in the original study, these 4 dimensions are both correlated and independent, representing distinct forms of intuitive–analytic thinking disposition. While the inverse scenario may be statistically possible due to suppressor effects, given the small-to-moderate correlations among the subscales of the 4-CTSQ, it is anticipated that the related subscales would not display contradictory patterns in terms of the outcomes they predict. Consequently, further research is necessary to investigate how the subscales of the 4-CTSQ predict psychological outcomes related to thinking style differences and to clarify the original findings that do not entirely align with the expectations in the existing literature.
Another limitation of the original study is that it relies mostly on self-report measures of intention. Previous research indicates that intention-based measures may not be entirely reliable (Sheeran, Reference Sheeran2002), and a significant disparity exists between intentions and actual behaviors (Sheeran and Webb, Reference Sheeran and Webb2016). Newton et al. (Reference Newton, Feeney and Pennycook2023) employed a truth discernment measure to assess the predictive power of the 4-CTSQ regarding behavioral outcomes. Their findings indicated that the capacity to distinguish between fake and true news, previously attributed to cognitive style differences in prior literature (Pennycook and Rand, Reference Pennycook and Rand2021), exhibited a significant association with all subdimensions of the scale. Furthermore, each subdimension demonstrated varying predictive strengths that aligned with expected directions in relation to truth discernment. However, this measure primarily assesses a cognitive skill at the self-report level rather than a specific behavioral outcome such as sharing post on social media, introducing the potential for bias. This limitation stems from the fact that truth discernment measures the alignment of one’s beliefs about news items with actual reality, rather than addressing the act of believing and sharing fake news on social media. When it comes to actual behavior, it is unclear whether an outcome will be produced in line with this cognitive skill. Therefore, it is beneficial to employ more robust measurement methods to evaluate the predictive power of the 4-CTSQ in relation to actual behavioral outcomes. Additionally, intentions and behaviors can stem from different psychological processes. For instance, individuals may not exhibit the tendencies they express at the intention level due to the influence of social desirability, focusing on personal interests when it comes to actual behaviors. Moreover, it is essential to determine whether outcomes purportedly linked to thinking style differences consistently reveal the same patterns for both intention and behavior and how the CTSQ subscales predict these aspects. For instance, the strong relationship between moral values such as cooperation and thinking style differences should be demonstrated with both intention and behavior measures to understand the predictive power of the 4-CTSQ comprehensively. Hence, evaluating intention and behavior measures in conjunction is necessary to comprehensively unveil the psychological outcomes of different thinking styles.
Lastly, it is imperative to examine the validity of the 4-CTSQ in different cultural contexts, particularly in non-WEIRD ones (Henrich et al., 2010). Newton’s study solely used data from U.S. and Canadian participants, representing a typical WEIRD sample. However, thinking style differences may manifest dissimilar patterns in diverse cultures and may be associated with distinct outcomes. For instance, the relationship between analytic thinking disposition and conservatism appears stronger in WEIRD cultures, but its stability in non-WEIRD cultures remains uncertain (Yilmaz and Alper, Reference Yilmaz and Alper2019). Another crucial cultural aspect to consider is the prevalent religion. The original study’s samples were drawn from a culture predominantly following Christianity, and the validity of the findings related to religion in cultures with different religious beliefs is uncertain. For example, Newton et al. (Reference Newton, Feeney and Pennycook2023) found that AOT was the most potent predictor of religious beliefs, while PIT was the most potent predictor of paranormal beliefs. However, considering the divergent historical processes and religious features of different religions, it remains unclear whether the same pattern would persist in a society dominated by another religion (e.g., Islam). Therefore, further research is needed to assess the cross-cultural validity of the 4-CTSQ and its applicability in diverse cultural contexts, particularly in non-WEIRD and religiously diverse societies.
1.3. The current study
In 2 pre-registered studies, we tested the claim that individuals differ along multiple dimensions of intuitive–analytic thinking styles with 4 factors, the replication of Newton et al.’s (Reference Newton, Feeney and Pennycook2023) original findings, and the validity of the 4-CTSQ in the non-WEIRD culture, Türkiye.Footnote 2 In the first study, we tested the construct validity of 4-CTSQ and examined the predictive power of scale on several measures, including performance-based cognitive reflection tests, base-rate problems, conspiratorial beliefs, COVID-19 conspiracy beliefs, religious beliefs, paranormal beliefs, morality as cooperation, and ideology. In the second study, we cross-validate the factor structure of the 4-CTSQ on the final version of the scale from the first study. Then, we test the stability of the initial findings using the previous measures and adding new ones. Additionally, we validate the earlier findings while incorporating new measures, including moral foundations, pro-vaccination attitudes, and prosociality. Moreover, we compare the explanatory capabilities of the 4-CTSQ with those of the need for cognition scale to explore their respective contributions to understanding thinking styles.
2. Study 1
We pre-registered all materials, analyses, and hypotheses before any data collection. The pre-registration form can be seen at https://osf.io/epyg6/?view_only=1339b0eaa249410292e349c5dac4c25a.
2.1. Participants
We collected data from 869 university students in exchange for extra credit and a gift draw. As pre-registered, we used a time limit as a stopping rule and collected data from as many participants as possible over 2 weeks. We excluded participants who completed the survey extremely fast or slow based on z-score for completion time; z > 3 and z < 3 (n = 13), inattentive participants (n = 131), and participants who were identified as a multivariate outliers on 4-CTSQ items (n = 30, χ 2(24) = 51.18, p = .001). The final dataset consists of 695 submissions (Mage = 25.36, SD = 7.77; 485 female).
2.2. Materials
2.2.1. 4-CTSQ
We used the 24-item 4-CTSQ developed by Newton et al. (Reference Newton, Feeney and Pennycook2023). 4-CTSQ has 4 subscales as follows, each consisting of 6 items: AOT, CMT, PIT, and PET. To create the Turkish version of the scale, we first translated all items into Turkish. Then, we had the existing translations checked by 2 experts and used the versions agreed upon among experts. In the original version of the questionnaire, the word ‘belief’ is mentioned in various items. Since this word evokes religious beliefs among religious individuals (Stanovich and Toplak, Reference Stanovich and Toplak2019), we used the word ‘opinion’ instead of belief in the Turkish version. 4-CTSQ uses a 6-point response scale (1 = strongly disagree, 6 = strongly agree). Since the items with differing wording tend to form separate factors (Kam and Meyer, Reference Kam and Meyer2023), Newton et al. (Reference Newton, Feeney and Pennycook2023) adopted a uniform directional approach for all subscale items to enhance strong reliability. Consequently, in the default scale configuration, higher scores across all subscales correspond to greater intuitiveness, while lower scores indicate greater reflectiveness (Newton et al., Reference Newton, Feeney and Pennycook2023). Thus, the original wording of all items pertaining to AOT and PET subscales needs to be reversed to ensure that subscales to correspond to the thinking style they originally expressed. Therefore, in the results, when we refer to a ‘preference for effortful thinking’, it indicates a preference for a level of effortful thinking, counter to the default wording of the items. Turkish version of the 4-CTSQ can be seen at Supplementary Materials.
2.2.2. Conspiratorial beliefs
We used the 15-item Generic Conspiracist Beliefs Scale (Brotherton et al., 2013) to measure conspiratorial beliefs. The scale focuses on individual differences in generic conspiracist ideation (e.g., ‘Evidence of alien contact is being concealed from the public’), and it was adapted to Turkish by Alper et al. (Reference Alper, Bayrak and Yilmaz2021). It uses a 5-point response scale (1 = definitely not true, 5 = definitely true). Cronbach’s alpha was found to be .89.
2.2.3. COVID-19 conspiracy beliefs
In addition to generic conspiracist ideation, we measured conspiracy beliefs about COVID-19. To measure specific conspiracy beliefs about COVID-19, we used a 7-item scale developed by Salali and Uysal (Reference Salali and Uysal2021). The scale includes COVID-19 conspiracy statements generated from mainstream and alternative websites. An example item was ‘The spread of the coronavirus is something a group of powerful people deliberately do to make money and/or gain control’. It uses a 5-point response scale (1 = strongly disagree, 5 = strongly agree). Cronbach’s alpha was .91.
2.2.4. Cognitive reflection
We used Cognitive Reflection Test (CRT; Frederick, Reference Frederick2005) and Cognitive Reflection Test-2 (CRT-2; Thomson and Oppenheimer, Reference Thomson and Oppenheimer2016) to measure reliance on reflective thinking. CRT consists of 3 mathematical questions with 4 choices, whereas CRT-2 includes 4 verbal questions. All questions have one correct answer, and total scores are calculated by adding the number of correct answers. We used scores from the 2 tests separately in analyses, and high scores correspond to high analytical thinking (e.g., reflection) skills. Cronbach’s alphas for CRT-1 and CRT-2 were .67 and .52, respectively.
2.2.5. Base-rate problems
In addition to performance-based CRT measurements, we also measured cognitive style differences via standard base-rate questions. We used 2 types of base-rate problems from De Neys and Glumicic (Reference De Neys and Glumicic2008). The measure consists of 3 incongruent problems, including stereotypes that conflict with the large base-rate group and 2 congruent problems with stereotypes that match the large base-rate group. Each problem has 2 choices, one corresponding to base-rate conflicting and the other to base-rate respecting answers. Cronbach’s alphas for incongruent problems and congruent problems were .73 and .48, respectively.
2.2.6. Analytic cognitive style
We calculated an overall analytic cognitive style score by averaging the responses on the items of CRT-1, CRT-2, and incongruent base-rate (e.g., conflict) problems. Cronbach’s alpha was .53.
2.2.7. Religious beliefs
To measure religious beliefs, we used a religious belief subscale of the Paranormal Belief Scale developed by Arslan (Reference Arslan2010). Based on the original Paranormal Belief Scale (Tobacyk and Milford, Reference Tobacyk and Milford1983), which is frequently used in the literature, Arslan (Reference Arslan2010) developed a new scale adapting its content to Turkish culture and the Islam religion. The religious belief subscale consists of 10 items and uses a 4-point response scale (1 = strongly disagree, 4 = strongly agree). The scale focuses on the level of belief in various phenomena in Islam (e.g., God, heaven, spirit, angel). Cronbach’s alpha was found to be .93.
2.2.8. Paranormal beliefs
We measured nonreligious paranormal beliefs with another subscale of the Paranormal Belief Scale (Arslan, Reference Arslan2010). It consists of 17 items on a 4-point response scale (1 = strongly disagree, 4 = strongly agree). In this sub-dimension, items measure the level of paranormal beliefs that are not directly included in Islam (e.g., Signs, superstitions, soothsaying, mind reading). Cronbach’s alpha was .87.
2.2.9. Morality as cooperation
To measure moral concerns, we employed the Morality as Cooperation Questionnaire (MAC-Q; Curry et al., Reference Curry, Chesters and Van Lissa2019a), which was developed within the theoretical framework of Morality as Cooperation Theory (MAC; Curry, Reference Curry, Shackelford and Hansen2016). MAC posits that morality originated as a result of diverse strategies devised to address the cooperative challenges that arose in the context of evolution (Curry, Reference Curry, Shackelford and Hansen2016). Curry et al. (Reference Curry, Mullins and Whitehouse2019b) conducted an analysis of ethnographic data spanning a 300-year period, representing 60 diverse societies and found that the 7 distinct moral dimensions are universally esteemed across most cultures (i.e., family, group, reciprocity, heroism, deference, fairness, property). Based on this finding, Curry et al. (Reference Curry, Mullins and Whitehouse2019b) introduced self-report measure of moral concerns, MAC-Q, which consists of 2 distinct sections: ‘Relevance’ and ‘Judgment’. The first section is designed to assess how relevant each moral dimension is in deciding whether something is right or wrong, for instance, asking whether actions such as ‘whether or not someone acted to protect their family’ are morally relevant. The second section is designed to gauge individuals’ agreement with various moral judgments, including statements like ‘people should be willing to do anything to help a member of their family’. However, both the original study (i.e., Curry et al., Reference Curry, Mullins and Whitehouse2019b) and an independent test of the MAC-Q (i.e., Yilmaz et al., Reference Yilmaz, Harma and Doğruyol2021) showed that only the relevance section exhibited a factor structure suitable for use by showing good fit indices. Thus, in the current study, we exclusively used the relevance scale of the MAC-Q, which comprises of 21 items representing 7 moral dimensions, with 3 items for each. MAC-Q was adapted into Turkish by Yilmaz et al. (Reference Yilmaz, Harma and Doğruyol2021). The participants responded to each item with a slider between 0 and 100 (0 = not at all relevant, 100 = extremely relevant). Cronbach’s alpha scores of subscales ranged between .65 and .81.
2.2.10. Demographic form
We used the demographic form to measure age, sex, education, perceived social status, religiosity (1 = not religious at all, 7 = very religious), and ideology (1 = very liberal, 7 = very conservative). Each variable was measured by a single item.
2.3. Data analytical strategy
We ran several confirmatory factor analyses (CFA) to test the construct validity of 4-CTSQ. All models were tested in Mplus Version 8 (Muthen and Muthen, Reference Muthen and Muthen2015) using maximum likelihood estimation. In line with Newton et al. (Reference Newton, Feeney and Pennycook2023), we first tested a single-factor model where all items were loaded on a single latent factor. Then, we tested the uncorrelated 4-factor model, where latent correlations were fixed to zero, the correlated 4-factor model, where latent correlations were freely estimated, and the correlated 4-factor higher-order model, where all latent factors were loaded on a higher-order latent factor.
We used normal theory weighted least squares with chi-square to evaluate the model fit. Following Hu and Bentler’s (Reference Hu and Bentler1999) 2-index presentation strategy, we also reported the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root means square residual (SRMR). Accordingly, RMSEA values smaller than .06 and CFI/TLI values larger than .95 represent an excellent fit, while RMSEA values smaller than .08 and CFI/TLI values larger than .90 represent a good model fit. Nested models were compared using the Wald test, and changes in TLI (ΔTLI) and RMSEA (ΔRMSEA) where changes larger than .002 in the expected direction are considerable (Meade et al., Reference Meade, Johnson and Braddy2008). We also used the Bayesian information criterion (BIC) to compare unnested models; a decrease larger than 10 represents a better model fit (Raftery, 1995). To test reliabilities, we used Hancock and Mueller’s (Reference Hancock, Mueller, Cudeck, du Toit and Sorbom2001) H value in which items are weighted based on their factor loadings, scores larger than .80 indicate stable latent factor structure.
2.4. Results
2.4.1. Construct validity
As presented in Table 1, the single-factor model revealed a poor fit to the data, while the remaining models approached a good model fit. The correlated model was superior to an uncorrelated model based on the Wald test (W (6) = 19.49, p = .003) and changes in fit indices (ΔTLI = .022, ΔRMSEA = .007). Modification indices suggested correlated error between 2 items (‘Either something is true, or it is false; there is nothing in-between’ and ‘There is no middle ground between what is true and what is false’). Since items are semantically similar, we decided to add an error correlation between items. As a result, the best-fitting model was modified correlated 4-factor model with a good fit to the data (χ 2 = 1083.79, df = 245, p < .001; CFI = .914; TLI = .903; RMSEA = .070, 90% CI [.066, .074]). Higher-order counterparts of correlated 4-factor models revealed a comparable fit with the first-order versions (ΔBIC’s < 10), suggesting that 4 factors could represent a higher-order structure. Standardized factor loadings and reliability coefficients for the final correlated 4-factor model were presented in Table 2.
Table 1 Fit indices of the 4-CTSQ
Note: The final model is bolded.
Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; PET, preference for effortful thinking; PIT, preference for intuitive thinking.
a 4-CTSQ higher-order factor was fixed at 1 in Model 3 to allow for model convergence.
Table 2 Standardized factor loadings for the 4-CTSQ
Note: Parenthetical values represent loadings for Study 2.
Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; PET, preference for effortful thinking; PIT, preference for intuitive thinking.
a Reverse coded items.
2.4.2. Criterion validity
We first calculated the correlation between the 4 subscales of 4-CTSQ. Correlations between the 4 dimensions were significant except for PIT and PET (r = .05, p = .221). All significant correlations were in the expected direction, with the strongest correlation between AOT and PIT (r = −.42, p < .001) in line with the original study (Table 3).
Table 3 Descriptive statistics for the 4-CTSQ dimensions
Note: Parenthetical values represent descriptive statistics for Study 2. The lower diagonal represents correlations for Study 1, and the upper diagonal represents correlations for Study 2.
* p < .001.
We then calculated the correlation between 4-CTSQ subscales and outcome variables.Footnote 3 All 4 dimensions were significantly correlated with analytic thinking style measures with few exceptions. Consistently, AOT (ranged between .17 and .26) and PET (ranged between .07 and .16) were positively correlated with analytic thinking performance scores, performance on stereotype challenging base-rate problems, and composite analytic cognitive style score, while CMT (ranged between −.20 and −.32) and PIT (ranged between −.03 and −.19) were negatively correlated with performance measures of thinking styles. A reverse pattern was evident across epistemically suspect belief measures; while AOT (ranged between −.15 and −.29) and PET (ranged between −.01 and −.15) were negatively correlated with paranormal, religious, conspiracy, and COVID-19 conspiracy beliefs, CMT (ranged between .07 and .26) and PIT (ranged between .12 and .22) were positively correlated with those belief measures. Across ideological measures, AOT and PET were negatively correlated with resistance to change (r AOT = −.35, p < .001, r PET = −.17, p < .001) and right-wing political orientation (r AOT = −.14, p < .001, r PET = −.10, p = .007) whereas CMT and PIT were positively correlated with resistance to change (r CMT = .43, p < .001, r PIT = .28, p < .001) and CMT was correlated with right-wing political orientation (r = .15, p < .001). Correlations between 4-CTSQ dimensions and MAC-Q dimensions were significant for most of the pairs and in the expected direction. The results of the correlation analyses are summarized in Table 4.
Table 4 Correlations between the 4-CTSQ dimensions and outcome variables
Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; MAC-Q, Morality-as-Cooperation Scale; MFQ, moral foundations Questionnaire; NFC, need for cognition; PET, preference for effortful thinking; PIT, preference for intuitive thinking; S1, study 1; S2, study 2.
* p < .05.
** p < .001.
To investigate the unique contribution of each 4-CTSQ subscale on outcome variables, we run a series of regression analyses (Table 5). Results demonstrated that AOT and CMT consistently predicted analytic thinking performance except for stereotype congruent base-rate problems (AOT βs ranged between .09 and .14, and CMT βs ranged between −.15 and −.25). Among the regression on belief measures, AOT (βs ranged between −.06 and −.19) and PIT (βs ranged between .12 and .26) significantly predicted all belief measures with one exception (AOT on conspiracy beliefs). Moreover, while AOT positively predicted resistance to change and political orientation, CMT predicted the same variables in the opposite direction. Regression analyses on MAC-Q dimensions also revealed a consistent pattern; PIT and CMT (except for fairness and property) positively predicted morality dimensions, while AOT and PET were mostly nonsignificant. Overall, AOT and CMT were predictive of analytic thinking performance, belief measures, and ideological measures, while PIT and CMT were predictive of morality dimensions.
Table 5 Standardized regression coefficients for the 4-CTSQ dimensions on outcome variables
Note: Parenthetical values represent descriptive statistics for Study 2.
Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; MAC-Q, Morality-as-Cooperation Scale; MFQ, moral foundations questionnaire; PET, preference for effortful thinking; PIT, preference for intuitive thinking; S1, study 1; S2, study 2.
* p < .05.
** p < .001.
2.5. Exploratory analysis
To explore whether the 4-CTSQ dimensions distinctively predict outcome measures or whether there is an underlying factor that accounts for the relationships with outcome measures, we ran a canonical correlation analysis between the 2 sets of variables. To further explore the relationship between 4-CTSQ dimensions and criterion variables, we ran a canonical correlation analysis between the 2 sets of variables. The analysis revealed that first (Wilks’s λ = .561, F(56, 2635.56) = 7.85, p < .001), second (Wilks’s λ = .103, F(39, 2008.45) = 2.76, p < .001), and third (Wilks’s λ = .039, F(24, 1358) = 1.61, p = .031) canonical variates were significant while the last one was not significant (Wilks’s λ = .018, F(11, 680) = 1.11, p = .348). Significant canonical variates explained 36%, 9%, and 4% of the variance, respectively. Accordingly, there were 3 factors (canonical correlates) to explain the relationships with the outcome measures. Therefore, relationships between 4-CTSQ dimensions and outcome measures cannot be attributed to a higher-order underlying cognitive thinking style factor.
3. Study 2
The first study’s findings provided preliminary implications for the 4-CTSQ to be a reliable and valid measure for detecting various psychological characteristics and beliefs based on thinking style differences. We conducted a second study to test the consistency and replicability of the existing findings in a new sample by further expanding our measurement tools. In the second study, we used the same measures from the first study as follows: 4-CTSQ, Conspiratorial Beliefs, COVID-19 Conspiracy Beliefs, Cognitive Reflection, Cognitive Style, Religious Beliefs, Paranormal Beliefs, Resistance to Change, and Demographic Form. We also expanded our measures with moral foundations, pro-vaccination, the need for cognition, and incentivized measures of prosociality. As in the first study, we pre-registered all materials, analyses, and hypotheses before the study. The pre-registration form can be seen at https://osf.io/jhkwz/?view_only=0c09cf0337c6466e8ca3b8a461301c77.
3.1. Participants
We reached 540 university students who participated study in exchange for extra course credit. We used a time limit as a stopping rule, as in Study 1, and collected data from as many participants as possible over 3 weeks. As pre-registered, we excluded participants who completed the survey extremely fast or slow based on z-score for completion time; z > 3 and z < 3 (n = 4), participants who failed in attention check question (n = 25), and participants who were identified as a multivariate outlier on 4-CTSQ items (n = 29, χ 2(24) = 51.18, p = .001). The final sample size was 486 (Mage = 21.91, SD = 2.41; 419 female).
3.2. Materials
3.2.1. Moral foundations questionnaire
We used the relevance section of the Moral Foundations Questionnaire (MFQ; Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011). The scale consists of 15 items representing 5 moral foundations (care, fairness, loyalty, authority, and sanctity), with 3 items for each. This section measures the extent to which various considerations are relevant to individuals’ thinking when deciding whether something is right or wrong. An example item was ‘Whether or not someone showed a lack of respect for authority’. We used the Turkish version of the MFQ, which was adapted into Turkish by Yilmaz et al. (Reference Yilmaz, Harma, Bahçekapili and Cesur2016a, Reference Yilmaz, Karadöller and Sofuoglu2016b). Participants responded to items with a 6-point scale (0 = not effective at all, 6 = very effective). Cronbach’s alpha scores of subscales ranged between .51 and .74.
3.2.2. Pro-vaccination attitudes
We measured pro-vaccination attitudes and intentions with 3 items as follows: ‘COVID-19 vaccines are safe’, ‘Generally, vaccines are safe’, and ‘When it is my turn, I will get the COVID-19 vaccine’. Participants responded to items with a 7-point response scale (1 = strongly disagree, 7 = strongly agree). Cronbach’s alpha was .86.
3.2.3. Prosociality
We used an incentivized donation task to measure prosocial behavior against the pandemic. The item asks each participant what percentage of money (200) they would want to give to individuals economically affected by the pandemic. Drawing upon prior literature (e.g., Charness et al., 2016), we employed a random lottery methodology, a cost-saving approach, wherein actual decisions are subjected to randomized compensation for some (but not all) participants.
3.2.4. Need for cognition
We used the Need for Cognition scale (NFC; Cacioppo et al., Reference Cacioppo, Petty and Feng Kao1984) to compare with CTSQ and examine which has more predictive validity. The scale was adopted into Turkish by Gülgöz and Sadowski (1995). It consists of 18 items on a 9-point response scale (−4 = extremely uncharacteristic of me; 4 = extremely characteristic of me). Cronbach’s alpha was found to be .91.
3.3. Results
3.3.1. Construct validity
We first cross-validated the factor structure of the 4-CTSQ scale on the final version of the scale in Study 1. CFA results revealed a good fit to the data for the modified correlated 4-factor model (χ 2 = 842.48, df = 245, p < .001; CFI = .921; TLI = .911; RMSEA = .071, 90% CI [.066, .076]). Standardized factor loadings and reliability coefficients for the scale are presented in Table 2.
3.3.2. Criterion validity
As in the first study, we first tested the correlation between subscales of 4-CTSQ (Table 3).Footnote 4 Results aligned with the first study except for the correlation between PIT and PET (r = −.15, p = .001). Again, the largest correlation was between AOT and PIT (r = −.47, p < .001). Correlations between 4-CTSQ subscales and thinking style measures supported the first study’s results with few exceptions (Table 4). Similarly, correlations of resistance to change and political ideology were comparable regarding direction and magnitude. While none of the correlations were significant for a behavioral measure of prosociality, all the correlations were significant for pro-vaccination attitudes in the expected direction (ranged between −.24 and .29). Among the correlations between moral foundations and 4-CTSQ dimensions, while AOT (ranged between −.24 and −.28) and PIT (ranged between −.11 and −.19) were negatively correlated with binding foundations; loyalty, authority, and purity, CMT (ranged between .12 and .28) and PIT (ranged between .16 and .21) were positively correlated with the same variables. Care and fairness were not correlated with the 4-CTSQ dimensions.
We also investigated the unique contribution of each 4-CTSQ dimension on outcome variables in a series of regression analyses. Regressions on analytic thinking style and belief measures revealed that the direction and magnitude of the standardized coefficients were in line with the first study except for CMT. Contrary to the first study’s results, CMT failed to predict analytic thinking performance measures; however, CMT is still predictive of belief measures. Regressions on ideological measures were mostly in line with the first study, except that PIT failed to predict resistance to change. Pro-vaccination attitudes were predicted by AOT (β = .21, p < .001) and PIT (β = −.13, p = .011, F (4, 481) = 13.95, p < .001). Regressions on moral foundations showed that most of the betas in predicting binding foundations (i.e., loyalty, authority, and purity) were significant. As expected, AOT and PET negatively predicted binding foundations, while CMT and PIT positively predicted those foundations (Table 5). On average, 4-CTSQ explained 9% of the variance on 18 outcome variables. We averaged the squared correlation between the need for cognition and the same outcome variables (2%) to compare the explanatory power of the 2 scales. Results of a related-samples Wilcoxon signed rank test indicated that 4-CTSQ, on average, explained more variance than the need for cognition scale (Z = 3.62, p = .002).
3.4. Exploratory analysis
As in the first study, we ran a canonical correlation analysis between 4-CTSQ dimensions and criterion variables. Results revealed that while first (Wilks’s λ = .637, F(56, 1818.70) = 5.92, p < .001) and second (Wilks’s λ = .253, F(39, 1386.60) = 1.62, p = .010) canonical variates were significant, third (Wilks’s λ = .204, F(24, 938) = 1.30, p = .151) and fourth (Wilks’s λ = .151, F(11, 470) = 0.99, p = .453) variates were not significant. Canonical variates explained 41%, 7%, 4%, and 2% of the variance, respectively.
4. Discussion
The purpose of the current research was twofold. First, we sought to investigate whether the Turkish version of the 4-factor multiple structures of thinking styles fits the data well. Second, we aimed to assess the replicability of Newton et al.’s (Reference Newton, Feeney and Pennycook2023) findings on outcome measures. The results provided novel evidence beyond the original study, supporting the claim that intuitive–analytic thinking styles vary across at least 4 distinct elements: Actively open-minded thinking, close-minded thinking, preference for effortful thinking, and preference for intuitive thinking. Consistent with Newton et al. (Reference Newton, Feeney and Pennycook2023), we found that these elements differentially predicted various outcomes, although we could not replicate all their findings. Furthermore, we validated the Turkish version of the 4-CTSQ and demonstrated that it is functional for understanding a wide range of beliefs in a non-WEIRD culture, Türkiye. To our knowledge, our research is the first one that independently attempts to replicate the original 4-factor structure of thinking styles and contributes to clarifying the theoretical and measurement complexities in the literature and enhancing our understanding of the dimensions of thinking style differences.
4.1. Epistemically suspect beliefs
We measured 3 beliefs about conspiracy and paranormal beliefs, which are prominent components of Epistemically Suspect Beliefs (Pennycook et al., Reference Pennycook, Fugelsang and Koehler2015), including conspiratorial beliefs, COVID-19 conspiracy beliefs, and paranormal beliefs. Our findings indicated that actively open-minded thinking was a negative predictor of conspiratorial beliefs and COVID-19 conspiracy beliefs, while preference for intuitive thinking positively predicted both. Furthermore, closed minded-thinking positively predicted COVID-19 conspiracy beliefs. However, preference for effortful thinking did not significantly predict either conspiracy measure. These results align with Newton et al. (Reference Newton, Feeney and Pennycook2023), suggesting that preference for intuitive and effortful thinking does not correspond to parallel factors within the same dimension. While preference for intuitive thinking demonstrated explanatory power for conspiratorial disposition, preference for effortful thinking did not exhibit characteristics of critical thinking and skepticism, which have been associated with decreased conspiracy beliefs (Dyer and Hall, Reference Dyer and Hall2019). Empirical support for this distinction can also be found in the literature. Faith in Intuition and the Need for Cognition have been identified as separate factors and generally do not show a strong correlation (Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996). In a recent study by Wu et al. (Reference Wu, Kuru, Campbell and Baruh2022), the Need for Cognition was positively associated with health misinformation, including conspiracy beliefs, whereas Faith in Intuition showed a negative relationship with these beliefs. Considering that the preference for intuitive thinking items was mainly derived from the Faith in Intuition scale and all preference for effortful thinking items were taken from the Need for Cognition scale, this finding strengthens the idea that these factors represent distinct cognitive characteristics rather than being opposites.
Similar patterns were observed in religious belief and paranormal belief outcomes. Actively open-minded thinking negatively predicted religious and paranormal beliefs, while Close-minded thinking positively predicted all measures except for paranormal belief in Study 1. However, there was not a consistent relationship between preference for effortful and intuitive thinking. Preference for intuitive thinking positively predicted all measures of religious and paranormal beliefs, while preference for effortful thinking had a significant effect only on the paranormal belief measure in Study 1. Thus, differentiation solely based on the preference for effortful thinking versus relying solely on intuition seems insufficient to fully account for the disparities between analytic and intuitive thinking. There is also evidence indicating that this holds true for analytic thinking measures based on performance. CRT is an insufficient tool to comprehensively capture both analytic and intuitive thinking styles. It primarily assesses reflective thinking and may not serve as a valid indicator of intuitive thinking (Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b). Hence, relying solely on CRT scores to distinguish between thinking styles may not provide a comprehensive understanding. Incorporating the preference for effortful thinking and preference for intuitive thinking dimensions offers a more nuanced and comprehensive insight into thinking styles than simply differentiating between low and high CRT scores.
4.2. Ideological orientation
Among the measures included in both studies, resistance to change, political ideology, and religious beliefs correspond to the main components of ideological orientation. Specifically in Türkiye, religiosity (Çarkoğlu, Reference Çarkoğlu2007) and resistance to change (Yilmaz and Saribay, Reference Yilmaz and Saribay2018) are prominent determinants of political preferences. Our findings demonstrated that the subscales of the 4-CTSQ have different predictive powers on these 3 measures of ideological orientation. Actively open-minded thinking and close-minded thinking consistently predicted all 3 components of ideological orientation across all measures in both studies. However, there was no such consistency for preference for intuitive thinking and effortful thinking. This suggests that actively open-minded thinking and close-minded thinking play a unique role in understanding the relationship between thinking style differences and ideological orientation (Baron, Reference Baron2019). This distinction would have been overlooked if the focus were solely on differentiating thinking styles based on effortful thinking and reliance on intuition. Although reflection and actively open-minded thinking are related cognitive features, they are not identical (Baron et al., Reference Baron, Gürçay, Metz, Toplak and Weller2017). This dissociation between the subscales of the 4-CTSQ concerning outcomes related to ideological orientation is supported by previous research. While social conservatism can be predicted through reflection scores based on CRT measures, some studies have found no relationship with economic conservatism (e.g., Baron, Reference Baron2015). On the other hand, research indicates that actively open-minded thinking has a stronger relationship with conservatism compared to CRT (e.g., Baron, Reference Baron2017; Svedholm-Häkkinen and Lindeman, Reference Svedholm-Häkkinen and Lindeman2018; Yilmaz and Saribay, Reference Yilmaz and Saribay2017c). Thus, actively open-minded thinking may be a better predictor of political attitudes, especially when religious commitments influence political beliefs (Baron, Reference Baron2019).
4.3. Prosociality and moral concerns
Beyond the intention measures of Newton et al. (Reference Newton, Feeney and Pennycook2023), we added an outcome measure focusing on actual behavior in Study 2. We asked participants how much of the money we gave them they would like to donate to a person economically affected by the COVID-19 pandemic. Thus, we aimed to measure prosociality with an incentivized donation task that includes the potential for real money losses and gains. This approach was essential as there is a recognized gap between intention and actual behavior (e.g., Sheeran and Webb, Reference Sheeran and Webb2016), and various studies suggest that thinking styles can lead to differences in intention versus behavior measures. However, we found neither a predictive role nor a correlation of any subscale of the 4-CTSQ on prosociality scores. This finding contrasts with previous studies that have suggested a relationship between thinking style differences and prosocial behaviors. Although there is a debate about whether analytic or intuitive thinking leads to more prosocial or selfish behavior, numerous studies indicated that intuition increases prosociality (e.g., Bear and Rand, Reference Bear and Rand2016; Kinnunen and Windmann, Reference Kinnunen and Windmann2013; Rand, Reference Rand2016). Therefore, it was expected that subscales of 4-CTSQ would somehow be related to prosociality. One possible reason for not finding a significant relationship may be related to the specific theme of our prosociality question. The COVID-19 pandemic was a global crisis that resulted in millions of people losing their lives, with negative effects observed not only in health but also in various aspects of life, including the economy and social spheres. Hence, participants’ responses to the prosociality question may have been influenced by this prominent theme. Another reason could be linked to the measurement method used. In the field of behavioral economics, prosociality is often assessed through economic games with well-defined instructions and rules (e.g., Ultimatum game, Trust game, Dictator game, and Public Goods Game). These games aim to measure prosocial tendencies in a manner that minimizes confounding factors. However, in our study, we simply asked participants whether they would be willing to donate a given amount of money, and this may not fully capture prosociality due to the simplicity of the measure. Therefore, we encourage future studies to systematically focus on behavioral measures of prosociality and explore the validity of the 4-CTSQ through the use of economic games. Such investigations will provide more comprehensive insights into the relationship between thinking style differences and prosocial behavior.
On the other hand, we found significant relationships between subscales of 4-CTSQ and morality measures, which are assessing moral commitments that enhance cooperation. Although these measures do not capture cooperative intentions or behaviors directly, such as paradigms that specifically address cooperation (e.g., public goods game), they do assess commitments that are influenced by cooperativeness. Moral behavior is fundamentally centered around cooperation with anonymous individuals, and the root of human morality lies in the motivations that promote cooperative actions (Curry, Reference Curry, Shackelford and Hansen2016; Haidt, Reference Haidt2012). Consequently, MAC-Q and the MFQ essentially assess levels of prosocial tendencies in different domains. Moreover, our measure of prosociality pertains to applied ethics, while the measures of moral concerns are aligned with normative ethics. In this regard, we found that subscales of 4-CTSQ exhibited differential and varying predictive patterns concerning cooperative intentions as measured by MAC-Q or MFQ. Particularly, preference for intuitive thinking positively predicted all 7 dimensions of MAC-Q, while actively open-minded thinking and close-minded thinking were consistently associated with binding foundations of MFQ. The association of open-minded thinking with binding foundations tended to be positive, whereas the association of close-minded thinking tended to be negative, in line with previous research (e.g., Baron et al., Reference Baron, Scott, Fincher and Metz2015). Therefore, it seems that 4-CTSQ was not found to explain variance in behavior-based measures of prosociality while predicting intention-based morality measures. In another recent experimental study conducted in Türkiye, Varol (Reference Varol2023) observed that the public goods game, assessed through economic games and MAC-Q, purported to measure cooperative intentions, was impacted differently following the earthquake threat. This highlights that MAC-Q cannot be employed as a measure of cooperation, and its correlation with actual behavior cannot be assumed without caution (Dogruyol et al., Reference Dogruyol, Velioglu, Bayrak, Acem, Isler and Yilmaz2023). Further studies, specifically focusing on prosociality’s intention and behavioral aspects, are required to elucidate the association between thinking style differences and prosocial tendencies more comprehensively.
4.4. Validity of the 4-CTSQ and cultural implications
We concluded that 4-CTSQ possesses acceptable psychometric properties and can be utilized in future studies within Türkiye, a non-WEIRD country (Muthukrishna et al., Reference Muthukrishna, Bell, Henrich, Curtin, Gedranovich, McInerney and Thue2020). Based on our results across all outcomes, similar to Newton et al. (Reference Newton, Feeney and Pennycook2023), actively open-minded thinking appears to be the most consistent predictor, while preference for effortful thinking exhibits the weakest predictive validity. This finding aligns with previous research, which suggests that having a propensity to question one’s intuitions and critically evaluate novel evidence may have a stronger impact than merely favoring effortful thinking (Baron, Reference Baron2019; Baron et al., Reference Baron, Scott, Fincher and Metz2015, Reference Baron, Isler, Yılmaz, Ottati and Stern2023; Newton et al., Reference Newton, Feeney and Pennycook2023). However, it is important to note that we were unable to fully replicate some of the findings of Newton et al. (Reference Newton, Feeney and Pennycook2023). In their study, they observed a positive association between close-minded thinking and religious belief but a negative relationship with paranormal belief, highlighting a critical difference between these related beliefs. In contrast, our findings did not indicate such a dissociation; close-minded thinking was positively related to both religious and paranormal beliefs. This finding can imply that this association can vary across different cultures, given that paranormal belief is regarded as a component of institutionalized religion in certain cultural contexts (Yilmaz, Reference Yilmaz2021). These potential outcomes should be investigated in forthcoming studies. Nevertheless, our results are consistent with previous studies that reported a negative link between analytic thinking and both religious and paranormal beliefs (e.g., Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b).
5. Conclusion
Our study supports that individuals differ along multiple dimensions of intuitive–analytic thinking styles with 4 related, but distinct factors and the pattern of correlations were explained with at least 2 factors. Generally coinciding with Newton et al. (Reference Newton, Feeney and Pennycook2023) findings, we showed that certain thinking styles are more pertinent to certain outcomes, implying that categorizing individuals as either ‘intuitive’ or ‘reflective’ oversimplifies the complexities of thinking styles. Therefore, relying on single-dimensional measures for intuitive–analytic thinking styles may result in inaccurate conclusions about the significance of these thinking styles. However, it should be noted that the outcome measures were not designed to capture the distinctions between the 4-CTSQ dimensions, and therefore, results were partially dependent on the outcome measures used in the study. In this study, we demonstrated that the 4-CTSQ can effectively predict a broad array of outcomes within a non-WEIRD cultural context. Looking ahead, future research could further test the predictive validity of the 4-CTSQ and elucidate the particularities of thinking styles’ multidimensional aspects by examining related outcomes not covered in this study. Additionally, future investigations should assess the 4-CTSQ’s capacity to predict real-life behaviors, such as academic achievement, across various time intervals between predictor and outcome, and within diverse cultural settings.
Supplementary material and data
Supplementary material and data for this article can be found at https://doi.org/10.1017/jdm.2023.45.
Funding statement
This research received no specific grant funding from any funding agency, commercial, or not-for-profit sectors.
Competing interest
The authors declare none.
Open access
