Hostname: page-component-7bb8b95d7b-fmk2r Total loading time: 0 Render date: 2024-10-03T19:09:42.807Z Has data issue: false hasContentIssue false

Multidimensional intuitive–analytic thinking style and its relation to moral concerns, epistemically suspect beliefs, and ideology

Published online by Cambridge University Press:  18 December 2023

Fatih Bayrak
Affiliation:
Department of Psychology, Baskent University, Ankara, Türkiye
Burak Dogruyol
Affiliation:
Department of Psychology, Kadir Has University, Istanbul, Türkiye
Sinan Alper
Affiliation:
Department of Psychology, Yaşar University, Izmir, Türkiye
Onurcan Yilmaz*
Affiliation:
Department of Psychology, Kadir Has University, Istanbul, Türkiye
*
Corresponding author: Onurcan Yilmaz; Email: onurcan.yilmaz@khas.edu.tr
Rights & Permissions [Opens in a new window]

Abstract

Literature highlights the distinction between intuitive and analytic thinking as a prominent cognitive style distinction, leading to the proposal of various theories within the framework of the dual process model. However, it remains unclear whether individuals differ in their thinking styles along a single dimension, from intuitive to analytic, or if other dimensions are at play. Moreover, the presence of numerous thinking style measures, employing different terminology but conceptually overlapping, leads to confusion. To address these complexities, Newton et al. suggested the idea that individuals vary across multiple dimensions of intuitive–analytic thinking styles and distinguished thinking styles between 4 distinct types: Actively open-minded thinking, close-minded thinking, preference for effortful thinking, and preference for intuitive thinking. They proposed a new measure for this 4-factor disposition, The 4-Component Thinking Styles Questionnaire (4-CTSQ), to comprehensively capture the psychological outcomes related to thinking styles; however, no independent test exists. In the current pre-registered studies, we test the validity of 4-CTSQ for the first time beyond the original study and examine the association of the proposed measure with various factors, including morality, conspiracy beliefs, paranormal and religious beliefs, vaccine hesitancy, and ideology in an underrepresented culture, Türkiye. We found that the correlated 4-factor model of 4-CTSQ is an appropriate measure to capture individual differences based on cognitive style. The results endorse the notion that cognitive style differences are characterized by distinct structures rather than being confined to two ends of a single continuum.

Type
Empirical Article
Creative Commons
Creative Common License - CCCreative Common License - BY
This is an Open Access article, distributed under the terms of the Creative Commons Attribution licence (https://creativecommons.org/licenses/by/4.0), which permits unrestricted re-use, distribution and reproduction, provided the original article is properly cited.
Copyright
© The Author(s), 2023. Published by Cambridge University Press on behalf of Society for Judgment and Decision Making and European Association of Decision Making

1. Introduction

A widely accepted notion in behavioral sciences posits that human beings possess 2 distinct cognitive processes, referred to as Type 1 and Type 2 (Evans, Reference Evans2003, Reference Evans2008; Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009; Kahneman, Reference Kahneman2011; Stanovich and West, Reference Stanovich and West2000). Type 1 involves autonomously operating subcognitive systems, which are evolutionarily prevalent in many animals, and operate through fast, automatic, effortless, and unconscious cognitive processes (Evans, Reference Evans2008). On the other hand, Type 2, representing a more human skill, is characterized by slow, deliberative, effortful, conscious, and controlled processes (Evans, Reference Evans2003) that rely on working memory, intelligence, and cultural factors (Barrett et al., Reference Barrett, Tugade and Engle2004; Evans, Reference Evans2003; Nisbett et al., Reference Nisbett, Peng, Choi and Norenzayan2001). Based on these distinct cognitive processes, thinking styles are commonly categorized into intuitive and analytic thinking. Examining these thinking styles and their related outcomes falls under the purview of the dual process model (DPM; Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009). Despite numerous studies exploring the effects of thinking styles across various domains (see Pennycook et al., Reference Pennycook, Fugelsang and Koehler2015 for a review) and the numerous DPM theories proposed to date (e.g., Budner, Reference Budner1962; Cacioppo and Petty, Reference Cacioppo and Petty1982; Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996; Kruglanski et al., Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013; Ståhl et al., Reference Ståhl, Zaal and Skitka2016), a theoretical consensus regarding the nature of cognitive process and thinking style differences remains ambiguous. It is still unclear whether people differ in their thinking styles on a single dimension, from intuitive to analytic, and whether there are other thinking styles (Newton et al., Reference Newton, Feeney and Pennycook2023). In addition, many different measures (e.g., cognitive reflection test, faith in intuition, need for closure, need for cognition, preference for intuition and deliberation, and importance of rationality) have been proposed so far within the framework of DPM, but the findings raise a question over the necessity of this complex variety concerning predicting very similar psychological factors and conceptual overlapping, albeit using different terminology. Consequently, there is a pressing need for clarifying structures of thinking style differences and an easy-to-use measure that comprehensively captures various aspects of thinking styles.

In this context, Newton et al. (Reference Newton, Feeney and Pennycook2023) asserted that individuals differ along multiple dimensions of intuitive–analytical thinking styles rather than simply varying between 2 ends of a single dimension. They delineated thinking styles into 4 components: Actively Open-minded Thinking, Close-minded Thinking, Preference for Intuitive Thinking, and Preference for Effortful Thinking. Introducing the 4-Component Thinking Styles Questionnaire (4-CTSQ) as a novel measure of cognitive style, Newton et al. (Reference Newton, Feeney and Pennycook2023) demonstrated its efficacy in elucidating diverse psychological outcomes associated with thinking styles, such as moral judgments, conspiracy beliefs, vaccination intentions, misperceptions about COVID-19, and truth discernment. Nevertheless, several aspects remain unexplored: (1) Newton’s 4-dimensional intuitive–analytic thinking styles proposition has not yet been tested beyond the original study, (2) certain findings contradict previous studies in the literature, and (3) the validity of the proposed measure in non-WEIRD cultures remains unvalidated. Thus, in 2 pre-registered studies, we aim to test the assertion that individuals differ along multiple dimensions of intuitive–analytic thinking styles, with 4 factors as proposed by Newton et al. (Reference Newton, Feeney and Pennycook2023). Additionally, we seek to replicate the original findings of Newton et al. (Reference Newton, Feeney and Pennycook2023) and assess the validity of the 4-CTSQ in a non-WEIRD culture, specifically Türkiye. Furthermore, we examine whether 4-CTSQ predicts various psychological outcomes associated with thinking styles, such as morality as cooperation, paranormal and religious beliefs, generic and COVID-19 conspiracy beliefs, religiosity, resistance to change, and ideology in a non-WEIRD culture. Furthermore, we compare the predictive power of the 4-CTSQ with existing performance-based measures of thinking styles in the literature, such as cognitive reflection tests and base-rate conflict problems.

1.1. Multiple DPM approaches and measures on thinking styles

Although the idea of DPM, albeit in a different form, was clearly proposed in the late 19th century (e.g., Bryan and Harter, Reference Bryan and Harter1899), recent interest in DPM approaches began with the cognitive revolution in the 1970s (Frankish and Evans, Reference Frankish, Evans, Evans and Frankish2009) and has since been extensively studied across diverse contexts, capturing the attention of scholars from various fields, including social cognition (e.g., Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996), psychology of learning (e.g., Yonelinas, Reference Yonelinas1994), developmental psychology (e.g., Klaczynski, Reference Klaczynski2004), and behavioral economics (e.g., Kahneman, Reference Kahneman2011). Within the framework of the DPM, multiple theoretical approaches have been introduced, each focusing on distinct aspects of thinking style differences and accompanied by their own conceptualizations and measures (see Evans, Reference Evans2003; Evans and Stanovich, Reference Evans and Stanovich2013; Osman, Reference Osman2004 for reviews).

Numerous scholars have developed conceptualizations and measurement scales that align with the disposition for reflective thinking. For instance, Cacioppo and Petty (Reference Cacioppo and Petty1982) introduced the Need for Cognition Scale, assessing one’s inclination for cognitive activities that require effort and their enjoyment of such activities. Going beyond the thinking more/less and reflection/impulsivity dualism, several scholars focused on the idea of what a general theory of good thinking should include (Baron et al., Reference Baron, Isler, Yılmaz, Ottati and Stern2023). In this regard, Baron (Reference Baron1993) elucidated a reasoning style marked by the proclivity to scrutinize new evidence in juxtaposition with one’s preexisting convictions, to allocate sufficient time to problem-solving endeavors before conceding abandoning, and to judiciously integrate the viewpoints of others in the process of shaping one’s own perspectives (Haran et al., Reference Haran, Ritov and Mellers2013). Then, different measurement tools have been proposed to measure this construct. For example, Stanovich and West (Reference Stanovich and West1997) emphasized the tendency to act with an open mind and revise existing beliefs upon new evidence, reflecting one of the implications of effortful thinking, and proposed the Actively Open-Minded Thinking Scale. In contrast, others sought to identify intuitive thinking dispositions using analogous conceptualizations but employing different measures, with measures like Epstein et al.’s (Reference Epstein, Pacini, Denes-Raj and Heier1996) Faith in Intuition Scale or Kruglanski et al.’s (Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013) Need for Closure Scale. Some scholars combined both dimensions, resulting in tools like Scott and Bruce’s (Reference Scott and Bruce1995) General Decision-Making Style Questionnaire, which assesses thinking style across 5 factors, and Betsch’s (Reference Betsch2004) Preference for Intuition and Deliberation Scale. Various comparable approaches have been proposed to date, with the primary objective of capturing thinking style differences based on the intuitive–analytic distinction (see Newton et al., Reference Newton, Feeney and Pennycook2023 for a comprehensive summary).

Drawing from this distinction, extensive literature shows that differences in intuitive–analytic thinking styles are associated with various outcomes, including beliefs, religiosity, ideology, and morality. For example, thinking style differences serve as a robust predictor of epistemically suspect beliefs: an increase in analytic thinking is linked to a decrease in conspiracy beliefs and paranormal beliefs (Alper et al., Reference Alper, Bayrak and Yilmaz2021; Bouvet and Bonnefon, Reference Bouvet and Bonnefon2015; Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012; Šrol, Reference Šrol2022; van Prooijen, Reference van Prooijen2017; Yelbuz et al., Reference Yelbuz, Madan and Alper2022). Studies also show that intuitive thinking is associated with religiosity (Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b; Yilmaz, Reference Yilmaz2021). Intuition positively correlates with religious belief (Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012), resulting in a stronger belief in God (Saribay et al., Reference Saribay, Yilmaz and Körpe2020; Shenhav et al., Reference Shenhav, Rand and Greene2012). A similar pattern emerges concerning ideology, which is closely related to religiosity (Caprara et al., Reference Caprara, Vecchione, Schwartz, Schoen, Bain, Silvester, Cieciuch, Pavlopoulos, Bianchi, Kirmanoglu, Baslevent, Mamali, Manzi, Katayama, Posnova, Tabernero, Torres, Verkasalo, Lönnqvist and Caprara2018; Piurko et al., Reference Piurko, Schwartz and Davidov2011). Conservatives tend to rely more on their intuition, whereas liberals tend to depend more on analytic thinking (Deppe et al., Reference Deppe, Gonzalez, Neiman, Jacobs, Pahlke, Smith and Hibbing2015; Eidelman et al., Reference Eidelman, Crandall, Goodman and Blanchar2012; Haidt, Reference Haidt2012; Jost, Reference Jost2017). In addition, some findings suggest that inducing intuitive thinking can diminish liberal tendencies and cause a shift toward conservatism (e.g., Skitka et al., Reference Skitka, Mullen, Griffin, Hutchinson and Chamberlin2002; Van de Vyver et al., Reference Van de Vyver, Houston, Abrams and Vasiljevic2016; Yilmaz and Saribay, Reference Yilmaz and Saribay2017a). The association between thinking style differences and moral values is a widely studied phenomenon in the literature: Analytic thinking strengthens individualizing moral foundations (e.g., Yilmaz and Saribay, Reference Yilmaz and Saribay2017b), while intuitive thinking exhibits a positive relationship with endorsing binding moral foundations (e.g., Pennycook et al., Reference Pennycook, Cheyne, Barr, Koehler and Fugelsang2014). Thus, existing literature indicates that the intuitive–analytic thinking dichotomy serves as an essential paradigm that may elucidate individual differences across various domains.

Nonetheless, there is an ongoing debate regarding the cognitive processes that determine an individual’s position along this single dimension (Newton et al., Reference Newton, Feeney and Pennycook2023). DPM theories converge on similar underlying constructs, namely intuitive and analytic thinking, albeit through different cognitive mechanisms. For example, some models emphasize the tendency to revise preexisting beliefs or engage in effortful thinking as indicative of reflective thinking (Cacioppo and Petty, Reference Cacioppo and Petty1982; Jarvis and Petty, Reference Jarvis and Petty1996; Krumrei-Mancuso and Rouse, Reference Krumrei-Mancuso and Rouse2016; Stanovich and West, Reference Stanovich and West2007), while others prioritize reliance on gut feelings or the need for certainty as indicative of intuitive thinking (e.g., Budner, Reference Budner1962; Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996; Kruglanski et al., Reference Kruglanski, Atash, De Grada, Mannetti and Pierro2013). Thus, approaches that address thinking style differences within a singular dimension based on analytical–intuitive thinking may commit a Jingle-jangle fallacy by oversimplifying the comprehensive nature of cognitive styles using identical denominations. In addition, the question of whether individuals vary in their thinking styles along a single dimension, ranging from intuitive to analytic, or whether other genuine dimensions exist remains unresolved (Newton et al., Reference Newton, Feeney and Pennycook2023). Furthermore, various DPM theories propose their own measurement tools, conceptualizing similar phenomena with different terms. However, these theories have remarkable similarities (Evans, Reference Evans2008), and the factors highlighted by these distinct approaches predominantly predict similar outcomes despite being labeled differently. Consequently, it is unclear how distinct their emphasized structures truly are from one another. Thus, despite the vast accumulation of empirical studies within the DPM literature providing insights into explaining individual differences, theoretical and measurement complexities still require further clarification. In response to this, Newton et al. (Reference Newton, Feeney and Pennycook2023) recently proposed a theoretical framework that addresses intuitive–analytic thinking disposition within a multidimensional structure and introduced a new measure of thinking style differences.

1.2. Four distinct styles of intuitive–analytic thinking and its comprehensive measure

To address the highlighted limitations, Newton et al. (Reference Newton, Feeney and Pennycook2023) conducted a study aimed at developing a comprehensive scale for assessing thinking style differences within the framework of DPM. They initially gathered 265 items from 12 different scales on thinking style and selected 50 candidate items for the new scale based on their correlation with the Cognitive Reflection Test (CRT; Frederick, Reference Frederick2005). Through this process, they established a 4-factor consisting of 24 items. As the CRT does not encompass all aspects of thinking styles (Hertzog et al., Reference Hertzog, Smith and Ariel2018) and multiple routes can lead to correct responses on the test (Bago and De Neys, Reference Bago and De Neys2019; Boissin et al., Reference Boissin, Caparos, Raoelison and De Neys2021; Raoelison et al., Reference Raoelison, Boissin, Borst and De Neys2021; Stupple et al., Reference Stupple, Pitchford, Ball, Hunt and Steel2017), Newton et al. (Reference Newton, Feeney and Pennycook2023) systematically examined factor structure and validity of the scale across 2 samples, considering various outcomes such as cognitive ability, cognitive style, performance measures of intuitive–analytic thinking, subjective happiness, empathizing, moral judgments, religious belief, conspiratorial belief, paranormal belief, and bullshit receptivity. Consequently, they established that the 24-item scale, featuring 4 correlated factors, namely actively open-minded thinking (AOT), close-minded thinking (CMT), preference for effortful thinking (PET), and preference for intuitive thinking (PIT), is a suitable instrument for measuring thinking style differences.

The AOTFootnote 1 focuses on profound questioning of preexisting beliefs and intuitions, along with a disposition toward reflectivity, leading to the consideration of alternative viewpoints based on evidence (Stanovich and Toplak, Reference Stanovich and Toplak2019). In contrast, CMT represents a thinking style that perceives truth in clear and distinct terms as black and white (Newton et al., Reference Newton, Feeney and Pennycook2023). PET refers to the inclination to engage in effortful cognitive activities, prioritizing deliberative thinking processes over automatic thoughts (Cacioppo and Petty, Reference Cacioppo and Petty1982). On the other hand, PIT reflects a preference for thinking based on intuitions and gut feelings (Betsch, Reference Betsch2004). Both AOT and PET can be classified as 2 subscales that stem from a foundation of analytic thinking, while the other 2 dimensions, CMT and PIT, correspond to 2 distinct manifestations of intuitive thinking. However, Newton et al. (Reference Newton, Feeney and Pennycook2023) emphasized that AOT and PET (or CMT and PIT) should not be evaluated as parallel factors operating on a continuum within a single dimension; rather, these factors represent distinct cognitive features that characterize different thinking styles.

Newton et al. (Reference Newton, Feeney and Pennycook2023) reported that the subscales exhibited varying strengths and directions in predicting the outcomes. AOT emerged as a negative predictor for all outcomes except for empathizing quotient and disgust-related moral dilemmas. CMT was a positive predictor for religious belief, subjective happiness, and moral dilemmas while serving as a negative predictor for paranormal belief. Conversely, PIT positively predicted all outcomes except for subjective happiness and religious belief. Lastly, PET positively predicted the empathizing quotient and subjective happiness.

Although the correlated 4-factor model fits the data better than the 2-factor model, paired factors are expected to be predictive in similar directions. Given that AOT and PET represent factors of analytic thinking disposition, while CMT and PIT indicate intuitive thinking, some of these results require clarification and replication. For example, Newton et al. (Reference Newton, Feeney and Pennycook2023) found that AOT was negatively associated with religious belief, whereas PIT and PET did not show any significant associations. However, prior research has identified the predictive role of PIT and PET on religious beliefs (e.g., Aarnio and Lindeman, Reference Aarnio and Lindeman2007; Dennin et al., Reference Dennin, Furman, Pretz and Roy2022). Moreover, CMT and PIT predicted paranormal belief in opposite directions. However, it is important to note that the prevailing expectation in the literature is for such relationships to generally align in a similar direction, even though the magnitudes of their effects may differ (e.g., Svedholm and Lindeman, Reference Svedholm and Lindeman2013). Furthermore, CMT positively predicted religious beliefs but negatively predicted paranormal beliefs. Nonetheless, it is important to acknowledge that the existing literature includes findings suggesting a close relationship between these 2 beliefs and significant overlap in their predictors (Aarnio and Lindeman, Reference Aarnio and Lindeman2007; Pennycook et al., Reference Pennycook, Cheyne, Seli, Koehler and Fugelsang2012; Willard and Norenzayan, Reference Willard and Norenzayan2013). Hence, the anticipation was that CMT would predict both religious beliefs and paranormal beliefs in a consistent direction. Newton et al. (Reference Newton, Feeney and Pennycook2023) interpreted these findings as an indication that various thinking styles are associated with different outcomes and emphasized that categorizing individuals solely as intuitive or reflective is an oversimplification. However, as stated in the original study, these 4 dimensions are both correlated and independent, representing distinct forms of intuitive–analytic thinking disposition. While the inverse scenario may be statistically possible due to suppressor effects, given the small-to-moderate correlations among the subscales of the 4-CTSQ, it is anticipated that the related subscales would not display contradictory patterns in terms of the outcomes they predict. Consequently, further research is necessary to investigate how the subscales of the 4-CTSQ predict psychological outcomes related to thinking style differences and to clarify the original findings that do not entirely align with the expectations in the existing literature.

Another limitation of the original study is that it relies mostly on self-report measures of intention. Previous research indicates that intention-based measures may not be entirely reliable (Sheeran, Reference Sheeran2002), and a significant disparity exists between intentions and actual behaviors (Sheeran and Webb, Reference Sheeran and Webb2016). Newton et al. (Reference Newton, Feeney and Pennycook2023) employed a truth discernment measure to assess the predictive power of the 4-CTSQ regarding behavioral outcomes. Their findings indicated that the capacity to distinguish between fake and true news, previously attributed to cognitive style differences in prior literature (Pennycook and Rand, Reference Pennycook and Rand2021), exhibited a significant association with all subdimensions of the scale. Furthermore, each subdimension demonstrated varying predictive strengths that aligned with expected directions in relation to truth discernment. However, this measure primarily assesses a cognitive skill at the self-report level rather than a specific behavioral outcome such as sharing post on social media, introducing the potential for bias. This limitation stems from the fact that truth discernment measures the alignment of one’s beliefs about news items with actual reality, rather than addressing the act of believing and sharing fake news on social media. When it comes to actual behavior, it is unclear whether an outcome will be produced in line with this cognitive skill. Therefore, it is beneficial to employ more robust measurement methods to evaluate the predictive power of the 4-CTSQ in relation to actual behavioral outcomes. Additionally, intentions and behaviors can stem from different psychological processes. For instance, individuals may not exhibit the tendencies they express at the intention level due to the influence of social desirability, focusing on personal interests when it comes to actual behaviors. Moreover, it is essential to determine whether outcomes purportedly linked to thinking style differences consistently reveal the same patterns for both intention and behavior and how the CTSQ subscales predict these aspects. For instance, the strong relationship between moral values such as cooperation and thinking style differences should be demonstrated with both intention and behavior measures to understand the predictive power of the 4-CTSQ comprehensively. Hence, evaluating intention and behavior measures in conjunction is necessary to comprehensively unveil the psychological outcomes of different thinking styles.

Lastly, it is imperative to examine the validity of the 4-CTSQ in different cultural contexts, particularly in non-WEIRD ones (Henrich et al., 2010). Newton’s study solely used data from U.S. and Canadian participants, representing a typical WEIRD sample. However, thinking style differences may manifest dissimilar patterns in diverse cultures and may be associated with distinct outcomes. For instance, the relationship between analytic thinking disposition and conservatism appears stronger in WEIRD cultures, but its stability in non-WEIRD cultures remains uncertain (Yilmaz and Alper, Reference Yilmaz and Alper2019). Another crucial cultural aspect to consider is the prevalent religion. The original study’s samples were drawn from a culture predominantly following Christianity, and the validity of the findings related to religion in cultures with different religious beliefs is uncertain. For example, Newton et al. (Reference Newton, Feeney and Pennycook2023) found that AOT was the most potent predictor of religious beliefs, while PIT was the most potent predictor of paranormal beliefs. However, considering the divergent historical processes and religious features of different religions, it remains unclear whether the same pattern would persist in a society dominated by another religion (e.g., Islam). Therefore, further research is needed to assess the cross-cultural validity of the 4-CTSQ and its applicability in diverse cultural contexts, particularly in non-WEIRD and religiously diverse societies.

1.3. The current study

In 2 pre-registered studies, we tested the claim that individuals differ along multiple dimensions of intuitive–analytic thinking styles with 4 factors, the replication of Newton et al.’s (Reference Newton, Feeney and Pennycook2023) original findings, and the validity of the 4-CTSQ in the non-WEIRD culture, Türkiye.Footnote 2 In the first study, we tested the construct validity of 4-CTSQ and examined the predictive power of scale on several measures, including performance-based cognitive reflection tests, base-rate problems, conspiratorial beliefs, COVID-19 conspiracy beliefs, religious beliefs, paranormal beliefs, morality as cooperation, and ideology. In the second study, we cross-validate the factor structure of the 4-CTSQ on the final version of the scale from the first study. Then, we test the stability of the initial findings using the previous measures and adding new ones. Additionally, we validate the earlier findings while incorporating new measures, including moral foundations, pro-vaccination attitudes, and prosociality. Moreover, we compare the explanatory capabilities of the 4-CTSQ with those of the need for cognition scale to explore their respective contributions to understanding thinking styles.

2. Study 1

We pre-registered all materials, analyses, and hypotheses before any data collection. The pre-registration form can be seen at https://osf.io/epyg6/?view_only=1339b0eaa249410292e349c5dac4c25a.

2.1. Participants

We collected data from 869 university students in exchange for extra credit and a gift draw. As pre-registered, we used a time limit as a stopping rule and collected data from as many participants as possible over 2 weeks. We excluded participants who completed the survey extremely fast or slow based on z-score for completion time; z > 3 and z < 3 (n = 13), inattentive participants (n = 131), and participants who were identified as a multivariate outliers on 4-CTSQ items (n = 30, χ 2(24) = 51.18, p = .001). The final dataset consists of 695 submissions (Mage = 25.36, SD = 7.77; 485 female).

2.2. Materials

2.2.1. 4-CTSQ

We used the 24-item 4-CTSQ developed by Newton et al. (Reference Newton, Feeney and Pennycook2023). 4-CTSQ has 4 subscales as follows, each consisting of 6 items: AOT, CMT, PIT, and PET. To create the Turkish version of the scale, we first translated all items into Turkish. Then, we had the existing translations checked by 2 experts and used the versions agreed upon among experts. In the original version of the questionnaire, the word ‘belief’ is mentioned in various items. Since this word evokes religious beliefs among religious individuals (Stanovich and Toplak, Reference Stanovich and Toplak2019), we used the word ‘opinion’ instead of belief in the Turkish version. 4-CTSQ uses a 6-point response scale (1 = strongly disagree, 6 = strongly agree). Since the items with differing wording tend to form separate factors (Kam and Meyer, Reference Kam and Meyer2023), Newton et al. (Reference Newton, Feeney and Pennycook2023) adopted a uniform directional approach for all subscale items to enhance strong reliability. Consequently, in the default scale configuration, higher scores across all subscales correspond to greater intuitiveness, while lower scores indicate greater reflectiveness (Newton et al., Reference Newton, Feeney and Pennycook2023). Thus, the original wording of all items pertaining to AOT and PET subscales needs to be reversed to ensure that subscales to correspond to the thinking style they originally expressed. Therefore, in the results, when we refer to a ‘preference for effortful thinking’, it indicates a preference for a level of effortful thinking, counter to the default wording of the items. Turkish version of the 4-CTSQ can be seen at Supplementary Materials.

2.2.2. Conspiratorial beliefs

We used the 15-item Generic Conspiracist Beliefs Scale (Brotherton et al., 2013) to measure conspiratorial beliefs. The scale focuses on individual differences in generic conspiracist ideation (e.g., ‘Evidence of alien contact is being concealed from the public’), and it was adapted to Turkish by Alper et al. (Reference Alper, Bayrak and Yilmaz2021). It uses a 5-point response scale (1 = definitely not true, 5 = definitely true). Cronbach’s alpha was found to be .89.

2.2.3. COVID-19 conspiracy beliefs

In addition to generic conspiracist ideation, we measured conspiracy beliefs about COVID-19. To measure specific conspiracy beliefs about COVID-19, we used a 7-item scale developed by Salali and Uysal (Reference Salali and Uysal2021). The scale includes COVID-19 conspiracy statements generated from mainstream and alternative websites. An example item was ‘The spread of the coronavirus is something a group of powerful people deliberately do to make money and/or gain control’. It uses a 5-point response scale (1 = strongly disagree, 5 = strongly agree). Cronbach’s alpha was .91.

2.2.4. Cognitive reflection

We used Cognitive Reflection Test (CRT; Frederick, Reference Frederick2005) and Cognitive Reflection Test-2 (CRT-2; Thomson and Oppenheimer, Reference Thomson and Oppenheimer2016) to measure reliance on reflective thinking. CRT consists of 3 mathematical questions with 4 choices, whereas CRT-2 includes 4 verbal questions. All questions have one correct answer, and total scores are calculated by adding the number of correct answers. We used scores from the 2 tests separately in analyses, and high scores correspond to high analytical thinking (e.g., reflection) skills. Cronbach’s alphas for CRT-1 and CRT-2 were .67 and .52, respectively.

2.2.5. Base-rate problems

In addition to performance-based CRT measurements, we also measured cognitive style differences via standard base-rate questions. We used 2 types of base-rate problems from De Neys and Glumicic (Reference De Neys and Glumicic2008). The measure consists of 3 incongruent problems, including stereotypes that conflict with the large base-rate group and 2 congruent problems with stereotypes that match the large base-rate group. Each problem has 2 choices, one corresponding to base-rate conflicting and the other to base-rate respecting answers. Cronbach’s alphas for incongruent problems and congruent problems were .73 and .48, respectively.

2.2.6. Analytic cognitive style

We calculated an overall analytic cognitive style score by averaging the responses on the items of CRT-1, CRT-2, and incongruent base-rate (e.g., conflict) problems. Cronbach’s alpha was .53.

2.2.7. Religious beliefs

To measure religious beliefs, we used a religious belief subscale of the Paranormal Belief Scale developed by Arslan (Reference Arslan2010). Based on the original Paranormal Belief Scale (Tobacyk and Milford, Reference Tobacyk and Milford1983), which is frequently used in the literature, Arslan (Reference Arslan2010) developed a new scale adapting its content to Turkish culture and the Islam religion. The religious belief subscale consists of 10 items and uses a 4-point response scale (1 = strongly disagree, 4 = strongly agree). The scale focuses on the level of belief in various phenomena in Islam (e.g., God, heaven, spirit, angel). Cronbach’s alpha was found to be .93.

2.2.8. Paranormal beliefs

We measured nonreligious paranormal beliefs with another subscale of the Paranormal Belief Scale (Arslan, Reference Arslan2010). It consists of 17 items on a 4-point response scale (1 = strongly disagree, 4 = strongly agree). In this sub-dimension, items measure the level of paranormal beliefs that are not directly included in Islam (e.g., Signs, superstitions, soothsaying, mind reading). Cronbach’s alpha was .87.

2.2.9. Morality as cooperation

To measure moral concerns, we employed the Morality as Cooperation Questionnaire (MAC-Q; Curry et al., Reference Curry, Chesters and Van Lissa2019a), which was developed within the theoretical framework of Morality as Cooperation Theory (MAC; Curry, Reference Curry, Shackelford and Hansen2016). MAC posits that morality originated as a result of diverse strategies devised to address the cooperative challenges that arose in the context of evolution (Curry, Reference Curry, Shackelford and Hansen2016). Curry et al. (Reference Curry, Mullins and Whitehouse2019b) conducted an analysis of ethnographic data spanning a 300-year period, representing 60 diverse societies and found that the 7 distinct moral dimensions are universally esteemed across most cultures (i.e., family, group, reciprocity, heroism, deference, fairness, property). Based on this finding, Curry et al. (Reference Curry, Mullins and Whitehouse2019b) introduced self-report measure of moral concerns, MAC-Q, which consists of 2 distinct sections: ‘Relevance’ and ‘Judgment’. The first section is designed to assess how relevant each moral dimension is in deciding whether something is right or wrong, for instance, asking whether actions such as ‘whether or not someone acted to protect their family’ are morally relevant. The second section is designed to gauge individuals’ agreement with various moral judgments, including statements like ‘people should be willing to do anything to help a member of their family’. However, both the original study (i.e., Curry et al., Reference Curry, Mullins and Whitehouse2019b) and an independent test of the MAC-Q (i.e., Yilmaz et al., Reference Yilmaz, Harma and Doğruyol2021) showed that only the relevance section exhibited a factor structure suitable for use by showing good fit indices. Thus, in the current study, we exclusively used the relevance scale of the MAC-Q, which comprises of 21 items representing 7 moral dimensions, with 3 items for each. MAC-Q was adapted into Turkish by Yilmaz et al. (Reference Yilmaz, Harma and Doğruyol2021). The participants responded to each item with a slider between 0 and 100 (0 = not at all relevant, 100 = extremely relevant). Cronbach’s alpha scores of subscales ranged between .65 and .81.

2.2.10. Demographic form

We used the demographic form to measure age, sex, education, perceived social status, religiosity (1 = not religious at all, 7 = very religious), and ideology (1 = very liberal, 7 = very conservative). Each variable was measured by a single item.

2.3. Data analytical strategy

We ran several confirmatory factor analyses (CFA) to test the construct validity of 4-CTSQ. All models were tested in Mplus Version 8 (Muthen and Muthen, Reference Muthen and Muthen2015) using maximum likelihood estimation. In line with Newton et al. (Reference Newton, Feeney and Pennycook2023), we first tested a single-factor model where all items were loaded on a single latent factor. Then, we tested the uncorrelated 4-factor model, where latent correlations were fixed to zero, the correlated 4-factor model, where latent correlations were freely estimated, and the correlated 4-factor higher-order model, where all latent factors were loaded on a higher-order latent factor.

We used normal theory weighted least squares with chi-square to evaluate the model fit. Following Hu and Bentler’s (Reference Hu and Bentler1999) 2-index presentation strategy, we also reported the comparative fit index (CFI), the Tucker–Lewis index (TLI), the root mean square error of approximation (RMSEA), and the standardized root means square residual (SRMR). Accordingly, RMSEA values smaller than .06 and CFI/TLI values larger than .95 represent an excellent fit, while RMSEA values smaller than .08 and CFI/TLI values larger than .90 represent a good model fit. Nested models were compared using the Wald test, and changes in TLI (ΔTLI) and RMSEA (ΔRMSEA) where changes larger than .002 in the expected direction are considerable (Meade et al., Reference Meade, Johnson and Braddy2008). We also used the Bayesian information criterion (BIC) to compare unnested models; a decrease larger than 10 represents a better model fit (Raftery, 1995). To test reliabilities, we used Hancock and Mueller’s (Reference Hancock, Mueller, Cudeck, du Toit and Sorbom2001) H value in which items are weighted based on their factor loadings, scores larger than .80 indicate stable latent factor structure.

2.4. Results

2.4.1. Construct validity

As presented in Table 1, the single-factor model revealed a poor fit to the data, while the remaining models approached a good model fit. The correlated model was superior to an uncorrelated model based on the Wald test (W (6) = 19.49, p = .003) and changes in fit indices (ΔTLI = .022, ΔRMSEA = .007). Modification indices suggested correlated error between 2 items (‘Either something is true, or it is false; there is nothing in-between’ and ‘There is no middle ground between what is true and what is false’). Since items are semantically similar, we decided to add an error correlation between items. As a result, the best-fitting model was modified correlated 4-factor model with a good fit to the data (χ 2 = 1083.79, df = 245, p < .001; CFI = .914; TLI = .903; RMSEA = .070, 90% CI [.066, .074]). Higher-order counterparts of correlated 4-factor models revealed a comparable fit with the first-order versions (ΔBIC’s < 10), suggesting that 4 factors could represent a higher-order structure. Standardized factor loadings and reliability coefficients for the final correlated 4-factor model were presented in Table 2.

Table 1 Fit indices of the 4-CTSQ

Note: The final model is bolded.

Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; PET, preference for effortful thinking; PIT, preference for intuitive thinking.

a 4-CTSQ higher-order factor was fixed at 1 in Model 3 to allow for model convergence.

Table 2 Standardized factor loadings for the 4-CTSQ

Note: Parenthetical values represent loadings for Study 2.

Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; PET, preference for effortful thinking; PIT, preference for intuitive thinking.

a Reverse coded items.

2.4.2. Criterion validity

We first calculated the correlation between the 4 subscales of 4-CTSQ. Correlations between the 4 dimensions were significant except for PIT and PET (r = .05, p = .221). All significant correlations were in the expected direction, with the strongest correlation between AOT and PIT (r = −.42, p < .001) in line with the original study (Table 3).

Table 3 Descriptive statistics for the 4-CTSQ dimensions

Note: Parenthetical values represent descriptive statistics for Study 2. The lower diagonal represents correlations for Study 1, and the upper diagonal represents correlations for Study 2.

* p < .001.

We then calculated the correlation between 4-CTSQ subscales and outcome variables.Footnote 3 All 4 dimensions were significantly correlated with analytic thinking style measures with few exceptions. Consistently, AOT (ranged between .17 and .26) and PET (ranged between .07 and .16) were positively correlated with analytic thinking performance scores, performance on stereotype challenging base-rate problems, and composite analytic cognitive style score, while CMT (ranged between −.20 and −.32) and PIT (ranged between −.03 and −.19) were negatively correlated with performance measures of thinking styles. A reverse pattern was evident across epistemically suspect belief measures; while AOT (ranged between −.15 and −.29) and PET (ranged between −.01 and −.15) were negatively correlated with paranormal, religious, conspiracy, and COVID-19 conspiracy beliefs, CMT (ranged between .07 and .26) and PIT (ranged between .12 and .22) were positively correlated with those belief measures. Across ideological measures, AOT and PET were negatively correlated with resistance to change (r AOT = −.35, p < .001, r PET = −.17, p < .001) and right-wing political orientation (r AOT = −.14, p < .001, r PET = −.10, p = .007) whereas CMT and PIT were positively correlated with resistance to change (r CMT = .43, p < .001, r PIT = .28, p < .001) and CMT was correlated with right-wing political orientation (r = .15, p < .001). Correlations between 4-CTSQ dimensions and MAC-Q dimensions were significant for most of the pairs and in the expected direction. The results of the correlation analyses are summarized in Table 4.

Table 4 Correlations between the 4-CTSQ dimensions and outcome variables

Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; MAC-Q, Morality-as-Cooperation Scale; MFQ, moral foundations Questionnaire; NFC, need for cognition; PET, preference for effortful thinking; PIT, preference for intuitive thinking; S1, study 1; S2, study 2.

* p < .05.

** p < .001.

To investigate the unique contribution of each 4-CTSQ subscale on outcome variables, we run a series of regression analyses (Table 5). Results demonstrated that AOT and CMT consistently predicted analytic thinking performance except for stereotype congruent base-rate problems (AOT βs ranged between .09 and .14, and CMT βs ranged between −.15 and −.25). Among the regression on belief measures, AOT (βs ranged between −.06 and −.19) and PIT (βs ranged between .12 and .26) significantly predicted all belief measures with one exception (AOT on conspiracy beliefs). Moreover, while AOT positively predicted resistance to change and political orientation, CMT predicted the same variables in the opposite direction. Regression analyses on MAC-Q dimensions also revealed a consistent pattern; PIT and CMT (except for fairness and property) positively predicted morality dimensions, while AOT and PET were mostly nonsignificant. Overall, AOT and CMT were predictive of analytic thinking performance, belief measures, and ideological measures, while PIT and CMT were predictive of morality dimensions.

Table 5 Standardized regression coefficients for the 4-CTSQ dimensions on outcome variables

Note: Parenthetical values represent descriptive statistics for Study 2.

Abbreviations: AOT, actively open-minded thinking; CMT, close-minded thinking; MAC-Q, Morality-as-Cooperation Scale; MFQ, moral foundations questionnaire; PET, preference for effortful thinking; PIT, preference for intuitive thinking; S1, study 1; S2, study 2.

* p < .05.

** p < .001.

2.5. Exploratory analysis

To explore whether the 4-CTSQ dimensions distinctively predict outcome measures or whether there is an underlying factor that accounts for the relationships with outcome measures, we ran a canonical correlation analysis between the 2 sets of variables. To further explore the relationship between 4-CTSQ dimensions and criterion variables, we ran a canonical correlation analysis between the 2 sets of variables. The analysis revealed that first (Wilks’s λ = .561, F(56, 2635.56) = 7.85, p < .001), second (Wilks’s λ = .103, F(39, 2008.45) = 2.76, p < .001), and third (Wilks’s λ = .039, F(24, 1358) = 1.61, p = .031) canonical variates were significant while the last one was not significant (Wilks’s λ = .018, F(11, 680) = 1.11, p = .348). Significant canonical variates explained 36%, 9%, and 4% of the variance, respectively. Accordingly, there were 3 factors (canonical correlates) to explain the relationships with the outcome measures. Therefore, relationships between 4-CTSQ dimensions and outcome measures cannot be attributed to a higher-order underlying cognitive thinking style factor.

3. Study 2

The first study’s findings provided preliminary implications for the 4-CTSQ to be a reliable and valid measure for detecting various psychological characteristics and beliefs based on thinking style differences. We conducted a second study to test the consistency and replicability of the existing findings in a new sample by further expanding our measurement tools. In the second study, we used the same measures from the first study as follows: 4-CTSQ, Conspiratorial Beliefs, COVID-19 Conspiracy Beliefs, Cognitive Reflection, Cognitive Style, Religious Beliefs, Paranormal Beliefs, Resistance to Change, and Demographic Form. We also expanded our measures with moral foundations, pro-vaccination, the need for cognition, and incentivized measures of prosociality. As in the first study, we pre-registered all materials, analyses, and hypotheses before the study. The pre-registration form can be seen at https://osf.io/jhkwz/?view_only=0c09cf0337c6466e8ca3b8a461301c77.

3.1. Participants

We reached 540 university students who participated study in exchange for extra course credit. We used a time limit as a stopping rule, as in Study 1, and collected data from as many participants as possible over 3 weeks. As pre-registered, we excluded participants who completed the survey extremely fast or slow based on z-score for completion time; z > 3 and z < 3 (n = 4), participants who failed in attention check question (n = 25), and participants who were identified as a multivariate outlier on 4-CTSQ items (n = 29, χ 2(24) = 51.18, p = .001). The final sample size was 486 (Mage = 21.91, SD = 2.41; 419 female).

3.2. Materials

3.2.1. Moral foundations questionnaire

We used the relevance section of the Moral Foundations Questionnaire (MFQ; Graham et al., Reference Graham, Nosek, Haidt, Iyer, Koleva and Ditto2011). The scale consists of 15 items representing 5 moral foundations (care, fairness, loyalty, authority, and sanctity), with 3 items for each. This section measures the extent to which various considerations are relevant to individuals’ thinking when deciding whether something is right or wrong. An example item was ‘Whether or not someone showed a lack of respect for authority’. We used the Turkish version of the MFQ, which was adapted into Turkish by Yilmaz et al. (Reference Yilmaz, Harma, Bahçekapili and Cesur2016a, Reference Yilmaz, Karadöller and Sofuoglu2016b). Participants responded to items with a 6-point scale (0 = not effective at all, 6 = very effective). Cronbach’s alpha scores of subscales ranged between .51 and .74.

3.2.2. Pro-vaccination attitudes

We measured pro-vaccination attitudes and intentions with 3 items as follows: ‘COVID-19 vaccines are safe’, ‘Generally, vaccines are safe’, and ‘When it is my turn, I will get the COVID-19 vaccine’. Participants responded to items with a 7-point response scale (1 = strongly disagree, 7 = strongly agree). Cronbach’s alpha was .86.

3.2.3. Prosociality

We used an incentivized donation task to measure prosocial behavior against the pandemic. The item asks each participant what percentage of money (200) they would want to give to individuals economically affected by the pandemic. Drawing upon prior literature (e.g., Charness et al., 2016), we employed a random lottery methodology, a cost-saving approach, wherein actual decisions are subjected to randomized compensation for some (but not all) participants.

3.2.4. Need for cognition

We used the Need for Cognition scale (NFC; Cacioppo et al., Reference Cacioppo, Petty and Feng Kao1984) to compare with CTSQ and examine which has more predictive validity. The scale was adopted into Turkish by Gülgöz and Sadowski (1995). It consists of 18 items on a 9-point response scale (−4 = extremely uncharacteristic of me; 4 = extremely characteristic of me). Cronbach’s alpha was found to be .91.

3.3. Results

3.3.1. Construct validity

We first cross-validated the factor structure of the 4-CTSQ scale on the final version of the scale in Study 1. CFA results revealed a good fit to the data for the modified correlated 4-factor model (χ 2 = 842.48, df = 245, p < .001; CFI = .921; TLI = .911; RMSEA = .071, 90% CI [.066, .076]). Standardized factor loadings and reliability coefficients for the scale are presented in Table 2.

3.3.2. Criterion validity

As in the first study, we first tested the correlation between subscales of 4-CTSQ (Table 3).Footnote 4 Results aligned with the first study except for the correlation between PIT and PET (r = −.15, p = .001). Again, the largest correlation was between AOT and PIT (r = −.47, p < .001). Correlations between 4-CTSQ subscales and thinking style measures supported the first study’s results with few exceptions (Table 4). Similarly, correlations of resistance to change and political ideology were comparable regarding direction and magnitude. While none of the correlations were significant for a behavioral measure of prosociality, all the correlations were significant for pro-vaccination attitudes in the expected direction (ranged between −.24 and .29). Among the correlations between moral foundations and 4-CTSQ dimensions, while AOT (ranged between −.24 and −.28) and PIT (ranged between −.11 and −.19) were negatively correlated with binding foundations; loyalty, authority, and purity, CMT (ranged between .12 and .28) and PIT (ranged between .16 and .21) were positively correlated with the same variables. Care and fairness were not correlated with the 4-CTSQ dimensions.

We also investigated the unique contribution of each 4-CTSQ dimension on outcome variables in a series of regression analyses. Regressions on analytic thinking style and belief measures revealed that the direction and magnitude of the standardized coefficients were in line with the first study except for CMT. Contrary to the first study’s results, CMT failed to predict analytic thinking performance measures; however, CMT is still predictive of belief measures. Regressions on ideological measures were mostly in line with the first study, except that PIT failed to predict resistance to change. Pro-vaccination attitudes were predicted by AOT (β = .21, p < .001) and PIT (β = −.13, p = .011, F (4, 481) = 13.95, p < .001). Regressions on moral foundations showed that most of the betas in predicting binding foundations (i.e., loyalty, authority, and purity) were significant. As expected, AOT and PET negatively predicted binding foundations, while CMT and PIT positively predicted those foundations (Table 5). On average, 4-CTSQ explained 9% of the variance on 18 outcome variables. We averaged the squared correlation between the need for cognition and the same outcome variables (2%) to compare the explanatory power of the 2 scales. Results of a related-samples Wilcoxon signed rank test indicated that 4-CTSQ, on average, explained more variance than the need for cognition scale (Z = 3.62, p = .002).

3.4. Exploratory analysis

As in the first study, we ran a canonical correlation analysis between 4-CTSQ dimensions and criterion variables. Results revealed that while first (Wilks’s λ = .637, F(56, 1818.70) = 5.92, p < .001) and second (Wilks’s λ = .253, F(39, 1386.60) = 1.62, p = .010) canonical variates were significant, third (Wilks’s λ = .204, F(24, 938) = 1.30, p = .151) and fourth (Wilks’s λ = .151, F(11, 470) = 0.99, p = .453) variates were not significant. Canonical variates explained 41%, 7%, 4%, and 2% of the variance, respectively.

4. Discussion

The purpose of the current research was twofold. First, we sought to investigate whether the Turkish version of the 4-factor multiple structures of thinking styles fits the data well. Second, we aimed to assess the replicability of Newton et al.’s (Reference Newton, Feeney and Pennycook2023) findings on outcome measures. The results provided novel evidence beyond the original study, supporting the claim that intuitive–analytic thinking styles vary across at least 4 distinct elements: Actively open-minded thinking, close-minded thinking, preference for effortful thinking, and preference for intuitive thinking. Consistent with Newton et al. (Reference Newton, Feeney and Pennycook2023), we found that these elements differentially predicted various outcomes, although we could not replicate all their findings. Furthermore, we validated the Turkish version of the 4-CTSQ and demonstrated that it is functional for understanding a wide range of beliefs in a non-WEIRD culture, Türkiye. To our knowledge, our research is the first one that independently attempts to replicate the original 4-factor structure of thinking styles and contributes to clarifying the theoretical and measurement complexities in the literature and enhancing our understanding of the dimensions of thinking style differences.

4.1. Epistemically suspect beliefs

We measured 3 beliefs about conspiracy and paranormal beliefs, which are prominent components of Epistemically Suspect Beliefs (Pennycook et al., Reference Pennycook, Fugelsang and Koehler2015), including conspiratorial beliefs, COVID-19 conspiracy beliefs, and paranormal beliefs. Our findings indicated that actively open-minded thinking was a negative predictor of conspiratorial beliefs and COVID-19 conspiracy beliefs, while preference for intuitive thinking positively predicted both. Furthermore, closed minded-thinking positively predicted COVID-19 conspiracy beliefs. However, preference for effortful thinking did not significantly predict either conspiracy measure. These results align with Newton et al. (Reference Newton, Feeney and Pennycook2023), suggesting that preference for intuitive and effortful thinking does not correspond to parallel factors within the same dimension. While preference for intuitive thinking demonstrated explanatory power for conspiratorial disposition, preference for effortful thinking did not exhibit characteristics of critical thinking and skepticism, which have been associated with decreased conspiracy beliefs (Dyer and Hall, Reference Dyer and Hall2019). Empirical support for this distinction can also be found in the literature. Faith in Intuition and the Need for Cognition have been identified as separate factors and generally do not show a strong correlation (Epstein et al., Reference Epstein, Pacini, Denes-Raj and Heier1996). In a recent study by Wu et al. (Reference Wu, Kuru, Campbell and Baruh2022), the Need for Cognition was positively associated with health misinformation, including conspiracy beliefs, whereas Faith in Intuition showed a negative relationship with these beliefs. Considering that the preference for intuitive thinking items was mainly derived from the Faith in Intuition scale and all preference for effortful thinking items were taken from the Need for Cognition scale, this finding strengthens the idea that these factors represent distinct cognitive characteristics rather than being opposites.

Similar patterns were observed in religious belief and paranormal belief outcomes. Actively open-minded thinking negatively predicted religious and paranormal beliefs, while Close-minded thinking positively predicted all measures except for paranormal belief in Study 1. However, there was not a consistent relationship between preference for effortful and intuitive thinking. Preference for intuitive thinking positively predicted all measures of religious and paranormal beliefs, while preference for effortful thinking had a significant effect only on the paranormal belief measure in Study 1. Thus, differentiation solely based on the preference for effortful thinking versus relying solely on intuition seems insufficient to fully account for the disparities between analytic and intuitive thinking. There is also evidence indicating that this holds true for analytic thinking measures based on performance. CRT is an insufficient tool to comprehensively capture both analytic and intuitive thinking styles. It primarily assesses reflective thinking and may not serve as a valid indicator of intuitive thinking (Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b). Hence, relying solely on CRT scores to distinguish between thinking styles may not provide a comprehensive understanding. Incorporating the preference for effortful thinking and preference for intuitive thinking dimensions offers a more nuanced and comprehensive insight into thinking styles than simply differentiating between low and high CRT scores.

4.2. Ideological orientation

Among the measures included in both studies, resistance to change, political ideology, and religious beliefs correspond to the main components of ideological orientation. Specifically in Türkiye, religiosity (Çarkoğlu, Reference Çarkoğlu2007) and resistance to change (Yilmaz and Saribay, Reference Yilmaz and Saribay2018) are prominent determinants of political preferences. Our findings demonstrated that the subscales of the 4-CTSQ have different predictive powers on these 3 measures of ideological orientation. Actively open-minded thinking and close-minded thinking consistently predicted all 3 components of ideological orientation across all measures in both studies. However, there was no such consistency for preference for intuitive thinking and effortful thinking. This suggests that actively open-minded thinking and close-minded thinking play a unique role in understanding the relationship between thinking style differences and ideological orientation (Baron, Reference Baron2019). This distinction would have been overlooked if the focus were solely on differentiating thinking styles based on effortful thinking and reliance on intuition. Although reflection and actively open-minded thinking are related cognitive features, they are not identical (Baron et al., Reference Baron, Gürçay, Metz, Toplak and Weller2017). This dissociation between the subscales of the 4-CTSQ concerning outcomes related to ideological orientation is supported by previous research. While social conservatism can be predicted through reflection scores based on CRT measures, some studies have found no relationship with economic conservatism (e.g., Baron, Reference Baron2015). On the other hand, research indicates that actively open-minded thinking has a stronger relationship with conservatism compared to CRT (e.g., Baron, Reference Baron2017; Svedholm-Häkkinen and Lindeman, Reference Svedholm-Häkkinen and Lindeman2018; Yilmaz and Saribay, Reference Yilmaz and Saribay2017c). Thus, actively open-minded thinking may be a better predictor of political attitudes, especially when religious commitments influence political beliefs (Baron, Reference Baron2019).

4.3. Prosociality and moral concerns

Beyond the intention measures of Newton et al. (Reference Newton, Feeney and Pennycook2023), we added an outcome measure focusing on actual behavior in Study 2. We asked participants how much of the money we gave them they would like to donate to a person economically affected by the COVID-19 pandemic. Thus, we aimed to measure prosociality with an incentivized donation task that includes the potential for real money losses and gains. This approach was essential as there is a recognized gap between intention and actual behavior (e.g., Sheeran and Webb, Reference Sheeran and Webb2016), and various studies suggest that thinking styles can lead to differences in intention versus behavior measures. However, we found neither a predictive role nor a correlation of any subscale of the 4-CTSQ on prosociality scores. This finding contrasts with previous studies that have suggested a relationship between thinking style differences and prosocial behaviors. Although there is a debate about whether analytic or intuitive thinking leads to more prosocial or selfish behavior, numerous studies indicated that intuition increases prosociality (e.g., Bear and Rand, Reference Bear and Rand2016; Kinnunen and Windmann, Reference Kinnunen and Windmann2013; Rand, Reference Rand2016). Therefore, it was expected that subscales of 4-CTSQ would somehow be related to prosociality. One possible reason for not finding a significant relationship may be related to the specific theme of our prosociality question. The COVID-19 pandemic was a global crisis that resulted in millions of people losing their lives, with negative effects observed not only in health but also in various aspects of life, including the economy and social spheres. Hence, participants’ responses to the prosociality question may have been influenced by this prominent theme. Another reason could be linked to the measurement method used. In the field of behavioral economics, prosociality is often assessed through economic games with well-defined instructions and rules (e.g., Ultimatum game, Trust game, Dictator game, and Public Goods Game). These games aim to measure prosocial tendencies in a manner that minimizes confounding factors. However, in our study, we simply asked participants whether they would be willing to donate a given amount of money, and this may not fully capture prosociality due to the simplicity of the measure. Therefore, we encourage future studies to systematically focus on behavioral measures of prosociality and explore the validity of the 4-CTSQ through the use of economic games. Such investigations will provide more comprehensive insights into the relationship between thinking style differences and prosocial behavior.

On the other hand, we found significant relationships between subscales of 4-CTSQ and morality measures, which are assessing moral commitments that enhance cooperation. Although these measures do not capture cooperative intentions or behaviors directly, such as paradigms that specifically address cooperation (e.g., public goods game), they do assess commitments that are influenced by cooperativeness. Moral behavior is fundamentally centered around cooperation with anonymous individuals, and the root of human morality lies in the motivations that promote cooperative actions (Curry, Reference Curry, Shackelford and Hansen2016; Haidt, Reference Haidt2012). Consequently, MAC-Q and the MFQ essentially assess levels of prosocial tendencies in different domains. Moreover, our measure of prosociality pertains to applied ethics, while the measures of moral concerns are aligned with normative ethics. In this regard, we found that subscales of 4-CTSQ exhibited differential and varying predictive patterns concerning cooperative intentions as measured by MAC-Q or MFQ. Particularly, preference for intuitive thinking positively predicted all 7 dimensions of MAC-Q, while actively open-minded thinking and close-minded thinking were consistently associated with binding foundations of MFQ. The association of open-minded thinking with binding foundations tended to be positive, whereas the association of close-minded thinking tended to be negative, in line with previous research (e.g., Baron et al., Reference Baron, Scott, Fincher and Metz2015). Therefore, it seems that 4-CTSQ was not found to explain variance in behavior-based measures of prosociality while predicting intention-based morality measures. In another recent experimental study conducted in Türkiye, Varol (Reference Varol2023) observed that the public goods game, assessed through economic games and MAC-Q, purported to measure cooperative intentions, was impacted differently following the earthquake threat. This highlights that MAC-Q cannot be employed as a measure of cooperation, and its correlation with actual behavior cannot be assumed without caution (Dogruyol et al., Reference Dogruyol, Velioglu, Bayrak, Acem, Isler and Yilmaz2023). Further studies, specifically focusing on prosociality’s intention and behavioral aspects, are required to elucidate the association between thinking style differences and prosocial tendencies more comprehensively.

4.4. Validity of the 4-CTSQ and cultural implications

We concluded that 4-CTSQ possesses acceptable psychometric properties and can be utilized in future studies within Türkiye, a non-WEIRD country (Muthukrishna et al., Reference Muthukrishna, Bell, Henrich, Curtin, Gedranovich, McInerney and Thue2020). Based on our results across all outcomes, similar to Newton et al. (Reference Newton, Feeney and Pennycook2023), actively open-minded thinking appears to be the most consistent predictor, while preference for effortful thinking exhibits the weakest predictive validity. This finding aligns with previous research, which suggests that having a propensity to question one’s intuitions and critically evaluate novel evidence may have a stronger impact than merely favoring effortful thinking (Baron, Reference Baron2019; Baron et al., Reference Baron, Scott, Fincher and Metz2015, Reference Baron, Isler, Yılmaz, Ottati and Stern2023; Newton et al., Reference Newton, Feeney and Pennycook2023). However, it is important to note that we were unable to fully replicate some of the findings of Newton et al. (Reference Newton, Feeney and Pennycook2023). In their study, they observed a positive association between close-minded thinking and religious belief but a negative relationship with paranormal belief, highlighting a critical difference between these related beliefs. In contrast, our findings did not indicate such a dissociation; close-minded thinking was positively related to both religious and paranormal beliefs. This finding can imply that this association can vary across different cultures, given that paranormal belief is regarded as a component of institutionalized religion in certain cultural contexts (Yilmaz, Reference Yilmaz2021). These potential outcomes should be investigated in forthcoming studies. Nevertheless, our results are consistent with previous studies that reported a negative link between analytic thinking and both religious and paranormal beliefs (e.g., Pennycook et al., Reference Pennycook, Cheyne, Koehler and Fugelsang2016a, Reference Pennycook, Ross, Koehler and Fugelsang2016b).

5. Conclusion

Our study supports that individuals differ along multiple dimensions of intuitive–analytic thinking styles with 4 related, but distinct factors and the pattern of correlations were explained with at least 2 factors. Generally coinciding with Newton et al. (Reference Newton, Feeney and Pennycook2023) findings, we showed that certain thinking styles are more pertinent to certain outcomes, implying that categorizing individuals as either ‘intuitive’ or ‘reflective’ oversimplifies the complexities of thinking styles. Therefore, relying on single-dimensional measures for intuitive–analytic thinking styles may result in inaccurate conclusions about the significance of these thinking styles. However, it should be noted that the outcome measures were not designed to capture the distinctions between the 4-CTSQ dimensions, and therefore, results were partially dependent on the outcome measures used in the study. In this study, we demonstrated that the 4-CTSQ can effectively predict a broad array of outcomes within a non-WEIRD cultural context. Looking ahead, future research could further test the predictive validity of the 4-CTSQ and elucidate the particularities of thinking styles’ multidimensional aspects by examining related outcomes not covered in this study. Additionally, future investigations should assess the 4-CTSQ’s capacity to predict real-life behaviors, such as academic achievement, across various time intervals between predictor and outcome, and within diverse cultural settings.

Supplementary material and data

Supplementary material and data for this article can be found at https://doi.org/10.1017/jdm.2023.45.

Funding statement

This research received no specific grant funding from any funding agency, commercial, or not-for-profit sectors.

Competing interest

The authors declare none.

Footnotes

1 While the AOT items, classified as a subdimension within the 4-CTSQ, underscore the importance of thorough examination of prevailing beliefs and intuitions, as well as the consideration of evidence-based alternative perspectives, it is worth noting that the original conceptualization of AOT encompasses a distinct scope. As originally conceptualized by Baron (Reference Baron1985), the fundamental notion of AOT entails a metacognitive disposition characterized by established norms guiding the thought process, standards for thinking itself, and their consistent application in accordance with an idealized model of optimal thinking (see Baron et al., Reference Baron, Isler, Yılmaz, Ottati and Stern2023 for a historical review of the concept).

2 While the utilization of a student sample in our study may not substantially impact the educational dimension, a principal facet of WEIRDness, as empirically elucidated, the Turkish sample exhibits a distinctly WEIRD character in significant contrast to the predominantly Western samples (Muthukrishna et al., Reference Muthukrishna, Bell, Henrich, Curtin, Gedranovich, McInerney and Thue2020) upon which the extant literature relies. The educational quality proffered in Türkiye does not precisely align with Western standards. Reports concerning this subject indicate that students in Türkiye face a disadvantage in comparison to their Western counterparts in the acquisition of fundamental cognitive competencies typically gained through education (e.g., Schleicher, Reference Schleicher2019). Apart from this, the presence of inter-religious diversity and the predominance of Christianity in WEIRD cultures inherently categorize any culture where Islam is the normative religious framework as non-WEIRD. However, it is crucial to remember that while WEIRD cultures may exhibit a certain degree of homogeneity (e.g., dominance of Christianity, individualism), the category of non-WEIRD cultures is not homogenous in itself; it is defined negatively, by not conforming to WEIRD standards. Therefore, although our study utilizes a student sample, it embodies non-WEIRD characteristics by virtue of originating from a culture shaped by a non-Christian religion.

3 The subscales of the 4-CTSQ scale are as follows: Actively open-minded thinking (AOT), Close-minded thinking (CMT), Preference for intuitive thinking (PIT), and Preference for effortful thinking (PET). As in the original study, items of the subscales have a consistent direction of wording. However, AOT and PET subscales were reverse-coded to ensure that they correspond to the thinking style they originally expressed. Thus, in the results, when we refer to ‘preference for effortful thinking’, it signifies a preference for a level of effortful thinking, contrary to wording of the items. Accordingly, a positive relationship between preference for effortful thinking and related variable, it essentially means that higher preference for effortful thinking scores associated with higher scores on that variable.

4 The scores on all subscales of the 4-CTSQ scale are aligned with the thinking styles as denoted in the original terminology of the concepts, following the reverse coding procedure outlined in the method section. For instance, when we mention a ‘preference for effortful thinking’, it conveys a preference for a certain degree of effortful thinking, counter to the phrasing of the items. Accordingly, a positive relationship between a preference for effortful thinking and a related variable, it essentially indicates that higher scores on the preference for effortful thinking are associated with higher scores on that variable.

References

Aarnio, K., & Lindeman, M. (2007). Religious people and paranormal believers: Alike or different? Journal of Individual Differences, 28(1), 19. https://doi.org/10.1027/1614-0001.28.1.1 CrossRefGoogle Scholar
Alper, S., Bayrak, F., & Yilmaz, O. (2021). Psychological correlates of COVID-19 conspiracy beliefs and preventive measures: Evidence from Turkey. Current Psychology, 40, 57085717. https://doi.org/10.1007/s12144-020-00903-0 CrossRefGoogle ScholarPubMed
Arslan, M. (2010). Paranormal inanç ölçeğinin Türkçe versiyonunun geliştirilmesi: Geçerlik ve güvenirlik çalışması. İnönü Üniversitesi İlahiyat Fakültesi Dergisi, 1(2), 2340.Google Scholar
Bago, B., & De Neys, W. (2019). The smart System 1: Evidence for the intuitive nature of correct responding on the bat-and-ball problem. Thinking & Reasoning, 25(3), 257299. https://doi.org/10.1080/13546783.2018.1507949 Google Scholar
Baron, J. (1985). Rationality and intelligence. New York: Cambridge University Press.CrossRefGoogle Scholar
Baron, J. (1993). Why teach thinking? – An essay. Applied Psychology: An International Review, 42, 191237.Google Scholar
Baron, J. (2015). Supplement to Deppe et al. (2015). Judgment and Decision Making, 10(4), 12.Google Scholar
Baron, J. (2017). Comment on Kahan and Corbin: Can polarization increase with actively open-minded thinking? Research & Politics, 4(1), 14. https://doi.org/10.1177/2053168016688122 Google Scholar
Baron, J. (2019). Actively open-minded thinking in politics. Cognition, 188, 818. https://doi.org/10.1016/j.cognition.2018.10.004 Google Scholar
Baron, J., Gürçay, B., & Metz, S. E. (2017). Reflection, intuition, and actively open-minded thinking. In Toplak, M., & Weller, J. (Eds.), Individual differences in judgment and decision making from a developmental context (pp. 107126). London: Routledge.Google Scholar
Baron, J., Isler, O., & Yılmaz, O. (2023). Actively open-minded thinking and the political effects of its absence. In Ottati, V., & Stern, C. (Eds.), Divided: Open-mindedness and dogmatism in a polarized world (pp. 162185). Oxford: Oxford University Press.Google Scholar
Baron, J., Scott, S., Fincher, K., & Metz, S. E. (2015). Why does the Cognitive Reflection Test (sometimes) predict utilitarian moral judgment (and other things)? Journal of Applied Research in Memory and Cognition, 4(3), 265284. https://doi.org/10.1016/j.jarmac.2014.09.003 CrossRefGoogle Scholar
Barrett, L. F., Tugade, M. M., & Engle, R. W. (2004). Individual differences in working memory capacity and dual-process theories of the mind. Psychological Bulletin, 130(4), 553573. https://psycnet.apa.org/doi/10.1037/0033-2909.130.4.553 Google Scholar
Bear, A., & Rand, D. G. (2016). Intuition, deliberation, and the evolution of cooperation. Proceedings of the National Academy of Sciences, 113(4), 936941. https://doi.org/10.1073/pnas.1517780113 Google Scholar
Betsch, C. (2004). [Preference for intuition and deliberation (PID): An inventory for assessing affect- and cognition based decision-making]. Zeitschrift für Differentielle und Diagnostische Psychologie, 25(4), 179197. https://doi.org/10.1024/0170-1789.25.4.179 Google Scholar
Boissin, E., Caparos, S., Raoelison, M., & De Neys, W. (2021). From bias to sound intuiting: Boosting correct intuitive reasoning. Cognition, 211, 104645. https://doi.org/10.1016/j.cognition.2021.104645 CrossRefGoogle ScholarPubMed
Bouvet, R., & Bonnefon, J. F. (2015). Non-reflective thinkers are predisposed to attribute supernatural causation to uncanny experiences. Personality and Social Psychology Bulletin, 41(7), 955961. https://doi.org/10.1177/0146167215585728 Google Scholar
Brotherton, R., French, C. C., & Pickering, A. D. (2013). Measuring Belief in Conspiracy Theories: The Generic Conspiracist Beliefs Scale. Frontiers in Psychology, 4. https://doi.org/10.3389/fpsyg.2013.00279 Google Scholar
Bryan, W. L., & Harter, N. (1899). Studies on the telegraphic language: The acquisition of a hierarchy of habits. Psychological Review, 6(4), 345375. https://psycnet.apa.org/doi/10.1037/h0073117 Google Scholar
Budner, S. N. Y. (1962). Intolerance of ambiguity as a personality variable 1. Journal of Personality, 30(1), 2950. https://doi.org/10.1111/j.1467-6494.1962.tb02303.x Google Scholar
Cacioppo, J. T., & Petty, R. E. (1982). The need for cognition. Journal of Personality and Social Psychology, 42(1), 116131. https://doi.org/10.1037/0022-3514.42.1.116 Google Scholar
Cacioppo, J. T., Petty, R. E., & Feng Kao, C. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48(3), 306307. https://doi.org/10.1207/s15327752jpa4803_13 Google Scholar
Caprara, G. V., Vecchione, M., Schwartz, S. H., Schoen, H., Bain, P. G., Silvester, J., Cieciuch, J., Pavlopoulos, V., Bianchi, G., Kirmanoglu, H., Baslevent, C., Mamali, C., Manzi, J., Katayama, M., Posnova, T., Tabernero, C., Torres, C., Verkasalo, M., Lönnqvist, J.-E., … Caprara, M. G. (2018). The contribution of religiosity to ideology: Empirical evidences from five continents. Cross-Cultural Research, 52(5), 524541. https://doi.org/10.1177/1069397118774233 Google Scholar
Charness, G., Gneezy, U., & Halladay, B. (2016). Experimental methods: Pay one or pay all. Journal of Economic Behavior & Organization, 131, 141150. https://doi.org/10.1016/j.jebo.2016.08.010 Google Scholar
Çarkoğlu, A. (2007). The nature of left–right ideological self‐placement in the Turkish context. Turkish Studies, 8(2), 253271. https://doi.org/10.1080/14683840701312245 Google Scholar
Curry, O. S. (2016). Morality as cooperation: A problem-centred approach. In Shackelford, T. K. & Hansen, R. D. (Eds.), The evolution of morality (pp. 2751). Cham: Springer.Google Scholar
Curry, O. S., Chesters, M. J., & Van Lissa, C. J. (2019a). Mapping morality with a compass: Testing the theory of ‘morality-as-cooperation’ with a new questionnaire. Journal of Research in Personality, 78, 106124. https://doi.org/10.1016/j.jrp.2018.10.008 Google Scholar
Curry, O. S., Mullins, D. A., & Whitehouse, H. (2019b). Is it good to cooperate? Testing the theory of morality-as-cooperation in 60 societies. Current Anthropology, 60(1), 4769. https://doi.org/10.1086/701478 Google Scholar
De Neys, W., & Glumicic, T. (2008). Conflict monitoring in dual process theories of thinking. Cognition, 106(3), 12481299. https://doi.org/10.1016/j.cognition.2007.06.002 CrossRefGoogle ScholarPubMed
Dennin, A., Furman, K., Pretz, J. E., & Roy, M. M. (2022). The relationship of types of intuition to thinking styles, beliefs, and cognitions. Journal of Behavioral Decision Making, 35(5), e2283. https://doi.org/10.1002/bdm.2283 Google Scholar
Deppe, K. D., Gonzalez, F. J., Neiman, J. L., Jacobs, C., Pahlke, J., Smith, K. B., & Hibbing, J. R. (2015). Reflective liberals and intuitive conservatives: A look at the Cognitive Reflection Test and ideology. Judgment and Decision making, 10(4), 314331. https://doi.org/10.1017/S1930297500005131 Google Scholar
Dogruyol, B., Velioglu, İ., Bayrak, F., Acem, E., Isler, O., & Yilmaz, O. (2023). Validation of the moral foundations questionnaire-2 in Türkiye and its relation to actual moral behavior (manuscript submitted for publication).Google Scholar
Dyer, K. D., & Hall, R. E. (2019). Effect of critical thinking education on epistemically unwarranted beliefs in college students. Research in Higher Education, 60, 293314. https://doi.org/10.1007/s11162-018-9513-3 Google Scholar
Eidelman, S., Crandall, C. S., Goodman, J. A., & Blanchar, J. C. (2012). Low-effort thought promotes political conservatism. Personality and Social Psychology Bulletin, 38(6), 808820. https://doi.org/10.1177/0146167212439213 CrossRefGoogle ScholarPubMed
Epstein, S., Pacini, R., Denes-Raj, V., & Heier, H. (1996). Individual differences in intuitive-experiential and analytical-rational thinking styles. Journal of Personality and Social Psychology, 71(2), 390405. https://doi.org/10.1037/0022-3514.71.2.390 CrossRefGoogle ScholarPubMed
Evans, J. S. B. (2003). In two minds: dual-process accounts of reasoning. Trends in Cognitive Sciences, 7(10), 454459. https://doi.org/10.1016/j.tics.2003.08.012 Google Scholar
Evans, J. S. B. (2008). Dual-processing accounts of reasoning, judgment, and social cognition. Annual Review of Psychology, 59, 255278. https://doi.org/10.1146/annurev.psych.59.103006.093629 Google Scholar
Evans, J. S. B., & Stanovich, K. E. (2013). Dual-process theories of higher cognition: Advancing the debate. Perspectives on Psychological Science, 8(3), 223241. https://doi.org/10.1177/1745691612460685 Google Scholar
Frankish, K., & Evans, J. S. B. T. (2009). The duality of mind: An historical perspective. In Evans, J. & Frankish, K. (Eds.), Two minds: Dual processes and beyond (pp. 129). Oxford: Oxford University Press.Google Scholar
Frederick, S. (2005). Cognitive reflection and decision making. Journal of Economic Perspectives, 19(4), 2542. https://doi.org/10.1257/089533005775196732 CrossRefGoogle Scholar
Graham, J., Nosek, B. A., Haidt, J., Iyer, R., Koleva, S., & Ditto, P. H. (2011). Mapping the moral domain. Journal of Personality and Social Psychology, 101(2), 366385. https://psycnet.apa.org/doi/10.1037/a0021847 Google Scholar
Gülgöz, S., & Sadowski, J. (1995). Düşünme ihtiyacı ölçeğinin türkçe uyarlamasıve öğrenci başarısı göstergeleri ile korelasyonu. Türk Psikoloji Dergisi, 10(35), 1524.Google Scholar
Haidt, J. (2012). The righteous mind: Why good people are divided by politics and religion. New York: Vintage.Google Scholar
Hancock, G. R., & Mueller, R. O. (2001). Rethinking construct reliability within latent variable systems. In Cudeck, R., du Toit, S., & Sorbom, D. (Eds.), Structural equation modeling: Present and future—A Festschrift in honor of Karl Joreskog (pp. 195216). Lincolnwood, IL: Scientific Software International.Google Scholar
Haran, U., Ritov, I., & Mellers, B. A. (2013). The role of actively open-minded thinking in information acquisition, accuracy, and calibration. Judgment and Decision making, 8(3), 188201. https://doi.org/10.1017/S1930297500005921 Google Scholar
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world? Behavioral and Brain Sciences, 33(2-3), 6183. https://doi.org/10.1017/s0140525x0999152x Google Scholar
Hertzog, C., Smith, R. M., & Ariel, R. (2018). Does the Cognitive Reflection Test actually capture heuristic versus analytic reasoning styles in older adults? Experimental Aging Research, 44(1), 1834. https://doi.org/10.1080/0361073X.2017.1398508 Google Scholar
Hu, L. T., & Bentler, P. M. (1999). Cut-off criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 155. https://doi.org/10.1080/10705519909540118 Google Scholar
Jarvis, W. B. G., & Petty, R. E. (1996). The need to evaluate. Journal of Personality and Social Psychology, 70(1), 172194. https://doi.org/10.1037/0022-3514.70.1.172 CrossRefGoogle Scholar
Jost, J. T. (2017). Ideological asymmetries and the essence of political psychology. Political Psychology, 38(2), 167208. https://doi.org/10.1111/pops.12407 Google Scholar
Kahneman, D. (2011). Thinking, fast and slow. New York: Macmillan.Google Scholar
Kam, C. C. S., & Meyer, J. P. (2023). Testing the nonlinearity assumption underlying the use of reverse-keyed items: A logical response perspective. Assessment, 30(5), 15691589. https://doi.org/10.1177/10731911221106775 Google Scholar
Kinnunen, S. P., & Windmann, S. (2013). Dual-processing altruism. Frontiers in Psychology, 4, 193. https://doi.org/10.3389/fpsyg.2013.00193 CrossRefGoogle ScholarPubMed
Klaczynski, P. A. (2004). A dual-process model of adolescent development: Implications for decision making, reasoning, and identity. Advances in Child Development and Behavior, 32, 73123. https://doi.org/10.1016/S0065-2407(04)80005-3 CrossRefGoogle ScholarPubMed
Kruglanski, A. W., Atash, M. N., De Grada, E., Mannetti, L., & Pierro, A. (2013). Need for Closure Scale (NFC). Measurement Instrument Database for the Social Science. https://www.midss.org/ Google Scholar
Krumrei-Mancuso, E. J., & Rouse, S. V. (2016). The development and validation of the comprehensive intellectual humility scale. Journal of Personality Assessment, 98(2), 209221. https://doi.org/10.1080/00223891.2015.1068174 Google Scholar
Meade, A. W., Johnson, E. C., & Braddy, P. W. (2008). Power and sensitivity of alternative fit indices in tests of measurement invariance. Journal of Applied Psychology, 93(3), 568592. https://psycnet.apa.org/doi/10.1037/0021-9010.93.3.568 Google Scholar
Muthen, L. K., & Muthen, B. O. (2015). Mplus. Statistical Analysis with Latent Variables: User’s Guide. Los Angeles, CA: Muthen & Muthen.Google Scholar
Muthukrishna, M., Bell, A. V., Henrich, J., Curtin, C. M., Gedranovich, A., McInerney, J., & Thue, B. (2020). Beyond Western, Educated, Industrial, Rich, and Democratic (WEIRD) psychology: Measuring and mapping scales of cultural and psychological distance. Psychological Science, 31(6), 678701. https://doi.org/10.1177/0956797620916782.Google Scholar
Newton, C., Feeney, J., & Pennycook, G. (2023). On the disposition to think analytically: Four distinct intuitive-analytic thinking styles. Personality and Social Psychology Bulletin. https://doi.org/10.1177/01461672231154886 Google Scholar
Nisbett, R. E., Peng, K., Choi, I., & Norenzayan, A. (2001). Culture and systems of thought: holistic versus analytic cognition. Psychological Review, 108(2), 291310. https://psycnet.apa.org/doi/10.1037/0033-295X.108.2.291 Google Scholar
Osman, M. (2004). An evaluation of dual-process theories of reasoning. Psychonomic Bulletin & Review, 11(6), 9881010. https://doi.org/10.3758/BF03196730 Google Scholar
Pennycook, G., Cheyne, J. A., Barr, N., Koehler, D. J., & Fugelsang, J. A. (2014). The role of analytic thinking in moral judgements and values. Thinking & Reasoning, 20(2), 188214. https://doi.org/10.1080/13546783.2013.865000 Google Scholar
Pennycook, G., Cheyne, J. A., Koehler, D. J., & Fugelsang, J. A. (2016a). Is the cognitive reflection test a measure of both reflection and intuition? Behavior Research Methods, 48, 341348. https://doi.org/10.3758/s13428-015-0576-1 Google Scholar
Pennycook, G., Cheyne, J. A., Seli, P., Koehler, D. J., & Fugelsang, J. A. (2012). Analytic cognitive style predicts religious and paranormal belief. Cognition, 123(3), 335346. https://doi.org/10.1016/j.cognition.2012.03.003 Google Scholar
Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). Everyday consequences of analytic thinking. Current Directions in Psychological Science, 24(6), 425432. https://doi.org/10.1177/0963721415604610 Google Scholar
Pennycook, G., & Rand, D. G. (2021). The psychology of fake news. Trends in Cognitive Sciences, 25(5), 388402. https://doi.org/10.1016/j.tics.2021.02.007 Google Scholar
Pennycook, G., Ross, R. M., Koehler, D. J., & Fugelsang, J. A. (2016b). Atheists and agnostics are more reflective than religious believers: Four empirical studies and a meta-analysis. Plos One, 11(4), e0153039. https://doi.org/10.1371/journal.pone.0153039 Google Scholar
Piurko, Y., Schwartz, S. H., & Davidov, E. (2011). Basic personal values and the meaning of left‐right political orientations in 20 countries. Political Psychology, 32(4), 537561. https://doi.org/10.1111/j.1467-9221.2011.00828.x Google Scholar
Raftery, A. E. (1995). Bayesian Model Selection in Social Research. Sociological Methodology, 25, 111. https://doi.org/10.2307/271063 Google Scholar
Rand, D. G. (2016). Cooperation, fast and slow: Meta-analytic evidence for a theory of social heuristics and self-interested deliberation. Psychological Science, 27(9), 11921206. https://doi.org/10.1177/0956797616654455 Google Scholar
Raoelison, M., Boissin, E., Borst, G., & De Neys, W. (2021). From slow to fast logic: The development of logical intuitions. Thinking & Reasoning, 27(4), 599622. https://doi.org/10.1080/13546783.2021.1885488 Google Scholar
Salali, G. D., & Uysal, M. S. (2021). Why some hesitate more: Cross-cultural variation in conspiracy beliefs, belief in science, and vaccine attitudes. MedRxiv. https://doi.org/10.1101/2021.07.09.21260228 Google Scholar
Saribay, S. A., Yilmaz, O., & Körpe, G. G. (2020). Does intuitive mindset influence belief in God? A registered replication of. Judgment and Decision Making, 15(2), 193202. https://doi.org/10.1017/s1930297500007348 Google Scholar
Schleicher, A. (2019). PISA 2018: Insights and interpretations. Paris: OECD Publishing.Google Scholar
Scott, S. G., & Bruce, R. A. (1995). Decision-making style: The development and assessment of a new measure. Educational and Psychological Measurement, 55(5), 818831. https://doi.org/10.1177/0013164495055005017 CrossRefGoogle Scholar
Sheeran, P. (2002). Intention—behavior relations: A conceptual and empirical review. European Review of Social Psychology, 12(1), 136. https://doi.org/10.1080/14792772143000003 Google Scholar
Sheeran, P., & Webb, T. L. (2016). The intention–behavior gap. Social and Personality Psychology Compass, 10(9), 503518. https://doi.org/10.1111/spc3.12265 Google Scholar
Shenhav, A., Rand, D. G., & Greene, J. D. (2012). Divine intuition: Cognitive style influences belief in God. Journal of Experimental Psychology: General, 141(3), 423. https://psycnet.apa.org/doi/10.1037/a0025391 CrossRefGoogle ScholarPubMed
Skitka, L. J., Mullen, E., Griffin, T., Hutchinson, S., & Chamberlin, B. (2002). Dispositions, scripts, or motivated correction? Understanding ideological differences in explanations for social problems. Journal of Personality and Social Psychology, 83(2), 470487. https://psycnet.apa.org/doi/10.1037/0022-3514.83.2.470 CrossRefGoogle ScholarPubMed
Šrol, J. (2022). Individual differences in epistemically suspect beliefs: the role of analytic thinking and susceptibility to cognitive biases. Thinking & Reasoning, 28(1), 125162. https://doi.org/10.1080/13546783.2021.1938220 CrossRefGoogle Scholar
Ståhl, T., Zaal, M. P., & Skitka, L. J. (2016). Moralized rationality: Relying on logic and evidence in the formation and evaluation of belief can be seen as a moral issue. Plos One, 11(11), e0166332. https://doi.org/10.1371/journal.pone.0166332 Google Scholar
Stanovich, K. E., & Toplak, M. E. (2019). The need for intellectual diversity in psychological science: Our own studies of actively open-minded thinking as a case study. Cognition, 187, 156166. https://doi.org/10.1016/j.cognition.2019.03.006 Google Scholar
Stanovich, K. E., & West, R. F. (1997). Reasoning independently of prior belief and individual differences in actively open-minded thinking. Journal of Educational Psychology, 89(2), 342357. https://psycnet.apa.org/doi/10.1037/0022-0663.89.2.342 Google Scholar
Stanovich, K. E., & West, R. F. (2000). Advancing the rationality debate. Behavioral and Brain Sciences, 23(5), 701717. https://doi.org/10.1017/S0140525X00623439 Google Scholar
Stanovich, K. E., & West, R. F. (2007). Natural myside bias is independent of cognitive ability. Thinking & Reasoning, 13(3), 225247. https://doi.org/10.1080/13546780600780796 Google Scholar
Stupple, E. J., Pitchford, M., Ball, L. J., Hunt, T. E., & Steel, R. (2017). Slower is not always better: Response-time evidence clarifies the limited role of miserly information processing in the Cognitive Reflection Test. Plos One, 12(11), e0186404. https://doi.org/10.1371/journal.pone.0186404 Google Scholar
Svedholm, A. M., & Lindeman, M. (2013). The separate roles of the reflective mind and involuntary inhibitory control in gatekeeping paranormal beliefs and the underlying intuitive confusions. British Journal of Psychology, 104(3), 303319. https://doi.org/10.1111/j.2044-8295.2012.02118.x Google Scholar
Svedholm-Häkkinen, A. M., & Lindeman, M. (2018). Actively open-minded thinking: Development of a shortened scale and disentangling attitudes towards knowledge and people. Thinking & Reasoning, 24(1), 2140. https://doi.org/10.1080/13546783.2017.1378723 Google Scholar
Thomson, K. S., & Oppenheimer, D. M. (2016). Investigating an alternate form of the cognitive reflection test. Judgment and Decision Making, 11(1), 99113. https://doi.org/10.1017/S1930297500007622 Google Scholar
Tobacyk, J., & Milford, G. (1983). Belief in paranormal phenomena: Assessment instrument development and implications for personality functioning. Journal of Personality and Social Psychology, 44(5), 10291037. https://psycnet.apa.org/doi/10.1037/0022-3514.44.5.1029 Google Scholar
Van de Vyver, J., Houston, D. M., Abrams, D., & Vasiljevic, M. (2016). Boosting belligerence: How the July 7, 2005, London bombings affected liberals’ moral foundations and prejudice. Psychological Science, 27(2), 169177. https://doi.org/10.1177/0956797615615584 Google Scholar
van Prooijen, J. W. (2017). Why education predicts decreased belief in conspiracy theories. Applied Cognitive Psychology, 31(1), 5058. https://doi.org/10.1002/acp.3301 Google Scholar
Varol, M. (2023). The effect of 2023 Türkiye – Syria Earthquake on cooperative behaviors and intentions. Unpublished Master Thesis, Kadir Has University, İstanbul.Google Scholar
Willard, A. K., & Norenzayan, A. (2013). Cognitive biases explain religious belief, paranormal belief, and belief in life’s purpose. Cognition, 129(2), 379391. https://doi.org/10.1016/j.cognition.2013.07.016 Google Scholar
Wu, Y., Kuru, O., Campbell, S. W., & Baruh, L. (2022). Explaining health misinformation belief through news, social, and alternative health media use: The moderating roles of need for cognition and faith in intuition. Health Communication, 38, 114. https://doi.org/10.1080/10410236.2021.2010891 Google Scholar
Yelbuz, B. E., Madan, E., & Alper, S. (2022). Reflective thinking predicts lower conspiracy beliefs: A meta-analysis. Judgment and Decision Making, 17(4), 720744. https://doi.org/10.1017/S1930297500008913 Google Scholar
Yilmaz, O. (2021). Cognitive styles and religion. Current Opinion in Psychology, 40, 150154. https://doi.org/10.1016/j.copsyc.2020.09.014 Google Scholar
Yilmaz, O., & Alper, S. (2019). The link between intuitive thinking and social conservatism is stronger in WEIRD societies. Judgment and Decision Making, 14(2), 156169. https://doi.org/10.1017/S1930297500003399 Google Scholar
Yilmaz, O., Harma, M., Bahçekapili, H. G., & Cesur, S. (2016a). Validation of the Moral Foundations Questionnaire in Turkey and its relation to cultural schemas of individualism and collectivism. Personality and Individual Differences, 99, 149154. https://doi.org/10.1016/j.paid.2016.04.090 Google Scholar
Yilmaz, O., Harma, M., & Doğruyol, B. (2021). Validation of Morality as Cooperation Questionnaire in Turkey, and its relation to prosociality, ideology, and resource scarcity. European Journal of Psychological Assessment, 37(2), 149160. https://doi.org/10.1027/1015-5759/a000627 Google Scholar
Yilmaz, O., Karadöller, D. Z., & Sofuoglu, G. (2016b). Analytic thinking, religion, and prejudice: An experimental test of the dual-process model of mind. The International Journal for the Psychology of Religion, 26(4), 360369. https://doi.org/10.1080/10508619.2016.1151117 Google Scholar
Yilmaz, O., & Saribay, S. A. (2017a). Analytic thought training promotes liberalism on contextualized (but not stable) political opinions. Social Psychological and Personality Science, 8(7), 789795. https://doi.org/10.1177/1948550616687092 Google Scholar
Yilmaz, O., & Saribay, S. A. (2017b). Activating analytic thinking enhances the value given to individualizing moral foundations. Cognition, 165, 8896. https://doi.org/10.1016/j.cognition.2017.05.009 Google Scholar
Yilmaz, O., & Saribay, S. A. (2017c). The relationship between cognitive style and political orientation depends on the measures used. Judgment and Decision Making, 12(2), 140147.CrossRefGoogle Scholar
Yilmaz, O., & Saribay, S. A. (2018). Lower levels of resistance to change (but not opposition to equality) is related to analytic cognitive style. Social Psychology, 49(2), 6575. https://doi.org/10.1027/1864-9335/a000328 Google Scholar
Yonelinas, A. P. (1994). Receiver-operating characteristics in recognition memory: Evidence for a dual-process model. Journal of Experimental Psychology: Learning, Memory, and Cognition, 20(6), 13411354. https://psycnet.apa.org/doi/10.1037/0278-7393.20.6.1341 Google Scholar
Figure 0

Table 1 Fit indices of the 4-CTSQ

Figure 1

Table 2 Standardized factor loadings for the 4-CTSQ

Figure 2

Table 3 Descriptive statistics for the 4-CTSQ dimensions

Figure 3

Table 4 Correlations between the 4-CTSQ dimensions and outcome variables

Figure 4

Table 5 Standardized regression coefficients for the 4-CTSQ dimensions on outcome variables

Supplementary material: File

Bayrak et al. supplementary material
Download undefined(File)
File 11 KB