2. Why Extant Evidence Does Not Support Causal Conclusions

Since the 1980s (Hoekema Reference Hoekema1986), philosophers have observed that undergraduates who major in philosophy tend to score remarkably well on standardized tests like the Graduate Record Examination (GRE) and the Law School Admission Test (LSAT). Indeed, this fact is widely advertised by philosophy departments at many institutions, and by the American Philosophical Association (APA 2014; 2019). Moreover, numerous studies have found that people who have studied philosophy tend to be more reflective, open-minded, and skilled in logical reasoning than those who have not done so (for a review, see Prinzing and Vazquez Reference Prinzing and Vazquez2024).

Unfortunately, although these results establish some striking differences between people who have and have not studied philosophy, they do not support claims about the effects of a philosophical education. This is because group differences of this kind can be explained by self-selection as well as by effects of philosophical study. That is, philosophers may be more reflective, open-minded, and logical because studying philosophy cultivated these dispositions and abilities. But it’s also possible that people who are already more reflective, open-minded, and logical are more likely to study philosophy in the first place. Of course, these are not exclusive possibilities. It may be that people who are more skilled with language and logic, more open-minded, and so on are more drawn to philosophy, but then studying philosophy further increases these abilities and dispositions.

This is not simply a problem “in theory.” Recent empirical work has found clear evidence of such self-selection. To illustrate, probably the most well-established difference between those who have and have not studied philosophy is that the former tend to be far more reflective (Byrd Reference Byrd2021; Livengood et al. Reference Livengood, Sytsma, Feltz, Scheines and Machery2010). “Reflectiveness” is typically measured using the Cognitive Reflection Test (Frederick Reference Frederick2005) or similar measures with questions that lure people into giving intuitive but incorrect responses. One such question asks, “If it takes 5 machines 5 minutes to make 5 widgets, how long will it take 100 machines to make 100 widgets?” For many people, “100 minutes” jumps to mind. Yet, a moment of reflection reveals that the correct answer is “5 minutes.” People who have studied philosophy are significantly more likely to correctly answer questions like this one, compared with those who haven’t. However, in prior work, we found that students in the first week of a Philosophy 101 course also scored dramatically higher than the population average on a version of the Cognitive Reflection Test (Prinzing and Vazquez Reference Prinzing and Vazquez2024). That is, these students who were at the very beginning of their philosophical education were already considerably more reflective than most people. Findings like these highlight the risk that observed differences between philosophers and non-philosophers might result largely or even entirely from self-selection.

Naturally, the ideal way to test for effects of philosophical study on intellectual abilities and dispositions would be to use a randomized, controlled experiment. With a large sample, randomization to treatment and control groups ensures that, on average, baseline characteristics do not differ across groups. This means that subsequent differences in the groups’ outcomes are highly unlikely to result from pre-existing differences and are highly likely to instead result from the treatment itself. Unfortunately, although a short-term and small-scale randomized experiment might be feasible, anything at a large-scale or over a long-term does not seem likely ever to take place. For instance, convincing a large group of college freshmen to allow a coin flip to decide whether they will major in philosophy is both practically and ethically fraught.

When it is not possible, not ethical, or simply not practicable to conduct a randomized experiment, some people will throw up their hands and abandon hope for knowledge of causation. But this reaction is premature. After all, there are many sciences, such as epidemiology, economics, and environmental science, in which randomized experiments are rare and, in many cases, impossible or imprudent. For this reason, an entire field of research, spanning numerous disciplines, has developed methods for supporting causal conclusions with non-experimental data (Pearl Reference Pearl2015). Within this field of causal inference research, scholars have developed a range of techniques that can be used to rule out selection effects or other sources of “confounding” (i.e., where two variables are correlated because they share a common cause, known as a confound).Footnote ¹

One particularly straightforward causal inference technique, called “covariate adjustment,” involves statistically controlling for confounds. The obvious limitation of this approach is that one can never know whether one has measured and controlled for all confounds. But there is an elegant way of effectively controlling for a very broad range of confounds, even without knowing exactly what they are. This is by controlling for baseline differences in the outcome of interest. To illustrate, suppose we wanted to test whether majoring in philosophy improves logical reasoning abilities. No doubt, a multitude of factors besides students’ majors will influence their scores on logic tests, anything from students’ educational opportunities in high school, to their parents’ levels of education, their household incomes, or other aspects of their socio-demographic backgrounds, etc. The impact of these factors will very likely have occurred by the time that the students arrive at college. Thus, if we measured students’ logical reasoning abilities both at the start of college and again at the end, then by statistically controlling for freshman year scores when examining differences in senior year scores, we can remove the influence of these confounds. In other words, if philosophy students’ performance on measures of intellectual abilities or dispositions were due to self-selection, then statistically controlling for baseline differences should make philosophy majors look unremarkable. Conversely, if philosophy students continue to score well on a variety of measures, even after controlling for baseline differences, then this would be at least some evidence that studying philosophy improved their scores.

3. How to Measure Good Thinking

One centrally important question for a study of this kind concerns the kinds of measures that could aptly gauge the potential effects of a philosophical education. Naturally, there are many different abilities and dispositions that can make a person a good thinker. Some of these might be well-captured by standardized tests, such as the aforementioned LSAT and GRE. After all, these are fairly comprehensive indexes of verbal, logical, and quantitative reasoning abilities. In fact, the LSAT is primarily a logic test (though it also includes a section that assesses reading comprehension). The GRE has a Verbal Reasoning section that assesses a person’s general facility with language, as well as a Quantitative Reasoning section, assessing mathematical ability. Since the study of philosophy is often claimed to teach students to use language skillfully, to make subtle distinctions and cogent arguments, and to recognize the logical relations among propositions and reason carefully from them, the LSAT and GRE Verbal each seem like fitting empirical metrics.

One potential objection to a focus on standardized tests (especially, but not exclusively the SAT) relates to accusations of racial and/or socio-economic biases (Eberle and Peltier Reference Eberle and Peltier1989). It was on the basis of such concerns that, around the time of the COVID-19 pandemic, numerous colleges and universities dropped standardized tests from their admissions processes. In the years since, many of these institutions have reversed course, and are now requiring applicants to submit test scores again. This is because performance on standardized tests reliably predicts academic success (Friedman et al. Reference Friedman, Sacerdote, Staiger and Tine2025; Leonhardt Reference Leonhardt2024), and removing them from consideration in admissions actually exacerbated the underrepresentation of minority groups (see, e.g., Schmill Reference Schmill2022). Thus, recognizing that all empirical measures are flawed, limited, and subject to measurement errors of various kinds, standardized tests appear to be some of the best available tools for measuring a wide range of valuable intellectual abilities (Currid-Halkett Reference Currid-Halkett2020).

Yet, philosophers also often claim that studying philosophy cultivates certain intellectual virtues like curiosity, intellectual humility, or open-mindedness. Alongside intellectual abilities, these sorts of dispositions also seem quite important for being a good thinker (King Reference King2021). After all, high test scores show that a person can reason well when prompted and incentivized to do so. But that leaves open the question of whether the person is generally disposed to use their abilities to earnestly and thoroughly pursue truth—to think carefully, critically, and with openness and humility. Naturally, such intellectual dispositions can be very difficult to quantify. Empirical measures of, e.g., curiosity, intellectual humility, and open-mindedness are generally going to be self-reports (e.g., Hoyle et al. Reference Hoyle, Davisson, Diebels and Leary2016; Price et al. Reference Price, Ottati, Wilson and Kim2015). And, in contrast to standardized tests, self-reports are subject to biases, such as “self-enhancement” bias (Weiner and Guenther Reference Weiner, Guenther, Zeigler-Hill and Shackelford2020), where individuals may try to present themselves in overly flattering ways.

In short, these two different kinds of measures have complementary strengths and weaknesses. Standardized tests are “objective” in the sense that they are immune to reporting biases. They also capture a broad range of important abilities but might be thought to reflect a relatively thin conception of good thinking. On the other hand, self-reports can capture dispositions like curiosity and open-mindedness, that seem to be important aspects of good thinking. But these are less “objective” in the aforementioned sense. Given these relative advantages and disadvantages, we would ideally like to see converging evidence from both kinds of measures. That is, although either result would be interesting in its own right, evidence that studying philosophy improves both test scores and self-reported intellectual dispositions would provide particularly strong evidence that the discipline makes people better thinkers.

4. Data and Measures for the Present Study

We analyzed data collected between 1990 and 2019 by the Higher Education Research Institute (HERI; https://heri.ucla.edu/) and Cooperative Institutional Research Program, based at the University of California, Los Angeles. Data from students graduating between 1994 and 2008 are publicly accessible in the HERI’s Data Archive (https://heri.ucla.edu/heri-data-archive/). To access the more recent data, we had to apply and pay a fee.

Participating students completed surveys at the start of their freshman year and end of their senior year, in which they reported on their academic majors, standardized test scores, sociodemographic backgrounds, and more. Incoming freshmen were asked to report their SAT scores and, prior to 2005, graduating seniors were also asked to report their GRE and LSAT scores (assuming they had taken these tests). Starting with the 2010 cohort, the surveys also included self-report measures called the “Habits of Mind” and “Pluralistic Orientation” scales. The individual questions in these scales are presented in Table 1. In their online documentation, the HERI describes these, respectively, as assessing “the behaviors and traits associated with academic success… [and] lifelong learning” and the “skills and dispositions appropriate for living and working in a diverse society” (https://www.heri.ucla.edu/PDFs/constructs/Appendix2017.pdf). But just looking at the survey questions themselves, the Habits of Mind scale seems to assess traits like curiosity (see, e.g., items 3, 7, 8), intellectual rigor (items 2, 9), and to some extent intellectual humility (item 5) and open-mindedness (item 10). The Pluralistic Orientation scale seems to be, in effect, a measure of open-mindedness.

Table 1.

Items from self-report measures

The HERI computes composite scores from each set of items. (Specifically, these are factor scores from item response theory models. Further details can be found online: https://heri.ucla.edu/cirp-constructs/.) For ease of interpretation, we standardized these composite scores, meaning that we scaled them to have a mean of 0 and a standard deviation of 1. Hence, a student with a score of 1 is one standard deviation above average, whereas someone with a score of -0.5 is half a standard deviation below average.

The full sample includes N = 649,511 students (including n = 4,843 philosophy majors), attending 804 colleges and universities around the United States. Of these, 59% identified as female, 35% as male, and 5% did not indicate a sex; 72% identified as White, 5% as Asian or Pacific Islander, 5% as Black or African American, 4% as Latino/a/x, 5% as mixed and 2% as another race or ethnicity, and 7% did not indicate a race or ethnicity. Because GRE and LSAT scores were only collected from students graduating between 1994 and 2004 (n = 392,858), and the Habits of Mind and Pluralistic Orientation scales were only administered to students graduating in 2010 or later (n = 122,352), some of our analyses use a subset of the full sample.

5. Empirical Findings

We first looked for evidence of selection effects by testing whether students who score higher on the SAT and the self-report measures of intellectual dispositions in the freshman year survey are more likely to major in philosophy. Second, we tested whether, at senior year, philosophy majors score higher than non-philosophy majors on the GRE, LSAT, Habits of Mind, and Pluralistic Orientation, even after controlling for baseline differences. Third, we compared the baseline-adjusted averages for specific majors, to see where philosophy places in the rankings. Finally, we looked for evidence of a distinct kind of selection effect related to the kinds of students who take the GRE and LSAT (explained further below) that could potentially have biased the standardized testing results.

Our analyses used mixed-effects regression models (logistic regressions for dichotomous outcomes) with random intercepts for institutions (i.e., the colleges and universities that students attended). We fit these models using the lme4 and lmerTest packages in R (Bates et al. Reference Bates, Maechler, Bolker, Steven Walker, Singmann and Dai2020; Kuznetsova, Brockhoff, and Christensen R. H. B. Reference Kuznetsova, Brockhoff and Christensen2017), computed estimated marginal means using the emmeans package (Lenth et al. Reference Lenth, Singmann, Love, Buerkner and Herve2018), and used multiple imputation to accommodate missing data with the mice package (Buuren and Groothuis-Oudshoorn Reference Buuren and Groothuis-Oudshoorn2011). The code used in these analyses is available online (https://osf.io/4s693). Below, we describe the results in colloquial English. More detailed results of the statistical models are given in Tables A1 and A2 in the Appendix.

5.2. Adjusting for Baseline Differences, Philosophers Still Outperform Their Peers

Next, we compared philosophy and non-philosophy majors’ scores on senior year standardized tests and self-report measures while adjusting for baseline differences, observed at the start of the freshman year. The idea, again, is that students’ intellectual abilities and dispositions are shaped by many factors beyond their academic majors, such as prior educational opportunities and socio-demographic backgrounds. These factors will also affect students’ scores at the start of college. So, by controlling for freshman year scores, we should be able to remove the influence of pre-college confounds, thereby giving a more accurate estimate of the treatment effects. Figure 1 plots the results.

Figure 1.

Baseline-adjusted average scores for philosophy and non-philosophy majors. Points and error bars indicate estimated marginal means with 95% confidence intervals, derived from mixed-effects regression models. For GRE Verbal, GRE Quantitative, and LSAT, means are adjusted for SAT scores. For Habits of Mind and Pluralistic Orientation, means are adjusted for scores in the freshman year survey.

Starting with the standardized tests, we ran three separate models for scores on the GRE Verbal, GRE Quantitative, and LSAT. Unsurprisingly, SAT scores were significantly, positively associated with GRE and LSAT scores across the board. But crucially, after accounting for SAT scores, philosophy majors scored significantly higher than non-philosophy majors on the GRE Verbal and LSAT. On the GRE Quantitative, by contrast, there was no significant difference between philosophy and non-philosophy majors. Turning to the self-report measures, we tested whether philosophy majors scored higher on the Habits of Mind and Pluralistic Orientation scales during senior year while adjusting for their scores during freshman year. Unsurprisingly, the freshman year scores were significant predictors in both cases. But, crucially, after accounting for these baseline differences, philosophy majors still scored significantly higher than non-philosophy majors on both Habits of Mind and Pluralistic Orientation.

These results indicate that, even after accounting for differences in pre-college verbal and quantitative reasoning abilities, philosophy majors display stronger verbal and logical (though not quantitative) reasoning abilities than their peers. More precisely, the SAT-adjusted average score for philosophy majors on the GRE Verbal (i.e., 573 out of a possible 800) is about 33 points higher than that of non-philosophy majors (i.e., 540). This amounts to a standardized mean difference (i.e., the difference between groups in terms of standard deviations) of 0.48, which, according to a common convention, would be considered a “medium-sized” effect (Cohen Reference Cohen1988). On the LSAT, the SAT-adjusted average score for philosophy majors (i.e., 157 out of a possible 180) is about 2 points higher than that of non-philosophy majors (i.e., 155). This is a standardized mean difference of 0.35, or a “small to medium-sized” effect. Similarly, senior philosophy majors reported greater levels of positive intellectual dispositions than their peers—encompassing curiosity, intellectual rigor, intellectual humility, and open-mindedness—even after controlling for reports of such dispositions in freshman year. These effects would conventionally be termed “small” effects (standardized mean differences of .24 for Habits of Mind and .21 for Pluralistic Orientation).

Now, although contrasting philosophy majors with non-philosophy majors enables a simple, statistically powerful comparison, it also conceals a great deal of diversity within the “non-philosophy” group. For all we have shown so far, there may be other majors that substantially outperform philosophy. Hence, to test the claim that there is something truly special about the study of philosophy, making it distinctively well-suited to cultivating good thinking, we compared philosophy majors with other specific majors, again while controlling for baseline levels of the outcomes. The sample includes students with 90 different majors, but some of these majors are represented by only small numbers of students. Hence, in these analyses, we excluded majors with fewer than 200 students. This ensures that each major’s average stably reflects characteristics of that major, rather than sampling variability. After these exclusions, the analyses involving the standardized tests included 57 majors and the analyses involving the self-reports included 62 majors.

Comparing the baseline-adjusted average scores for all of these different majors revealed something truly striking. Although philosophy majors are unremarkable when it comes to the GRE Quantitative (placing 30^th out of 57), they rank first on the GRE Verbal, first on the LSAT, first on the Habits of Mind scale, and sixth on the Pluralistic Orientation scale. These rankings are presented in Figures 2-3.

Figure 2.

SAT-adjusted average scores on standardized tests for specific majors. Points and error bars indicate estimated marginal means with 95% confidence intervals derived from mixed-effects regression models. Philosophy is highlighted with red.

Figure 3.

Baseline-adjusted average scores on self-report measures for specific majors. Points and error bars indicate estimated marginal means with 95% confidence intervals derived from mixed-effects regression models. Philosophy is highlighted with red.

6. Implications and Conclusion

These findings support a popular and venerable argument for the value of philosophy. That argument, as discussed in Section 1, rests on a crucial, empirical premise—namely, that studying philosophy makes people better thinkers. In a recent review, we concluded that there was an absence of evidence concerning this premise (Prinzing and Vazquez Reference Prinzing and Vazquez2024). Here, we brought to bear a far larger body of data than any prior study and conducted more rigorous analyses. The data included records from over half a million undergraduate students, attending over 800 institutions across the United States. Moreover, because we have data from these students both at the start and end of their time in higher education, we were able to account for pre-existing differences when comparing philosophy majors with their peers. In this way, we controlled for a host of potential confounds when estimating the effects of philosophical study.

Our results indicated that students with better verbal reasoning abilities and more curiosity, intellectual rigor, and open-mindedness are more likely to major in philosophy. They also indicated that, after adjusting for baseline differences, philosophy majors outperform other students on these measures. In fact, on average, philosophy majors score higher than all other majors on the GRE Verbal and LSAT, as well as a self-report measure designed to assess good habits of mind. Short of a randomized experiment—which for various ethical and practical reasons is unlikely ever to take place—these findings arguably constitute the clearest and strongest kind of empirical support that we will find for the claim that studying philosophy makes students better thinkers.

One important caveat comes from the possibility that certain aspects of the intellectual abilities assessed by the GRE or LSAT are not well assessed by the SAT. For example, unlike the LSAT, the SAT does not include a section dedicated specifically to logical reasoning. Thus, it may be that controlling for SAT scores does not fully account for pre-college differences in logical reasoning abilities. One way to overcome this limitation would be to have students complete the exact same tests (which could be based on the LSAT or other tests, such as the California Critical Thinking Skills Test) at the start and end of their education. Future work may also be able to extend these findings by exploring whether specific forms of philosophical study might affect students differently. For example, do students who focus on ethics show different effects from those who focus on metaphysics? What about students in more analytic versus continental departments? Another interesting avenue would be to look for a “dose-response” relationship—for example by testing whether philosophy majors differ from philosophy minors (who might be thought to have received a smaller “dose” of philosophy).

One limitation of these findings is that, although the Habits of Mind and Pluralistic Orientation items capture some of the behaviors and motivations characteristic of intellectual virtues, they do not speak to the objects, occasions, and means of their exercise. Thus, they do not capture the full profile of intellectual virtue as philosophers traditionally conceive of it (King Reference King2021). To determine whether students of philosophy use their intellectual abilities for the right reasons, with the right means, on the right occasions, and directed at the right objects would require further, and potentially a very different kind of, evidence.

Relatedly, there are many intellectual virtues that one could examine, and the measures used here only touched on a few. For example, one of the items from the Habits of Mind scale could be seen as tapping into intellectual humility (item 5, see Table 1). But there are other, more effective measures of intellectual humility that one could use (see, e.g., Hoyle et al. Reference Hoyle, Davisson, Diebels and Leary2016). And, naturally, one could also consider whether studying philosophy supports virtues like intellectual autonomy, fair-mindedness, intellectual courage, or others (King Reference King2021).

Finally, there are numerous other kinds of claims about the value of philosophy, besides its ability to cultivate intellectual abilities or dispositions. Some would claim that philosophical study supports students’ holistic formation as persons (Standish and Saito Reference Standish, Saito and Thompson2023), or that it fosters autonomy (Standish Reference Standish and Marples1999), or bestows them with “powerful knowledge” (Young and Muller Reference Young and Muller2013). These are plausibly laudable educational goals, but our findings do not speak to these claims. Perhaps future work could investigate these sorts of outcomes—though, there is room for reasonable skepticism about the prospects for empirically assessing them.

Nonetheless, the intellectual abilities and dispositions assessed in this study seem both useful and admirable. Skill with logic enables people to make better inferences, which should plausibly lead them to form more true and fewer false beliefs. Facility with language enables people to articulate their beliefs more clearly to others. The dispositions critical for lifelong learning and open, humble thinking are plausibly useful for people in all walks of life. (Though, the degree to which the various measures used here predict specific outcomes later in life could itself be tested empirically.) Hence, if philosophical study cultivates these abilities and dispositions, as our findings suggest, then this is good news indeed for philosophy. In the highly technocratic and bureaucratized world of the 21^st century academy, the ability to point to such measurable outcomes is often necessary to maintain institutional support for departments and programs. Hence, our findings may have some utility for those advocating for the discipline.

Finally, although we have focused specifically on intellectual outcomes, these same sorts of abilities and dispositions are thought to have substantial implications for the political life of pluralistic societies. For decades, political philosophers have argued that dispositions that facilitate rigorous and autonomous reflection are essential not only for individual flourishing but also for sustaining democratic norms and institutions (Brighouse Reference Brighouse2006; Gutmann Reference Gutmann1995). According to this view, the kinds of intellectual capacities that we have examined support, for example, the ability to tolerate and engage charitably with diverse perspectives and to find common ground within pluralistic societies, and that these are, in turn, essential for the health of democracy (Lynch Reference Lynch2019; Nussbaum Reference Nussbaum2024; Samaržija and Cassam Reference Samaržija and Cassam2023).

Although our study does not directly assess these sorts of civic outcomes, it is noteworthy that philosophy majors ranked highly on a measure like the Pluralistic Orientation scale, which was designed specifically to assess “skills and dispositions appropriate for living and working in a diverse society” and is regarded as an important indicator of the success of a liberal education (Hurtado and DeAngelo Reference Hurtado and DeAngelo2012). It would be worthwhile, in future work, to explore these broader civic and moral effects of philosophical study more directly. After all, it is one thing to form sharp, analytical thinkers, and quite another to cultivate intellectually virtuous citizens inclined to use their minds responsibly in service of the common good.