The limitations of traditional diagnostic systems for mental disorders – such as the Diagnostic and Statistical Manual of Mental Disorders (DSM; American Psychiatric Association, 2013) – have received increasing attention in the 21st century (e.g. Insel et al., Reference Insel, Cuthbert, Garvey, Heinssen, Pine, Quinn and Wang2010). For example, the heterogeneity within traditional diagnostic categories means that studying these constructs can obscure causes, treatment effects and outcomes that are specific to one symptom or a tightly bound syndrome nested within a traditional diagnostic category (e.g. Bentall, Wickham, Shevlin, & Varese, Reference Bentall, Wickham, Shevlin and Varese2012; Fried & Nesse, Reference Fried and Nesse2015a; Parker, Reference Parker2005). Similarly, the overlapping symptoms between diagnoses mean that studying one diagnosis at a time results in lost opportunities to identify mechanisms associated with symptoms or syndromes that cut across multiple disorders (e.g. Barlow, Sauer-Zavala, Carl, Bullis, & Ellard, Reference Barlow, Sauer-Zavala, Carl, Bullis and Ellard2014). Describing these patterns of overlap in the symptom-level structure of the DSM-5 – and understanding how pervasive they are – could thus provide new insights into symptoms that have high or low specificity for differentiating syndromes and associated mechanisms.
Several studies have examined the descriptive symptom-level structure of traditional diagnostic systems, with a particular focus on understanding comorbidity among diagnoses. For example, Borsboom, Cramer, Schmittmann, Epskamp, and Waldorp (Reference Borsboom, Cramer, Schmittmann, Epskamp and Waldorp2011) generated a network of symptom-level overlap in DSM-IV-TR, examining the ‘small world of psychopathology’ whereby shared symptoms resulted in observed connections among the majority of diagnoses. Tio, Epskamp, Noordhof and Borsboom (Reference Tio, Epskamp, Noordhof and Borsboom2016) subsequently used the same approach to examine a network of the symptom-level overlap in the ICD-10. Most recently, Forbes (Reference Forbes2023) examined whether the repetition of symptoms among a subset of DSM-5 disorders is likely to be inflating the surface similarity of diagnoses in a way that artificially reinforces dimensions based on patterns of disorder covariation or comorbidity (i.e. in the Hierarchical Taxonomy of Psychopathology; Kotov et al., Reference Kotov, Krueger, Watson, Achenbach, Althoff, Bagby and Zimmerman2017). Each of these studies focused on disorder-level overlap and comorbidity based on the idea that considerable overlap of symptoms among major depressive disorder (MDD) and generalised anxiety disorder, for example, makes it more likely an individual will meet criteria for both diagnoses at the same time.
By contrast, the present study takes a descriptive approach to untangling the elements of psychopathology, to address five research questions: (1) How many distinct symptoms comprise the hundreds of diagnoses defined in DSM-5? (2) What proportion of these symptoms repeat across multiple diagnoses and/or chapters? (3) What patterns are evident in the symptom overlap among diagnoses within and between different chapters? (4) Are some chapters of psychopathology more prone to symptom repetition than others? (5) And, finally, which symptoms show the greatest non-specificity as indicators of varied manifestations of psychopathology? Laying bare these patterns represents an essential step towards characterising the heterogeneity and homogeneity in the constructs our field has been studying for decades.
Method
The first stage of coding aimed to distil the constituent symptomsFootnote 1 of the diagnoses in chapters 1–19 of section II of the DSM-5. Similar to the approaches described in Borsboom et al. (Reference Borsboom, Cramer, Schmittmann, Epskamp and Waldorp2011) and Tio et al. (Reference Tio, Epskamp, Noordhof and Borsboom2016), the diagnostic criteria for all diagnoses and specifiers were reduced to their core symptoms. In this process, disjunctive criteria were split into separate symptoms (e.g. ‘insomnia or hypersomnia nearly every day’ was split into insomnia and hypersomnia). Only symptoms relevant to adult psychopathology were included (e.g. child-only symptoms, such as ‘Is often truant from school’ for conduct disorder, were not included). Symptoms were separated from their causes and consequences – including associated distress and impairment – as well as from descriptive information about symptom onset, duration, frequency and severity. Further, symptoms were only listed once per diagnosis to avoid artificially introducing repetition (e.g. psychomotor agitation is listed twice in the criteria for bipolar I disorder, but only listed once in the constituent symptoms for the diagnosis).
Primary disorders with any symptoms described in their diagnostic criteria were included at the outset, as well as specifiers that listed any additional symptoms for the corresponding disorder or chapter. Specifiers were treated akin to discrete diagnoses, rather than collapsed into the criteria for the relevant disorders, and were only listed once for each chapter (e.g. the specifiers for depressive disorders – such as ‘with anxious distress’ and ‘with melancholic features’ – can be appended to all of the diagnoses listed in the depressive disorders chapter, but were listed only once each for the chapter). Specifiers were not included if they only specified a cause, context, pattern of comorbidity with other conditions or disorders, subset/mixing of symptoms already listed in the primary disorder, onset, illness course, frequency, severity, duration, episodicity or familial patterns. Other specified and unspecified disorders were typically excluded for these same reasons, but were included if new symptoms were introduced or a novel syndrome was described (e.g. night eating syndrome was included, as described under other specified feeding or eating disorder). Online Supplementary Table S1 lists the 85 primary disorders that were not directly represented in the current analyses, 82 of which included no additional symptoms.
There were several cases with ambiguity regarding the symptoms comprising a diagnosis. For example, neurocognitive disorders list criteria like ‘evidence of significant cognitive decline’; for these disorders, the symptom examples listed under each of the neurocognitive domains were used (American Psychiatric Association, 2013, 593–595), guided by the specific indicators or domains listed in the diagnostic criteria. Other very broad symptoms were either mapped onto subsets of closely related symptoms, or onto the examples of symptoms listed in the text. For example, in adjustment disorders, ‘disturbance of conduct’ was coded to comprise the corresponding symptoms of conduct disorder, and ‘emotional symptoms’ was coded to comprise the specific examples of these symptoms listed in the specifiers.
The resulting list of symptoms was then coded for content overlap using both qualitative content coding and natural language processing (NLP). Initially, identical symptoms were identified and coded as such. Following this, conceptual redundancy was coded by four members of the research team in a three-step process. Symptoms were first assigned to classes of affective, behavioural, cognitive and/or somatic symptoms, and then to subcategories (e.g. affective symptoms were coded into low mood, elevated mood, fluctuating mood, anxious mood, angry mood and restricted affect). Symptoms within each subcategory were then coded for redundancy using the heuristic of whether the same self-report item could capture multiple symptoms (i.e. that the symptoms represent the same subjective experience, such as depressed mood v. low mood). This process was repeated and refined throughout multiple stages (i.e. we estimate the full list of symptoms was manually screened for repetition more than 90 times in total).
We also used NLP to screen for semantic matches that the manual coding may have missed. To do so, we built a computational model with the goal to identify when two symptoms described in the DSM-5 had the same meaning based on their position in a high-dimensional representation of semantic similarity. After filtering out the pairs of symptoms that had been identified as completely identical, the model scored the 566 580 remaining possible pairs of symptoms from 0 (very dissimilar) to 1 (semantically identical) using a pre-trained model that was fine-tuned on 1067 pairs of non-identical symptoms manually coded as ‘definitely the same’ and ‘definitely different’. A fivefold cross-validation framework was then used to assess how well the model performed, with mean precision and recall across the folds of 0.772, an F1 score of 0.766, and area under the curve of 0.859 – indicating that the model was quite good at identifying semantically identical descriptions and ranking pairs of symptoms by semantic similarity for further manual checking. The top 1000 pairs with the highest semantic similarity scores were manually checked for additional matches by two researchers, identifying 26 new matching symptom pairs. See the Online Supplementary materials for more information on this process. At the end of both stages of coding for content overlap, there were 3096 matching symptom pairs.
Results
All told, 202 diagnosesFootnote 2 are directly represented here, including 135 primary disorders and 76Footnote 3 specifiers or other specified disorders with additional symptoms. We identified a total of 1419 constituent symptoms, and our qualitative and computational content overlap analyses identified 628 distinct symptoms in this list. The full dataset showing all symptoms and redundancy coding is available on the Open Science Framework (https://osf.io/r5vqk/).
Figure 1 shows the patterns of symptom repetition among the 202 diagnoses. While repetition appears to be pervasive, the majority (n = 397, 63.2%) of the 628 distinct symptoms are unique to a single diagnosis. The other, non-unique symptoms (n = 231, 36.8%) occur an average of 4.4 times (standard deviation = 3.4, median = 3, range = 2–22), a total of 1022 times, and together make up 72.0% of the symptoms listed in all of the diagnostic criteria. Of these 231 symptoms that overlap between diagnoses, 163 (70.6%) repeat within the same chapter, 155 (67.1%) repeat between multiple chapters, and 87 (37.7%) repeat both within and between chapters.
Figure 1. Mapping repetition among all symptoms in the DSM-5. Two hundred and two primary disorders and specifiers are represented, and diagnostic criteria have been split into a list of 1419 constituent symptoms. Each dot on the circumference is a symptom; its size represents symptom frequency. Lines linking symptoms map the repetition among diagnoses within and between chapters.
Figure 2 explicates these patterns of within- and between-chapter symptom overlap (see Online Supplementary Figs S1–S19 for individual panels for each chapter, with symptom and diagnosis labels). Table 1 describes some patterns of repetition at both the diagnosis and symptom level within each chapter. Overall, of the 202 diagnoses represented, 140 (69.3%) have at least one symptom that repeats in another diagnosis – 118 (58.4%) in a diagnosis in another chapter. Further, 75 diagnoses (37.1%) have every symptom repeating in at least one other diagnosis – 47 (23.3%) have every symptom repeating in other chapters. Finally, 62 diagnoses (30.7%) have no symptom overlap (i.e. the corresponding symptoms are listed only once in the DSM-5); notably, 35 of these diagnoses include only a single symptom.
Figure 2. Symptom repetition within each chapter. Each circular plot represents a chapter, the rings within each circle correspond to diagnoses or specifiers, and each dot is a symptom. Joined dots falling along the same radius represent a symptom repeating within the chapter. Filled dots repeat in other chapters; empty dots only occur in one chapter. Copies of all plots are available with much more detail in the online Supplementary materials (online Supplementary Figures S1–S19), and the inset panel shows an example of such a detailed plot for sexual dysfunctions.
Table 1. Summary of symptom repetition in each chapter of the DSM-5
Note. Results corresponding to maximum possible symptom repetition are bolded; those corresponding to minimum possible repetition are underlined.
Some domains are more prone to symptom repetition than others (see Fig. 2 and Table 1). For example, none of the diagnoses described in the elimination disorders, gender dysphoria or paraphilic disorders chapters have any symptoms that repeat in other diagnoses, whereas all diagnoses in the bipolar and related disorders, trauma- and stressor-related disorders, dissociative disorders, neurocognitive disorders and personality disorders chapters have at least one symptom that repeats in another diagnosis.
While 12 (63.2%) of the chapters have more than half of their distinct symptoms unique to a single diagnosis, six chapters (31.6%) have more than half of their symptoms repeating in other chapters: bipolar and related disorders, schizophrenia spectrum and other psychotic disorders, depressive disorders, trauma- and stressor-related disorders, neurocognitive disorders, and disruptive, impulse control and conduct disorders. By contrast, some chapters have very few (<10%) of their symptoms repeating in other chapters: elimination disorders, gender dysphoria, paraphilic disorders, feeding and eating disorders, sexual dysfunctions and obsessive-compulsive related disorders (OCRDs).
Focusing on repetition within chapters, only bipolar and related disorders have most of the symptoms in the chapter repeating in multiple diagnoses. Repetition within chapters is also relatively common for substance-related and addictive disorders (45.3% of symptoms repeating in the chapter), trauma- and stressor-related disorders (37.3%), depressive disorders (33.3%) and neurocognitive disorders (33.0%). Interestingly, there are several chapters with substantial symptom repetition between chapters, but little-to-no repetition within the chapter: neurodevelopmental disorders, anxiety disorders, somatic symptom and related disorders, personality disorders, and disruptive, impulse control and conduct disorders all have only 5–8% of symptoms repeating within their chapter, v. 30–56% of symptoms repeating between chapters.
Finally, we answer the question of which symptoms show the greatest non-specificity by examining symptom repetition among all diagnoses and across chapters (Table 2 and Fig. 3). A noteworthy finding is that the symptoms in the DSM-5 that repeat most frequently, and that repeat across most chapters, are dominated by symptoms of MDD. Specifically, 10 of the top 15 most non-specific symptoms in the DSM-5 appear in the diagnostic criteria for MDD (see Table 2).
Table 2. Top 15 most non-specific symptoms based on highest frequency and repetition across multiple classes of psychopathology
Note. Sorted by the number of diagnoses in which the symptom occurs.
*Denotes symptoms that are part of the diagnostic criteria for major depressive disorder. Online Supplementary Table S2 expands on this list to include all symptoms that repeat across chapters.
Figure 3. Map of symptoms that repeat across chapters, sorted by number of chapters in which the symptom occurs. Each ring represents a chapter, and the dots on the ring are distinct symptoms in that chapter that repeat in other chapters. Joined dots falling along the same radius denote a symptom repeating between chapters. Symptoms in the diagnostic criteria for major depressive disorder are marked with numbers to highlight the considerable overlap depression symptoms show. A detailed version of this plot with symptom labels is presented in online Supplementary Figure S20; another version that includes all 628 symptoms – regardless of whether they repeat across chapters – is presented in online Supplementary Figure S21.
Further examination of MDD symptoms showed that all 20 disaggregated symptoms repeat in other chapters, ranging from 5 to 22 total occurrences each. Even when excluding the five occurrences of a major depressive episode in the diagnostic criteria for various diagnoses (i.e. MDD, schizoaffective disorder, bipolar I, bipolar II and cyclothymic disorder), and excluding all depressive disorders as well as all specifiers for bipolar and related disorders and depressive disorders, 14 of the 20 symptoms for MDD still repeat in 34 other diagnoses across eight chapters.
Discussion
Symptom repetition is perhaps not as pervasive as it first appears in Fig. 1: Nearly two-thirds of the distinct symptoms are unique to a single diagnosis, and 30% of the diagnoses analysed were uncontaminated by repetition – including the entirety of three (albeit small) chapters of paraphilic disorders, elimination disorders and gender dysphoria. Feeding and eating disorders, sexual dysfunctions and OCRDs were also relatively self-contained. However, the 231 symptoms that do repeat – spanning 140 diagnoses and 16 chapters – have interesting stories to tell.
The repetition within chapters often appears to be by design: bipolar-related disorders all consist of hypo/manic and depressive episodes; acute stress disorder and posttraumatic stress disorder describe largely overlapping responses to traumatic experiences; substance use disorders reflect the same core criteria regardless of the substance being used; and neurocognitive disorders are all comprised of deficits in the cognitive domains of complex attention, executive function, learning and memory, language, perceptual-motor skills and social cognition. The repetition in depressive disorders is less aligned to these features of other chapters with substantial within-chapter repetition, and also differs from anxiety disorders, which share a common affective core like depressive disorders but have no overlapping symptoms.
Outside of these examples, most chapters have more repetition with other classes of psychopathology than among their constituent diagnoses – often markedly so. For example, symptoms of neurodevelopmental disorders, anxiety disorders, somatic symptom and related disorders, personality disorders, and disruptive, impulse control and conduct disorders all had substantial repetition in other chapters (30–56%) but relatively little repetition among diagnoses within the chapter (5–8%). This may reflect efforts by the committees overseeing each chapter to ensure the diagnoses are clearly distinguishable – a process that is not, to our knowledge, implemented across chapters.
By contrast, the symptom repetition between chapters appears to be less purposeful. In total, 155 symptoms repeated between chapters, listed 742 times across 118 diagnoses in 16 chapters, corresponding to marked non-specificity for many of these symptoms. The criteria for one diagnosis in particular stood out: the symptoms that occur most frequently and across the most chapters are overwhelmingly those of MDD. Even after excluding all closely related diagnoses (i.e. the five diagnoses with a major depressive episode in their criteria as well as the two depressive disorders and six specifiers for bipolar and depressive disorders with overlapping symptoms), MDD symptoms still repeat in 35 diagnoses spanning anxiety disorders, trauma- and stressor-related disorders, somatic symptom and related disorders, feeding and eating disorders, sleep–wake disorders, substance-related and addictive disorders, neurocognitive disorders and personality disorders. This level of non-specificity in the diagnostic criteria leads us to wonder how meaningful it is to study MDD as a unitary construct, particularly in the context of other literature illustrating the heterogeneity and low predictive validity of MDD diagnoses (e.g. Fried, Flake, & Robinaugh, Reference Fried, Flake and Robinaugh2022; Fried & Nesse, Reference Fried and Nesse2015b; McGlinchey, Zimmerman, Young, & Chelminski, Reference McGlinchey, Zimmerman, Young and Chelminski2006; Winter et al., Reference Winter, Leenings, Ernsting, Sarink, Fisch, Emden and Hahn2022; Zimmerman, Chelminski, McGlinchey, & Young, Reference Zimmerman, Chelminski, McGlinchey and Young2006a; Zimmerman, McGlinchey, Young, & Chelminski, Reference Zimmerman, McGlinchey, Young and Chelminski2006b).
Speculating about potential mechanisms that account for the non-specificity, perhaps MDD symptoms are psychological responses to stress, similar to how fever – a symptom that also cuts across numerous diagnostic categories – reflects an inflammatory response to cell damage or stress (OpenStax, 2019). This idea mirrors work in the psychological test literature where measurement of general distress appears saturated with depression symptoms (e.g. Ben-Porath & Tellegen, Reference Ben-Porath and Tellegen2020; Kessler et al., Reference Kessler, Andrews, Colpe, Hiripi, Mroczek, Normand and Zaslavsky2002). In a similar vein, MDD symptoms like sleep problems, difficulty concentrating and low mood may recapitulate the distress and impairment associated with most DSM-5 diagnoses, rather than identifying a coherent syndrome that corresponds to specific causes, mechanisms or treatment needs (Parker, Reference Parker2005).
Regardless of the underlying mechanisms, the pervasiveness of MDD symptoms throughout the DSM-5 likely hampers diagnostic accuracy in at least two ways. First, it may do so due to misattribution of symptoms in other diagnoses to MDD – or vice versa (Horvath & Todd, Reference Horvath and Todd2023; Zimmerman et al., Reference Zimmerman, Chelminski, McGlinchey and Young2006a). Second, it may inflate rates of comorbidity due to symptom overlap with other diagnoses, which makes it easier to receive multiple diagnoses with only a limited set of presenting symptoms, particularly for diagnoses that share many symptoms like generalised anxiety disorder (e.g. Zbozinek et al., Reference Zbozinek, Rose, Wolitzky-Taylor, Sherbourne, Sullivan, Stein and Craske2012). The inflated surface similarity of diagnoses that share symptoms with MDD may contribute to the higher comorbidity rates of MDD that have been observed with diagnoses that share symptoms, compared to those that do not (e.g. Hasin et al., Reference Hasin, Sarvet, Meyers, Saha, Ruan, Stohl and Grant2018). These same types of bias will also apply to the repetition of other symptoms throughout the DSM-5.
Limitations and future directions
It is important to reiterate that these findings are based on a purely descriptive analysis of the diagnostic criteria laid out in the DSM-5. Therefore, an important caveat is that we made subjective decisions in deciding whether symptoms overlap or not. Although we aimed to mitigate this limitation by using NLP, others may have made different decisions. For example, it can be argued that the phenomenology of difficulties concentrating in MDD v. major neurocognitive disorder v. tobacco withdrawal are qualitatively distinct, regardless of whether they are described using similar language in the DSM. To facilitate alternative interpretations of the patterns we have described, the data are publicly available (https://osf.io/r5vqk/). Further, this study cannot speak to the causes or consequences of symptom overlap within and between chapters, which warrants consideration in the development of the next edition of the DSM. Others have already highlighted the pitfalls of adding non-specific symptoms to new and existing diagnoses, increasing comprehensiveness at the cost of specificity (Stanton, Reference Stanton2020). Empirical work is needed to understand whether the symptoms in the DSM can be disentangled to form distinct homogeneous syndromes, and whether such a change would improve the reliability, validity and utility of the diagnoses. Such work could also determine whether the patterns of (non)specificity observed here are reflected in patterns of covariation among the symptoms and diagnoses.
Conclusion
To the extent that diagnoses are unintentionally repackaging the same information, symptom repetition represents an insidious confound for research and practice. The heterogeneity within and homogeneity between diagnoses suggest there may be cross-cutting symptoms or symptom clusters that could offer a better framework of phenotypes for research on biomarkers and mechanisms (e.g. Insel et al., Reference Insel, Cuthbert, Garvey, Heinssen, Pine, Quinn and Wang2010); active ingredients and specific processes in psychotherapy (e.g. Hofmann & Hayes, Reference Hofmann and Hayes2019; Wolpert, Pote, & Sebastian, Reference Wolpert, Pote and Sebastian2021); and reconceptualisation of the diagnosis and classification of psychopathology (e.g. Kotov et al., Reference Kotov, Krueger, Watson, Achenbach, Althoff, Bagby and Zimmerman2017). Ultimately, more empirical work on fine-grained clinical phenomena promises to improve on the reliability and validity of the DSM-5 constructs that frame much of our research and practice.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291723002544.
Acknowledgements
The authors would like to thank Anton Harris for all his work on redundancy coding, and Hannah Woodbridge for her work on breaking down the DSM-5 into its constituent symptoms.
Financial support
This work was supported by a Macquarie University Research Fellowship (grant number 9201701144) and an Investigator Grant from the Australian National Health and Medical Research Council (grant number GNT1194292).
Competing interests
None.
Open access
