Differentiated nomological networks of internalizing, externalizing, and the general factor of psychopathology (‘p factor’) in emerging adolescence in the ABCD study

Background Structural models of psychopathology consistently identify internalizing (INT) and externalizing (EXT) specific factors as well as a superordinate factor that captures their shared variance, the p factor. Questions remain, however, about the meaning of these data-driven dimensions and the interpretability and distinguishability of the larger nomological networks in which they are embedded. Methods The sample consisted of 10 645 youth aged 9–10 years participating in the multisite Adolescent Brain and Cognitive Development (ABCD) Study. p, INT, and EXT were modeled using the parent-rated Child Behavior Checklist (CBCL). Patterns of associations were examined with variables drawn from diverse domains including demographics, psychopathology, temperament, family history of substance use and psychopathology, school and family environment, and cognitive ability, using instruments based on youth-, parent-, and teacher-report, and behavioral task performance. Results p exhibited a broad pattern of statistically significant associations with risk variables across all domains assessed, including temperament, neurocognition, and social adversity. The specific factors exhibited more domain-specific patterns of associations, with INT exhibiting greater fear/distress and EXT exhibiting greater impulsivity. Conclusions In this largest study of hierarchical models of psychopathology to date, we found that p, INT, and EXT exhibit well-differentiated nomological networks that are interpretable in terms of neurocognition, impulsivity, fear/distress, and social adversity. These networks were, in contrast, obscured when relying on the a priori Internalizing and Externalizing dimensions of the CBCL scales. Our findings add to the evidence for the validity of p, INT, and EXT as theoretically and empirically meaningful broad psychopathology liabilities.

In the present study, we build upon and extend previous work by examining the nomological networks of the p and INT and EXT specific factors in baseline data from the ABCD multisite study (Volkow et al., 2018) of 118 759-to10-year-old emerging adolescents. While recent work by our group and others also has sought to define the structure of psychopathology in this sample (Clark et al., 2020;Michelini et al., 2019;Moore et al., 2020), these studies have focused primarily comparing alternative structural models of the Child Behavior Checklist (CBCL) (Clark et al., 2020;Michelini et al., 2019;Moore et al., 2020) and validated the derived factors focusing on a small number of criterion variables with the goal of comparing the different factors (Clark et al., 2020;Michelini et al., 2019;Moore et al., 2020). The current study will examine the convergent and divergent associations between the general ( p) and specific (EXT and INT) factors and a comprehensive, multi-domain pool of criterion measures. We focus on delineating the full unique and overlapping profiles of these latent liabilities, as well as comparing their respective nomological network with those that emerge from the original CBCL Internalizing and Externalizing scales. The ABCD study involves a comprehensive assessment battery (Barch et al., 2018), covering demographics, psychopathology, temperament, family history of substance use and psychopathology, socio-environment (school and family environment), and cognitive ability. Multiple assessment modalities are also included, such as youth-, parent-, and teacher-report questionnaires, interviews, and behavioral task performance. Moreover, the ABCD study used a multi-stage probability sampling strategy, and with sample weighting approximates a representative US population sample of 9-and 10-year olds (Garavan et al., 2018), thus improving generalizability (Deutz et al., 2020;Snyder, Young, & Hankin, 2019b). Results from our investigation found that p, INT, and EXT exhibit distinct and well-differentiated nomological networks that are readily interpreted in terms of neurocognitive, temperamental, and social factors. These nomological networks were, in contrast, obscured when relying on the a priori internalizing and externalizing dimensions of the CBCL scales.

Participants
The study used data collected from the ABCD Study, a large-scale study of youth aged 9-10 years (N = 10 645), recruited from 21 research sites across the USA (Barch et al., 2018;Garavan et al., 2018;Volkow et al., 2018). These data were collected from baseline visits between 1 September 2016 and 15 November 2018. The data used in this report came from ABCD Release 2.01, DOI: 10.15154/1504041. The sample was roughly genderbalanced (47.6% female) with a mean age of 9.93 years (S.D. = 0.62 years). Around half (50.7%) of the sample was White, with the remaining participants identifying themselves as Hispanic (19.7%), African-American (14.5%), Other/Multi-racial (9.8%), or Asian (2.1%). Data on ethnicity was missing for 3.2% of the sample. Approximately two-thirds of youth (65.5%) came from

3052
Sarah J. Brislin et al. households in which the parents were married. Most parents (83.0%) reported at least some college and most households reported an annual income of at least $50 000 (62.6%).

Measures
The Child Behavior Checklist (Achenbach et al., 2000) (Clark et al., 2020), rendering decisions about which specific modeling strategy to adopt less consequential.

Family history variables
Family History of Substance Problems was computed from the ABCD's Family History Assessment (Barch et al., 2018). A threshold is established for a family member counting as an affected case based on the number of serious problems that person has had due to alcohol use and substance use. The following coding was used: 0 = neither parent met the threshold; 1 = one or more parents met the threshold.
Family History of Psychopathology Based on previously published protocols (Milne et al., 2008(Milne et al., , 2009), a family history composite score was constructed from responses for ABCD's Family History Assessment (Barch et al., 2018; see Supplement for further description).

Area deprivation index
Area deprivation index (ADI) scores were available for each participant for up to three residences. A weighted average of ADI scores was computed based on months lived at each residence.

The Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) psychopathology
The Kiddie-Structured Assessment for Affective Disorders and Schizophrenia for DSM-5 (KSADS-5) is a structured, diagnostic interview that was administered to parents via computer in reference to their child (Barch et al., 2018;Kobak, Kratochvil, Stanger, & Kaufman, 2013, see online Supplementary Methods for detailed description of coding).
The Prodromal Questionnaire-Brief Version (PQ-B; Loewy, Bearden, Johnson, Raine, and Cannon, 2005) is a youth-report measure designed to index subclinical prodromal psychosis risk phenotypes.
The Brief Problem Monitoring Form (BPM; Piper, Gray, Raber, and Birkett, 2014) is an abbreviated, 18-item version of the CBCL that was sent to teachers for completion. Completion rate was 35% (N = 4495). This measure produces three t-scores regarding youth psychopathology: Internalizing Problems, Externalizing Problems, and Attention Problems.

Trait measures
The behavioral inhibition system/behavioral activation system (BIS/BAS) measure (Pagliaccio et al., 2016) is a 24-item scale designed to assess three facets of behavioral activation, reflecting positive affect: Drive, Fun Seeking, and Reward Responsiveness, and a Behavioral Inhibition scale, indexing sensitivity to punishment.
A 20-item youth short version of the Urgency, Premeditation (lack of), Perseverance (lack of), Sensation Seeking, Positive Urgency, Impulsive Behavior Scale (UPPS-P), developed for the use in the ABCD study (Barch et al., 2018) was administered via self-report at baseline to index trait impulsivity yielding five subscales: Negative Urgency, Positive Urgency, Lack of Perseverance, Lack of Planning, and Sensation Seeking.

School environment
Youth reported on School Risk and Protective Factors (SRPF) to assess their connection to the school environment (Zucker et al., 2018). This measure was taken from the PhenX Toolkit, yielding three subscales: School Environment, School Involvement, and School Disengagement scale.

Social functioning
Parents and youth reported on the prosocial behavior of the youth using the Prosocial Behavior Scale, which is a 3-item scale formed from the 'Strengths and Difficulties Questionnaire' (SDQ; Goodman, 2001).

Family environment
Parent and child both rated the quality of the family environment with the Family Conflict subscale from the PhenX Toolkit Family Environment Scale (Zucker et al., 2018). Youth reported on their perceived level of parental monitoring using the Parental Monitoring Survey, a scale developed to assess parents' efforts to keep track of their child's whereabouts (Zucker et al., 2018).

Psychological Medicine
Neurocognition General Neurocognition (GN) scores were computed by fitting a bifactor model to behavioral tasks from the NIH toolbox, the Rey Auditory Verbal Learning Task, the WISC-V, and the 'Little Man' task . Exploratory factor analyses suggested that three broad factors characterized these tasks, corresponding to memory, speed/ flexibility, and reasoning; in the bifactor model, these three factors served as the specific factors. The bifactor model fit well by conventional standards: χ 2 = 443.16, df = 34, p < 0.001; RMSEA = 0.032; CFI = 0.990; TLI = 0.983; SRMR = 0.017 (West, Taylor, & Wu, 2012). GN factor scores (i.e. scores for the general factor in the bifactor model) were generated using maximum a posteriori scoring (MacCallum, 2009).

Data analytic strategy
All analyses were performed using R. Data were weighted to correspond to the American Community Survey proportions and analyses accounted for clustering within the collection site and family (Heeringa & Berglund, 2020) †1 . All SEM analyses were performed with laavan (open-source code at https://github.com/ SripadaLab/ABCD_nomological_networks). Factor scores were estimated in factor models as described above, and regressions involving these factors were concurrently performed (in the latent space), controlling for the following covariates: participant sex, race, parent education, parent marital status, household income. Associations between CBCL Internalizing Problems and Externalizing Problems t-scores (gender normed) with the same set of independent variables were examined using multilevel models, with the same covariates excluding sex. When examining associations with demographic variables, all demographic variables were entered into one model. Given the large sample size, we chose to use the conservative alpha level a False Discovery Rate Benjamini-Hochburg corrected p < 0.001 to determine significance.

Results
p. Regarding demographics and family history (Table 1, Fig. 1a) p scores were higher for males relative to females, white relative to African-American, Hispanic, and Asian youth, children in lower income households, children with unmarried parents, and children with a higher family loading of mental health and substance use problems. p factor scores were significantly associated with all KSADS diagnoses, prodromal psychotic symptoms, and teacher ratings of Internalizing, Externalizing, and Attention problems (  Fig. 2a). Lastly, p factor scores were associated with the worse school environment, lower school involvement, greater school disengagement, less prosocial behavior, more family conflict, and less parental monitoring (Table 3, Fig. 2b).
INT. When examining associations between demographic and family variables and INT factor scores (Table 1, Fig. 1a), INT scores were higher for females compared to males and lower for African-American youth compared to white youth. INT factor scores were also higher for youth from households with higher levels of parental education and with a higher family loading of mental health problems. Regarding associations with psychopathology, INT factor scores were higher for individuals with depression, suicide/self-harm, anxiety disorders, and teacher ratings of internalizing problems (Table 2, Fig. 1b). INT scores were significantly lower for individuals with an attention deficit hyperactivity disorder (ADHD) diagnosis, consistent with a negative association with teacher ratings of EXT and attention problems. INT factor scores were significantly associated with higher BIS and lower scores on all BAS facet scales, as well as lower Positive Urgency, Lack of Planning, and Sensation Seeking scores, reflecting a more emotionally and behaviorally constrained personality (Table 2, Fig. 2a). INT scores were also associated with higher scores on GN. Lastly, higher INT factor scores were associated with more family conflict per parent report (Table 3, Fig. 2b).
EXT. When examining associations between demographic and family history variables and EXT factor scores (Table 1, Fig. 1a), EXT scores were higher in males compared to females, African-American compared to white youth, and youth with unmarried parents. EXT factor scores were also higher for individuals who had at least one parent with a substance use problem. Regarding DSM-5 psychopathology (Table 2, Fig. 1b), youth diagnosed with conduct disorder (CD), oppositional defiant disorder, or bipolar disorder scored higher on the EXT factor than those without diagnoses. These findings are consistent with positive associations between EXT factor scores and teacher ratings of externalizing problems. Showing discriminant associations, youth that was diagnosed with generalized or social anxiety scored lower on the EXT factor than those that did not meet criteria for diagnosis. EXT factor scores were associated with lower BIS scores, higher BAS-Drive and Fun Seeking scores, and higher Negative Urgency, Positive Urgency, Lack of Planning, and Sensation Seeking scores (Table 2, Fig. 2a). Lastly, EXT factor scores were less prosocial behavior and more family conflict.

CBCL Internalizing Problems and Externalizing Problems
In contrast to findings for INT (which accounts for the variance due to p), the CBCL Internalizing Problems scale showed a relatively non-specific pattern of associations, with positive associations with mood and anxiety disorders as expected, but also with CD, ODD, and ADHD (Table 2). In addition, the Internalizing Problems scores were associated with higher levels of trait impulsivity, as evidenced by positive associations with UPPS subscales, counterintuitive to conceptualizations of internalizing proneness as reflecting high levels of constraint. Furthermore, in contrast to EXT, the CBCL Externalizing Problems scale again showed a non-specific pattern of associations with nearly all DSM-5 psychopathology, including, somewhat counterintuitively, internalizing disorders such as depression and anxiety disorders. Scores on the Externalizing Problems scale were also associated with higher scores on the prodromal psychosis scale, teacher-reported internalizing, externalizing, and attention problems, impulsivity, and lower scores on GN.

Discussion
We delineated the nomological networks of three major dimensions that emerge in hierarchical models of psychopathology, the general factor of psychopathology ('p factor') and specific † The notes appear after the main text.
INT and EXT factors, in a large, diverse sample of emerging adolescents. We found p, INT, and EXT were each associated with nomological networks readily distinguishable along four axes: neurocognition ( p negatively associated, INT positively associated, EXT unrelated), fear/distress emotions ( p and INT positively associated), impulsivity ( p and EXT positively associated),     and social adversity ( p and EXT positively associated). Notably, these distinct nomological networks were obscured when looking at the CBCL Internalizing and Externalizing Problems scores without accounting for their substantial overlap (i.e. without accounting for a shared variance from a superordinate p factor). Overall, our results show that three major dimensions that emerge from hierarchical models of psychopathology form clearly distinguishable and readily interpretable nomological networks, adding to the evidence that they help to 'carve nature at its joints'. Our results highlight distinct cognitive and temperamental profiles associated with p, INT, and EXT, perhaps offering insights into the psychological and neural mechanisms that drive these broad liabilities. As a summary, Table 4 highlights associations between p/INT/EXT and four dimensions of importance: GN, two emotion/impulse dimensions: a fear/distress dimension (indexed by BIS and KSADS symptom profile), an impulsivity dimension (indexed by UPPS and BAS subscales and KSADS symptom profile), and a social adversity dimension (indexed by School Risk and Protective Factors and Family Conflict scales). p's position in this space reflects more pervasive alterations involving low neurocognition as well as elevations in both fear/distress and impulsivity profiles in the context of high social and environmental adversity. This might reflect a state in which there are both globally elevated emotionsnegative and positiveand impulses as well as reduced executive control capacities to modulate these impulses (Carver et al., 2017;Deutz et al., 2020). EXT and INT, in contrast, reflect domain-specific alterations: elevated impulsivity profile in EXT and elevated inhibition profile in INT. INT is in addition associated with elevated neurocognition, which in turn might help explain INT's complex pattern of associations. For example, it is possible that higher neurocognition contributes to higher fear/distress symptoms due to greater prospection and associated ruminative worry (Penney, Miedema, & Mazmanian, 2015), but at the same time, it is protective for attention problems (i.e. lower BPM Attention and ADHD) due to enhanced cognitive control. Overall, while our pattern of findings is consistent with previous studies (Bloemen et al., 2018;Brandes et al., 2019;Carver et al., 2017;Castellanos-Ryan et al., 2016;Deutz et al., 2020;Hankin et al., 2017;Martel et al., 2017;Michelini et al., 2019;Moore et al., 2020;Tackett et al., 2013), this study is among the first to position these liabilities for psychopathology along these cognitive, temperament, and environmental continua simultaneously, providing novel insights into their interrelationships.
The p factor is conceptualized as a broad liability to all forms of prevalent psychiatric symptomatology. Interestingly, the p factor's nomological network was similarly extensive across domains and included: a higher load of family history of psychopathology (consistent with a high heritability of p; Allegrini et al., 2020;Selzam et al., 2018); reduced family income; worse family, school, and neighborhood environment; reduced cognitive ability; and elevated emotional responses (including fear/distress emotions and impulsivity). In addition, p was associated with a broad array of KSADS diagnoses. Importantly, p's nomological network was established through instruments anchored in youth-, parent-, and teacher-report as well as task-based behavioral performance,  Note. Standardized betas from regression analyses with p, INT, and EXT and include the following covariates: participant sex, race, parent education level, marital status, and household income. For CBCL Scale analyses standardized betas are from separate multilevel models (MLMs) and sex was excluded as a covariate as scales were t-scored by sex. Analyses were weighted to correspond to American Community Survey proportions and clustered by testing site and family ID. ADHD = Attention Deficit/Hyperactivity Disorder; ODD = Oppositional Defiant Disorder. * False Discovery Rate corrected p < 0.001; bolded values |β|⩾ 0.20, denote at least a medium effect size. suggesting rater-style alone cannot explain the breadth of p-associated risk variables. In contrast to p, nomological associations of the specific INT and EXT were generally more domainspecific. INT and EXT diverged from each other in demographic associations, with males and African-American youth having higher EXT, and females having higher INT. These distinct patterns of nomological associations for p, INT, and EXT raise additional questions about their ultimate drivers (e.g. environment, genes, environment × gene interactions) and perhaps identify targets for prevention efforts that can impact the emergence of broad psychopathology. It is notable that the nomological networks of p, INT, and EXT were obscured when examining a priori Internalizing and Externalizing Problems scales, which demonstrated non-specific associations when not controlling for common variance due to p. That is, the dimensions derived from the bifactor structural model of psychopathology yielded improved convergent and divergent validity compared to the standard CBCL-based measures of Internalizing and Externalizing. For example, internalizing is broadly conceptualized in terms of elevated inhibition and threat sensitivity, while externalizing is conceptualized in terms of disinhibition and reduced sensitivity to threats (Krueger, McGue, & Iacono, 2001;Nigg, 2017). Yet, CBCL Internalizing scores are positively associated with CD, while CBCL Externalizing scores were counterintuitively positively associated with multiple anxiety disorders. In contrast, the INT and EXT factors that control for the common variance of p exhibit the relationships predicted by prior theory (INT inversely associated with CD; EXT inversely associated anxiety disorders). More broadly, INT and EXT exhibit sharply differentiated, interpretable relationships with a host of variables (teacher-rated externalizing and attention problems, BAS-Drive, GN) that are, in contrast, not visible with the CBCL Internalizing and Externalizing Problems scales. Previous research has raised a concern about the 'perils of partialing' (Lynam, Hoyle, & Newman, 2006), wherein well-defined and interpretable scales can sometimes lose their original interpretation when portions of their variance are partialed away. Results from this study suggest that partialing via a bifactor model is not a 'peril' in this case. Instead, here partialing yields more interpretable INT, EXT, and p constructs, which exhibit more differentiated nomological networks that are better aligned with prior theory.
A striking finding in our results is that nomological associations of INT often diverged from those of the p factor. For example, p was related to higher attention problems, higher behavioral activation and impulsivity, and lower GN, while INT was associated with fewer attention problems, lower behavioral activation, and higher GN. This observation is noteworthy because the p factor represents a liability to all prevalent  Note. Standardized betas from regression analyses with p, INT, and EXT and include the following covariates: participant sex, race, parent education level, marital status, and household income. For CBCL Scale analyses standardized betas are from separate multilevel models (MLMs) and sex was excluded as a covariate as scales were t-scored by sex. Analyses were weighted to correspond to American Community Survey proportions and clustered by testing site and family ID. * False Discovery Rate corrected p < 0.001; bolded values |β| ⩾ 0.20, denote at least a medium effect size. psychiatric symptoms, which includes internalizing spectrum symptoms. Put another way, our results show that two underlying factors contribute to observed fear and distress symptoms measured by the CBCL: a broad liability p factor and a narrow liability INT factor. But importantly, these two liabilities are embedded in highly differentiated, and in several cases, highly divergent, nomological networks. It is notable that the literature on anxiety and depression are similarly mixed, with some studies linking these internalizing disorders to variables associated with p (e.g. lower cognitive abilities; Levin et al., 2007;Rapport et al., 2001;Snyder, 2013), while other studies link them to variables associated with INT (e.g. higher cognitive abilities; Karpinski, Kolb, Tetreault, and Borowski, 2018;Penney et al., 2015). Results of the current study serve to resolve some of these tensions by showing that observed internalizing symptoms likely reflect equifinality wherein distinct underlying liabilities (i.e. p-based liability v.
INT-based liability) lead to similar clinical presentations (i.e. internalizing symptoms). However, future work replicating these findings to confirm their robustness and using person-centered analyses are needed to fully resolve these findings. Some limitations of the current work warrant mention. First, our hierarchical model of psychopathology was derived from the parent-report CBCL. While studies show parent-report on youth psychopathology is valid at this age; obtaining reports from multiple informants on psychopathology has been found to improve associations with relevant outcomes (Clark, Durbin, Hicks, Iacono, & McGue, 2017). In future waves of ABCD data collection, the youth themselves will complete the CBCL which will strengthen the model estimates of p, INT, and EXT. Second, our study captures nomological networks among 9and 10-year-olds. There is some evidence for the rank-order stability of the p factor and specific factors across adolescence (Castellanos-Ryan et al., 2016;McElroy, Belsky, Carragher, Fearon, & Patalay, 2018;, but follow-up investigations in future waves of ABCD data are needed to confirm the stability of the observed networks across time.
In sum, the current study demonstrates distinct, divergent, and interpretable patterns of nomological associations for p, INT, and EXT in pre-adolescents in the racially and economically diverse ABCD sample. These findings set the stage for future studies in the ABCD sample, leveraging longitudinal waves of data to trace the progression of psychopathology through adolescence, and leveraging multi-modal data to extend the nomological networks described here to encompass biological variables including genes and neurocircuitry.
Acknowledgements. Data used in the preparation of this article were obtained from the Adolescent Brain Cognitive Development (ABCD) Study (https://abcdstudy.org), held in the NIMH Data Archive (NDA). This is a multisite, longitudinal study designed to recruit more than 10 000 children age 9-10 and follow them over 10 years into early adulthood. The ABCD Study is supported by the National Institutes of Health and additional federal partners under award numbers U01DA041022, U01DA041028, U01DA041048, U01DA041089, U01DA041106, U01DA041117, U01DA041120, U01DA041134, U01DA041148, U01DA041156, U01DA041174, U24DA041123, and U24DA041147. A full list of supporters is available at https://abcdstudy.org/nih-collaborators. A listing of participating sites and a complete listing of the study investigators can be found at https://abcdstudy.org/principal-investigators.html. ABCD consortium investigators designed and implemented the study and/or provided data but did not necessarily participate in analysis or writing of this report. This manuscript reflects the views of the authors and may not reflect the opinions or views of the NIH or ABCD consortium investigators. The ABCD data repository grows and changes over time. The ABCD release used in this report came from NDA Study 721, 10.15154/1504041, which can be found at https://nda.nih.gov/study. html?id=721. The specific NDA study associated with the data reported in this manuscript is: NDA Study 981, 10.15154/1519364.This work was supported by the following grants from the US National Institutes of Health, the National Institute on Drug Abuse, and the National Institute on Alcohol Abuse and Alcoholism: R01MH107741 (CS), U01DA041106 (CS, LH, MH, BH), T32 AA007477 (SB, DC). In addition, CS was supported by a grant from the Dana Foundation David Mahoney Neuroimaging Program.

Conflicts of interest. None.
Ethical standards. The authors assert that all procedures contributing to this work comply with ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Supplementary material. The supplementary material for this article can be found at https://doi.org/10.1017/S0033291720005103 Notes 1 Parallel analyses were run on unweighted data and results were not appreciably different.