Introduction
Internalizing and externalizing problems are among the most common, costly, and burdensome issues facing children and adults across the lifespan (Forbes et al., Reference Forbes, Tackett, Markon and Krueger2016). Externalizing disorders, typically encompassing symptoms of aggression, conduct problems, and oppositionality, have a global prevalence of over 5% (Polanczyk et al., Reference Polanczyk, Salum, Sugaya, Caye and Rohde2015). Moreover, untreated externalizing problems lead to future problems such as academic difficulties (Shi & Ettekal, Reference Shi and Ettekal2021), social delinquency, incarceration, and substance abuse (Krueger et al., Reference Krueger, Hobbs, Conway, Dick, Dretsch, Eaton, Forbes, Forbush, Keyes, Latzman, Michelini, Patrick, Sellbom, Slade, South, Sunderland, Tackett, Waldman, Waszczuk, Wright, Zald, Watson and Kotov2021; Loeber et al., Reference Loeber, Farrington, Stouthamer-Loeber, Van Kammen and Jessor1998). Internalizing psychopathology, which encompasses anxiety and depression, has an even higher prevalence ranging from 20 to 25% in children and adolescents, and has greatly increased since the COVID-19 pandemic (Racine et al., Reference Racine, McArthur, Cooke, Eirich, Zhu and Madigan2021). Thus, it is crucial to identify processes that prevent the development of internalizing and externalizing psychopathology.
General psychopathology
Diagnoses from the internalizing and the externalizing spectra show a high rate of co-occurrence (Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2014; Clark et al., Reference Clark, Hicks, Angstadt, Rutherford, Taxali, Hyde, Weigard, Heitzeg and Sripada2021; Murray et al., Reference Murray, Eisner and Ribeaud2016). Internalizing and externalizing problems are most accurately modeled dimensionally rather than as categorical phenomena (Markon et al., Reference Markon, Chmielewski and Miller2011), and numerous factor analytic modeling studies have identified strong covariation among internalizing, externalizing, and thought-disordered dimensions of psychopathology (Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2014; Cervin et al., Reference Cervin, Norris, Ginsburg, Gosch, Compton, Piacentini, Albano, Sakolsky, Birmaher, Keeton, Storch and Kendall2021; Choate et al., Reference Choate, Bornovalova, Hipwell, Chung and Stepp2022; Gluschkoff et al., Reference Gluschkoff, Jokela and Rosenström2019; Lahey et al., Reference Lahey, Applegate, Hakes, Zald, Hariri and Rathouz2012). The strong covariation between these problems has led to the hypothesis that internalizing, externalizing, and thought-disordered problems share a common cause – a higher-order factor called “p factor” – that accounts for their covariation (Caspi et al., Reference Caspi, Houts, Belsky, Goldman-Mellor, Harrington, Israel, Meier, Ramrakha, Shalev, Poulton and Moffitt2014; Forbes et al., Reference Forbes, Tackett, Markon and Krueger2016; Krueger & Eaton, Reference Krueger and Eaton2015). The p factor can be modeled in various ways, including a higher-order factor model or a bifactor model. The general factor of psychopathology reflects what is common among dimensions of psychopathology. The general factor is operationalized as the common variance among internalizing and externalizing psychopathology indicators (Ree et al., Reference Ree, Carretta and Teachout2015; Watts, Meyer, et al., Reference Watts, Meyer, Greene, Wood, Trull, Steinley and Sher2021; Zinbarg et al., Reference Zinbarg, Revelle, Yovel and Li2005). Specific psychopathology (e.g., externalizing) in a bifactor model is represented by the residual correlations among the specific psychopathology facets that are not accounted for by the general factor.
Given the high cost and burden of internalizing and externalizing psychopathology, it is important to identify early processes that lead to development of specific internalizing or externalizing problems versus co-occurring internalizing and externalizing problems. The present study considers reactive and control processes of temperament that may help explain differing developmental pathways to psychopathology.
Temperamental reactive and control processes in the development of general and specific psychopathology
Temperament is defined as constitutionally based ways in which individuals regulate and react to their environment (Rothbart & Bates, Reference Rothbart and Bates2006). Individual differences in temperament are early appearing, biologically based, and relatively stable across development (Rothbart & Bates, Reference Rothbart and Bates2006). The structure of temperament is broadly categorized by three relatively orthogonal dimensions – two reactivity dimensions: positive and negative emotionality; and one control dimension: self-regulation (Rothbart & Bates, Reference Rothbart and Bates2006; Rothbart, Reference Rothbart2011). Positive emotionality reflects the tendency to experience positive emotions, e.g., enthusiasm and joy, often reflecting extraversion (Watson et al., Reference Watson, Clark and Carey1988). Negative emotionality reflects a propensity toward anger/frustration, fear, and sadness (Rothbart & Bates, Reference Rothbart and Bates2006). Self-regulation reflects a child’s ability to regulate behavior, cognition and emotions, and often includes subordinate constructs of executive functioning, attentional control, and effortful control. In the present study, we focus on facets of the negative emotionality and self-regulation dimensions of temperament given their robust associations with psychopathology (e.g., Eisenberg et al., Reference Eisenberg, Sadovsky, Spinrad, Fabes, Losoya, Valiente, Reiser, Cumberland and Shepard2005; Muris & Ollendick, Reference Muris and Ollendick2005). One of the most widely studied predictors of children’s adjustment is immediacy preference, which is a facet of self-regulation and is a control process (Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996).
Immediacy preference
Immediacy preference is the selection of a smaller, immediate reward over a larger, distal one (Mischel & Ebbesen, Reference Mischel and Ebbesen1970; Stephens & Anderson, Reference Stephens and Anderson2001). The inverse of immediacy preference is delay preference (Rachlin & Jones, Reference Rachlin and Jones2008), which is frequently called delay of gratification. Immediacy preference is often assessed with a self-imposed waiting task (Metcalfe & Mischel, Reference Metcalfe and Mischel1999) that is designed to assess a person’s ability or preference to resist the temptation of the immediate reward (i.e., gratification) in favor of a more motivationally salient, distal reward. In a temperament framework, delay of gratification is considered a control process because it describes a self-regulatory process by which an individual might suppress a dominant response in favor of a subordinate one (Moran et al., Reference Moran, Lengua and Zalewski2013). This conceptualization of delay of gratification as a control process of temperament is supported by prior research (Bjorklund & Kipp, Reference Bjorklund and Kipp1996; Moran et al., Reference Moran, Lengua and Zalewski2013; Murray & Kochanska, Reference Murray and Kochanska2002). We use the term “delay of gratification” when referring to the duration of waiting in a self-imposed waiting task, whereas we use the term “immediacy preference” when referring to the process as a risk factor. Immediacy preference has been shown to be associated with deficits in self-control, a common impairment in individuals with psychopathology (Kidd et al., Reference Kidd, Palmeri and Aslin2013; Michaelson & Munakata, Reference Michaelson and Munakata2020).
Immediacy preference (i.e., a stronger relative preference for an immediate reward) has been associated with many facets of psychopathology, including impulsive decision making and externalizing psychopathology. Several studies have found that immediacy preference was associated with both parent- and teacher-reported aggressive and delinquent behaviors in children ranging from ages 3 to 13 years (Campbell & von Stauffenberg, Reference Campbell and von Stauffenberg2009; Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996). A 40-year follow-up study found that adults with greater immediacy preference in childhood continued to show more externalizing problems compared to their peers who had a delay preference in childhood (Casey et al., Reference Casey, Somerville, Gotlib, Ayduk, Franklin, Askren, Jonides, Berman, Wilson, Teslovich, Glover, Zayas, Mischel and Shoda2011). Numerous studies have identified a strong association of immediacy preference with externalizing problems and delinquent outcomes (Campbell & von Stauffenberg, Reference Campbell and von Stauffenberg2009; Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996).
Previous studies have implicated immediacy preference as a risk factor that is specific to externalizing problems, i.e., not internalizing problems (Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996). However, little work has examined whether immediacy preference is associated with general psychopathology. Watts et al. (Reference Watts, Duncan and Quan2018) used the Eunice Kennedy Shriver National Institute of Child Health and Development (NICHD) Study of Early Child Care and Youth Development (SECCYD) sample, the same sample as the present study. Importantly, Watts and colleagues found that when accounting for numerous covariates, including family background, home environment, and early cognitive ability, immediacy preference was no longer significantly associated with behavioral outcomes (Watts et al., Reference Watts, Duncan and Quan2018). However, the behavioral outcomes variable was a total behavior problems score computed as an average of internalizing and externalizing problems. Thus, the study did not allow for determining whether immediacy preference was associated with general versus specific psychopathology. Partitioning internalizing and externalizing problems is especially important because there might be reason to expect that immediacy preference is more strongly associated with externalizing problems compared to internalizing problems (Campbell & von Stauffenberg, Reference Campbell and von Stauffenberg2009; Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996).
Another study using the SECCYD sample (Deutz et al., Reference Deutz, Geeraerts, Belsky, Deković, van Baar, Prinzie and Patalay2020) examined the association between antecedent factors, including immediacy preference, and specific and general psychopathology at later ages (8 and 14 years old). Results indicated that immediacy preference was not associated with later general or specific psychopathology. However, the study did not examine immediacy preference in relation to the development (i.e., change over time) of behavior problems. Moreover, the study examined only mother- and self report, and did not include perspectives from other informants such as fathers.
In sum, much remains unknown regarding the association between immediacy preference and psychopathology. For instance, it is not known the degree to which immediacy preference in early childhood predicts general versus specific psychopathology in adolescence (Deutz et al., Reference Deutz, Geeraerts, Belsky, Deković, van Baar, Prinzie and Patalay2020). In addition to immediacy preference, the present study considers temperamental negative emotionality, a reactive process, which has been proposed as a transdiagnostic risk factor for psychopathology (Mikolajewski et al., Reference Mikolajewski, Allan, Hart, Lonigan and Taylor2013; Weissman et al., Reference Weissman, Bitran, Miller, Schaefer, Sheridan and McLaughlin2019). Transdiagnostic risk factors contribute to the etiology and maintenance of a broad range of emotional and behavioral difficulties (Egan et al., Reference Egan, Wade and Shafran2011). Transdiagnostic risk factors can occur within a given specific factor – e.g., perfectionism is a risk factor for eating disorders, anxiety, and depression (Egan et al., Reference Egan, Wade and Shafran2011) – or can reflect risk for general psychopathology, meaning they contribute to a broader range of behaviors (Lynch et al., Reference Lynch, Sunderland, Newton and Chapman2021).
Temperamental negative emotionality
Temperamental negative emotionality is an individual’s characteristic reaction to stimuli, and includes the tendency to display overt sadness, irritability, fear/withdrawal, distress, and/or anger, and may be characterized by somatic or autonomic physiological reactivity in response to stimuli (Fox, Reference Fox1989; Rothbart & Derryberry, Reference Rothbart and Derryberry1981). As such, negative emotionality is considered a reactive process within a temperament framework (Eisenberg et al., Reference Eisenberg, Fabes, Guthrie, Murphy, Maszk, Holmgren and Suh1996; Moran et al., Reference Moran, Lengua and Zalewski2013). Numerous studies have found that negative emotionality is highly correlated with internalizing and externalizing psychopathology (Leaberry et al., Reference Leaberry, Rosen, Slaughter, Reese and Fogleman2019; McLaughlin & Nolen-Hoeksema, Reference McLaughlin and Nolen-Hoeksema2011; Steinberg & Drabick, Reference Steinberg and Drabick2015), and evidence suggests that negative emotionality plays a causal role in the development of internalizing and externalizing psychopathology (Lilienfeld, Reference Lilienfeld2003). There is some evidence that negative emotionality is associated with general psychopathology (i.e., p factor; Hankin et al., Reference Hankin, Davis, Snyder, Young, Glynn and Sandman2017). Negative emotionality is thought to evolve and change in its manifestation, much like symptoms of psychopathology more broadly (e.g., Aldao et al., Reference Aldao, Gee, De Los Reyes and Seager2016; De Los Reyes et al., Reference De Los Reyes, Henry, Tolan and Wakschlag2009, Reference De Los Reyes, Bunnell and Beidel2013; Mischel & Shoda, Reference Mischel and Shoda1995; Pettersson et al., Reference Pettersson, Lahey, Larsson and Lichtenstein2018; Rutter & Sroufe, Reference Rutter and Sroufe2000).
A previous study found that negative emotionality, as observed in a frustration task, was associated with both internalizing and externalizing problems, and partially accounted for their covariation (Mikolajewski et al., Reference Mikolajewski, Allan, Hart, Lonigan and Taylor2013). Negative emotionality predicts psychopathology (Brandes et al., Reference Brandes, Herzhoff, Smack and Tackett2019; Briggs-Gowan et al., Reference Briggs-Gowan, Carter, Bosson-Heenan, Guyer and Horwitz2006; Hawes et al., Reference Hawes, Carlson, Finsaas, Olino, Seely and Klein2020) and they overlap in behavioral indicators of distress (e.g., dysthymia and depression; Greene & Eaton, Reference Greene and Eaton2017), suggesting that they conceptually overlap to a degree. However, prior literature has indicated that temperament/personality constructs differ from psychopathology. Temperament is primarily concerned with the how of a behavior (e.g., how intensely a child cries), whereas psychopathology focuses on the content of the behavior (e.g., what does the child cry about; Bates et al., Reference Bates, Schermerhorn, Petersen, Lewis and Rudolph2014; De Pauw & Mervielde, Reference De Pauw and Mervielde2010; Lemery et al., Reference Lemery, Essex and Smider2002; Thomas & Chess, Reference Thomas and Chess1977). However, to our knowledge, no previous study has examined control and reactive temperamental processes simultaneously in predicting general and specific psychopathology in the same model.
Effortful control, an aspect of self-regulation which comprises immediacy preference, and negative emotionality are related but separate aspects of temperament (Eisenberg et al., Reference Eisenberg, Sadovsky, Spinrad, Fabes, Losoya, Valiente, Reiser, Cumberland and Shepard2005; Rothbart et al., Reference Rothbart, Ahadi, Hershey and Fisher2001). Children with less effortful control tend to experience greater adjustment difficulties in the face of stress, thus leading them to show more negative emotionality (Moran et al., Reference Moran, Lengua and Zalewski2013; Muris & Ollendick, Reference Muris and Ollendick2005). A prior study using the same sample as the present study found that anger was associated with higher mother-reported externalizing problems. This same study found that higher levels of effortful control at 54 months indirectly predicted lower levels of externalizing problems at age 15 years (Crockett et al., Reference Crockett, Wasserman, Rudasill, Hoffman and Kalutskaya2018). Furthermore, when examined together, negative emotionality and effortful control have shown additive effects on psychopathology (Eisenberg et al., Reference Eisenberg, Fabes, Guthrie, Murphy, Maszk, Holmgren and Suh1996, Reference Eisenberg, Guthrie, Fabes, Shepard, Losoya, Murphy, Jones, Poulin and Reiser2000, Reference Eisenberg, Sadovsky, Spinrad, Fabes, Losoya, Valiente, Reiser, Cumberland and Shepard2005). Relatedly, one study indicated that an imbalance in approach behavior – a reactive process – and control processes was associated with externalizing problems, indicating an interaction of control and reactive processes in externalizing problems (Jonas & Kochanska, Reference Jonas and Kochanska2018). Surprisingly, few studies have simultaneously examined reactive and control systems, such as immediacy preference and negative emotionality, despite evidence that they likely influence each other (Moran et al., Reference Moran, Lengua and Zalewski2013; Rothbart & Bates, Reference Rothbart and Bates2006). To our knowledge, no previous studies have examined these two processes simultaneously as transdiagnostic risk factors for dimensions of psychopathology across childhood to adolescence. However, one barrier to identifying early mechanisms in the development of later psychopathology is that the behavioral manifestations of psychopathology change across development.
Heterotypic continuity
A salient issue in developmental psychopathology is that the behavioral manifestations of psychopathology change across development, a phenomenon called heterotypic continuity (Cicchetti & Rogosch, Reference Cicchetti and Rogosch2002). Heterotypic continuity occurs when the same psychological phenomenon manifests as different behaviors across development (Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018, Reference Petersen, Choe and LeBeau2020). For example, externalizing problems in children often appear as tantruming and overt oppositionality, whereas in adolescents and adults, externalizing behaviors become more covert and tend to take the form of indirect behaviors (e.g., substance use; Mikolajewski et al., Reference Mikolajewski, Allan, Hart, Lonigan and Taylor2013; Miller et al., Reference Miller, Vaillancourt and Boyle2009; Petersen & LeBeau, Reference Petersen and LeBeau2022). Patterson (Reference Patterson1993) described externalizing problems using the analogy of a “chimera,” a mythological creature with a goat’s body that, with development, grows the head of a lion and the tail of a serpent. This was meant to highlight that although individual differences in externalizing behavior are relatively stable across time, externalizing behavior manifests in different ways across the lifespan (Patterson, Reference Patterson1993). This notion of changing manifestations of behavior across development has also been identified in internalizing problems (e.g., Avenevoli & Steinberg, Reference Avenevoli and Steinberg2001; Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018; Weems, Reference Weems2008). For example, separation anxiety and fear of animals is common in younger children, whereas social anxiety is more common in adolescence (Weems, Reference Weems2008).
Consistent with the developmental issues framework (Sroufe, Reference Sroufe, Cassidy and Shaver2016), the changing behavioral manifestation of psychopathology reflects a combination of time-varying genetic and environmental factors, such as school entry transition, in combination with varying developmental tasks and greater experience-dependent capacity. For example, developmental tasks in preschool (e.g., self-regulation) differ from those in adolescence (e.g., peer acceptance), which changes how behavior problems tend to manifest. Thus, for externalizing and internalizing problems, the underlying construct persists across development, but their behavioral manifestations change. However, there are key challenges in identifying early mechanisms in the development of psychopathology.
A key challenge of identifying early mechanisms in the development of psychopathology deals with longitudinal assessment. It is difficult to examine internalizing and externalizing psychopathology across a lengthy span of development in meaningful ways because behavioral manifestations of psychopathology change across development (McElroy et al., Reference McElroy, Belsky, Carragher, Fearon and Patalay2018). Assessment and analysis become even more challenging when considering that internalizing and externalizing psychopathology often co-occur (Pettersson et al., Reference Pettersson, Lahey, Larsson and Lichtenstein2018). Heterotypic continuity poses challenges for measurement because different measures from different informants across ages are needed to capture developmental changes (Petersen & LeBeau, Reference Petersen and LeBeau2022). If the measures do not align with the changes in the construct’s manifestation, studies will yield faulty conclusions (Chen & Jaffee, Reference Chen and Jaffee2015; Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018, Reference Petersen, LeBeau and Choe2021). In addition to using different measures across ages, it is also important to consider using different informants.
Different informants across development
Before children enter schooling, the most accurate informants on the children’s behavior tend to be their parents and caregivers, as reflected in the proliferation of parent- and caregiver-report measures for early childhood (Achenbach & Rescorla, Reference Achenbach and Rescorla2000). Many have argued that using multiple informants is the best approach for assessing child psychopathology (De Los Reyes & Makol, Reference De Los Reyes, Makol and Mihura2021, Reference De Los Reyes, Makol and Asmundson2022; Makol et al., Reference Makol, Youngstrom, Racz, Qasmieh, Glenn and De Los Reyes2020; Watts, Makol, et al., Reference Watts, Meyer, Greene, Wood, Trull, Steinley and Sher2021). In early-to-mid-childhood, when children attend school and preschool, teachers are important informants on children’s behavior, because teachers help account for children’s behavior across multiple contexts (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). Context of measurement is important. Prior research has shown that parent and teacher reports of disruptive behaviors that occur in both home and school tend to result in stronger convergence, whereas context-specific behaviors (i.e., disruptive only at home, not school) result in weaker convergence (De Los Reyes & Makol, Reference De Los Reyes, Makol and Mihura2021; Hartley et al., Reference Hartley, Zakriski and Wright2011; Kwon et al., Reference Kwon, Kim and Sheridan2012). A review on correlations between mother, father, and teacher ratings of ADHD symptoms in children and adolescents indicated that correlations between mother and father ratings of inattention and hyperactivity–impulsivity were high (r = .67–.70). Correlations between parent and teacher ratings of inattention and hyperactivity–impulsivity were somewhat lower (r = .28–.47). Taken together, evidence indicates that context matters in ratings of observed psychopathology symptoms (Martel et al., Reference Martel, Markon and Smith2017).
Furthermore, when children enter their adolescent years, adolescents become more reliable reporters on their internal experience, which is particularly meaningful for internalizing symptoms such as anxiety and depression that may be less overtly visible to outside observers (Damme et al., Reference Damme, Norton, Briggs-Gowan, Wakschlag and Mittal2022). Taken together, it is important to consider multiple informants on children’s externalizing and internalizing symptoms across development to help account for (a) differing manifestations of behavior in multiple contexts (e.g., school and home) and (b) rater-specific bias.
Developmental scaling
The challenge is in how to meaningfully combine the scores from the different measures as rated by the various informants so that the scores are on a comparable metric for assessing children’s change over time. The combination of heterotypic continuity and having different informants across ages poses important challenges. Heterotypic continuity requires age-differing measures to account for the changing manifestation of the construct. Similarly, informants are differentially capable of rating various aspects of the child’s behavior (e.g., relational aggression versus social anxiety) in different contexts (e.g., home versus school), thus requiring different measures for different types of informants (e.g., parents, teachers, self report) and at different ages. For example, the Child Behavior Checklist 1.5–5 (Achenbach & Rescorla, Reference Achenbach and Rescorla2000) and the Caregiver–Teacher Report Form (Achenbach & Rescorla, Reference Achenbach and Rescorla2001) both assess children at the same age range, but the Caregiver–Teacher Report Form includes slightly different question content aimed at examining school rather than home context for behaviors. Furthermore, the Child Behavior Checklist 6–18 (Achenbach & Rescorla, Reference Achenbach and Rescorla2000) includes age-differing items (compared to the ages 1.5–5 form) to maintain developmentally appropriate content, e.g., substance use, that assess problems specific to mid-childhood to adolescence. The Teacher’s Report Form (Achenbach & Rescorla, Reference Achenbach and Rescorla2001) also adjusts its question content to account for development. Therefore, a single informant from one context (e.g., mother in the home context) might only capture one potentially biased view of a given child’s behavior (De Los Reyes & Kazdin, Reference De Los Reyes and Kazdin2005). By contrast, including multiple informants across multiple contexts reduces the impact of informant and context-specific biases.
Traditionally, studies have largely ignored heterotypic continuity when examining children’s development of psychopathology (Chen & Jaffee, Reference Chen and Jaffee2015; Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018, Reference Petersen, LeBeau and Choe2021). Many studies have addressed the challenge of differing measures and informants by using (a) only those ages where the same measure or items are assessed and (b) only those informants who provide ratings across the full age span. However, it is problematic to exclude ages because of a developmental change in the manifestation of the construct. Excluding ages due to a change in measurement would exclude important developmental periods and transitions associated with meaningful developmental change, such as the transition from preschool to school entry or the transition to adolescence. Moreover, it is problematic to exclude informants merely because an informant did not provide ratings across all ages of the study. For instance, in a study from early childhood to adolescence, this would result in the exclusion of self report, which is important for assessing internalizing problems in adolescence. It is thus crucial to leverage approaches that use all available information from all possible informants at all possible ages to get the best estimate of people’s development on a comparable scale.
Developmental scaling is a recommended approach to ensure statistical equivalence of scores across different measures (Kolen & Brennan, Reference Kolen and Brennan2014). Developmental scaling approaches have been used successfully to place scores from different measures and informants onto the same scale (Petersen & LeBeau, Reference Petersen and LeBeau2022). Developmental scaling has been widely used in educational psychology to link children’s academic achievement scores across ages (e.g., Kenyon et al., Reference Kenyon, MacGregor, Li and Cook2011; McArdle, Reference McArdle2009; Murayama et al., Reference Murayama, Pekrun, Lichtenfeld and Vom Hofe2013). However, relatively few studies have used developmental scaling approaches to study social development. For instance, few studies have used developmental scaling to study development of internalizing psychopathology (Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018) and externalizing psychopathology (Petersen & LeBeau, Reference Petersen and LeBeau2022).
One approach to developmental scaling uses item response theory (IRT). A two-parameter IRT model estimates item difficulty and discrimination. Item difficulty, also called severity, is the point of median probability at which 50% of respondents endorse a given response. Item discrimination is how well the item distinguishes between the high and low levels of a given construct. Based on the items’ difficulty and discrimination parameters, one can generate an item characteristic curve, which represents the expected score on the item as a function of the person’s level on the latent psychopathology factor. Combining the individual items, one can generate a test characteristic curve, which represents the expected score on the measure as a function of the person’s level on the latent psychopathology factor. To link any given pair of measures and raters, IRT uses scaling parameters to minimize the differences between the two test characteristic curves of the common items across the two measures. The scaling parameters are determined as the linear transformation (i.e., intercept and slope parameter) of the test characteristic curves of the common items between the two measures, that, when applied to the second measure, minimize differences between the test characteristic curves of the common items. Essentially, the scaling parameters minimize the differences in the probability of a rater endorsing the age- and rater-common items across the two measures. That is, IRT links measures’ scales based on the severity and discrimination of the age- and rater- common items. IRT uses the age- and rater-common items to set the common scale. However, all items for a given rater at a given age are used to estimate a person’s score on the common scale.
We are aware of only one prior study that has linked scores from different measures and raters across ages (Petersen & LeBeau, Reference Petersen and LeBeau2022). And, to our knowledge, no prior studies have used developmental scaling to link scores from different measures and raters for multiple dimensions of psychopathology. Performing developmental scaling of multiple dimensions of psychopathology leverages the strong covariation between internalizing and externalizing problems to obtain more accurate estimates of each. Researchers have called for studies that implement developmental scaling for multiple dimensions of psychopathology simultaneously (Tackett & Hallquist, Reference Tackett and Hallquist2022).
This approach of having multiple informants and measures and placing their scores on the same scale is essential to accurately estimate changes in externalizing and internalizing symptoms, given considerable differences in informant reports and measure scales (Petersen & LeBeau, Reference Petersen and LeBeau2022). Developmental scaling allows for individuals to have their informant and self-reported psychopathology mapped onto growth curves, to show people’s change in psychopathology across a long development span. To our knowledge, only one study has used developmental scaling from multiple informants to account for heterotypic continuity in order to study individuals’ development of psychopathology (Petersen & LeBeau, Reference Petersen and LeBeau2022).
The present study
The present study examines whether immediacy preference and negative emotionality, control and reactive processes, respectively, predict the development of externalizing and internalizing problems across early childhood to adolescence, using a large sample. We leverage multi-informant ratings from mothers, fathers, teachers, afterschool caregivers, other caregivers, and self report. This is the first study to link scores from multiple measures, raters, and psychopathology dimensions onto the same scale. Prior literature has emphasized the importance of accounting for internalizing and externalizing symptoms concurrently to be a more ecologically valid representation of psychopathology (Ruggero et al., Reference Ruggero, Kotov, Hopwood, First, Clark, Skodol, Mullins-Sweatt, Patrick, Bach, Cicero, Docherty, Simms, Bagby, Krueger, Callahan, Chmielewski, Conway, De Clercq, Dornbach-Bender, Eaton, Forbes, Forbush, Haltigan, Miller, Morey, Patalay, Regier, Reininghaus, Shackman, Waszczuk, Watson, Wright and Zimmermann2019). This is the first study to account for heterotypic continuity of multiple dimensions of psychopathology simultaneously to borrow information from each in the estimation of the other, for more accurate estimates given considerable covariation between internalizing and externalizing problems.
Our study has three primary aims: 1) describe the trajectories of internalizing and externalizing problems across a lengthy developmental span. We use an IRT approach to developmental scaling that places the scores from age-differing measures and raters of internalizing and externalizing problems onto the same scale to account for heterotypic continuity and effects of informant type. 2) We aim to evaluate whether measures of reactive and control processes of temperament predict trajectories of internalizing and externalizing problems. The developmental scaling approach allows us to chart children’s trajectories of internalizing and externalizing problems across ages 2–15 years, and to examine immediacy preference and negative emotionality as predictors of children’s trajectories. 3) Using a bifactor model, we aim to determine whether immediacy preference and negative emotionality factors predict general versus specific psychopathology.
We hypothesize that immediacy preference will predict general psychopathology and specific externalizing problems, but not specific internalizing problems, because immediacy preference has been more strongly associated with externalizing problems compared to internalizing problems in prior work (Campbell & von Stauffenberg, Reference Campbell and von Stauffenberg2009; Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996). Little work has examined the association between immediacy preference and general psychopathology, but the strong association between immediacy preference and externalizing problems may drive this association. In addition, we hypothesize that temperamental negative emotionality will predict general psychopathology and specific internalizing problems, but not specific externalizing problems. Negative emotionality, like general psychopathology, is often thought to reflect a general liability (Forbes et al., Reference Forbes, Rapee and Krueger2019; Phillips et al., Reference Phillips, Brock, James, Nelson, Espy and Nelson2022). Additionally, the link between negative emotionality and general psychopathology along with internalizing problems has been established in prior literature (Castellanos-Ryan et al., Reference Castellanos-Ryan, Brière, O’Leary-Barrett, Banaschewski, Bokde, Bromberg, Büchel, Flor, Frouin, Gallinat, Garavan, Martinot, Nees, Paus, Pausova, Rietschel, Smolka, Robbins, Whelan, Schumann and Conrod2016; Lahey et al., Reference Lahey, Moore, Kaczkurkin and Zald2021; Lahey, Reference Lahey2009; Olino et al., Reference Olino, Dougherty, Bufferd, Carlson and Klein2014; Tackett et al., Reference Tackett, Lahey, van Hulle, Waldman, Krueger and Rathouz2013).
Method
Children (N = 1,364) and their families were recruited for the NICHD SECCYD study in 1991 from 31 hospitals near one of 10 locations in the United States: Little Rock, AR; Irvine, CA; Lawrence and Topeka, KS; Boston, MA; Morganton and Hickory, NC; Charlottesville, VA; Seattle, WA; and Madison, WI. Children were recruited at birth and were followed for data collection in four total phases with multiple timepoints at each phase (Phase I, ages 0–3; Phase II, though 1st grade; Phase III, through 6th grade; Phase IV, through 9th grade) until they were 15 years old. The present study involves behavior problem ratings that were assessed near-annually from ages 2–15 years (except ages 13 and 14). The sample was 48% female, 80.4% White, 12.9% Black, 6.1% Hispanic, 1.6% Asian American, 0.4% American Indian, and 4.7% of “other” ethnicity. At intake, the mother’s age ranged from 18 to 46 years of age (M = 28.11, SD = 5.63), 77% of households had fathers living in the home, and there was an average of 4.27 people living in the household (SD = 1.17). For more information about the study methods and participants, see NICHD Early Child Care Research Network (2005).
Exclusion criteria included: (1) The mother was younger than 18 years of age at the time of the child’s birth; (2) the family did not anticipate remaining in data collection for at least three years; (3) the child had obvious disabilities at birth and/or remained in the hospital for more than seven days after birth; or (4) the mother was not able to speak conversational English. At enrollment, trained researchers visited the family homes, and families were scheduled for periodic data collection. During a given phase, research assistants visited family homes, childcare, and invited families to the laboratory playroom to collect observations and administer study measures.
Measures
Analysis scripts and a data dictionary of study variables were published at https://osf.io/yz4we/. Descriptive statistics and correlations of study variables are in Table 1.
Correlation matrix of model variables
Note. The correlations and descriptive statistics are presented from data are in long format, where each participant has multiple rows: i.e., one row for each informant-by-timepoint combination. “Age” in years; “INR” = income-to-needs ratio; “Min” = lowest score in the sample; “Max” = highest score in the sample. † p < .10; *p < .05; **p < .01; ***p < .001; all ps two-tailed.
Behavior problems
Children’s externalizing and internalizing behavior problems were rated by mothers, fathers, teachers, afterschool caregivers, other caregivers (e.g., daycare workers and babysitters), and self report. Ratings from different informants and measures were used, depending on the child’s age. Behavior problem ratings were completed on the following Achenbach measures: Child Behavior Checklist 2–3 (CBCL 2–3; Achenbach, Reference Achenbach1992), Child Behavior Checklist 4–18 (CBCL 4–18; Achenbach, Reference Achenbach1991a, Reference Achenbach1991b), Caregiver–Teacher Report Form (C–TRF; Achenbach & Rescorla, Reference Achenbach and Rescorla2000), Teacher’s Report Form (TRF; Achenbach, Reference Achenbach1991a), and Youth Self-Report (YSR; Achenbach, Reference Achenbach1991b). The ages at which each rater provided ratings is provided in Table 2. The measures that were completed by each rater at each age are depicted in Figure 1.
Depiction of how the scores from various raters and measures were linked at different ages. Note. Raters are depicted in the rows, and the child’s age (in years) is depicted in the columns. Different shapes indicate different measures (square = Child Behavior Checklist 4–18; rounded square = Child Behavior Checklist 2–3; circle = Teacher’s Report Form; diamond = Caregiver–Teacher Report Form; hexagon = Youth Self-Report). A solid arrow indicates that scores from the same measure were linked using all items (i.e., all items were common items; e.g., mothers’ ratings at ages 6 and 8). A broken arrow indicates that scores from different measures were linked using the common items (e.g., mothers’ ratings at ages 3 and 4). The direction of the arrow indicates the measure to which the other was linked (e.g., mothers’ ratings at age 8 were linked to mothers’ ratings at age 6). The solid black box indicates the referent measure (mothers’ ratings at age 6) to which every other measure was linked either directly or indirectly. The gray bounding boxes indicate that scores from different raters were linked using the common items (e.g., self-report ratings at age 15 were linked to mothers’ ratings at age 15).
The child’s age when each rater provided ratings of the child’s behavior problems
Note. “x” indicates the measure was collected at the specified age. “*” indicates the referent age and rater.
Items were rated as 0, 1, or 2 corresponding to “not true,” “somewhat or sometimes true,” or “very true or often true,” respectively. The Externalizing scale of the CBCL 2–3 includes the Aggressive Behavior and Destructive Behavior subscales. The Externalizing scale of the CBCL 4–18, TRF, and YSR includes the Aggressive Behavior and Delinquent Behavior subscales. The Externalizing scale of the C–TRF includes the Aggressive Behavior and Attention Problems subscales. The scales include symptoms of breaking rules, cruelty, aggression, and destruction of property.
The Internalizing scale of the CBCL 2–3 includes the Anxious/Depressed and Withdrawn subscales. The Internalizing scale of the CBCL 4–18, TRF, and YSR includes the Anxious/Depressed, Somatic Complaints, and Withdrawn subscales. The Internalizing scale of the C–TRF includes the Anxious/Obsessive, Fears, and Depressed/Withdrawn subscales.
The Achenbach scales are widely used, and the scores show strong reliability (internal consistency, test–retest reliability, and interrater reliability) and validity (content, construct, and criterion-related validity; Sattler, Reference Sattler2022). Internal consistency estimates by age and rater of the present study are in Supplementary Table S1.
Due to the wide age range of children and adolescents included in the study, we took steps to ensure that the same construct was assessed on the same scale, across time. The number of common items among different measures are in Table 3. To account for developmental changes in both internalizing and externalizing problems, we used an IRT approach to developmental scaling, consistent with previous research (described later; Kolen & Brennan, Reference Kolen and Brennan2014; Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018; Petersen & LeBeau, Reference Petersen and LeBeau2022). Developmental scaling linked behavior problem scores across ages and informants onto the same scale. One-year cross-time stability estimates by rater are in Supplementary Table S2. Full descriptive statistics of externalizing and internalizing problems by age and rater are in Table 4, and correlations among internalizing and externalizing problems by rater are in Table 5. Furthermore, the percentage of participants with behavior ratings, by rater type, are in Supplementary Table S3.
The number of common items for each pair of measures
Note. “CBCL” = Child Behavior Checklist, “C–TRF” = Caregiver–Teacher Report Form, “TRF” = Teacher’s Report Form, “YSR” = Youth Self-Report. The top table presents the number of common items on the Externalizing scale. The bottom table presents the number of common items on the Internalizing scale. Numbers on the diagonal represent the total number of items in the Externalizing scale (top table) or Internalizing scale (bottom table) for that measure (e.g., the CBCL 4–18 has 33 items on the Externalizing scale and 31 items on the Internalizing scale). Numbers below the diagonal represent, for that pair of measures, the number of items that are common to both of the measures. The number of unique items can be calculated by subtracting the number of common items from the total number of items. For instance, the CBCL 4–18 has 6 unique externalizing items when compared with the TRF (i.e., 33 total items minus 27 common items). Conversely, the TRF has 7 unique externalizing items when compared with the CBCL 4–18 (i.e., 34 total items minus 27 common items).
Descriptive statistics of externalizing and internalizing problems by age and rater
Note. “–” indicates not applicable because the particular rater did not provide ratings at the given time point. Means and standard deviations (SDs) for externalizing problems are the top number in each box, whereas means and SDs for internalizing problems are the bottom number.
Correlation matrix of externalizing problem scores (below diagonal) and internalizing problem scores (above diagonal) by rater
Note. “n/a” indicates not applicable because the two raters did not provide ratings at the same time point(s). Diagonal represents within-informant correlations between externalizing and internalizing problems scores.
† p < .10; *p < .05; **p < .01; ***p < .001; all ps two-tailed.
Predictors
Delay of gratification
Delay of gratification was assessed in the present study with a self-imposed waiting task when the participants were 54 months old. An experimenter elicited the child’s preference in treats. Then, the child was told that they would engage in a game where the experimenter would leave the child in the room with the preferred treat. Further, the child was told that if they waited until the experimenter returned, the child could eat the treat and receive an additional portion of treat as reward for waiting the full length of the task. The child was also instructed that if they ring a bell, that will signal to the experimenter that the child does not want to wait, and that the child would only receive the portion of treats presented and no extra portions. The recorded experimental trial had a 7-minute ceiling, and the child’s score was the total number of seconds the child waited until ringing the bell, eating the treat, or 7 minutes (whichever came first). The self-imposed waiting task is one of the most widely used performance-based tasks of inhibitory control and motivational self-regulation in psychological research.
Negative emotionality
Temperamental negative emotionality was assessed from mother- and other caregiver report at 54 months using a modified version of the Children’s Behavior Questionnaire (CBQ; Rothbart et al., Reference Rothbart, Ahadi, Hershey and Fisher2001). Raters on the CBQ were asked to rate how well each item described the child in the past 6 months using a 7-point scale where 1 = extremely untrue and 7 = extremely true. Of the 196 items in the original CBQ, mothers completed 80 items and other caregivers completed 48 items. We used the Negative Affect scale on the CBQ, which encompasses the following subscales: Anger/Frustration (10 items), Sadness (10 items), and, among mother report, Fear (10 items). Other caregivers did not complete the Fear subscale. Internal consistency estimates were α = .82 for mothers’ reports and α = .93 for other caregivers’ reports. Mothers’ ratings were modestly correlated with other caregivers’ ratings (r[761] = .09, p = .017). To incorporate a multi-informant perspective into the estimation of the child’s negative emotionality, we averaged ratings across raters. Negative emotionality as assessed by the CBQ has been widely used and has shown strong internal consistency, rank-order stability, and construct validity (Rothbart et al., Reference Rothbart, Ahadi, Hershey and Fisher2001).
Due to potential inflation of prediction of psychopathology outcomes due to overlap in item content between the CBQ and the numerous measures of child psychopathology, we dropped items from the mother report and other-caregiver-report CBQ that we judged to conceptually overlap with items from the Internalizing or Externalizing scales of the CBCL. We dropped two items from the mother-report CBQ: one item that assessed frustration and one item that assessed sadness. We dropped one item from the other-caregiver-report CBQ that assessed temper tantrums. Two items from the mother report and two items from the other caregiver report had similar item content but were distinct enough to retain, e.g., an item pertaining to getting irritated when making a mistake versus being afraid to make a mistake. In this case, both items conceptually assess behaviors pertaining to making a mistake, but one behavior is anticipatory whereas the other is reactive; thus, we considered them conceptually distinct.
Covariates
Several demographic characteristics were examined as covariates in the growth curve and bifactor models, including the child’s sex (1 = girl, 0 = boy), race, and ethnicity, and the income-to-needs ratio of the child’s family. These covariates were selected because they have been shown to robustly impact associations between predictors and child behavior problems (e.g., Petersen et al., Reference Petersen, LeBeau and Choe2021; Shi et al., Reference Shi, Ettekal, Deutz and Woltering2020). Race was a dummy coded variable where African American children were compared to other races. Ethnicity was dummy coded such that Hispanic children were compared to non-Hispanic children.
Statistical analysis
Developmental scale of externalizing and internalizing problems
We used multidimensional IRT (M-IRT) and linking to create a single uniform developmental scale (i.e., developmental scaling) for externalizing and internalizing problems that spans multiple years of development. We conducted this linking in five steps: (1) Fit M-IRT models at each age and for each rater type separately. (2) Link the measures’ scores over time within each rater type. (3) Link scores across raters. (4) Calculate latent factor scores on the linked scale. (5) Use linked factor scores in growth curve and bifactor models. We describe this procedure in detail below. Full description of linking details are in Supplementary Appendix S1.
Step 1. Fit M-IRT models at each age and for each rater type separately
We used the multidimensional graded response IRT model using the mirt package (Chalmers, Reference Chalmers2012) in R 3.6.1 (R Core Team, 2022) to estimate item parameters. The mirt package uses a maximum likelihood expectation-maximization algorithm to estimate item parameters. The maximum likelihood estimation procedure uses all available data for each item and provides valid inferences if the data are missing at random or completely at random. The graded response model is a generalized version of the two-parameter logistic model for dichotomous outcomes, accommodating polytomous items that are ordinal in nature. The multidimensional graded response model adds the ability to include multiple latent factors (i.e., externalizing and internalizing problems) – and their covariance – in the same model. This multidimensional graded response IRT model is conceptually like a two-factor categorical confirmatory factor analysis approach (fit to ordinal data) with the internalizing and externalizing latent factors allowed to covary, and with no cross loadings. That is, internalizing and externalizing problem items were included in the same model, but they were allowed to load onto distinct latent factors. The externalizing and internalizing problem items in the current study were questionnaire items rated from 0 to 2. We used the externalizing problems latent factor as the reference group and allowed the mean and variance for internalizing problems latent factor to be estimated freely. Setting the externalizing factor as the reference group, along with linking both internalizing and externalizing items in the same model, placed the internalizing and externalizing problem scores onto the same mathematical scale across ages and raters.
There may be shifts in the externalizing or internalizing problem constructs over time due to natural developmental changes (Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018). The present study spans a wide age range (ages 2–15 years). When spanning a wide age range, it is considered safer to fit a separate model at each age rather than a single model that spans all ages because a model that spans across a wide age range is more likely to violate IRT dimensionality assumptions (Kolen & Brennan, Reference Kolen and Brennan2014). We fit two latent factors corresponding to the constructs of interest: i.e., externalizing and internalizing problems. IRT assumes that each latent factor (e.g., externalizing problems) is unidimensional, which is more likely at a single time point than across all time points in the same model. Thus, we fit a separate IRT model at each age and for each rater type in the present study. This approach was also applied by Petersen et al. (Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018) and by Petersen & LeBeau (Reference Petersen and LeBeau2022) in their creation of a developmental scale for internalizing and externalizing problems, respectively, across a wide age range.
Step 2. Link the measures’ scores over time within each rater type
After successful estimation of the individual IRT models, we used multidimensional linking methodology to create the developmental scale for externalizing and internalizing problems. Developmental scaling is a form of data harmonization that aims to place two measures that assess the same construct but differ based on severity and discrimination onto the same scale. One way to create a developmental scale is to link the two measures. The strength of the linking is enhanced if there are items that overlap across the two measures, often referred to as common items. Developmental scaling based on item parameter invariance theory assumes that any difference in item parameter estimates can be rescaled onto a single unified metric with a linear transformation across adjacent ages. Based on this assumption, the item parameters, and the resulting latent factor scores of externalizing and internalizing problems can be linked across ages by comparing and linearly transforming differences in discrimination and severity of the common items across adjacent ages.
We used multidimensional developmental scaling techniques to link the measures’ scores over time within each rater type. We used the plink package (Weeks, Reference Weeks2010) in R to perform the linking by using the multidimensional test characteristic function procedure with an oblique Procrustes rotation (Oshima et al., Reference Oshima, Davey and Lee2000). The oblique rotation method allowed the latent factors – externalizing and internalizing problems – to be correlated. For linking, we used a multidimensional Stocking-Lord procedure (Stocking & Lord, Reference Stocking and Lord1983). The Stocking-Lord linking procedure iteratively estimates linking constants by minimizing differences in the aggregate scores across common items.
To estimate the Stocking-Lord parameters, we set the reference age at 6 years for each rater because age 6 was the first age when most rater types (except other caregivers and self report) provided ratings of the child’s externalizing and internalizing problems. We set the reference rater to be the mother because the mother typically provided the most ratings across the developmental age span. The reference age and rater pair set the scale to which the item parameters at subsequent ages and for other raters were transformed. In other words, we transformed the estimated item parameters at all ages and for all raters to be on the same scale as the item parameters estimated for mothers’ ratings at 6 years of age. To achieve this, we first linked the item parameters across ages within rater type. We performed the process of linking iteratively by chaining together multiple linking constants across the age span. First, for a given rater type, we estimated Stocking-Lord linking constants that linked the item parameters at age 7 to be on the same scale as that rater type’s item parameters at age 6. We estimated additional linking constants between adjacent age spans, for example between 5 and 6 years of age, 7 and 8 years of age, and so on. We used two estimated scaling constants including an intercept parameter, B, and a slope parameter, A, to link the item parameters onto the reference scale.
After successfully estimating the linking constants, we then transformed all item parameters to be on the age 6 scale for the given rater. Min (Reference Min2007) provides further technical details on the multivariate linking terms. To shift all item parameters to a common age 6 scale, we applied all previous adjacent scaling constants to the item parameters. For example, when shifting the item parameter estimates for 7-year-olds to the age 6 scale, we used a single set of scaling constants. However, when shifting the item parameters for 8-year-olds, we used two sets of scaling constants: first, we transformed the item parameter estimates for 8-year-olds to the scale of the 7-year-olds, and then we transformed them a second time to be on the age 6 scale. See Figure 1 for a visualization of the linking process. We performed this step of the linking process separately for each row in the figure (i.e., within rater types; horizontal arrows).
Step 3. Link scores across raters
After creating developmental scales across ages within rater types, we linked scores across raters at age 6 (except for the other caregivers’ reports collected at age 2 and self report collected at age 15). As described above, we set the mother as the reference rater. We used a similar process as in step 2; we estimated Stocking-Lord linking constants to link the item parameters across raters within a single age. For example, we estimated a set of linking constants to link the item parameters of the fathers’ ratings to the item parameters of mothers’ ratings at age 6 to ensure that their factor scores were on the same scale. This step moved the developmental scales for fathers, teachers, and afterschool caregivers to the mothers’ scale, anchored at age 6, while preserving the developmental scale created within rater types in step 2. The process of linking scores across raters is depicted in Figure 1 with the gray bounding boxes (vertical arrows).
Step 4. Calculate latent factor scores on the linked scale
After successfully placing item parameter estimates onto a single developmental scale (for all raters and ages), we calculated children’s latent externalizing and internalizing problem scores with expected a posteriori factor scores. The linking in the previous two steps scaled the factor scores to be on the single developmental scale while retaining changes in means and variances over time and across raters. The linking constants by measure and age are in Supplementary Table S4.
In sum, the linking of scores within a rater type created a developmental scale for scores from that rater type, so each rater type had their own trajectory (see Figure 2). We then, ultimately, linked each rater type’s developmental scale (directly or indirectly) to the mothers’ ratings at age 6, so that each rater type’s trajectory was on the same developmental scale. Examples of linked scores across raters and years are depicted with test characteristic curves in Supplementary Figures S1 through S4. The test characteristic curves of the linked scores across raters and years were highly similar (and more similar than the test characteristic curves of the pre-linked scores), indicating that we successfully linked scores across raters and years to be on the same scale.
Mean factor scores by age, rater, and type of behavior problem.
Differential item functioning
Post-linking estimates of scale-level DIF between measures used to link scores across different raters and ages are in Supplementary Table S5. Tests of differential item functioning (DIF) by age showed no major concerns at the scale level after linking (see Supplementary Appendix S2). The distribution of DIF effect size statistics between ages by rater type are in Supplementary Figure S5. We also conducted analyses to examine potential differential item functioning by sex and race (see Supplementary Appendix S3). There was some differential item functioning between males and females, and between Whites and Non-Whites. However, most instances of DIF were differences in severity (i.e., uniform DIF) rather than differences in discrimination (non-uniform DIF). Thus, covariate adjustment within the growth models should adjust for group-level differences in the factor score differences by sex and race.
Secondary analysis of aggression and delinquent behavior
As a secondary analysis, we also examined aggression and delinquent subdimensions of externalizing problems given their differing associations with risk factors (J. Murray & Farrington, Reference Murray and Farrington2010; Wall & Barth, Reference Wall and Barth2005). Thus, we also conducted developmental scaling with aggression and delinquent behavior (see Supplementary Appendix S4).
Step 5. Use linked factor scores in growth curve and bifactor models
After linking factor scores from all raters and at all ages to be on the scale of mothers’ ratings at age 6, we used the linked factor scores as the child’s estimated level of behavior problems for a given rater and age in subsequent growth curve and bifactor models.
Growth curve modeling
We modeled children’s trajectories of behavior problems using mixed models. We estimated mixed models using the lmer function of the lme4 package (D. Bates et al., Reference Bates, Mächler, Bolker and Walker2015) in R. We modeled externalizing problems and internalizing problems separately given their different developmental courses. Because our goal was to predict behavior problems in adolescence, we set the intercepts to be at the last time point (age 15), consistent with prior research. First, we established the form of change of children’s behavior problems over time by comparing linear, quadratic, and cubic polynomials. To reduce the potential for multicollinearity among the polynomial terms, we used orthogonal polynomials calculated using the poly function in R. We used chi-square difference tests to compare nested models.
Upon establishing the form of the trajectory, we added dummy-coded fixed effects for the rater type: mother, father, teacher, afterschool caregiver, other caregiver, or self report. We set mother report to be the reference group because mothers provided the most ratings on average. Then, we added fixed effects for the covariates. After adding covariates, we then added the focal predictors of interest, the child’s negative emotionality and delay of gratification. For parsimony and interpretability, we examined the predictors in relation to the intercepts and linear slopes. Model formulas are in Supplementary Appendix S5.
We determined the importance of focal predictors using R 2 statistics to evaluate how much variation in behavior problems was explained by the rater predictors, demographic predictors, and focal predictors of interest. We computed R 2 statistics defined by Nakagawa and Schielzeth (Reference Nakagawa and Schielzeth2013). In Supplementary Appendix S6, we describe tests of systematic missingness and how missing data were handled.
Bifactor model
We estimated a bifactor model at the last timepoint in the study (i.e., age 15) to determine whether negative emotionality and delay of gratification predicted general versus specific psychopathology. First, we fit a bifactor model at age 15 years with only externalizing and internalizing items and no predictors. The bifactor model included a latent factor for the general factor of psychopathology, in addition to latent factors for externalizing problems and internalizing problems. The latent factors for internalizing and externalizing problems for each participant were estimated using the developmentally scaled factor scores from all informant types who provided ratings at age 15 (mother-, father-, and self report). The latent factors were set to be uncorrelated, so the general factor represented the covariation among all externalizing and internalizing items. By contrast, the specific psychopathology factors – i.e., externalizing problems and internalizing problems – represented the covariation among the items within that dimension after extracting the variance accounted for by the general factor. The externalizing latent factor represented unique externalizing variance, and the internalizing latent factor represented unique internalizing variance. Upon establishing a well-fitting bifactor model, we examined the focal predictors of interest. Predictors were allowed to predict the three latent factors. Then, we added covariates.
Bifactor models were fit in lavaan (Rosseel, Reference Rosseel2012) in R. Models with diagonally weighted least squares (WLSMV) were unable to be estimated due to sparse cells in some response categories for some items. Therefore, models used maximum likelihood estimation and a probit link with robust standard errors (MLR-probit) to account for the nonnormally distributed data, which has shown comparable power to WLSMV and better control for Type I error when using ordinal data (Cuhadar & Kalkan, Reference Cuhadar and Kalkan2023; Suh, Reference Suh2015). Missing data were handled with full information maximum likelihood estimation, which uses all available data and is the gold standard approach for handling missingness when data are missing at random or completely at random. We evaluated model fit with the root mean square error of approximation (RMSEA), Robust estimate of comparative fit index (CFI), and standardized root mean square residual (SRMR). Model fit was considered good if RMSEA ≤ .05, CFI ≥ .95, and SRMR ≤ .08; model fit was considered acceptable if RMSEA ≤ .08, CFI ≥ .90, and SRMR ≤ .10 (Bentler & Bonett, Reference Bentler and Bonett1980; Hu & Bentler, Reference Hu and Bentler1999; Schermelleh-Engel et al., Reference Schermelleh-Engel, Moosbrugger and Müller2003; Schreiber et al., Reference Schreiber, Nora, Stage, Barlow and King2006).
Due to the sparse cells in some response categories for some items, we structured the multi-informant ratings at age 15 in long form by rater, to leverage ratings from all raters at age 15: mothers, fathers, and self-report. That is, each participant had up to three rows (one for each rater). To account for the non-independence of multiple observations per participant, we used the participant as a cluster variable, which calculates robust standard errors using a Huber-White sandwich estimator (Huber, Reference Huber1967; White, Reference White1980).
The scale of each latent factor was set using the effects coding method (Little et al., Reference Little, Slegers and Card2006) so that the average of the items’ factor loadings was equal to one. Composite reliability, indexed by coefficient omega, was estimated using the semTools package (Jorgensen et al., Reference Jorgensen, Pornprasertmanit, Schoemann and Rosseel2022) in R.
Results
The present study sought to achieve three aims: First, we sought to describe people’s trajectories of externalizing and internalizing problems from ages 2–15 years after performing developmental scaling to put the scores from the different measures, informants, and constructs onto the same scale. Second, we examined whether negative emotionality and immediacy preference predict the slopes from ages 2–15 and intercepts (i.e., ending levels) of people’s trajectories of externalizing and internalizing problems at age 15. Third, we examined whether negative emotionality and immediacy preference predict general psychopathology versus specific psychopathology – i.e., unique externalizing problems or unique internalizing problems – at age 15.
Aim 1: Describe the trajectories
Unconditional means model
An unconditional means model (i.e., random intercepts) demonstrated that individual differences in intercepts accounted for 34% of the variance in children’s trajectories of externalizing problems and 18% of the variance in children’s trajectories of internalizing problems. A model with random intercepts and random linear slopes fit better than a model with only random intercepts and accounted for 40% of the variance in externalizing problems and 22% of the variance in internalizing problems (externalizing: Δχ2[3] = 1,809.63; internalizing: Δχ2[3] = 408.11; ps < .001).
Functional form
To determine the best-fitting functional form, we compared models with random linear, quadratic, and cubic slopes. A model with random linear and quadratic slopes fit better than a model with only random linear slopes (externalizing: Δχ2[4] = 1,543.94; internalizing: Δχ2[4] = 1,937.47; ps < .001). A model with random cubic slopes did not converge due to small variance of the random cubic term. Thus, we selected the quadratic model as the best-fitting functional form of growth.
Rater type
Then, we added the rater type (e.g., mother, father, teacher) as a dummy-coded predictor of the trajectories to account for systematic differences as a function of rater type. Mothers served as the reference rater group. The model with rater type as a predictor fit better than a model without rater type (externalizing: Δχ2[13] = 2,927.75; internalizing: Δχ2[13] = 8,780.80; ps < .001). The model with rater type predicting linear and quadratic slopes fit better than models with rater type predicting only linear slopes (externalizing: Δχ2[4] = 417.34; internalizing: Δχ2[4] = 69.95; ps < .001). Thus, for the baseline growth model, we selected the model in which the rater type predicted the intercepts, linear slopes, and quadratic slopes to allow different curvature by rater type.
Baseline growth model
Individuals’ growth curves from the baseline growth model are depicted in Figure 3. Model results are in Supplementary Table S6.
Children’s model-implied growth curves of internalizing and externalizing problems by rater. Note. “All-Rated” refers to the model-implied ratings for the “average” rater.
For externalizing problems, the model explained 47% of the variance (fixed effects: 10%; random effects: 37%). Ratings by fathers, teachers, and afterschool caregivers had lower intercepts compared to ratings by mothers. Self-report had higher intercepts than ratings by mothers. When setting the intercepts to the first timepoint when the target informant type provided ratings, ratings by other caregivers had higher intercepts than ratings by mothers (see Supplementary Appendix S9). In summary, mothers tended to rate their child as showing more externalizing problems than did fathers, afterschool caregivers, and teachers (except for teachers’ ratings after age 8; see Figure 2); however, mothers tended to rate their child as showing fewer externalizing problems than did other caregivers or self-report. On average, externalizing problems decreased across ages 2–15, but trajectories differed by rater. Mothers’ ratings showed average decreases across ages 2–15, whereas teachers’ ratings showed modest average increases from ages 4–8 and then stayed relatively stable with slight declines from ages 8–11.
For internalizing problems, the model explained 47% of the variance (fixed effects: 24%; random effects: 23%). Ratings by fathers, teachers, and afterschool caregivers had lower intercepts compared to ratings by mothers. Self-report had higher intercepts than ratings by mothers. When setting the intercepts to the first timepoint when the target informant type provided ratings, ratings by other caregivers had higher intercepts than ratings by mothers (see Supplementary Appendix S9). In summary, mothers tended to rate their child as showing more internalizing problems than did fathers, afterschool caregivers, and teachers; however, mothers tended to rate their child as showing fewer internalizing problems than did other caregivers or self-report. On average, internalizing problems showed decreases from ages 2–10 and increases from ages 10–15, but trajectories differed by rater. Mothers’ ratings were relatively stable from ages 2–15, whereas teachers’ ratings showed modest average increases from ages 4–8 and then stayed relatively stable from ages 8–11.
Demographic covariates
Then, we added demographic characteristics to the model as covariates. Model results are in Supplementary Table S7. For externalizing problems, the fixed effects explained 12% of the variance (Δ2%). Compared to girls, boys had higher intercepts but did not differ in slopes. Relative to other races, African American children had higher intercepts but did not differ in slopes. Relative to non-Hispanic children, Hispanic children did not differ in intercepts but showed marginally significant differences in slopes in terms of shallower decreases with age. A lower income-to-needs ratio was associated with higher intercepts but not different slopes. In other words, boys, African Americans, and children from families with a lower income-to-needs ratio had higher ratings of externalizing problems. Compared to non-Hispanic children, Hispanic children showed lesser decreases in ratings of externalizing problems across development.
For internalizing problems, the fixed effects in the baseline growth model explained 24% of the variance (Δ <1%). Compared to boys, girls had higher intercepts and had steeper increases over time. Relative to other races, African American children had steeper decreases over time but did not differ in intercepts. Relative to non-Hispanic children, Hispanic children did not differ in intercepts or slopes. A lower income-to-needs ratio was associated with higher intercepts but not different slopes. In other words, girls and children from families with a lower income-to-needs ratio had higher ratings of internalizing problems. Girls showed steeper increases in internalizing problems compared to boys; African Americans showed steeper decreases in internalizing problems compared to non-African Americans.
Aim 2: Predicting the trajectories
Then, we added negative emotionality and delay of gratification as predictors in the model. Model results are in Supplementary Table S8.
For externalizing problems, the fixed effects in the baseline growth model explained 17% of the variance. Thus, the predictors of interest collectively accounted for ∼5% of additional variance. Higher negative emotionality was associated with higher intercepts but steeper declines in externalizing problems over time. Greater delay of gratification was associated with lower intercepts but not with differences in slopes. Prototypical growth curves of externalizing problems as a function of delay of gratification are in Figure 4.
Children’s model-implied externalizing problems trajectory as a function of duration of time waited in a self-imposed waiting task.
For internalizing problems, the fixed effects in the baseline growth model explained 26% of the variance. Thus, the predictors of interest collectively accounted for ∼3% of additional variance. Higher negative emotionality was associated with higher intercepts but steeper declines in internalizing problems over time. Greater delay of gratification was associated with lower intercepts but not with differences in slopes.
Sensitivity analysis results: growth curve models
For results of sensitivity analyses of growth curves, see Supplementary Appendix S9. Early cognitive ability was associated with lower intercepts of both internalizing and externalizing problems. When accounting for early cognitive ability as a covariate, its presence attenuated the previously significant associations between delay of gratification and intercepts of internalizing and externalizing problems. When examining mother-only ratings of internalizing and externalizing problems, negative emotionality and delay of gratification did not predict differences in slopes in internalizing and externalizing problems that were present when examining ratings from all informants. Results regarding intercepts did not differ significantly.
When excluding ratings prior to 54 months of age, results did not substantively change. When examining anger/frustration and fear, separately, instead of a negative emotionality composite, anger/frustration was strongly associated with higher externalizing and internalizing problem intercepts, and steeper declines in externalizing problems and, at a trend-level, internalizing problems. By contrast, fear was not associated with intercepts or slopes of externalizing problems but was strongly associated with higher intercepts and steeper declines in internalizing problems. When examining mother and caregiver report of negative emotionality separately, mother-reported negative emotionality was associated with higher intercepts of internalizing and externalizing problems, and steeper decreases in internalizing but not externalizing problems. Caregiver report had similar results as mother report, except that caregiver-reported negative emotionality predicted steeper declines in internalizing problems at a trend-level. Finally, when examining aggressive and delinquent behavior separately, associations with predictors and aggressive behavior factor did not substantively change from the primary analyses. Associations between predictors and delinquent behaviors also did not substantively change from the primary analyses.
Aim 3: Predicting general versus specific psychopathology at age 15
Given associations between the risk factors and ending levels of externalizing and internalizing problems at age 15, we examined the risk factors in relation to general versus specific psychopathology at age 15. The bifactor model with all items (externalizing: 33 items; internalizing: 33 items) did not fit. Thus, we modified the model by dropping items with low endorsement rates (externalizing: 7 items; internalizing: 0 items), leaving 26 externalizing items and 33 internalizing items. Then, we removed loadings onto the specific factors that were not significant and positive (externalizing: 8 items; internalizing: 1 item), leaving 18 externalizing items and 32 internalizing items that loaded onto the respective specific factor. The model fit well according to RMSEA (.051) and SRMR (.050) but did not fit well according to CFI (.796). Thus, we made model modifications.
We allowed item residuals to be correlated for which the modification index was large (Δχ2 > 20), indicating local non-independence of items, if the modification was also consistent with theory (i.e., both items were within the same domain). This led to 104 correlated residuals out of 649 possible within-domain correlated residuals. The model fit well according to RMSEA (.034) and SRMR (.047) and showed acceptable fit according to CFI (.907).
Items’ factor loadings on the general factor and specific externalizing and internalizing factors are in Supplementary Table S9. The proportion of variance in ratings that was accounted for by the general factor (i.e., the explained common variance, ECV) was .69. The ECV of specific factors (ECVs) for the specific externalizing factor and specific internalizing factor was .09 and .22, respectively. Coefficient omega was .73, .21, and .42 for the general factor, specific externalizing factor, and specific internalizing factor, respectively. Thus, the reliability was acceptable for the general factor but was low for the specific factors, as evidenced by items’ relatively weak factor loadings on the specific factors. Findings in relation to unique externalizing and unique internalizing problems should therefore be interpreted carefully.
Then, we added the predictors: negative emotionality and delay of gratification. Regression coefficients are in Supplementary Table S10. Negative emotionality was positively associated with the general factor and unique externalizing problems; it was not significantly associated with unique internalizing problems. Delay of gratification was negatively associated with the general factor but was positively associated with unique internalizing problems; it was not significantly associated with unique externalizing problems.
Then we added covariates. Regression coefficients are in Supplementary Table S11. The association between negative emotionality and the general factor remained significantly associated even when controlling for covariates. However, the association between negative emotionality and unique externalizing was attenuated to trend-level significance after controlling for covariates. The association between negative emotionality and unique internalizing problems remained nonsignificant. The association between delay of gratification and externalizing problems remained nonsignificant. However, delay of gratification was no longer significantly associated with the general factor or unique internalizing problems after controlling for covariates When compared to boys, girls showed lower ratings of general psychopathology, and higher ratings of unique externalizing and internalizing problems. When compared to non-African Americans, African Americans showed lower ratings of unique internalizing and externalizing problems and, to a trend-level, higher ratings of general psychopathology. When compared to non-Hispanics, Hispanics showed higher ratings of unique externalizing problems and, to a trend-level, general psychopathology. A higher income-to-needs ratio was associated with lower ratings of general psychopathology and higher ratings of unique internalizing problems, at a trend-level. Compared to mother report, fathers rated their child as showing greater general psychopathology, but lower unique externalizing problems unique internalizing problems, all at a trend-level. Compared to mother report, self-report showed higher ratings of general psychopathology and unique internalizing and externalizing problems.
Sensitivity analysis results: Bifactor models
Full results from the bifactor model sensitivity analyses are in Supplementary Appendix S10. When including early cognitive ability as an additional covariate in a bifactor framework, the association between delay of gratification and general and specific psychopathology remained nonsignificant. The association between negative emotionality and specific externalizing problems became statistically significant, after previously being associated at a trend-level. Regression coefficients, including early cognitive ability, are in Supplementary Table S12. When examining anger/frustration and fear, separately, as predictors of general and specific psychopathology, anger/frustration was not associated with unique internalizing problems, but predicted general psychopathology and unique externalizing problems more strongly than did fear. Fear was not significantly associated with general or specific psychopathology. Mother report of negative emotionality significantly predicted general and unique externalizing problems, and, at a trend-level, unique internalizing problems. Caregiver report of negative emotionality did not predict general or unique behavior problems. When examining predictors of unique aggressive and delinquent behavior, negative emotionality predicted unique delinquent behavior, but not unique aggressive behavior. Delay of gratification was not significantly associated with unique aggressive or delinquent behavior. Finally, when separately estimating bifactor models derived from mother report and self report, model fit according to CFI was poor for self-report but was acceptable for mother report. Predictors and covariates were added to these analyses and the full results are in Supplementary Appendix S10. However, we caution the interpretation of the findings from the self-report model due to poor model fit. For mother report, negative emotionality was positively associated with general psychopathology and unique externalizing problems, but was not associated with unique internalizing problems. Delay of gratification was negatively associated with general psychopathology and was positively associated with unique internalizing problems, but was not associated with unique externalizing problems.
Discussion
The present study had three goals: First, we aimed to describe people’s trajectories of externalizing and internalizing problems from ages 2–15 years while accounting for heterotypic continuity via developmental scaling. Second, we examined whether negative emotionality and immediacy preference predicted the slopes from ages 2–15 and intercepts (i.e., ending levels) of people’s trajectories of externalizing and internalizing problems at age 15. Third, we examined whether negative emotionality and immediacy preference predicted general psychopathology versus specific psychopathology at age 15.
Developmental trajectories
The present study is the first to chart the trajectories of externalizing and internalizing psychopathology concurrently across a lengthy developmental span while linking ratings from multiple raters. Similar to previous findings, ratings of externalizing problems decreased across development (e.g., Keiley et al., Reference Keiley, Bates, Dodge and Pettit2000; Leve et al., Reference Leve, Kim and Pears2005; Petersen et al., Reference Petersen, Bates, Dodge, Lansford and Pettit2015). Boys showed higher levels of externalizing problems than girls at age 15, but they did not differ in slopes.
For internalizing problems, results indicated an average decrease from ages 2 to 10 and an average increase from ages 10 to 15 years. An increase in internalizing symptoms at around 10–15 years of age maps onto pubertal development, and is expected given previous research (e.g., Hamlat et al., Reference Hamlat, Snyder, Young and Hankin2019). Mothers tended to endorse higher levels of both internalizing and externalizing problems (for the child/adolescent) compared to ratings by fathers, teachers, and afterschool caregivers; mothers tended to endorse lower rates of internalizing and externalizing problems compared to other caregivers and self-report. Mothers’ ratings of internalizing problems tended to be relatively stable (on average) from 2 to 15 years of age, whereas teachers’ ratings showed average increases from 4 to 8 years, and then were relatively stable (on average) from 8 to 11 years of age. Thus, developmentally scaled trajectories of internalizing and externalizing problems from ages 2 to 15 years were consistent with the expected age-related decrease in externalizing problems, and with the age-related increase in internalizing problems that aligns with pubertal development. Girls showed steeper increases in internalizing problems than boys and showed higher levels at age 15.
Reactive and control processes as predictors of trajectories
We examined only one dimension of reactive (negative emotionality) and one dimension of control (immediacy preference) processes as predictors of psychopathology trajectories. Higher informant-reported negative emotionality assessed as the average of mother and other caregiver report at 54 months was associated with steeper declines in externalizing and internalizing problems across development but remained associated with higher ending levels of externalizing and internalizing problems at age 15, controlling for covariates – though effect sizes were small. These results indicate that negative emotionality assessed in early childhood is associated with relatively high levels of internalizing and externalizing problems that endure across childhood to adolescence. In a sensitivity analysis examining the subscales of negative emotionality – fear and anger/frustration – in place of negative emotionality, they showed differential associations with trajectories of internalizing and externalizing problems. As would be expected based on prior studies (e.g., Crockett et al., Reference Crockett, Wasserman, Rudasill, Hoffman and Kalutskaya2018), anger/frustration was associated with externalizing problems but also internalizing problems, whereas fear was strongly associated with only internalizing problems. These results highlight the importance of considering multiple aspects of negative emotionality.
Immediacy preference assessed at 54 months was associated with higher ending levels of externalizing and internalizing problems at age 15, controlling for covariates. That is, a delay preference, i.e., longer wait times in a self-imposed waiting task, was associated with lower ratings of externalizing and internalizing problems at age 15 – though effect sizes were small. The association between immediacy preference and externalizing problems is consistent with previous research (e.g., Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996; Mischel et al., Reference Mischel, Shoda and Rodriguez1989; Peake, Reference Peake2017). However, immediacy preference was not associated with changes in externalizing or internalizing problems across development. These results point to the small, but potentially meaningful, association of specific facets of reactive and control processes with levels of internalizing and externalizing psychopathology across childhood and adolescence.
Predictors of mother-reported trajectories differed from predictors of trajectories derived from multiple raters, such that negative emotionality did not predict slopes of internalizing or externalizing problems, and delay of gratification was not significantly associated with lower intercepts of internalizing problems. All other results were consistent with primary analyses. These results indicate that analyses including only mother ratings were just as well suited as when using multiple raters to detect that negative emotionality was associated with higher levels but not slopes of behavior problems, and that delay of gratification did not predict slopes across the developmental span. Of note, mother-reported negative emotionality was not able to detect differences in predictions of slopes, which provides further evidence for the utility of examining multiple perspectives of the child’s behavior.
When accounting for early cognitive ability in a sensitivity analysis, the associations between delay of gratification and the intercepts of internalizing and externalizing problems were attenuated to non-significance and trend-level significance, respectively. Early cognitive ability such as verbal skills are thought to support executive function – whose deficits underlie externalizing problems – and delay of gratification. Given these findings, future studies should examine the role of early cognitive ability and executive functioning when examining reactive and control processes as predictors of psychopathology. Prior studies have shown that executive functioning may play a key role in the relations of reactive and control processes on behavior problems (Ursache et al., Reference Ursache, Blair, Stifter and Voegtline2013; Watts et al., Reference Watts, Duncan and Quan2018).
General versus specific risk factors for psychopathology at age 15
In a bifactor model, negative emotionality at 54 months was significantly associated with general psychopathology and unique externalizing problems, but not unique internalizing problems at age 15 years. When accounting for covariates, the association between negative emotionality and unique externalizing was attenuated to trend-level significance. Sensitivity analysis elucidated that among symptoms of externalizing problems, delinquent behavior, but not aggressive behavior, appears to be most strongly related to negative emotionality. Furthermore, anger/frustration, not fearfulness, was strongly associated with general psychopathology and unique externalizing problems. A possible interpretation is that negative emotionality as assessed by the CBQ is overt and observable in nature, and thus may reflect more externalizing problems (Eisenberg et al., Reference Eisenberg, Valiente, Spinrad, Cumberland, Liew, Reiser, Zhou and Losoya2009). Furthermore, when accounting for covariates, the association between negative emotionality and general psychopathology remained significant. Interestingly, negative emotionality is often considered associated with internalizing psychopathology (Greene & Eaton, Reference Greene and Eaton2017), but our findings indicate that when accounting for general psychopathology, there was no association between childhood negative emotionality and later internalizing problems. The exception was that when examining mother- vs. caregiver-reported negative emotionality as predictors, only mother-reported negative emotionality was significantly associated with general psychopathology, unique externalizing problems, and, at a trend-level, unique internalizing problems.
One potential explanation for the consistently strong relation between negative emotionality and general psychopathology is that general psychopathology might be functionally interpreted as negative emotionality, such that temperamental negative emotionality may partially reflect a trait-like version of what is shared between internalizing and externalizing problems. An implication of these findings is that negative emotionality assessed at young ages may be a clinically relevant factor to consider when assessing psychopathology, as previous work has indicated (Forbes et al., Reference Forbes, Rapee and Krueger2019).
In a sensitivity analysis in which early cognitive ability was included as an additional covariate along with demographic characteristics, the association between negative emotionality and general psychopathology was attenuated to trend-level significance and the association with unique externalizing problems returned to statistical significance with the inclusion of early cognitive ability as a covariate. These findings suggest that other variables, potentially early cognitive ability, or demographic or environmental features, might partially account for the association between negative emotionality and psychopathology, which otherwise show strong associations.
Immediacy preference, i.e., shorter wait time, assessed at 54 months was associated with higher general psychopathology and with lower ratings of unique internalizing problems at age 15 years. Surprisingly, there was no association between immediacy preference and unique externalizing problems. Although the association between shorter wait times and lower ratings of internalizing problems was not hypothesized, previous research has observed such an association (Ho et al., Reference Ho, Dang, Odum, DeHart and Weiss2022). The authors interpreted the association as possibly reflecting that choosing an immediate reward over a distal one is adaptive in reducing feelings of anxiety or depression. Alternatively, the finding could reflect that internalizing problems may be characterized by over-regulation (Murray & Kochanska, Reference Murray and Kochanska2002). The finding that there was no association between immediacy preference and unique externalizing problems was contrary to what many studies have found (e.g., Ip et al., Reference Ip, Jester, Sameroff and Olson2019; Krueger et al., Reference Krueger, Caspi, Moffitt, White and Stouthamer-Loeber1996). However, these studies have not examined the association between immediacy preference and unique externalizing problems by controlling for the general factor of psychopathology. Nevertheless, previous studies have shown that after accounting for the general factor, self-regulation and executive functioning are not strongly associated with unique psychopathology (Bloemen et al., Reference Bloemen, Oldehinkel, Laceulle, Ormel, Rommelse and Hartman2018). However, consistent with Watts et al. (Reference Watts, Duncan and Quan2018), when controlling for covariates, the associations between immediacy preference and psychopathology outcomes were attenuated. This may reflect that individual differences in delay of gratification may be partially accounted for by demographic and background factors, including sex, culture, and socioeconomic status.
Watts et al. (Reference Watts, Duncan and Quan2018) examined the association between immediacy preference and internalizing and externalizing psychopathology. However, Watts et al., assessed a summation of internalizing and externalizing symptoms, whereas we derived general psychopathology using bifactor models, which allowed us to distinguish shared and unique psychopathology. This difference is important because we found that the signs of the association of delay of gratification with general psychopathology (positive) versus unique internalizing problems (negative) were in the opposite direction. The sign difference in the association with immediacy preference would not have been observable with a total behavior problems score, as was used in Watts et al. (Reference Watts, Duncan and Quan2018), or by examining total externalizing problems or total internalizing problems, as we separately examined in the present study. Others have argued against the use of behavior composites because they do not allow for partitioning variance between levels (Michaelson & Munakata, Reference Michaelson and Munakata2020). Therefore, accounting for the shared variance of internalizing and externalizing psychopathology may help identify potentially divergent effects of immediacy preference on various forms of psychopathology.
In a separate model including predictors and covariates together, immediacy preference was no longer associated with unique internalizing problems or the general factor. These results indicate that much of the variance in the association of immediacy preference with internalizing and general psychopathology may be accounted for demographic and socioeconomic factors. This result was similar to those found by Watts et al. (Reference Watts, Duncan and Quan2018), suggesting that immediacy preference may partially reflect environmental and dispositional processes. Although Watts et al. (Reference Watts, Duncan and Quan2018) focused primarily on children of mothers without college degrees, we leveraged a larger sample while linking scores from multiple informants to obtain a more robust estimate.
The numerous sensitivity analyses (growth curve models: Supplementary Appendix S9; bifactor models: Supplementary Appendix S10) indicated utility in examining subdimensions of constructs. Negative emotionality, in particular anger/frustration, was a robust predictor of externalizing problems across many sensitivity analyses. It was also useful to examine subdimensions of externalizing problems, including delinquent versus aggressive behavior; negative emotionality was more strongly associated with delinquent than aggressive behavior. Furthermore, when examining the association between temperament and psychopathology, accounting for covariates such as early cognitive ability provided potential alternative explanations for future work to consider. The effect sizes of findings in the sensitivity analyses were generally small in magnitude (β = −.01–.23); therefore, we caution that smaller samples might be underpowered to detect these associations.
Explained common variance
Partitioning unique and general psychopathology also allows estimating the proportion of variance in psychopathology ratings that was accounted for by general and specific psychopathology factors in the sample. The general factor accounted for ∼69% of the reliable variance in ratings of psychopathology at age 15 while specific externalizing problems accounted for ∼9% and specific internalizing problems accounted for ∼22%. This indicates that the general factor was substantially stronger than the specific factors at age 15. Because internalizing problems increased whereas externalizing problems decreased from childhood to adolescence, it is unsurprising that the specific internalizing factor had a larger ECVs compared to the specific externalizing factor. These results are important to consider, because predictors of unique externalizing psychopathology at age 15 may not be as robust given that there is little strength in ratings of unique externalizing problems at this age. By contrast, the unique internalizing and general factors were much stronger at age 15.
Comparison to prior studies using SECCYD dataset
Prior researchers have studied internalizing and externalizing problems using the same dataset as the present study. Fanti & Heinrich (Reference Fanti and Henrich2010) examined pure and co-occurring mother-reported internalizing and externalizing problems from age 2 to 12. In their study, “pure” referred to a latent class where individuals endorsed high levels in externalizing or internalizing problems but low levels in the other. “Co-occurring” referred to a latent class where individuals endorsed high levels of both. This study found that children with a difficult temperament were more likely to be classified in a co-occurring group compared to “pure” externalizing or internalizing problems, which is consistent with our findings with general psychopathology. They also found similar results where early cognitive development difficulties were associated most strongly with pure chronic externalizing problems compared to other groups, and were less likely to be associated with pure internalizing problems (Fanti & Henrich, Reference Fanti and Henrich2010).
Cao et al. (Reference Cao, Liang and Zhou2021) examined various inhibitory control tasks, including immediacy preference, as mediators between early tobacco smoke exposure and an average composite of mother- and father-reported internalizing and externalizing problems at the earlier time point of 6th grade. They found that delay of gratification did not predict internalizing or externalizing problems in 6th grade, which differed from our results. Wang & Liu (Reference Wang and Liu2021) examined internalizing and externalizing problems at multiple timepoints of 1st, 3rd, 4th, 5th, and 6th grade from teacher ratings. Unlike our linking approach, they estimated a latent factor score for intercepts and slopes of internalizing and externalizing problems composed of scores from each grade as manifest variables. In this study, executive functioning and social competence measured at 1st grade were the predictors of interest (Wang & Liu, Reference Wang and Liu2021). The researchers found that poor executive functioning and poor social competence both predicted higher intercepts of internalizing and externalizing problems (Wang & Liu, Reference Wang and Liu2021).
In comparison to prior studies, the present study uses multidimensional linking to chart trajectories across multiple raters and uses robust methods of bifactor modeling to evaluate predictors of general and specific psychopathology. Furthermore, like Cao et al. (Reference Cao, Liang and Zhou2021) who examined “hot” and “cool” inhibitory control as risk factors, our study examines different facets of a given construct, i.e., reactive and control processes.
To our knowledge, there are two prior studies, Deutz et al. (Reference Deutz, Geeraerts, Belsky, Deković, van Baar, Prinzie and Patalay2020) and McElroy et al. (Reference McElroy, Belsky, Carragher, Fearon and Patalay2018), that examined factor analytic models of general psychopathology using the SECCYD dataset. Deutz et al. (Reference Deutz, Geeraerts, Belsky, Deković, van Baar, Prinzie and Patalay2020) found that immediacy preference did not predict separately analyzed mother- or self-reported general psychopathology after controlling for covariates, similar to our study; however, we also included father report, and we linked scores across raters. Furthermore, we examined these questions for general and specific psychopathology with bifactor modeling. McElroy et al. (Reference McElroy, Belsky, Carragher, Fearon and Patalay2018) examined phenotypic stability in the general and specific factors across all measurements of mother-reported psychopathology from ages 2 to 14, but did not examine predictors of general versus specific psychopathology.
Strengths
The present study had key strengths. First, we examined children’s internalizing and externalizing problems from multiple informants. Second, we examined a lengthy span of development in a large and diverse sample. Third, we used a robust IRT approach to developmental scaling to estimate trajectories of internalizing and externalizing psychopathology on the same scale using ratings at different ages, from numerous raters, and from different measures. Fourth, we examined aspects of reactive and control processes – negative emotionality and immediacy preference – along with demographic characteristics in association with general versus specific psychopathology. The present study provides novel and important contributions to the theory of reactive and control processes of temperament as predictors and the measurement and analysis of psychopathology across a lengthy developmental span.
Limitations
The present study also had weaknesses. First, we are unable to make causal inferences due to the observational design of the study. Although we examined negative emotionality and immediacy preference at 54 months in relation to psychopathology at age 15, we are unable to rule out the possibility of the reverse direction of effect, or the possibility of unmeasured third variables. Second, the reliability of the specific psychopathology factors was relatively low, which may have hindered our ability to detect associations with unique externalizing or internalizing problems. It will be important for future work to identify ways of assessing specific psychopathology that yield greater internal consistency.
Third, the linking approach used in the study assumes that item parameters and factor scores are linearly related across measures, raters, and periods of measurements. However, the results indicated that the approach to linking was successful across measures, raters, and timepoints (see Supplement Figures S2–S5). The linking assumption of linearity is between item parameters at adjacent ages; the linking does not assume that the change is linear across the entire age span – the changes can be larger or smaller at various ages as needed to adjust for differences. Our developmental scaling approach linked scores across the age- and rater-common items at the aggregate construct level rather than at the item level. Thus, our approach would not be well-suited for interpreting scores for any individual behavior/item across time or raters. Moreover, our approach would also not be appropriate if there is not a common factor (e.g., externalizing problems) that influences scores across the home and school contexts. Our approach may also potentially overlook meaningful qualitative differences that can occur from one year to the next (e.g., transition from elementary to middle school; Shi & Ettekal, Reference Shi and Ettekal2021).
Fourth, negative emotionality and psychopathology were both derived from the same method (i.e., informant report), whereas immediacy preference was assessed using a laboratory task. Shared method variance might inflate associations between ratings of negative emotionality and psychopathology, when compared to more context-dependent performance on an immediacy preference task (Brock et al., Reference Brock, Rimm-Kaufman and Wanless2014; Makol et al., Reference Makol, Youngstrom, Racz, Qasmieh, Glenn and De Los Reyes2020). Finally, there are potential limitations to the sample. Data collection for the NICHD SECCYD study began approximately 30 years ago, which may impact the relevance of the findings to current understanding of temperament and psychopathology. Moreover, the sample is not nationally representative in terms of race or ethnicity, thus limiting the potential generalizability of the findings to the U.S. population. However, the sample is diverse economically and geographically.
Conclusion
The findings from the present study extend previous work (Petersen & LeBeau, Reference Petersen and LeBeau2022) that has linked ratings of psychopathology from multiple informants across a wide age range. Very few studies have used developmental scaling to study the development of psychopathology across a lengthy span (Petersen et al., Reference Petersen, Lindhiem, LeBeau, Bates, Pettit, Lansford and Dodge2018; Petersen & LeBeau, Reference Petersen and LeBeau2022). The present study demonstrates a useful approach to link ratings of internalizing and externalizing psychopathology across time and across raters. The approach was valuable because it allowed us to chart children’s development in internalizing and externalizing problems across a lengthy developmental span and to examine early risk factors of their trajectories. Additionally, findings demonstrate the importance of partitioning general versus specific psychopathology.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0954579424000713.
Open access
