Introduction
Attention-deficit/hyperactivity disorder (ADHD) is consistently associated with an increased risk of several adverse health and social outcomes, including poor education achievement, risky sexual behaviours and premature mortality (Cortese et al., Reference Cortese, Faraone, Bernardi, Wang and Blanco2013; Chang et al., Reference Chang, Lichtenstein, D'Onofrio, Sjolander and Larsson2014; Dalsgaard et al., Reference Dalsgaard, Ostergaard, Leckman, Mortensen and Pedersen2015; Faraone et al., Reference Faraone, Asherson, Banaschewski, Biederman, Buitelaar, Ramos-Quiroga, Rohde, Sonuga-Barke, Tannock and Franke2015). ADHD might begin in childhood and persist throughout adulthood, or it may remit spontaneously in around half of the cases (Caye et al., Reference Caye, Spadini, Karam, Grevet, Rovaris, Bau, Rohde and Kieling2016b). Recent evidence suggested that subthreshold symptoms can get worse over time, causing the emergence of a full-blown syndrome only in adulthood (Caye et al., Reference Caye, Sibley, Swanson and Rohde2017), although the topic is still under debate in the literature (Cooper et al., Reference Cooper, Hammerton, Collishaw, Langley, Thapar, Dalsgaard, Stergiakouli, Tilling, Davey Smith, Maughan, O'Donovan, Thapar and Riglin2018; Manfro et al., Reference Manfro, Santoro, Polanczyk, Gadelha, Pan, Bressan, Brietzke, Talarico, Belangero, Rohde and Salum2018). Although some risk factors for the persistence or emergence of adult ADHD are known (Caye et al., Reference Caye, Spadini, Karam, Grevet, Rovaris, Bau, Rohde and Kieling2016b, Reference Caye, Swanson, Thapar, Sibley, Arseneault, Hechtman, Arnold, Niclasen, Moffitt and Rohdec), the attending psychiatrist is currently unable to correctly predict the course of the disorder based on clinical assessments of children or to propose a preventive intervention for those at risk.
One issue might be the inability to combine what is already known about risk factors. Although mental disorders arise from multiple risk factors, previous studies frequently define risk for targeted preventive interventions on the basis of a single risk factor, for instance, an affected first-degree relative or presence of subthreshold symptoms (Brent et al., Reference Brent, Brunwasser, Hollon, Weersing, Clarke, Dickerson, Beardslee, Gladstone, Porta, Lynch, Iyengar and Garber2015; Taylor et al., Reference Taylor, Valentine, Sellman, Bransby-Adams, Daley and Sayal2015; Buntrock et al., Reference Buntrock, Ebert, Lehr, Smit, Riper, Berking and Cuijpers2016). Meanwhile, multivariable risk scores such as the Framingham risk score for cardiovascular disease have been one of the main frameworks for the study of preventive strategies in other areas of medicine.
Our aim was to develop and validate a multivariable risk calculator that estimates the individual risk of ADHD in late adolescence/young adulthood based on childhood characteristics. ADHD lends itself easily to the development of a risk calculator for the following reasons: First, its adverse health and social consequences are well established (Asherson et al., Reference Asherson, Buitelaar, Faraone and Rohde2016). Second, it is widely accepted that its roots are in early childhood, although some argue the full syndrome might develop later in some individuals (Moffitt et al., Reference Moffitt, Houts, Asherson, Belsky, Corcoran, Hammerle, Harrington, Hogan, Meier, Polanczyk, Poulton, Ramrakha, Sugden, Williams, Rohde and Caspi2015; Agnew-Blais et al., Reference Agnew-Blais, Polanczyk, Danese, Wertz, Moffitt and Arseneault2016; Caye et al., Reference Caye, Rocha, Anselmi, Murray, Menezes, Barros, Goncalves, Wehrmeister, Jensen, Steinhausen, Swanson, Kieling and Rohde2016a). Third, being a neurodevelopmental disorder, early intervention has the potential to change brain development and improve later clinical outcomes (Shaw et al., Reference Shaw, Lerch, Greenstein, Sharp, Clasen, Evans, Giedd, Castellanos and Rapoport2006). Fourth, there is substantive evidence to support a priori hypotheses about specific childhood risk factors (Caye et al., Reference Caye, Spadini, Karam, Grevet, Rovaris, Bau, Rohde and Kieling2016b).
Method
Our methods follow well-established probability models in medicine and recommendations of the Transparent Reporting of a multivariable prediction model for Individual Prognosis Or Diagnosis (TRIPOD) statement (Collins et al., Reference Collins, Reitsma, Altman and Moons2015). We developed the predictive model in one a priori selected sample and validated it independently in three external samples (TRIPOD analysis type 3). We selected the Avon Longitudinal Study of Parents and Children (ALSPAC) cohort as the generating sample based on the following a priori defined criteria: population-based sample, largest sample.
Samples and participants
ALSPAC
The ALSPAC is a prospective birth cohort study in the UK. Pregnant women with expected delivery dates between 1 April 1991 and 31 December 1992 were invited to participate. Ethical approval for the study was obtained from the ALSPAC Ethics and Law Committee and the Local Research Ethics Committees. Further details on assessments can be found elsewhere (Boyd et al., Reference Boyd, Golding, Macleod, Lawlor, Fraser, Henderson, Molloy, Ness, Ring and Davey Smith2013). Please note that the study website contains details of all the data that are available through a fully searchable data dictionary (http://www.bris.ac.uk/alspac/researchers/data-access/data-dictionary/). For the current study, we included 5113 subjects that were assessed for ADHD in childhood (age 7 or 10) and in the last available assessment (age 17).
E-Risk
The Environmental Risk (E-Risk) Longitudinal Twin Study is a prospective birth cohort study designed to represent the UK population. In 1999–2000, investigators enrolled 1116 families with same-sex 5-year-old twins (N = 2232) born from 1 January 1994 to 4 December 1995 (Moffitt and Team, Reference Moffitt and Team2002). The study was approved by the Joint South London and Maudsley and the Institute of Psychiatry Research Ethics Committee, and parents gave informed written consent. Further details can be found elsewhere (Moffitt and Team, Reference Moffitt and Team2002). For the analyses, we included 2040 subjects with data on ADHD in childhood (ages 5, 7, 10 or 12) and in young adulthood (age 18).
Pelotas 1993
The 1993 Pelotas Birth Cohort is a prospective longitudinal birth cohort set in Brazil. In 1993, mothers of all children born in the city of Pelotas were contacted and 5249 children were enrolled. The study was approved by the institutional review board of the Federal University of Pelotas, and participants provided written informed consent. Further information on the cohort design can be found elsewhere (Goncalves et al., Reference Goncalves, Assuncao, Wehrmeister, Oliveira, Barros, Victora, Hallal and Menezes2014). For the current study, we included 4039 participants that had complete ADHD assessment at age 18–19 years old.
MTA
The Multimodal Treatment Study of Children with ADHD (MTA) is the largest clinical trial and observational follow-up conducted with children with ADHD. In the first phase of the study, investigators enrolled 579 children aged 7–10 years old with ADHD and assigned them to 14 months of one of four groups of management. Two years after baseline, 515 consented to enter an observational follow-up and a local normative comparison group of 289 classmates (258 without ADHD) was added. Assessments were conducted at 12, 14 and 16 years after baseline. Informed consent (parental permission and child assent) was obtained for all participating families, using forms approved by both local institutional review boards and the NIH. Detailed design and methods have been presented in previous publications (1999). We included 717 subjects with any complete ADHD assessment in young adulthood (mean age 24).
Assessment and definition of the outcome variable
In each sample, the outcome was a dichotomous ADHD definition in late adolescence or young adulthood. In ALSPAC, participants' parents completed the hyperactive subscale of the Strengths and Difficulties Questionnaire (SDQ-HS) at 17 years of age. The scale showed excellent discrimination against a DSM-IV diagnosis derived from the Development and Well-Being Assessment (DAWBA) conducted in a subsample of 1673 participants (area under the curve (AUC) = 0.89, 95% CI 0.81–0.96). The best cut-off score to define diagnosis was at least six points on the SDQ-HS (sensitivity = 83.3% and specificity = 83.3%). In the E-Risk, ADHD was ascertained at age 18 years using structured interviews based on full DSM-5 criteria (Agnew-Blais et al., Reference Agnew-Blais, Polanczyk, Danese, Wertz, Moffitt and Arseneault2016). In the MTA sample, ADHD symptoms were derived from the parents’ Conners Adult ADHD Rating Scale (CAARS). At least five DSM-5 symptoms of inattention and/or hyperactivity were required for the symptom criteria. Impairment was evaluated with the Impairment Rating Scale (IRS), which has strong psychometrics and accurately identifies impairment in adults with ADHD (Sibley et al., Reference Sibley, Pelham, Molina, Gnagy, Waxmonsky, Waschbusch, Derefinko, Wymbs, Garefino, Babinski and Kuriyan2012). This diagnostic approach was chosen because it has better diagnostic accuracy than a semi-structured interview in this sample (Sibley et al., Reference Sibley, Swanson, Arnold, Hechtman, Owens, Stehli, Abikoff, Hinshaw, Molina, Mitchell, Jensen, Howard, Lakes and Pelham2017b). In the Pelotas cohort, trained psychologists interviewed the participants at 18–19 years old with a structured interview for ADHD based on DSM-5 criteria (Caye et al., Reference Caye, Rocha, Anselmi, Murray, Menezes, Barros, Goncalves, Wehrmeister, Jensen, Steinhausen, Swanson, Kieling and Rohde2016a). A strict age-at-onset criterion was not required to define ADHD in young adulthood to take into account recent evidence suggesting a significant prevalence of late-onset ADHD presentation (Moffitt et al., Reference Moffitt, Houts, Asherson, Belsky, Corcoran, Hammerle, Harrington, Hogan, Meier, Polanczyk, Poulton, Ramrakha, Sugden, Williams, Rohde and Caspi2015; Agnew-Blais et al., Reference Agnew-Blais, Polanczyk, Danese, Wertz, Moffitt and Arseneault2016; Caye et al., Reference Caye, Rocha, Anselmi, Murray, Menezes, Barros, Goncalves, Wehrmeister, Jensen, Steinhausen, Swanson, Kieling and Rohde2016a).
Assessment and definition of predictor variables
We selected the following predictor variables assessed in childhood: female sex, socioeconomic status, mother's depression, intelligence quotient, maltreatment, ADHD symptoms, depressive symptoms, oppositional defiant behaviour and conduct disorders, and single-parent family. All predictors were collected before age 12, with the exception of intelligence in Pelotas, which was measured at age 18. Their selection was based on the extensive review of previous reports in the literature and a meta-analysis conducted by our group (Moffitt et al., Reference Moffitt, Houts, Asherson, Belsky, Corcoran, Hammerle, Harrington, Hogan, Meier, Polanczyk, Poulton, Ramrakha, Sugden, Williams, Rohde and Caspi2015; Agnew-Blais et al., Reference Agnew-Blais, Polanczyk, Danese, Wertz, Moffitt and Arseneault2016; Caye et al., Reference Caye, Rocha, Anselmi, Murray, Menezes, Barros, Goncalves, Wehrmeister, Jensen, Steinhausen, Swanson, Kieling and Rohde2016a, Reference Caye, Spadini, Karam, Grevet, Rovaris, Bau, Rohde and Kielingb). We have included all variables that were available across the four samples with some level of comparability, without performing univariate analysis or stepwise techniques for variable selection. Definition of predictors was defined a priori according to relevant literature in the field. Further details are provided in online eTable 1.
Statistical analysis
When developing a predictive model in multiple samples, a recommended approach consists in selecting and tuning the best model in one a priori selected sample and assessing its performance in the remaining independent samples for external validity. Because the evaluation of internal performance within the same sample where the model was derived is affected by overfitting, internal validation optimism correction should be performed. Among the most accepted techniques for internal validation is bootstrap resampling.
We have developed the predictive model in the ALSPAC cohort. We ran a logistic regression including outcome (ADHD at last assessment) as the dependent variable and all eligible predictor variables as covariates. We inspected linearity assumptions of continuous variables by plotting the predictor and the logit of the outcome, and trough Box–Tidwell regressions. We derived the model using linear splines of equal sample sizes (with knots at 25th, 50th and 75th percentiles) in the ADHD symptom variable, and this model had better fit indices (AIC, BIC). Multiple imputation with chained equations (ten imputations) using the remaining predictors was used to deal with missing values in the predictor variables. We used a fixed number of ten iterations and assessed convergence with trace plots. In the ALSPAC cohort, for each of the 1000 bootstrap resamples, we have performed pooled regression coefficient estimates and variance across imputations with the command mi estimate in Stata (Rubin, Reference Rubin1987). We evaluated the predictive discrimination of the probability model calculating the area under the receiver operating characteristic curve (c-statistic) of the estimated probability against the actual outcome as an index of model performance. We have assessed optimism of internal validation with bootstrap inference using 1000 replications with the R package rms (Harrell et al., Reference Harrell, Lee and Mark1996). We have assessed internal and external model calibration with calibration curves, plotting predicted probabilities against observed frequencies. Extreme predictions at the right end of the distribution (highest risk) including <1% of the sample at risk were excluded of the calibration analyses to avoid instability of the estimates, and these ranges are not shown in each graph. Multiple imputation and model generation were conducted in Stata MP 13.0. Finally, we tested the predictive discrimination of the same predictors using Machine Learning approaches with the R package caret (see eMethods).
We performed several sensitivity analyses to assess the robustness of our findings. We analysed the performance (measured by the c-statistic) of the model among individuals who endorsed a very low number of ADHD symptoms at baseline (operationalised as equal or below the median of each population) in ALSPAC, E-Risk and Pelotas samples. We had also analysed the performance (measured by the c-statistic) of the model excluding one variable at each time. Finally, we present the variation of the predicted probability within fixed levels of ADHD symptoms to assess the contribution of the remaining variables to the model.
Results
The number of participants with a dichotomous definition of adult ADHD and the frequency of childhood predictors in each sample can be found in Table 1.
Frequency of young adulthood ADHD and of childhood predictors across the four samples
ADHD, attention-deficit hyperactivity disorder; ODD, oppositional defiant disorder; CD, conduct disorder; SD, standard deviation; IQR, interquartile range; IQ, intelligence quotient.
a Definition of lifetime depression of the mother was designed to be very sensitive, either by multiple assessments and/or by applying a very low threshold (further details on Table S1 of Supplementary material).
b ALSPAC: number of DSM-IV depressive items endorsed. E-Risk, MTA: Children's Depressive Inventory (CDI) score. Pelotas: emotional subscale score of the SDQ.
c ALSPAC, E-Risk, MTA: number of DSM-IV ADHD items endorsed. Pelotas: hyperactivity subscale score of the SDQ.
Note: reported values before multiple imputation. Because each factor may have missing values, we report total number of participants and a proportion where the denominator is the total number of valid subjects.
Performance of the predictive model in the generating sample
All variables entered in the probabilistic model were used for the calculation of the estimated risk of the individual (Table 2). Only ADHD symptoms were corrected with splines. The predictive model discriminated between adult ADHD v. no adult ADHD with an AUC of 0.82 (Bootstrap-corrected 95% CI 0.80–0.83, p < 0.001), which indicates very good discrimination (Fig. 1). Correction for optimism with bootstrapping yielded an AUC of 0.81. The calibration plot showed that the predicted probability and the observed frequency of adult ADHD closely agreed throughout the entire range of risk (0 to around 50%, Fig. 2). The bias-corrected calibration curve was nearly identical (eFig. 1). The AUC varied within a range of 0.74–0.82 in sensitivity analyses taking out one predictor at a time (eTable 2 in Supplementary material). Proposed probability cut-offs are presented with sensitivity, specificity, positive predictive value and negative predictive value in eTable 3 in Supplementary material.
The probability model in the generating sample (n = 5113)
OR, odds ratio; ODD, oppositional defiant disorder; CD, conduct disorder; ADHD, attention-deficit hyperactivity disorder; BC, Bootstrap-corrected.
a We report the OR for a ten-point change in the intelligence quotient scale.
b Due to the OR of 1.00 for depressive symptoms, we have omitted this variable from the online calculator.
Receiver operating characteristic curves in each each cohort plotting Sensitivity and 1-Specificity for the predicted probabilities generated by the risk calculator against adult ADHD as the classificatory variable.
Calibration curves in each cohort plotting the predicted probabilities generated by the risk calculator (x-axis) against observed adult ADHD frequency (y-axis). Dashed diagonal line represents perfect calibration.
Performance of the predictive model in a validating cohort sample in the same country
In the E-Risk study, the predictive model discriminated between adult ADHD v. no adult ADHD with an AUC of 0.75 (Bootstrap-corrected 95% CI 0.71–0.78, p < 0.001), which indicates fair discrimination (Fig. 1). The calibration plot showed reasonable agreement between predicted and observed event frequencies, especially in the lower range of risk (Fig. 2). The discrimination was the same when restricting the sample to randomly selected non-siblings (eTable 4 in Supplementary material).
Performance of the predictive model in a validating sample in a middle-income country
In the Pelotas cohort, the predictive model discriminated between adult ADHD vs. no adult ADHD with an AUC of 0.57 (Bootstrap-corrected 95% CI 0.55–0.60, p < 0.001), which indicates poor discrimination (Fig. 1). There was a low agreement between estimated probability and observed frequency of the outcome (Fig. 2).
Performance of the predictive model in a validating clinical sample in a country with similar income
In the MTA, the predictive model discriminated between adult ADHD v. no adult ADHD with an AUC of 0.76 (Bootstrap-corrected 95% CI 0.73–0.80, p < 0.001) (Fig. 1). The calibration plot showed that predicted probability and observed frequency of adult ADHD closely agreed throughout the entire range of risk (0 to around 70%, Fig. 2), although the model had underestimated event frequency consistently.
Performance of the predictive model within participants with very low endorsement of ADHD symptoms in childhood
We tested the performance of the model for predicting late-onset ADHD in population samples, among only participants that endorsed few ADHD symptoms in childhood – the median or lower number of symptoms in their respective populations. The model had fair discrimination in these subgroups, except for the Pelotas sample in which the model already had poor discrimination (Table 3).
Performance of the score for individuals with very low ADHD childhood symptoms
BC, Bootstrap-corrected.
ROC analyses were done only in participants with low endorsement of ADHD symptoms in childhood. Low endorsement was defined as a median number of symptoms or below the median of their respective population (ALSPAC: two or less ADHD symptoms; E-Risk: one or zero ADHD symptoms; Pelotas: the median or less than median (4) in the hyperactivity subscale of the SDQ).
Performance of the predictive model removing one predictor at a time
We tested the model taking out one predictor at a time (eTable 2). The most relevant individual predictor was the level of ADHD symptoms in childhood. However, the model still had fair performance in the model without ADHD symptoms in childhood, with an AUC of 0.74 (95% CI 0.72–0.76, p < 0.001).
Variation of the predicted probability within fixed levels of ADHD symptoms
We assessed the predicted probabilities of an adult ADHD diagnosis at any fixed level of ADHD symptoms, considering a maximum variation of the remaining factors (see eFig. 2). The observed variance indicates that ADHD symptoms are not the only relevant predictive factor in the model. These findings analysed together clearly indicate that this is not a model based on just one variable.
Specificity of the predictive model in predicting ADHD
Considering that E-risk is the population cohort with the most comprehensive assessment of comorbid mental disorders, we tested the model's discrimination predicting adult anxiety and major depressive disorder. The performance was significantly lower than for ADHD, showing specificity for ADHD compared to other forms of adult psychopathology (eTable 5 in Supplementary material).
Risk calculator and robustness of findings
Predictive discrimination estimates using three different Machine Learning approaches were almost the same (see eTable 6 in Supplementary material). In a secondary analysis, we also have developed one comprehensive predictive model with all samples at once, using site as one more predictor variable (see eTable 7; eFig. 3). A risk calculator can be found at http://www.ufrgs.br/prodah/adhd-calculator/.
Discussion
The widespread use of tools that predict clinical outcomes in medical practice has promoted the development and testing of preventive interventions, but this approach has been rarely attempted for mental health (Bitton and Gaziano, Reference Bitton and Gaziano2010). We generated a probability model to predict adult ADHD in a large birth cohort in the UK, with very good discrimination – AUC of 0.81 after optimism correction – and calibration. This performance is compared to the most used clinical tools in medicine (Morrow et al., Reference Morrow, Antman, Charlesworth, Cairns, Murphy, de Lemos, Giugliano, McCabe and Braunwald2000). Recent attempts for mental health reported risk scores with good calibration (Fusar-Poli et al., Reference Fusar-Poli, Rutigliano, Stahl, Davies, Bonoldi, Reilly and McGuire2017; Hafeman et al., Reference Hafeman, Merranko, Goldstein, Axelson, Goldstein, Monk, Hickey, Sakolsky, Diler, Iyengar, Brent, Kupfer, Kattan and Birmaher2017). These studies lacked, however, a consistent external validation with completely independent samples.
Our next step was to validate the score in independent samples. First, we tested the score in another UK birth cohort, the E-Risk. Its performance in predicting adult ADHD was similar. This is an important finding because several risk models in mental health did not replicate well even in samples from similar settings (Kivipelto et al., Reference Kivipelto, Ngandu, Laatikainen, Winblad, Soininen and Tuomilehto2006; Anstey et al., Reference Anstey, Cherbuin, Herath, Qiu, Kuller, Lopez, Wilson and Fratiglioni2014). Since data generated in population samples frequently do not translate to clinical samples (Weissman et al., Reference Weissman, Brown and Talati2011), we tested the performance of the score in the MTA study, the largest clinical trial ever conducted for ADHD. As for ALSPAC and E-risk, the score worked well with good discrimination and calibration.
We then tested the score in a third birth cohort from Brazil. We observed that the score was much less accurate with an AUC of 0.57. This finding is not surprising, since previous evidence suggests that the predictive discrimination of risk tools is lower in diverse sociocultural and ethnic populations (Chia et al., Reference Chia, Gray, Ching, Lim and Chinna2015). However, since predictor factors assessment in Pelotas was the most heterogeneous, observed low discrimination might have been an effect of measurement error.
Models that predict a diagnosis of chronic disorders often include premorbid signs and symptoms of the disease as predictive factors. For example, the factor that increased discrimination the most in the recently published calculator for psychosis was the index diagnosis when presenting to secondary care, where psychotic disorders had the greatest weight compared to other disorders such as mood disorders (Fusar-Poli et al., Reference Fusar-Poli, Rutigliano, Stahl, Davies, Bonoldi, Reilly and McGuire2017). Although this is a valid approach, other variables must also add to prediction, otherwise models would be tautological. Therefore, we also validated the score in subjects with a low endorsement of ADHD symptoms in childhood. The performance was good even in this sensitivity analysis. In addition, we assessed the probabilities of an adult ADHD diagnosis at any fixed level of ADHD symptoms, allowing maximum variation of the remaining factors. Finally, we checked discrimination of the model removing each factor at once. Findings suggested that although ADHD symptoms are the most important overall predictor, the complete model works as a necessary refinement and a model without ADHD symptoms has good discrimination as well.
We also conducted other secondary analyses to assess the robustness of our findings. We tested the impact of using other statistical methods on our results. We observed that the discrimination of the prediction models remained stable regardless of chosen statistical methods. Finally, we tested the hypothesis of whether the score was specific for the prediction of ADHD. This is an important proof-of-concept: personalised medicine has always been a challenge for the area of psychiatry, as it has been shown consistently that most identified biomarkers and risk factors associated with one mental disorder are also associated with several others (Cross-Disorder Group of the Psychiatric Genomics C et al., Reference Lee, Ripke, Neale, Faraone, Purcell, Perlis, Mowry, Thapar, Goddard, Witte, Absher, Agartz, Akil, Amin, Andreassen, Anjorin, Anney, Anttila, Arking, Asherson, Azevedo, Backlund, Badner, Bailey, Banaschewski, Barchas, Barnes, Barrett, Bass, Battaglia, Bauer, Bayes, Bellivier, Bergen, Berrettini, Betancur, Bettecken, Biederman, Binder, Black, Blackwood, Bloss, Boehnke, Boomsma, Breen, Breuer, Bruggeman, Cormican, Buccola, Buitelaar, Bunney, Buxbaum, Byerley, Byrne, Caesar, Cahn, Cantor, Casas, Chakravarti, Chambert, Choudhury, Cichon, Cloninger, Collier, Cook, Coon, Cormand, Corvin, Coryell, Craig, Craig, Crosbie, Cuccaro, Curtis, Czamara, Datta, Dawson, Day, De Geus, Degenhardt, Djurovic, Donohoe, Doyle, Duan, Dudbridge, Duketis, Ebstein, Edenberg, Elia, Ennis, Etain, Fanous, Farmer, Ferrier, Flickinger, Fombonne, Foroud, Frank, Franke, Fraser, Freedman, Freimer, Freitag, Friedl, Frisen, Gallagher, Gejman, Georgieva, Gershon, Geschwind, Giegling, Gill, Gordon, Gordon-Smith, Green, Greenwood, Grice, Gross, Grozeva, Guan, Gurling, De Haan, Haines, Hakonarson, Hallmayer, Hamilton, Hamshere, Hansen, Hartmann, Hautzinger, Heath, Henders, Herms, Hickie, Hipolito, Hoefels, Holmans, Holsboer, Hoogendijk, Hottenga, Hultman, Hus, Ingason, Ising, Jamain, Jones, Jones, Jones, Tzeng, Kahler, Kahn, Kandaswamy, Keller, Kennedy, Kenny, Kent, Kim, Kirov, Klauck, Klei, Knowles, Kohli, Koller, Konte, Korszun, Krabbendam, Krasucki, Kuntsi, Kwan, Landen, Langstrom, Lathrop, Lawrence, Lawson, Leboyer, Ledbetter, Lee, Lencz, Lesch, Levinson, Lewis, Li, Lichtenstein, Lieberman, Lin, Linszen, Liu, Lohoff, Loo, Lord, Lowe, Lucae, MacIntyre, Madden, Maestrini, Magnusson, Mahon, Maier, Malhotra, Mane, Martin, Martin, Mattheisen, Matthews, Mattingsdal, McCarroll, McGhee, McGough, McGrath, McGuffin, McInnis, McIntosh, McKinney, McLean, McMahon, McMahon, McQuillin, Medeiros, Medland, Meier, Melle, Meng, Meyer, Middeldorp, Middleton, Milanova, Miranda, Monaco, Montgomery, Moran, Moreno-De-Luca, Morken, Morris, Morrow, Moskvina, Muglia, Muhleisen, Muir, Muller-Myhsok, Murtha, Myers, Myin-Germeys, Neale, Nelson, Nievergelt, Nikolov, Nimgaonkar, Nolen, Nothen, Nurnberger, Nwulia, Nyholt, O'Dushlaine, Oades, Olincy, Oliveira, Olsen, Ophoff, Osby, Owen, Palotie, Parr, Paterson, Pato, Pato, Penninx, Pergadia, Pericak-Vance, Pickard, Pimm, Piven, Posthuma, Potash, Poustka, Propping, Puri, Quested, Quinn, Ramos-Quiroga, Rasmussen, Raychaudhuri, Rehnstrom, Reif, Ribases, Rice, Rietschel, Roeder, Roeyers, Rossin, Rothenberger, Rouleau, Ruderfer, Rujescu, Sanders, Sanders, Santangelo, Sergeant, Schachar, Schalling, Schatzberg, Scheftner, Schellenberg, Scherer, Schork, Schulze, Schumacher, Schwarz, Scolnick, Scott, Shi, Shilling, Shyn, Silverman, Slager, Smalley, Smit, Smith, Sonuga-Barke, St Clair, State, Steffens, Steinhausen, Strauss, Strohmaier, Stroup, Sutcliffe, Szatmari, Szelinger, Thirumalai, Thompson, Todorov, Tozzi, Treutlein, Uhr, van den Oord, Van Grootheest, Van Os, Vicente, Vieland, Vincent, Visscher, Walsh, Wassink, Watson, Weissman, Werge, Wienker, Wijsman, Willemsen, Williams, Willsey, Witt, Xu, Young, Yu, Zammit, Zandi, Zhang, Zitman, Zollner, Devlin, Kelsoe, Sklar, Daly, O'Donovan, Craddock, Sullivan, Smoller, Kendler and Wray2013). We observed that the score was specific for ADHD, not predicting major depressive disorder or anxiety disorders.
Previous cohort investigations included in the present study did not find a significant childhood DSM dichotomous ADHD diagnosis in the trajectory of late-onset ADHD (Agnew-Blais et al., Reference Agnew-Blais, Polanczyk, Danese, Wertz, Moffitt and Arseneault2016; Caye et al., Reference Caye, Rocha, Anselmi, Murray, Menezes, Barros, Goncalves, Wehrmeister, Jensen, Steinhausen, Swanson, Kieling and Rohde2016a). Thus, it might seem surprising that childhood ADHD symptoms predict adult ADHD. The MTA report also highlighted the importance of child ADHD subthreshold symptoms in adult ADHD in cases where formal DSM diagnosis was not found in childhood (Sibley et al., Reference Sibley, Rohde, Swanson, Hechtman, Molina, Mitchell, Arnold, Caye, Kennedy, Roy and Stehli2017a). Since this approach was not the main focus of previous cohort studies (ADHD subthreshold symptoms), this might explain why childhood ADHD symptoms predict adult ADHD even in cohorts where childhood dichotomous diagnosis was not relevant for adult ADHD.
Our findings should be interpreted considering a set of limitations. First, the design and assessments of different samples were not uniform, limiting the discrimination of the score in the validating samples. Adult ADHD, for instance, was measured with a scale rather than with a structured interview in the generating sample, but not in the validating samples. It is possible, therefore, that the proposed estimated predictive discrimination in validating samples might actually be an underestimation. Further validating efforts with assessments that more closely resemble those of the generating sample might observe higher AUCs. However, this could also be seen as the strength of the study, since observed discrimination indices are considered good, even with different methodologies implemented in individual studies. Second, there was attrition in the generating sample's assessments. Nevertheless, potential selection bias does not appear to affect the prediction of outcomes in this cohort, as shown in previous publications (Boyd et al., Reference Boyd, Golding, Macleod, Lawlor, Fraser, Henderson, Molloy, Ness, Ring and Davey Smith2013). Also, we have used multiple imputation techniques to deal with missing values. Third, the observed positive predictive value in selected cut-offs reaches a maximum of 61.8%, while the negative predictive value is much higher throughout prediction. Although this might be considered insufficient, we ought to remember that the positive predictive value depends much on the prevalence of the studied condition, and we are working with population-based samples where the base rate of the condition is low. As a comparison, the Framingham risk score, which is also a tool developed in the general population, yields a positive predictive value of up to 30–40%. The risk score for bipolar disorder reports a positive predictive value of up to 32%, even among offspring of bipolar patients (a high-risk sample). Fifth, it is important to note that other variables that are related to ADHD could have been a part of the risk score like prematurity and ADHD in first-degree relatives. However, they were not available for testing in the four data sets and our guide for risk factors was evidence-based guided by a previous meta-analysis (Caye et al., Reference Caye, Spadini, Karam, Grevet, Rovaris, Bau, Rohde and Kieling2016b). Accordingly, the predicted probability provided by the model should be considered an estimate probability obtained with a pre-specified set of variables.
What is the clinical utility of this score, provided that previous literature already has shown that most variables included in our model that are non-specific risk factors for mental disorders and ADHD symptoms in childhood, as expected, are key predicted risk factors for adult ADHD? No the previous effort combined all these variables in a single risk calculator. Therefore, the only information that clinicians could offer was that some variables, like comorbidity with CD/ODD in childhood, increase the risk of persistence of ADHD. By using this calculator, attending clinicians can identify high-risk individuals to inform parents and guide decisions.
Thus, we propose a multivariable risk model to predict ADHD in young adulthood based on childhood factors that have good discrimination in both population and clinical settings. Clinicians can use the model to guide long-term decisions based on the identification of children at high risk for future adult ADHD diagnosis. Also, it provides a framework for testing the effectiveness of preventive interventions focused on high-risk individuals. Furthermore, the score might be used to identify at-risk individuals for investigating neurobiological features including brain development. The lower discrimination observed in a middle-income country urges the discussion of how globally generalisable are the risk models that are currently being widely used in clinical practice. Indeed, even the well-established Framingham cardiovascular risk model is being subjected to criticism for its wide variation in performance across different populations. Therefore, future attempts to improve the current model should include setting-specific recalibration analyses that should then be translated to specific risk calculators to be used across different settings. Also, we suggest that cohorts use more standardised methods of collection of predictors and outcomes in psychiatry for the study of risk factors, so that we can disentangle whether failure to replicate is due to the heterogeneity of methods or population. Hence, our work adds to the need for validation of risk models in low- and middle-income countries.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S2045796019000283
Availability of data and material
Due to constraints on the data sharing permissions of the samples included in this study, we are not allowed to share the data for public use.
Acknowledgements
We are extremely grateful to all the families who took part in this study, the midwives for their help in recruiting them, and the whole ALSPAC team, which includes interviewers, computer and laboratory technicians, clerical workers, research scientists, volunteers, managers, receptionists and nurses.
Contributions
All authors equally contributed to the conceptual design and planning the analyses of the current study. AC, TBM and ICP analysed the data and the remaining authors interpreted and supervised the analyses. AC wrote the first draft, and the remaining authors revised until the final version of the manuscript was submitted. LAR coordinated the work and was the main supervisor of all the steps of this work.
Financial support
ALSPAC: The UK Medical Research Council and Wellcome (grant ref: 102215/2/13/2) and the University of Bristol provide core support for ALSPAC. E-Risk: The Environmental Risk (E-Risk) Longitudinal Twin Study is funded by grant G1002190 from the United Kingdom Medical Research Council UKMRC. Additional support was provided by grant HD077482 from the National Institute of Child Health and Human Development and the Jacobs Foundation. Pelotas 1993: funded by the National Council for Scientific and Technological Development (CNPq, Brazil) and the Hospital de Clinicas de Porto Alegre (HCPA), Porto Alegre, Brazil. This article is based on the data from the study ‘Pelotas Birth Cohort, 1993’ conducted by Postgraduate Program in Epidemiology at Universidade Federal de Pelotas with the collaboration of the Brazilian Public Health Association (ABRASCO). MTA: The Multimodal Treatment Study of Children with ADHD (MTA) was an NIMH cooperative agreement randomised clinical trial, continued under an NIMH contract as a follow-up study and finally under a NIDA contract.
Conflict of interest
James M. Swanson acknowledges research support, advisory board membership, speaker's bureau membership and/or consulting for Alza, Richwood, Shire, Celgene, Novartis, Celltech, Gliatech, Cephalon, Watson, CIBA, UCB, Jansen-Cilag, McNeil and Eli-Lilly. Christian Kieling receives authorship royalties from ArtMed and Manole. Luis A. Rohde has received Honoraria, has been on the speakers' bureau/advisory board and/or has acted as a consultant for Eli-Lilly, Janssen-Cilag, Novartis and Shire in the last 3 years. He receives authorship royalties from Oxford Press and ArtMed. He also received travel awards for taking part of 2014 APA and 2015 WFADHD meetings from Shire. The ADHD and Juvenile Bipolar Disorder Outpatient Programs chaired by him received unrestricted educational and research support from the following pharmaceutical companies in the last 3 years: Eli-Lilly, Janssen-Cilag, Novartis and Shire. Other authors report no conflict of interest.
Open access
