Introduction
Students’ mental health has attracted much more research after COVID-19-related lockdowns than ever before (Arsandaux et al., Reference Arsandaux, Montagni, Macalli, Texier, Pouriel, Germain, Mebarki, Kinouani, Tournier, Schuck and Tzourio2021; Bennett et al., Reference Bennett, Heron, Gunnell, Purdy and Linton2022; Gestsdottir et al., Reference Gestsdottir, Gisladottir, Stefansdottir, Johannsson, Jakobsdottir and Rognvaldsdottir2021; Hernández-Torrano et al., Reference Hernández-Torrano, Ibrayeva, Sparks, Lim, Clementi, Almukhambetova, Nurtayev and Muratkyzy2020; Nuñez et al., Reference Nuñez, Pallasch and Radtke2022; Robinson et al., Reference Robinson, Sutin, Daly and Jones2022; Villani et al., Reference Villani, Pastorino, Molinari, Anelli, Ricciardi, Graffigna and Boccia2021; Voltmer et al., Reference Voltmer, Köslich-Strumann, Walther, Kasem, Obst and Kötter2021; Yamamoto et al., Reference Yamamoto, Uchiumi, Suzuki, Sugaya, Murillo-Rodriguez, Machado, Imperatori and Budde2022). Prospective studies showed that young adults (generally defined as people aged between 18 and 25 years) enroled in university were affected by symptoms of poorer mental health than other working peers or adults well before restrictive measures were implemented (Blanco et al., Reference Blanco, Okuda, Wright, Hasin, Grant, Liu and Olfson2008; Granieri et al., Reference Granieri, Franzoi and Chung2021); in fact, as previously reported, higher education is a risk factor for depressive symptoms, anxiety symptoms and suicidal thoughts (Karyotaki et al., Reference Karyotaki, Cuijpers, Albor, Alonso, Auerbach, Bantjes, Bruffaerts, Ebert, Hasking, Kiekens, Lee, McLafferty, Mak, Mortier, Sampson, Stein, Vilagut and Kessler2020). Furthermore, young adulthood is also a crucial transition phase, accompanied by the pursuit of self-efficacy and economic independence, higher education and new social needs. The Global Burden of Disease Study also evidenced that disability-adjusted life-years are at their highest in young adulthood (GBD 2019 Mental Disorders Collaborators, 2022), in line with epidemiological findings indicating this period of life as the typical age of onset of most mental disorders (Solmi et al., Reference Solmi, Radua, Olivola, Croce, Soardo, Salazar de Pablo, Il Shin, Kirkbride, Jones, Kim, Kim, Carvalho, Seeman, Correll and Fusar-Poli2022). To date, research on student mental health found that university support services, when in place, probably had little impact on the well-being of the general student population (Barnett et al., Reference Barnett, Arundell, Saunders, Matthews and Pilling2021; Eisenberg et al., Reference Eisenberg, Hunt, Speer and Zivin2011) due to various reasons, namely, low rates of the number of counsellors per student (generating long waitlists – (Blanco et al., Reference Blanco, Okuda, Wright, Hasin, Grant, Liu and Olfson2008; Cohen et al., Reference Cohen, Graham and Lattie2022; Lueck and Poe, Reference Lueck and Poe2021)), high levels of personal stigma hindering help-seeking behaviours (Eisenberg et al., Reference Eisenberg, Downs, Golberstein and Zivin2009) and few systemic interventions to address the determinants of (poor) mental health and prompt an early intervention to assess and treat the symptoms before they develop into a full-blown mental disorder (Upsher et al., Reference Upsher, Nobili, Hughes and Byrom2022) or treat effectively the students with chronic symptoms of mental disorders (Zivin et al., Reference Zivin, Eisenberg, Gollust and Golberstein2009).
Although essential to tailor population-specific psychological interventions (Barnett et al., Reference Barnett, Arundell, Saunders, Matthews and Pilling2021), little prospective evidence has been published regarding the prediction of mental health problems in university students (Ebert et al., Reference Ebert, Buntrock, Mortier, Auerbach, Weisel, Kessler, Cuijpers, Green, Kiekens, Nock, Demyttenaere and Bruffaerts2019; Suldo et al., Reference Suldo, Thalji and Ferron2011; Tyssen et al., Reference Tyssen, Vaglum, Grønvold and Ekeberg2008). A thorough assessment of student population needs and their psychological symptoms (Kitzrow, Reference Kitzrow2009) – leading to faster and more reliable diagnosis work-ups – is the first step for a pragmatic approach to design and evaluate the efficacy of specific mental health interventions. However, some studies investigated students’ mental health through variables strictly related to generalized anxiety disorder and depressive disorders, with little indulging in assessing demographic or other determinants of psychological well-being (Sheldon et al., Reference Sheldon, Simmonds-Buckley, Bone, Mascarenhas, Chan, Wincott, Gleeson, Sow, Hind and Barkham2021).
To provide better services, prospective studies that evaluate risk factors at baseline (ideally at university/college enrolment) would be helpful to predict those students expected to need help in the short, medium and long term. Several contributions regarding the role of the COVID-19 pandemic evidenced (Meda et al., Reference Meda, Pardini, Slongo, Bodini, Zordan, Rigobello, Visioli and Novara2021; Nomura et al., Reference Nomura, Yamazaki, Maeda, Hirayama, Ono, Fushimi, Mishima and Yamamoto2022; Weber et al., Reference Weber, Schulze, Bolzenkötter, Niemeyer and Renneberg2022) that such studies can be (i) conducted with little or no specific funding and (ii) they can help better understand the mental disorder development trajectory in a population at high risk of distress and maladjustment.
The present work is part of a larger prospective study named ECOS (vide infra) that evaluates the depressive, anxiety, eating and obsessive–compulsive symptoms of students and their determinants. Here, we report prospective data collected at enrolment and after 6 months of hundreds of Italian university students who completed several questionnaires investigating mental health problems. The objectives of this study were to establish the percentage of students suffering from severe symptoms common to depression, anxiety, obsessive–compulsive disorder and suicidal ideation at baseline; to elucidate the sociodemographic variables (e.g., gender, financial situation and field of study (Berger et al., Reference Berger, Franke, Hofmann, Sperth and Holm-Hadulla2015; Volpe et al., Reference Volpe, Ventriglio, Bellomo, Kadhum, Lewis, Molodynski, Sampogna and Fiorillo2019)) associated with poor mental health at baseline and to evaluate the risk factors (e.g., depressive symptoms at enrolment) that dent the chances of symptoms improvement or otherwise make symptom aggravation probable. Moreover, we wanted to establish which self-report scale could explain a substantial part of current and future mental health problems. To do so, given the a priori knowledge of factors associated – mainly identified through cross-sectional studies – with worse mental health outcomes in students (Limone and Toto, Reference Limone and Toto2022), we first employed binomial regression modelling to evince the association between demographic-individual factors and severe depressive, anxiety, obsessive–compulsive symptoms and suicidal ideation. Lastly, given the paucity of studies providing evidence for which risk factors are associated with future poorer mental health, we opted to adopt a supervised machine learning approach to automatically determine which factors at baseline could be leveraged to predict a change in symptoms (new onset, stability, improvement or worsening) after 6 months.
Materials and methods
Study cohort
All procedures described in this research were approved by the University of Padova Psychology Ethical Committee (Area 17 – ECOS: Eating, Compulsive and Obsessive Symptoms in Young Adults Protocol Ref. 3005) under the latest version of the Declaration of Helsinki. A significant part of the procedures described herein was also employed in two previous articles derived from the same protocol (Meda et al., Reference Meda, Pardini, Slongo, Bodini, Zordan, Rigobello, Visioli and Novara2021; Novara et al., Reference Novara, Pardini, Visioli and Meda2022). Participants provided written informed consent to the study. Recruitment took place in Padova, Italy, between October 2019 and October 2020. From October 2019 to March 2020, at the start of different teaching classes, we presented the objectives of the prospective study in person, and the attending students were given a card with an URL. By accessing the URL, students could provide their informed consent and participate in the study.
From March 2020 onward, due to COVID-related restrictions, the study was described via prerecorded videos or remote presentations at the beginning of teaching classes. Throughout the slide show, students had the possibility to copy the URL redirecting to the informed consent. Enrolment and follow-up continued throughout 2022. After obtaining informed consent, participants were required to complete a demographic schedule, as well as self-reported mental health questionnaires on the REDCap web application (Harris et al., Reference Harris, Taylor, Thielke, Payne, Gonzalez and Conde2009). Every 6 months since enrolment, the participants were automatically contacted by email and asked to participate in another data collection (for a total of six contacts and possible evaluations). That email contained a unique and personal e-address for each participant to access the platform and complete the questionnaires another time. With that unique link, the anonymous identity of the participant could be tracked longitudinally. A total of 1,902 students agreed to participate in the study (21.3% response rate, approximately 9,000 students were invited to participate) in the first cross-sectional study. Of this pool, 1,388 participants matched the target population characteristics (Italian native-speaker students aged 18–30 years); participants were excluded if they were non-native speakers, were outside the previous age range, or completed the questionnaires only in part. No other inclusion or exclusion criteria were applied. Students who participated in the study are (or were) enroled in Medicine and Surgery, Psychology/Mental Health and Neuroscience, Pharmacy and other health sciences, STEM sciences, Arts and Humanities and Law, Economics and Political Sciences.
At the second cross-sectional study (6 months after the first questionnaire administration), of the students (N = 1388) who completed the first wave of questionnaires, 768 students agreed to participate in the study a second time. Of these students, 557 were deemed eligible according to the previously outlined criteria. Sample characteristics at first cross-sectional and comparison between the subsample of dropouts to second-time participants are reported in Tables 1 and 2, respectively.
Sample characteristic at baseline
BAI = Beck Anxiety Inventory; BDI = Beck Depression Inventory; BMI = body mass index; MD = mental disorders; OC = obsessive–compulsive; OCI-R = Obsessive–Compulsive Inventory-Revised. Statistical significance was assessed with t-test (for data with normal distribution), Mann-Whitney U test (for data with non-normal distribution) or Chi-square test for frequency data. Test characteristics (i.e., degrees of freedom are not reported for ease of readability). Further testing corrected for possible confounders can be found in the results section.
Regression model for severe depressive symptoms
* p < 0.05; **p < 0.01; ***p < 0.001.
BIC = Bayesian Information Criterion; BDI = Beck Depression Inventory-II.
Measures
Participants were asked to report their demographic information, years of education, enrolment status, if they have ever been diagnosed with a mental health condition or other medical disorder, illness or disease or if they had a family history of mental health issues. Further questions investigated lifestyle habits, such as dietetic regimen, medication prescription, past/current psychotherapy and frequency of worry about their economic/financial conditions. The latter information was assessed with a 5-point Likert scale, where 0 = never worry (nil); 1 = rarely worried (mild); 2 = sometimes worried (moderate); 3 = frequently worried (high); 4 = always worried (extreme). To specifically design this measure, we defined the item similarly to the Worry Domains Questionnaire (Tallis et al., Reference Tallis, Eysenck and Mathews1992). All demographic variables were treated as categorical.
Five self-report questionnaires were administered: the Beck Depression Inventory-II (BDI-II; Beck et al., Reference Beck, Steer and Brown2011; Sica and Ghisi, Reference Sica, Ghisi and Lange2007) to assess depression characteristics (in our sample, this measure maintained a high internal consistency, Cronbach’s α = 0.91 [range: 0.91–0.92]); the Beck Anxiety Inventory (BAI; Sica and Ghisi, Reference Sica, Ghisi and Lange2007; Steer and Beck, Reference Steer, Beck, Zalaquett and Wood1997) for measuring the physical and cognitive symptoms of anxiety (Cronbach’s α = 0.92 [range: 0.91–0.93]); the Obsessive–Compulsive Inventory-Revised (OCI-R; Abramowitz and Deacon, Reference Abramowitz and Deacon2006; Sica et al., Reference Sica, Ghisi, Altoè, Chiri, Franceschini, Coradeschi and Melli2009); the Eating Disorder Inventory-3 (EDI-3; Garner, Reference Garner2004; Giannini et al., Reference Giannini, Pannocchia, Dalle Grave, Muratori and Vignone2008 and the Eating Habits Questionnaire (EHQ; Gleaves et al., Reference Gleaves, Graham and Ambwani2013; Novara et al., Reference Novara, Pardini, Pastore and Mulatti2017) [52–71]. For ease of readability and consistency, in this article, we will discuss only the results related to depressive, anxious and obsessive symptoms, as well as suicidal ideation (the latter measured by item 9 of the BDI-II (Desseilles et al., Reference Desseilles, Perroud, Guillaume, Jaussent, Genty, Malafosse and Courtet2012; Green et al., Reference Green, Brown, Jager-Hyman, Cha, Steer and Beck2015)).
Statistical analysis
Anonymized data were downloaded from the REDCap platform and curated using RStudio-R version 4.1.2 (R Core Team (2019)). In this work, we investigated the variables associated with depressive, anxious and obsessive–compulsive symptoms and suicidal ideation at baseline, as well as baseline predictors of mental health outcomes at the first follow-up (6 months after baseline). We analyse factors associated with more severe symptoms of depression, anxiety and obsessive–compulsive behaviour. For the purpose of binomial regression analysis and using random forest algorithms, we labelled participants with severe symptomatology (=1) or without it (=0), as measured with self-report measures. The reasons for employing a stringent cut-off are twofold: first, putative cut-offs for distinguishing nil, mild and moderate severity of symptoms are less reliable; second, the number of possible changes (and stability) in the severity is four times the number of possible changes with a single cut-off (see also below in the ‘Prospective assessment’ section). The cut-offs applied were based on the validation procedure of the single psychometric tools. They are as follows: 95th percentile of the BDI-II score (i.e., female score above 20 and male score above 19); score above 26 for the BAI scale (derived from the validation study, for both genders) and score above 28 for the OCI-R scale (derived from the validation study based on the AUC, for both genders). Suicidal ideation was measured with item 9 of the BDI scale (Desseilles et al., Reference Desseilles, Perroud, Guillaume, Jaussent, Genty, Malafosse and Courtet2012; Green et al., Reference Green, Brown, Jager-Hyman, Cha, Steer and Beck2015), with any score above 0 meaning the presence of some kind of suicidal ideation. The rationale for this choice is supported by literature, which shows little advantage in distinguishing between active or passive suicidal ideation when assessing risk (Liu et al., Reference Liu, Bettis and Burke2020).
First, we implemented a regression modelling approach to evaluate the weight of several independent variables on the outcomes of interest, while covarying for possible confounders. In this case, we used a binomial distribution to model the data and show the association between severe symptoms and independent variables. Here we report only the models with the lowest Bayesian Information Criterion (BIC – an index of model fitting: the lower the BIC, the lower the variance left unexplained by the model, the better the fit), which were identified through a stepwise selection approach (Raftery, Reference Raftery1995). We report the β estimate for each variable of the model with the lowest BIC. The estimation of a variable represents the importance of that variable in changing the questionnaire scores.
Prospective assessment
Then, to identify the predictors of severity changes from baseline to 6 months after enrolment, we implemented a random forest algorithm (supervised machine learning), which extracted from the dataset the independent variables at baseline that played a statistically significant role in determining the change or stability of the severity of the symptoms (i.e., stable ‘well-being’ – not severe at baseline and after 6 months; stable ‘severity’ – severe both at baseline and after 6 months; improvement – from severe to not severe symptoms in 6 months; worsening – from not severe to severe in 6 months). In this way, the machine learning algorithm was asked to predict four different outcomes. The algorithm was first trained using 80% of the dataset and then tested to assess its performance on the remaining 20% of the dataset. Random forest explainer package (Paluszynska et al., Reference Paluszynska, Biecek and Jiang2020) aided the visualization of random forest characteristics and predictive performance.
Results
A total of 1,388 students participated in the study (992 – 71.4% females; aged 21.01 ± 2.05). Some relevant statistically significant differences between the genders emerged with an uncorrected direct comparison approach (Table 1). Females were more likely to be underweight (8.2%) than males (2.8%), who, on the other hand, reported being overweight more frequently (15.19%) than females (6.7%). Most participants were recruited before the first COVID-19-related lockdown in Italy (which lasted from the second week of March 2020 to the first week of May 2020).
Females were also more likely to suffer from any physical disorder (17%) or eating disorder (6.9%) than males (8.8% and 0%, respectively). Moreover, median scores for depressive and anxiety symptomatology were higher in females (11, interquartile range [6–19] and 13, IQR [7–22], respectively) than males (9, IQR [4–16] for both measures). However, the frequency of severe depression symptoms was not different between sexes: only the frequency of severe anxiety symptoms was higher in females than in males. It is noteworthy that the frequency of suicidal ideation was not different between the sexes.
Importantly, there were no statistically significant differences in the frequency of self-reported, clinician-diagnosed mental health condition diagnosis between sexes or family history of mental disorders.
For a quick comparison (uncorrected for confounder variables), we reported in Figure 1 the percentage of severe symptoms (of depression, anxiety or obsessive–compulsive symptomatology), as well as the frequency of suicidal ideation per gender and field of study.
Frequency of severe depressive, anxiety symptoms and suicidal ideation among students per field of study and gender.
Significant variables associated with severe symptoms of depression
We define several binomial regression models to assess the contribution of demographic variables and clinically relevant characteristics to the higher severity of depression, anxiety, obsessive–compulsive symptoms and the presence of suicidal ideation.
The model that best described the severity of depressive symptoms, as measured with the Beck Depression Inventory, is reported in Table 2. We evidenced an association between high and extreme frequency of worry for one’s economic situation, with the odds of having severe symptoms of depression approximately tripled (odds ratio [OR] for high frequency of worry: 3.11 [1.88–5.15], p < 0.001; OR for extreme frequency of worry: 3.51 [1.53–7.57], p < 0.01). Additionally, severe depressive symptoms were associated with having followed at least one diet during life (probability of severe depressive symptoms increased by approximately 70%; OR = 1.77 [1.31–2.39], p < 0.001). The largest association was between suicidal ideation and depression severity, with the former increasing the odds of depression severity by approximately nine times (OR = 9.29 [6.71–12.86], p < 0.001). We also evinced some factors associated with a lower likelihood of manifesting severe symptoms: either commuting or living off-site (residence outside of Padova by more than 50 km), as well as whether enrolment in the study took place before the first ever COVID-19-related lockdown (i.e., the restrictive measures put in place from March to May 2020). Both variables approximately halved the chances of participants reporting severe symptoms of depression. The performance of the model was decent, with R 2 = 0.310.
Significant variables associated with severe anxiety symptoms and obsessive–compulsive symptoms
For each of the other measures (i.e., anxiety severity, obsessive–compulsive symptoms severity and presence of suicidal ideation), we reported two models: one agnostic with respect to the BDI-II total score and its subscale score, the other instead leveraged this information to produce a better fit of the data to the model. The reason for describing these two models (i.e., with or without the information produced by the BDI-II) is that the BAI, OCI-R scores, and the presence of suicidal ideation (measured with item number nine of the BDI-II) might be influenced by the presence of depression/depressive symptoms (i.e., the higher the burden of depressive symptoms, the higher the likelihood of these scores being elevated).
We found one variable associated with severe symptoms of anxiety: suicidal ideation. Its presence increased twofold (Table 3A, OR = 2.09 [1.49–2.93], p < 0.001, Table 2) the odds of severe symptoms. Using the score of BDI-II and its subscales, the previous association was no longer significant and replaced by the somatic subscale score of BDI-II (OR = 1.09 [1.07–1.12], p < 0.001 – meaning that, for each point on the scale, the odds of severe anxiety symptoms increased by approximately 9%). However, for both models, performance was very low (R 2 = 0.034 or 0.07, respectively).
Regression models for severe anxiety or obsessive–compulsive symptoms
* p < 0.05; **p < 0.01; ***p < 0.001.
BIC = Bayesian Information Criterion; BAI = Beck Anxiety Inventory; BDI = Beck Depression Inventory-II; OCI-R = Obsessive–Compulsive Inventory-Revised.
Regarding obsessive–compulsive symptoms, we found that the frequency of economic worry increased by two to three times the odds of severe symptoms (Table 3B, moderate frequency OR = 1.97 [1.18–3.30], p < 0.01; high frequency OR = 3.08 [1.75–5.42], p < 0.001; extreme frequency was not significant in this model); furthermore, enrolment before lockdown decreased the odds of severe symptoms by 50% (OR = 0.55 [0.39–0.77], p < 0.001). Finally, suicidal ideation increased the probability of severe symptoms by two times and a half – OR of 2.41 ([1.71–3.42], p < 0.001). The model fit was low (R 2 = 0.113). However, considering the BDI-II total score in the model provided a better fit (although still low, R 2 = 0.2) for each additional point on the scale, the odds of having severe symptoms increased by 10% ([1.08–1.11], p < 0.001).
Significant variables associated with current suicidal ideation
A large number of variables were significantly associated with suicidal ideation: we highlighted a protective role of enrolment before lockdown (Table 4, OR = 0.56 [0.4–0.78], p < 0.001; meaning that before lockdowns, the risk of suicidal ideation was halved with respect to after restrictive measures were effective) and marginally significant protection linked to enrolment during lockdown (OR = 0.58, [0.36–0.95], p < 0.05). Having no history of diagnosed mental disorder was found to be a significant protective factor with a large effect (OR = 0.28 [0.2–0.4], p < 0.001; i.e., reducing the chances of suicidal ideation by approximately 4 times).
Regression models for current suicidal ideation
* p < 0.05; **p < 0.01; ***p < 0.001.
BIC = Bayesian Information Criterion; BDI = Beck Depression Inventory-II; BMI = body mass index.
Regarding risk factors, male gender increased the odds of suicidal ideation approximately twofold (OR = 2.18 [1.57–3.01], p < 0.001). Also, the frequency of economic worry was identified as a possible risk factor, with increasing odds of suicidal ideation, as frequency varied from moderate to extreme (for the latter, a 10-fold increase was evidenced [5–21.5], p < 0.001). A family history of mental disorder was associated with 60% higher odds ([1.17–2.2], p < 0.01) of suicidal ideation. A statistically marginal significance of BMI was shown (being obese increased the odds of suicidal ideation about three times, OR = 2.93 [1.01–8.5], p < 0.05).
When covarying for BDI-II scores, we found that the score to the cognitive subscale of depression (which measured cognitive distortions) increased the odds of suicidal ideation by 36% for each additional point to the subscale ([1.31–1.42], p < 0.001) – in this case, for the scoring of the cognitive subscale, we excluded item 9 pertaining to suicidal ideation due to collinearity with the outcome measure. The absence of a history of mental disorders maintained a significant protective role, although the size of the effect was reduced (OR = 0.58 [0.38–0.9], p < 0.05). However, a marginal significance was achieved for the family history of mental disorder (if present, the risk of suicidal ideation increased by 48%; OR = 1.48 [1.00–2.18], p < 0.05). It should be noted that the size of the effect of the male sex increased, with the odds of suicidal ideation in men being 2.84 times higher than in women ([1.87–4.32], p < 0.001). The R 2 for this model was 0.46 and should be considered highly acceptable (considering the volatility of the outcome measure).
Comparison between follow-up and drop-out samples
Six months after enrolment, 557 participants completed a second wave of questionnaires (Table 5). We compared the samples of participants who participated in follow-up or dropped out. Follow-up completers were significantly more likely to have a family history of mental disorders, study Medicine and Surgery and be enroled before lockdown. No other statistically significant variables were evidenced.
Comparison between follow-up completers and dropouts
BAI = Beck Anxiety Inventory; BDI = Beck Depression Inventory; BMI = body mass index; MD = mental disorders; OC = Obsessive–Compulsive; OCI-R = Obsessive–Compulsive Inventory-Revised. Statistical significance was assessed with t-test (for data with normal distribution), Mann-Whitney U test (for data with non-normal distribution) or Chi-square test for frequency data. Test characteristics (i.e., degrees of freedom) are not reported for ease of readability.
Prediction of severe depressive symptoms after 6 months with supervised machine learning
We implemented a random forest algorithm to determine which variables at baseline could be leveraged to predict a change (or stationarity) in depressive symptoms. The algorithm was trained to predict four possible outcomes: stationarity of severe symptoms, stationarity of well-being, improvement of symptoms or worsening symptoms: 79 participants experienced a worsening of symptoms – 45 from nil-mild symptoms to moderate symptoms or from moderate to severe; 34 from nil-mild symptoms to severe symptoms; 77 participants experienced a symptom improvement; 310 remained in a state of well-being, whereas 91 participants still experienced severe symptoms. The random forest trained with 80% of the dataset (n = 461) identified cognitive and somatic subscale scores of BDI-II, frequency of economic worry and field of study as significant variables to classify participants into the four classes (Figure 2A). Furthermore, the interaction between the two subscales (Figure 2B) showed that the predictive capacity of the algorithm to classify participants is heavily influenced by extreme scores: the chances of participants still struggling with severe symptoms after 6 months from enrolment reached a probability of almost 1 when both subscale scores were higher than 20; similarly, the probability of being without severe symptoms (subpanel ‘Still Ok’ of Figure 2A) reached almost 1 when both subscale scores are below 10. When tested in the remaining 20% (n = 96) of the sample, the overall precision of the model was 0.77 (95% CI: 0.67–0.85), which was statistically significant (p < 0.001), although with lower precision in predicting participants whose symptoms worsened (balanced precision: 0.49) than participants stationarily well (0.85). In particular, the positive predictive value of worsening symptoms after 6 months of enrolment was 0 (whereas the negative predictive value was 0.89). Further details on model characteristics can be found in the Supplementary Appendix.
Significant variables extracted by random forest algorithms to predict severe depressive symptoms at follow-up. (A) Multiway plot showing the relative importance of the top 10 variables used to predict severe depressive symptoms at follow-up (6 months after enrolment). The variables in red are statistically significant. A higher score in accuracy decrease or gini decrease (which is a measure of how each variable contributes to the homogeneity of the nodes and leaves) reflects the relative importance of that variable in the model (i.e., if the variable is located in the upper right corner of the plot – like cognitive symptoms – it means that removing that variable from the model significantly worsens the model prediction capability). (B) Predictions of the random forest of severe depressive symptoms depending on values of the BDI-II subscale scores at baseline. Participants could be classified into four classes: improvement from baseline, worsening from baseline or stability (either steady severe symptoms or steady well-being). For each subplot predictions range from 0 (deep blue) to 1 (red). For example, the probability of ‘still struggling’ reaches almost 1 (certainty) if the subscale scores at the beginning of the study are high.
Prediction of suicidal ideation after 6 months with supervised machine learning
A random forest algorithm was trained to predict four possible outcomes: stationarity of suicidal ideation, stationarity of no suicidal ideation and improvement (suicidal ideation at baseline, not at follow-up) or worsening symptoms (no suicidal ideation at baseline, present at follow-up – termed ex novo): 34 participants report ex novo suicidal ideation, while 48 participants reported absence of suicidal ideation at follow-up; 402 without suicidal ideation both at baseline and at follow-up; 73 reported suicidal ideation both at baseline and follow-up. The random forest trained with 80% of the data set (n = 461) identified cognitive and somatic subscale scores of BDI-II, frequency of economic worry, field of study and residence (in Padova, commuting or off-site), as significant variables to classify participants into the four classes (Figure 3A). Noteworthily, although suicidal ideation at baseline could be leveraged by the model to improve predictive capability, it was not statistically significant. Furthermore, the interaction between the two subscales of the BDI (Figure 3B) showed that the predictive capacity of the classification algorithm is again heavily influenced by extreme scores. The chances of participants not reporting suicidal ideation were high for cognitive scores below 10, with little contribution of somatic symptoms scores. The model classification performance was low for the other classes (improvement, worsening or suicidal ideation at baseline and follow-up). When tested in the remaining 20% (n = 96) of the sample, the overall precision of the model was 0.91 (95% CI: 0.84–0.96), which was statistically significant (p < 0.001). However, the balanced accuracy for ex novo suicidal ideation was 0.5 with a negative predictive value of 0.96 but a sensitivity of 0. Balanced accuracy for participants who were stationary – without suicidal ideation – was 0.92. Further details on model characteristics can be found in the Supplementary Appendix.
Significant variables extracted by random forest algorithms to predict suicidal ideation at follow-up. (A) Multiway plot showing the relative importance of the top 10 variables used to predict suicidal ideation at follow-up. The variables in red are statistically significant. A higher score in the decrease in accuracy or the decrease in gini reflects the relative importance of that variable in the model. (B) Predictions of the random forest of suicidal ideation according to the values of the BDI-II subscale scores at baseline.
Discussion
Since the COVID-19 pandemic and the resulting lockdowns, mental health among students has attracted more research than ever before (Li et al., Reference Li, Wang, Wu, Han and Huang2021; Robinson et al., Reference Robinson, Sutin, Daly and Jones2022). Several prospective studies before the pandemic highlighted that higher education is a risk factor for depressive symptoms, anxiety symptoms and suicidal thoughts (Adams et al., Reference Adams, Saunders, Keown-Stoneman and Duffy2021; Storrie et al., Reference Storrie, Ahern and Tuckett2010).
However, there is little prospective evidence on the prediction of mental health problems. This would be the first step in a pragmatic approach to predicting which students are likely to need assistance in the short term, medium term and long term.
In this work, we assessed students’ depressive, anxiety, obsessive–compulsive symptoms and suicidal ideation and their possible determinants. We collected demographic and self-report measures from 1,388 Italian university students who completed questionnaires on mental health problems at enrolment and after 6 months. We (i) report a high percentage of students suffering from severe symptoms of mental health problems; (ii) identify the factors associated with poor mental health at baseline (for depressive symptoms: economic worry, enrolment time with respect to lockdowns, having followed at least one diet and suicidal ideation; for suicidal ideation: male gender, family or own history of mental disorder(s) and cognitive depressive symptoms; for obsessive–compulsive and anxiety symptoms: severity of depressive symptoms severity); (iii) highlight the risk factors that could predict a lack of improvement in symptoms or an increased likelihood of symptom worsening (cognitive symptoms and somatic symptoms of depression, economic worry, field of study and residence for both severity of depressive symptoms and suicidal ideation). Lastly, we showed which self-report measures explained a significant part of data variance related to current and future mental health problems.
We found that the percentage of students suffering from severe depressive symptoms ranged between 22% (in females) and 17% in males. That means that in our sample, 1 out of 4–6 had scored higher than the 95th percentile of normative scores for the BDI-II (the questionnaire used to assess depressive symptoms). Another extremely distressing finding was that 22% and approximately 20% (males and females, respectively) endorsed suicidal ideation in the 2 weeks before questionnaire completion. Figures for severe anxiety and obsessive–compulsive symptoms are a tad lower (severe anxiety percentages are approximately between 9% and 17%; severe OC symptoms are 14%). These findings are generally in line with previous descriptive studies on the percentage of students with mental health problems (Robinson et al., Reference Robinson, Sutin, Daly and Jones2022; Sheldon et al., Reference Sheldon, Simmonds-Buckley, Bone, Mascarenhas, Chan, Wincott, Gleeson, Sow, Hind and Barkham2021), although our results were, to some extent, more worrying than expected. We consider this finding secondary to the normalization of mental health problems after the wave of COVID-19 studies, which reported that an eye-watering percentage of young adults struggled with their mental health (before, during and after lockdowns) and on which an astounding amount of media reports were recorded. Future studies could test this ‘de-stigmatization’ hypothesis by measuring mental health problems and the degree of personal and social stigma in university students and compare it to historical data to assess if a sort of Papageno effect (Etzersdorfer and Sonneck, Reference Etzersdorfer and Sonneck1998; Niederkrotenthaler et al., Reference Niederkrotenthaler, Voracek, Herberth, Till, Strauss, Etzersdorfer, Eisenwort and Sonneck2010) for general mental health symptoms could be in place (Papageno effect, strictly speaking, is ‘the [positive] influence that mass media can have by responsibly reporting on suicide and presenting non-suicide alternatives to crises’).
In our study, we found that people who reported high or extreme levels of worry about the economic situation were three times more likely to experience severe symptoms of depression compared to those who reported lower levels of worry. The effect between the economic situation and depressive symptoms is a bidirectional vicious cycle (Ridley et al., Reference Ridley, Rao, Schilbach and Patel2020). The results of the random forest algorithm we employed also evidenced this link, as it identified economic worry (which is, however, distinct from poverty or actual financial distress) as a useful predictor to assess future (after 6 months) depression severity. Specifically, the algorithm showed that students with low or no worries about their economic situation had a greater chance of feeling relatively well with respect to their peers in distress about their finances. There is evidence from previous research (specifically on depressive symptoms and suicidal behaviour) that fear for expected financial losses is sufficient to trigger a cascade leading to depressive symptoms and suicidal thoughts (Fiksenbaum et al., Reference Fiksenbaum, Marjanovic, Greenglass and Garcia-Santos2017).
However, the association with the largest effect size (i.e., the strongest association) is between suicidal ideation and the severity of depression, with the former increasing the likelihood of severe depression symptoms by about nine times. This has to be contextualized by taking into account that suicidal ideation, measured with BDI, is a measure of depression itself (and a crucial diagnostic criteria for major depressive disorder, as conceptualized by DSM-5 (Cai et al., Reference Cai, Xie, Zhang, Cui, Lin, Sim, Ungvari, Zhang and Xiang2021; Liu et al., Reference Liu, Pan, Zhu, Meng, Tian, Li and Li2022)).
Regarding the evaluation of the factors associated with severe symptoms of anxiety, for every one-point increase on the BDI-II somatic subscale, the odds of being severely anxious increase by approximately 9%. In relation to obsessive–compulsive symptoms, we discovered that every extra point on the BDI-II scale increased the chance of developing symptoms by 10%. There could be two interpretations to this finding: the fact that higher scores to the BDI-II tend to increase other scale scores (as a rising tide lifts all boats) and/or both phenomena are comorbid (Jenkins et al., Reference Jenkins, Ducker, Gooding, James and Rutter-Eley2021; Norton et al., Reference Norton, Temple and Pettit2008; Torres et al., Reference Torres, Cruz, Vicentini, Lima and Ramos-Cerqueira2016). Further studies should consider measuring obsessive, anxiety and depressive symptoms with more than one scale to eventually unravel the contagion effects that a single scale score could exert on the other measures. In other words, in our study, the BDI-II scale was the fastest and useful tool for assessing mental health, but probably for the aforementioned reasons.
When covarying for the depression scores, some variables were significantly associated with the presence of suicidal ideation: we found that the score on the cognitive subscale of depression (which measured the variables related to cognitive distortions) increased the odds of suicidal ideation by 36% for each additional point on the subscale. Marginal significance for mental disorders in the family was evidenced, whereas no history of mental disorders played a significant protective role. It is noteworthy that self-identification with the male gender increased the likelihood of suicidal ideation by 2.84 times. As reported in the literature, depressive symptoms exert a strong effect on suicidal ideation (Konick and Gutierrez, Reference Konick and Gutierrez2005; Wang et al., Reference Wang, Shi and Luo2017). Regarding gender, recent evidence contrasted the view that female gender posits a greater risk of suicidal ideation (Eskin et al., Reference Eskin, Voracek, Stieger and Altinyazar2011; Rogers and Joiner, Reference Rogers and Joiner2017). The fact that young adult males are at higher risk of suicidal ideation has been getting more traction lastly (Lima et al., Reference Lima, Messias, Brito, Ferreira, Barbosa, Pinho, Brito and Silveira2021; Yarar et al., Reference Yarar, Bulut and Demirbaş2023). These can be better understood in light of the more extensive use of statistical tools, taking into account multiple variables simultaneously and thus examining the corrected weights of gender on suicidal ideation (Gui et al., Reference Gui, Sun and Zhou2022), although cultural differences need to be thoroughly assessed, as they can shift the weights in favour of males or females (Kaggwa et al., Reference Kaggwa, Najjuka, Favina, Griffiths and Mamun2023).
Lastly, we employed random forest algorithms to evince what factors at baseline were predictive of a change in depressive symptoms (i.e., still severe symptoms, worsening symptoms, improvement of symptoms and still no severe symptoms) and suicidal ideation (i.e., still suicidal ideation, ex novo suicidal ideation, improvement of suicidal ideation or still no suicidal ideation). For both measures, the supervised machine learning algorithm identified the scores on the cognitive and somatic subscales of the BDI, the frequency of economic worry, the field of study and the residence at baseline (the latter only for suicidal ideation) as significant variables to leverage to predict the change (or stability) of depressive symptoms. Noteworthily, suicidal ideation at baseline could be leveraged by the machine to predict suicidal ideation after 6 months, but it did not yield a strictly statistically significant role. To our knowledge, there have been several studies implementing machine learning models leveraging previous suicidal ideation to predict future suicidal ideation (Benjet et al., Reference Benjet, Borges, Miah, Albor, Gutiérrez‐García, Zavala Berbena, Guzmán, Vargas‐Contreras, Hermosillo de la Torre, Hernández Uribe, Quevedo, Covarrubias Díaz, Martínez Ruiz, Valdés‐García, Martínez Jerez and Mortier2022; Liao et al., Reference Liao, Wang, Zhou, Zhao, Li, Guo, Ji, Lv, Zhang, Zhang, Deng, Chen, Li and Qiu2022; Malone et al., Reference Malone, Zhao, Liu, Song, Sen and Scott2021), although time frames from baseline to follow-up did not coincide with our 6-month time frame. Substantially, previous suicidal ideation is an important factor, posing a greater risk of future suicidal ideation and suicidal behaviour (Bafna et al., Reference Bafna, Rogers and Galynker2022), but in our analysis, this prediction did not reach canonical statistical significance (although it should be taken into account that the machine learning model would lose a great proportion of its accuracy without data from previous suicidal ideation – Figure 3A). Machine learning in future studies could help find new associations between the variables, although limitations must be considered: when dealing with rare outcomes (e.g., new onset of suicidal ideation after several months from baseline), algorithms learn better to predict the more common outcome (e.g., still no suicidal ideation). We think this unbalance in the data (i.e., outcomes of stability being over-represented with respect to changes in symptomatology) can be effectively dealt with through multicentric collaborations. This approach would simultaneously address two tricky situations: cross-cultural/geographic differences and small sample sizes.
Strengths and limitations
The main strengths of this study are the prospective design, the analysis of various psychological dimensions through the compilation of validated questionnaires, the large cohort size and the careful acquisition of demographic variables. However, it should be considered that self-reporting questionnaires are not diagnostic instruments, and therefore they might not detect relevant changes in the mental health of participating subjects. Other limits concern the gender distribution of the sample, the response rate and the percentage of follow-up completers. About 1 out of 5 students invited to participate in the study completed all questionnaires. Moreover, approximately 1/3 of the individuals agreed to participate in the study again 6 months after enrolment. Although the baseline differences between follow-up completers and dropouts are negligible in terms of the outcomes of interest, the prospective evidence presented herein must be taken with caution, and it needs further support from more research for the following reasons: a sampling bias could skew the results due to self-selection of the participants, an issue similar to other studies in this field; although the percentage of follow-up completers for online participants is approximately in line with other studies characterized by time-demanding online questionnaires (de Leeuw et al., Reference de Leeuw, Hox and Luiten2018; Saleh and Bista, Reference Saleh and Bista2017), this could be seen as a possible source of participant self-selection. Lastly, it should be mentioned that our sample consists mainly of female participants. It is worth noting that females tend to be more represented than males in the fields of study we sampled. However, the relevant issue of gender bias in willingness to participate in research studies has already been pointed out (Smith, Reference Smith2008), but the consequences of its effect might be field-specific. In our case, a mixture of skewed gender percentages and self-selection of male participants could have contributed to our finding of the higher likelihood of suicidal ideation in males or, on the other hand, that differences could be due to cross-gender factors that we did not measure. It is worth mentioning that, although valid, we determined suicidal ideation solely on the basis of the BDI-II item 9 response (Desseilles et al., Reference Desseilles, Perroud, Guillaume, Jaussent, Genty, Malafosse and Courtet2012; Green et al., Reference Green, Brown, Jager-Hyman, Cha, Steer and Beck2015).
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S2045796023000550.
Availability of Data and Materials
The anonymized dataset and code for analysis can be retrieved at https://www.researchgate.net/profile/Nicola-Meda-2.
Acknowledgements
The authors would like to extend gratitude to all students who, without monetary incentives, gave their time and essential contribution to the study. The authors extend their thanks to all the Professors and Department directors who helped us spread awareness of the study. Lastly, the authors would like to thank Dr Irene Slongo, Dr Giulia Fattorini and Dr Christian Durante for help with participant recruitment. The authors thank Michele Gasparini (Evoé, Padova, Italy) for hosting the kick-off meeting of this project.
Competing interest
Nicola Meda, Susanna Pardini, Francesco Visioli and Caterina Novara declare none.
Ethical standards
ECOS: Eating, Compulsive and Obsessive Symptoms in Young Adults Protocol Ref. 3005.
Authors’ contributions
Conceptualization of the study was by N.M., S.P., F.V. and C.N. Methodology was prepared by N.M., S.P., F.V. and C.N. Investigation was done by N.M., S.P., F.V. and C.N. Software was prepared by P.R. and N.M. Formal analysis was performed by N.M. Visualization was done by N.M. Supervision was done by F.V. and C.N. Writing – original draft was by N.M. Writing – review and editing was by N.M., S.P., F.V. and C.N.
Open access
