Introduction
Schools play a central role in adolescent social lives, which is closely related to adolescent mental health (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018, Reference Shinde, Weiss, Khandeparkar, Pereira, Sharma, Gupta and Patel2020). Adolescent self-rated school climate is an indicator of the quality of social environments in schools (e.g. interpersonal relationships among students with teachers/peers, and participation in school activities/decisions) (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018), which, therefore, can be considered a proximal social determinant for adolescent mental health. Indeed, longitudinal studies have shown strong associations of school climate with adolescent mental health (e.g. lower depressive symptoms [DS]) (Hinze et al., Reference Hinze, Montero-Marin, Blakemore, Byford, Dalgleish, Degli Esposti and Kuyken2024; Raniti, Rakesh, Patton, & Sawyer, Reference Raniti, Rakesh, Patton and Sawyer2022), and a prior cluster randomized controlled trial (RCT) observed large effects of school climate intervention on adolescent mental health (e.g. d = −1.19 on DS [Shinde et al., Reference Shinde, Weiss, Khandeparkar, Pereira, Sharma, Gupta and Patel2020]). These observations suggest that school climate requires greater research focus to improve adolescent mental health, considering that adolescents spend the majority of their time in schools with increasing rates of enrolment and retention globally (United Nations Educational, Scientific and Cultural Organization, 2017).
The COVID-19 pandemic appeared to cause diverse changes in social lives through public health measures (e.g. restrictions on social contact and recreational activities), which may lead to aggravated adolescent mental health (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024; Racine et al., Reference Racine, McArthur, Cooke, Eirich, Zhu and Madigan2021). This may suggest that the negative effects of the pandemic differed according to the quality of social environments that individuals felt. However, no study has examined whether the impacts of the pandemic differed according to levels of school climate, which is a major indicator of the quality of adolescent social environments. This is a large knowledge gap to better understand the mechanisms of the negative effects of the pandemic on adolescent mental health and to improve public health measures under future pandemics.
We previously reported the impacts of pandemic on adolescent mental health (i.e. DS [Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024] and psychotic experiences [PEs] [DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024]) without considering school climate, using data from the Tokyo Teen Cohort (TTC), an ongoing study following 3171 adolescents in metropolitan Tokyo (Ando et al., Reference Ando, Nishida, Yamasaki, Koike, Morimoto, Hoshino, Scientific and Team2019). COVID-19 occurred midway through fourth wave (age 16) data collection (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024), providing a unique opportunity to understand the effects of COVID-19 in a sophisticated design; the data from TTC can compare the outcomes of the pre-pandemic group with that of the during-pandemic group while accounting for levels of the outcomes at previous three waves (age 10, 12, 14). To date, the TTC is the only adolescent cohort experiencing such a natural experimental situation. With this TTC dataset, the current study aimed to examine whether the pandemic effects differed according to adolescent self-rated school climate.
Methods
Study design and procedure
The TTC is an ongoing population-based cohort study (Ando et al., Reference Ando, Nishida, Yamasaki, Koike, Morimoto, Hoshino, Scientific and Team2019), consisting of random sampling of adolescents born during 2002–2004, using the Basic Resident Register of the Tokyo metropolitan area with oversampling of adolescents in lower-income households. Data was collected when the participants were aged 10 (T1: N = 3171), 12 (T2: N = 3007), 14 (T3: N = 2667), and 16 (T4: N = 2614). The TTC study protocol was approved by the Ethics Committees of the Tokyo Metropolitan Institute of Medical Science (#12–35), the University of Tokyo (#10057), and SOKENDAI (Graduate University for Advanced Studies; #2012002). Written informed consent was obtained from all parents. The authors assert that all procedures contributing to this work comply with the ethical standards of the relevant national and institutional committees on human experimentation and with the Helsinki Declaration of 1975, as revised in 2008.
Measurements
School climate
School climate was self-rated using an adapted version of the 28-item Beyond Blue School Climate Questionnaire (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018, Reference Shinde, Weiss, Khandeparkar, Pereira, Sharma, Gupta and Patel2020) (score range: 0–28) at T4. This questionnaire assesses the following components of school climate: ‘Supportive Teacher Relationships’ (e.g. ‘In this school, teachers and students really trust one another’), ‘Student Belonging’ (e.g. ‘Most other students accept me as I am’), ‘Student Participation in School Activities and Decisions’ (e.g. ‘Students have a say in decisions affecting them at this school’), and ‘Personal Commitment to Academic Values’ (e.g. ‘Doing well in school is important to me’). Higher scores indicate a better-perceived school climate. The original Beyond Blue School Climate Questionnaire does not define a specific time frame for most items, except for one item: ‘Thinking of my teachers this term, I really like’ (in Japan, a term typically lasts 3–4 months). Consistent with this, we did not specify a time frame for the adapted version used in this study. Additionally, we assessed school climate-related statuses using original questions (e.g. ‘Does your homeroom teacher help your classmates who are not feeling well?’) at T1. Detailed questions are shown in online Supplementary Tables S1 and S2.
Depressive symptoms (DS)
DS were self-rated using the Short Mood and Feelings Questionnaire (Thapar & McGuffin, Reference Thapar and McGuffin1998) (score range: 0–26) at T1–T4, with higher scores indicating worsening symptoms. Scores for participants with one missing item (missing rate: 0.98–1.83% of the total sample across T1–T4) were imputed using person-mean scores (Eyre et al., Reference Eyre, Bevan Jones, Agha, Wootton, Thapar, Stergiakouli and Riglin2021; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024). For the analysis of DS, our previous study restricted the study sample to adolescents with valid responses to DS at T4 and at least at T1–T3 (N = 2034) (Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024). Among them, the proportion of missing data on school climate was low (N = 148: 7.3%).
Psychotic experiences (PEs)
PEs were self-rated using the items derived from the schizophrenia section of the Diagnostic Interview Schedule for Children (Costello, Edelbrock, & Costello, Reference Costello, Edelbrock and Costello1985), with an added question on visual hallucinations (‘Have you ever seen things that other people could not see?’). Each item was scored on a three-point scale: no (0); maybe (0.5); and yes, definitely (1), and an aggregated score was calculated (score range: 0–5), which was treated as a continuous variable for statistical analysis (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024). For the analysis of PEs, our previous study restricted the study sample based on the same criterion as DS (N = 1935). Among them, the proportion of missing data on school climate was low (N = 54: 2.8%).
Statistical analysis
The COVID-19 pandemic occurred midway through T4 (March 2020), naturally dividing participants into pre-pandemic and during-pandemic groups (coded post-hoc based on the assessment date) (Fig. 1). Our previous studies reported the negative effects of the COVID-19 pandemic on DS and PEs by analyzing differences in each outcome between participants in the pre-pandemic and during-pandemic groups at T4 while controlling for within-person changes in each outcome across surveys using mixed-effect models (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024). The models were adjusted for chronological age at T4 and slope of elapsed time (in months) from the start of the COVID-19 pandemic in Japan (calculated based on the assessment date), without considering the influence of school climate. Based on the natural trajectories of each outcome, we adjusted for linear and quadratic slopes in the analysis of DS (Kwong et al., Reference Kwong, Manley, Timpson, Pearson, Heron, Sallis and Leckie2019) and for linear slope in the analysis of PEs (Staines et al., Reference Staines, Healy, Coughlan, Clarke, Kelleher, Cotter and Cannon2022).
Figure 1. Phases of the COVID-19 and related measures in Tokyo. During the nationwide school closure period (Tokyo Metropolitan Government, 2023), nearly all schools across Japan were closed (Ministry of Education, Culture, Sports, Science and Technology, 2020a, 2020b, 2020c), and students were not permitted to attend in-person classes. During this time, schools provided textbooks and teaching materials for self-study. A minority of schools also offered digital educational content, such as TV broadcasts (24%) and digital textbooks (29%) (Ministry of Education, Culture, Sports, Science and Technology, 2020d). However, only a small proportion (5%) of schools implemented online education via video conferencing platforms (Ministry of Education, Culture, Sports, Science and Technology, 2020d). The nationwide school closure was imposed only once during the pandemic period in Japan (Tokyo Metropolitan Government, 2023).
To examine whether the effects of the pandemic differed according to adolescent self-rated school climate, we further included the school climate score (continuous) and its interaction with the indicator of the during-pandemic group in the model. Both variables were multiplied by another indicator of the T4 survey, because the pandemic and its interaction affected DS and PEs only at T4 (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024). Additionally, to examine whether this effect modification of school climate differed by phases of the pandemic, the during-pandemic period (consisting of the overall COVID-19 pandemic period) was divided into three periods (school closure/late first-year/second-year). Mixed-effects models are maximum likelihood-based analyses and robust for missing data in outcomes under the assumption of ‘missing at random’ (Bell & Fairclough, Reference Bell and Fairclough2014). The significance level was set at α = 0.05. All analyses were conducted using R version 4.3.2, with lmerTest package.
Results
The school climate score and its related statuses (assessed at T4 and T1, respectively) did not show significant differences between the pre-pandemic and during-pandemic groups in both datasets for DS (Table 1 and online Supplementary Table S1) and PEs (Table 1 and online Supplementary Table S2). Scores for DS and PEs did not significantly differ between the two groups across all surveys, except for the DS score at T4, when the pandemic occurred (Table 1). Given the non-normal distribution of these continuous outcomes, we calculated the median and mean absolute deviation (online Supplementary Table S3), which are more robust measures for non-normally distributed data and potential outliers. We compared the scores between the pre-pandemic and during-pandemic groups using a non-parametric test (i.e. Mann–Whitney U test) (online Supplementary Table S3). The significance of group differences remained consistent, with only the DS score at T4 showing a significant difference between the two groups. There were slight significant differences in age at T4 between the pre-pandemic and during-pandemic groups (0.4 and 0.3 years in the datasets for DS and PEs, respectively). Detailed information on the sample characteristics is available elsewhere (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024). In brief, demographic variables assessed, including household income and recreational drug use such as cannabis, were not significantly different between the groups at all timepoints. On the other hand, there were small significant differences in proportions (less than 4%) for low household income throughout the surveys between samples included and excluded in the dataset for DS. Moreover, the age at T4 was slightly higher (0.2 years) among the samples excluded compared to those included in the dataset for PEs.
Table 1. Characteristics of study participants included in the current study by timing of age 16 survey (pre-pandemic group or during-pandemic group)
DS, depressive symptoms; PEs, psychotic experiences; T1, age 10 survey; T2, age 12 survey; T3, age 14 survey; T4, age 16 survey.
a p values were derived from t test for continuous variables and from chi-square test for dichotomous variables.
For the analysis of DS, the interaction effect between during-pandemic and school climate was significant (unstandardized coefficient [B] = −0.166, 95% confidence interval [CI] −0.225 to −0.107) (Table 2). Significant interactions were also observed during late first-year (B = −0.183, 95% CI −0.254 to −0.112) and second-year (B = −0.243, 95% CI −0.387 to −0.100) (Table 3). The negative effects of the COVID-19 pandemic were not significant on DS among adolescents who had high (approximately 80 percentile or more) school climate score at any phase of the pandemic (Fig. 2).
Table 2. Moderation effects of school climate on associations between the COVID-19 pandemic and adolescent mental health during overall COVID-19 pandemic period
B, Unstandardized coefficient; CI, confidence interval; DS, depressive symptoms; PEs, psychotic experiences; T1, age 10 survey; T2, age 12 survey; T3, age 14 survey; T4, age 16 survey.
a Centered at the start time of the COVID-19 pandemic in Japan.
b Centered at age 16.
c Number of participants in pre-pandemic group.
d Number of participants in during-pandemic group.
*p < 0.05; **p < 0.01; ***p < 0.001.
Table 3. Moderation effects of school climate on associations between the COVID-19 pandemic and adolescent mental health, during school closure, late first-year, and second-year of the pandemic
B, Unstandardized coefficient; CI, confidence interval; DS, depressive symptoms; PEs, psychotic experiences; T1, age 10 survey; T2, age 12 survey; T3, age 14 survey; T4, age 16 survey.
a Centered at the start time of the COVID-19 pandemic in Japan.
b Centered at age 16.
c Number of participants in pre-pandemic group.
d Number of participants in during-pandemic group according to the ‘stage of pandemic.’.
*p < 0.05; **p < 0.01; ***p < 0.001.
Figure 2. Visualization of differences in the effects of the COVID-19 pandemic on adolescent mental health by adolescent self-rated school climate. *p < 0.05, **p < 0.01, ***p < 0.001. The effects of the COVID-19 on depressive symptoms during (a) Overall COVID-19 pandemic period and (b) during School closure, Late first-year, and Second-year. The effects of the COVID-19 on psychotic experiences during (c) Overall COVID-19 pandemic period and (d) during School closure, Late first-year, and Second-year. Error bars represent 95% confidence intervals.
Similar results were obtained for the analysis of PEs. The interaction effect between during-pandemic and school climate was significant (B = −0.020, 95% CI −0.028 to −0.012) (Table 2). Significant interactions were also observed during late first-year (B = −0.022, 95% CI −0.031 to −0.012) and second-year (B = −0.022, 95% CI −0.040 to −0.003) (Table 3). The negative effects of the COVID-19 pandemic were not significant on PEs among adolescents who had high (approximately 80 percentile or more) school climate score at any phase of the pandemic (Fig. 2).
Discussion
This is the first study to examine whether the impacts of the COVID-19 pandemic on DS and PEs among adolescents differed according to the levels of adolescent self-rated school climate, which is considered to be a proximal social determinant of adolescent mental health. We found that the negative effects of the pandemic both on DS and PEs were significantly lower among adolescents who experienced a better school climate, and that the negative effects of the pandemic were not significant among adolescents who scored high school climate (80 percentile or more). These results may suggest that quality of social environments that individuals feel can buffer the impact of actual diverse changes in social lives, extending the suggested mechanism that diverse social changes caused by the pandemic lead to aggravated adolescent mental health (DeVylder et al., Reference DeVylder, Yamaguchi, Hosozawa, Yamasaki, Ando, Miyashita and Nishida2024; Hosozawa et al., Reference Hosozawa, Ando, Yamaguchi, Yamasaki, DeVylder, Miyashita and Nishida2024; Racine et al., Reference Racine, McArthur, Cooke, Eirich, Zhu and Madigan2021).
School climate score itself (Table 1) and prior levels of its related statuses (online Supplementary Tables S1 and S2) were not significantly different between the pre-pandemic and during-pandemic groups. These results may suggest that supportive school social environments (e.g. better interpersonal relationships among students with teachers/peers) (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018) can be maintained even when the pandemic caused diverse changes in social lives; in schools, for example, many school events/activities (e.g. school festivals, field trips, and talking in lunch time) were cancelled/restricted due to public health measures (e.g. restrictions on social contact and recreational activities) (Takaku, Shobako, & Nakata, Reference Takaku, Shobako and Nakata2024). Improving school climate in daily basis is considered to be crucial to improve adolescent mental health (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018, Reference Shinde, Weiss, Khandeparkar, Pereira, Sharma, Gupta and Patel2020; Yamaguchi et al., Reference Yamaguchi, Ando, Miyashita, Usami, Yamasaki, Endo and Nishida2023), and such improved supportive environments can also play protective roles for difficult situations such as the pandemic.
School climate can be improved through daily activities. For example, a large cluster RCT before the COVID-19 pandemic reported a promising effect of a school climate intervention program that included whole-school activities (e.g. a letterbox system providing a platform for students to raise concerns, complaints, and suggestions), group-level activities (e.g. forming peer groups to assist in organizing various activities such as skit presentations and role plays during school assemblies), and individual-level activities (e.g. one-on-one counseling for students), which resulted in a significant improvement in school climate (d = 2.23) (Shinde et al., Reference Shinde, Weiss, Varghese, Khandeparkar, Pereira, Sharma and Patel2018). These activities, which focus on creating supportive school environments, also significantly reduced adolescent DS (d = −1.19), an effect size much larger than that associated with commonly used individual psychotherapy methods, such as cognitive behavioral therapy (d = −0.21) (Werner-Seidler et al., Reference Werner-Seidler, Spanos, Calear, Perry, Torok, O'Dea and Newby2021, meta-analysis). Thus, school climate presents a vital target for interventions aimed at improving adolescent mental health, including DS (Singla, Shinde, Patton, & Patel, Reference Singla, Shinde, Patton and Patel2021). However, evidence for interventions specifically targeting school climate remains limited. Further research is needed to establish a robust evidence base regarding the effectiveness of school climate interventions on adolescent mental health across diverse populations.
The current study had several limitations. First, although TTC dataset was a unique to examine the impacts of the COVID-19 with sophisticated design, caution may be required to interpret the results as causal effects. For example, we did not directly assess COVID-19 infection, and thus, we cannot rule out a direct biological pathway from COVID-19 infection to adolescent mental health. However, the infection rate among adolescents (ages 10–19) during the pandemic period at T4 (from March 2020 to July 2021) was low, at less than 1.2%, in Tokyo (Bureau of Public Health, Tokyo Metropolitan Government, 2022; Statistics Division, Bureau of General Affairs, Tokyo Metropolitan Government, 2020, 2021), where participants in the TTC were recruited. Furthermore, several studies have reported negligible or small impacts of COVID-19 infection on mental health (e.g. anxiety/depression symptoms [Odds ratio = 1.08] [Klaser et al., Reference Klaser, Thompson, Nguyen, Sudre, Antonelli, Murray and Steves2021] and PEs [Odds ratio = 1.36] [Oh et al., Reference Oh, Goehring, Rajkumar, Besecker, Zhou and DeVylder2021a, Reference Oh, Schiffman, Marsh, Zhou, Koyanagi and DeVylder2021b]). Collectively, these findings indicate that COVID-19 infection likely had a minimal effect on our results. Second, the during-pandemic group was slightly older than the pre-pandemic group. However, the age differences were minimal (0.4 and 0.3 years in the datasets for DS and PEs, respectively), and age was not significantly associated with outcome levels in the analyses for both DS and PEs (Tables 2 and 3). Consequently, these age differences are unlikely to have introduced substantial bias in the estimates. Third, dropout rates were significantly higher among adolescents who were older (0.2 years) at T4 and from lower-income households (less than 4% difference). However, these differences were small and may have not led to much bias in the estimates. Finally, the TTC sample mostly consisted of participants of Asian ethnicity living in Japanese metropolitan areas, limiting generalizability of the results to other populations.
Conclusions
The impacts of the COVID-19 on mental health significantly attenuated among adolescents who felt better school climate, suggesting that supportive school social environments can play a protective role for the negative effects of diverse social changes caused by the pandemic among adolescents. School climate interventions should be conducted to improve quality of social environments in schools, which are central places in adolescent daily lives.
Supplementary material
The supplementary material for this article can be found at https://doi.org/10.1017/S0033291724003192
Acknowledgements
The authors would like to wholeheartedly thank the families that participated in the Tokyo Teen Cohort for their cooperation.
Funding statement
This work was supported by Japan Society for the Promotion of Science (JSPS) KAKENHI (SyY, grant number JP20H01777), (AN, grant number JP20H03951, JP21H05173, JP23H05472, JP24H00666), (KK, grant number JP21H05171, JP21H05174), (MH, grant number JP21K10487), (SA, grant number JP22H05211), (MM, grant number JP23H03174), (SaY, grant number JP24K16821, JP24H00917); JST RISTEX Japan (SyY, grant number JPMJRS24K1); UTokyo Institute for Diversity and Adaptation of Human Mind (UTIDAHM); and the International Research Center for Neurointelligence (WPI-IRCN) at The University of Tokyo Institutes for Advanced Study (UTIAS). The funding sources had no role in the study design; collection, analysis, and interpretation of data; writing of the report; and the decision to submit the article for publication.
Competing interests
Dr Ando has received personal fees from Shionogi, Janssen, Takeda Pharmaceuticals, and Sumitomo Pharma outside of the submitted work. Prof. Kasai has received grants from Shionogi, Sumitomo, Eli Lilly Japan, Daiichi-Sankyo, Tanabe-Mitsubishi, Otsuka, Takeda, and Teijin; grant evaluation fees from Takeda and Astellas; and honoraria for lectures from Takeda, Sumitomo, Otsuka, Meiji Seika Pharma, Fujifilm Wako Pure Chemicals, and Ricoh outside of the submitted work. Drs. Yamaguchi, DeVylder, Yamasaki, Miyashita, Hosozawa, Baba, Niimura, Nakajima, Usami, Hiraiwa-Hasegawa, and Nishida report no financial interests or potential conflicts of interest.
Open access
