To address the high prevalence of anxiety in children and adolescents, a number of social and emotional learning programs have been implemented in schools in Australia and internationally (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011; Neil & Christensen, Reference Neil and Christensen2007). Social emotional learning is defined by the Collaborative for Academic Social and Emotional Learning (CASEL, 2003) as the process of developing the skills, behaviours and attitudes that students need to effectively manage their emotions, cognitions, and social behaviour. As such, social and emotional learning programs explicitly teach students skills for coping during times of adversity (Jones & Bouffard, Reference Jones and Bouffard2012; Lowry-Webster, Barrett, & Dadds, Reference Lowry-Webster, Barrett and Dadds2012). In the face of schools’ limited resources, questions have been raised as to whether it is more effective to deliver social and emotional learning programs only to children who are ‘at risk’ or experiencing early symptoms of distress (known as a targeted approach), or whether programs should be delivered to all students regardless of whether they are experiencing distress (known as the universal approach; Neil & Christensen, Reference Neil and Christensen2007; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). The current study aimed to compare universal and targeted approaches of delivering a specific social and emotional learning program, with a focus on whether either delivery method was effective in reducing anxiety.
Approximately one in six Australian children are currently experiencing levels of anxiety severe enough to interfere with their everyday functioning (Boyd, Gullone, Kostanski, Ollendick, & Shek, Reference Boyd, Gullone, Kostanski, Ollendick and Shek2000; Dadds, Spence, Holland, Barrett, & Laurens, Reference Dadds, Spence, Holland, Barrett and Laurens1997). Approximately 6.9% of Australian children aged from 4 to 11 years have been diagnosed with an anxiety disorder (Lawrence et al., Reference Lawrence, Johnson, Hafekost, Boterhoven, Sawyer, Ainley and Zubrick2015), and lifetime prevalence of anxiety disorders are estimated at 20% (McEvoy, Grove, & Slade, Reference McEvoy, Grove and Slade2011). More than half of people who suffer from anxiety disorders experience their first symptoms by 11 years of age, meaning that prevention and early intervention are particularly pertinent (Kessler et al., Reference Kessler, Berglund, Demler, Jin, Merikangas and Walters2005). If left untreated, anxiety disorders in children tend to be characterised by relatively persistent symptom rates and low rates of remission (Cohen, Cohen, & Brook, Reference Cohen, Cohen and Brook1993). The prevalence of anxiety also has negative impacts on society; anxiety disorders are associated with the highest total cost in comparison with all other mental disorders (Rice & Miller, Reference Rice and Miller1998; Wittchen, Reference Wittchen2002). Due to the adverse effects of anxiety for the individual and society, the prevention of anxiety in childhood and adolescence has attracted growing interest from researchers and practitioners. Social and emotional learning programs have been found to decrease levels of anxiety symptoms and reduce the number of students experiencing clinically significant anxiety disorders (Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008; Teubert & Pinquart, Reference Teubert and Pinquart2011). More broadly, a meta-analytic review by Durlak and colleagues (Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011) found that social and emotional learning programs led to a number of positive outcomes for students, including increased social and emotional competence, improved academic performance, and decreases in both externalising and internalising symptoms.
Despite the promising outcomes of social and emotional learning programs for anxiety symptoms and positive adjustment, research suggests that many schools are not implementing social and emotional learning programs, or are using these programs with poor fidelity (Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011). One barrier to administrating social and emotional learning programs in schools is that schools often lack the resources or time within their curriculum (Ballet & Kelchtermans, Reference Ballet and Kelchtermans2009; Greenberg et al., Reference Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnik and Elias2003). As such, it is important for research to examine the most effective methods for delivering social and emotional learning programs (Greenberg et al., Reference Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnik and Elias2003).
Social and emotional learning programs (and prevention programs more generally) are usually classified into four levels of delivery: universal, selected, indicated, and treatment groups (Horowitz & Garber, Reference Horowitz and Garber2006; Neil & Christensen, Reference Neil and Christensen2007). In school settings, universal programs generally refer to programs that are delivered to all students regardless of symptoms. The universal approach involves delivering social and emotional learning programs to all students, regardless of whether they are currently experiencing behavioural, social or emotional problems (Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). Some schools employ a whole-school approach when delivering universal programs, although more commonly such programs involve one grade level. On the other hand, selected programs target children and adolescents who are deemed ‘at risk’ due to observed behaviours, socio-economic factors, or other factors such as family history (Rapee et al., Reference Rapee, Wignall, Sheffield, Kowalenko, Davis, McLoone and Spence2006). Indicated programs target children or adolescents who are experiencing mild-to-moderate symptoms (Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). Finally, treatment groups are made up of individuals who have been diagnosed with a disorder (Horowitz & Garber, Reference Horowitz and Garber2006). The indicated and selected groups are often overlapping, as children and adolescents who are classified as high risk are often experiencing some kind of distress, whether or not these symptoms are observable to others. For the purpose of the current study, the term ‘targeted group’ refers to interventions or programs comprised of selected and indicated participants. While evidence suggests that both universal and targeted methods of delivery have positive outcomes (Neil & Christensen, Reference Neil and Christensen2007; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008) there is no extant study that directly compares these different methods of delivery. The purpose of the current study is to compare these delivery methods by investigating outcomes for students delivered the same program in either a targeted approach or a universal approach. This article will now discuss the potential advantages of each approach before outlining the design, aims and hypotheses of the current study.
Benefits of the Targeted Approach
One potential benefit of delivering targeted group social and emotional learning programs is that it allows the program to be tailored to the needs of different groups. These smaller groups allow for a higher staff-to-student ratio, therefore increasing the one-on-one attention received by the participants. In a more homogenous cohort and small group environment, there might be increased empathy, bonding and opportunities to form social relationships with like-minded others (Rapee et al., Reference Rapee, Wignall, Sheffield, Kowalenko, Davis, McLoone and Spence2006). Initial evidence suggests that students report having positive experiences in small group social and emotional learning programs. Semple, Reid, and Miller (Reference Semple, Reid and Miller2005) found that all participants in their targeted program, where children were selected based on high anxiety, expressed pleasure at being part of the program. A study by Rapee and colleagues (Reference Rapee, Wignall, Sheffield, Kowalenko, Davis, McLoone and Spence2006), who implemented both targeted and universal social and emotional learning programs aimed at preventing depression, found that participants were more satisfied with the targeted group format. Rapee and colleagues attributed this to the smaller group environments leading to opportunities to share experiences with like-minded others. However, the researchers acknowledged that a potential confounding variable could be that the targeted group was facilitated by a mental health professional while the universal program was delivered by teachers.
Another potential advantage of the targeted group approach is that targeted social and emotional learning programs have resulted in higher effect sizes for a range of outcomes, including gains in social and emotional competency and reductions in anxiety, when compared with universal programs (Jones & Bouffard, Reference Jones and Bouffard2012; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). However, this higher effect size may result from these at-risks groups having lower baseline scores of social and emotional competence, and therefore more room for improvement (Haney & Durlak, Reference Haney and Durlak1998; Jones & Bouffard, Reference Jones and Bouffard2012).
Benefits of the Universal Approach
An advantage of the universal delivery of social and emotional learning is that it avoids the risk of some children who are experiencing distress going unnoticed, as the program is delivered to all students. Flett and Hewitt (Reference Flett and Hewitt2013) found that often children are ‘flying under the radar’ in the sense that they are experiencing psychological distress, but they are not receiving support. Flett and Hewitt report that only a small proportion of students needing intervention actually receive it. In accordance, Merikangas and colleagues (Reference Merikangas, He, Burstein, Swendsen, Avenevoli, Case and Olfson2011) found that only one in five adolescents who experience clinically significant levels of anxiety receive treatment. Research suggests that anxious children are more likely to ‘fly under the radar’ because internalising disorders are much less observable to others in comparison to overt behavioural problems (Dwyer, Nicholson, & Battistutta, Reference Dwyer, Nicholson and Battistutta2006; Papandrea & Winefield, Reference Papandrea and Winefield2011). Papandrea and Winefield (Reference Papandrea and Winefield2011) found that teachers feel ill equipped to recognise the symptoms of internalising disorders due to inadequate mental health training. Furthermore, they found that students experiencing anxiety are more likely to go unnoticed by teachers because they tend not to be disruptive in the classroom. This is especially concerning, given that teachers are most commonly responsible for referring students to targeted social and emotional programs (Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008).
Further complicating the identification of anxious students is that some children and adolescents are deliberately masking their distress. Flett and Hewitt (Reference Flett and Hewitt2013) found that some anxious children and adolescents have personality styles that promote self-concealment and perfectionism. Others may hide their distress in order to avoid the social stigma associated with mental illness (Bowers, Manion, Papadopoulos, & Gauvreau, Reference Bowers, Manion, Papadopoulos and Gauvreau2013). Even psychometrically sound screening tools may not identify these students, due to the child or adolescent's desire to hide their problems (Flett & Hewitt, Reference Flett and Hewitt2013).
Another potential benefit of universal delivery is that even children who are not currently experiencing anxiety symptoms can benefit from social and emotional learning programs (Schonert-Reichl et al., Reference Schonert-Reichl, Oberle, Lawlor, Abbott, Thomson, Oberlander and Diamond2015; Seligman, Ernst, Gillham, Reivich, & Linkins, Reference Seligman, Ernst, Gillham, Reivich and Linkins2009). For example, in mindfulness-based social and emotional learning programs, benefits were apparent for children and adolescents across the full range of wellbeing, including those who were currently considered well adjusted (Kuyken et al., Reference Kuyken, Weare, Ukoumunne, Vicary, Motton, Burnett and Huppert2013; Schonert-Reichl et al., Reference Schonert-Reichl, Oberle, Lawlor, Abbott, Thomson, Oberlander and Diamond2015). Similarly, Lowry-Webster and colleagues (Reference Lowry-Webster, Barrett and Dadds2012) found that a universally implemented social and emotional learning program based on cognitive behaviour therapy principles led to reductions in anxiety symptoms for students, regardless of their risk status. Outcomes of social and emotional programs, such as increased peer acceptance, increased empathy among students and increases in academic performance, are likely to be beneficial to well-adjusted students in the present moment (Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011). Additionally, while non-anxious students may not necessarily experience a decline in anxiety due to low baseline levels, it is plausible that they will be better able to manage state anxiety using coping strategies taught in social and emotional learning programs.
In addition to immediate positive impacts, universally delivered social and emotional learning programs better equip children to cope with adversity in the future. As such, the universal approach fosters resilience in students. Resilience is best described as the process of ‘bouncing back’ after encountering negative events, challenges or adversity and return to almost the same level of emotional wellbeing (Johnson, Reference Johnson2008). Schools are often not in a position to reduce many of the risk factors for psychopathology that children and adolescents face (Fergus & Zimmerman, Reference Fergus and Zimmerman2005). As such, in school settings, promoting resilience is best achieved by developing students’ internal assets and resources, which they can draw upon in times of adversity (Fergus & Zimmerman, Reference Fergus and Zimmerman2005). Social and emotional learning fosters adaptive coping skills, emotional regulation, improved self-concept and helpful thinking strategies, all of which act as protective factors that promote positive adjustment in students.
The Current Study
Universal and targeted social and emotional learning programs have both been found to have promising results for reducing anxiety (Durlak et al., Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). Overall, while both approaches have merit, a direct comparison of targeted and universally implemented SEL programs will be useful in informing which approach leads to the best outcomes for students and is the most effective use of resources. The aim of the current study is to evaluate and compare the anxiety outcomes of both a targeted and a universally implemented mindfulness-based social and emotional learning program. Based on the literature outlined above, it was predicted that the targeted group delivery method would produce greater reductions in anxiety, compared to the universal delivery method. However, as many researchers have postulated that this difference in effect size is likely due to anxious children having higher baseline anxiety scores, it was tentatively hypothesised that anxious children in the universally delivered program would see a similar decrease in anxiety to those students in the targeted group program.
One issue with focusing only on the anxiety outcomes of this study is that such an approach might not adequately capture the benefits and drawbacks of each approach. For example, this study is also interested in capturing information about whether the program might have assisted well-adjusted students by providing skills to manage state anxiety and future adversity. The current study aims to better explain the outcomes of both methods of delivery, using semi-structured interviews to explore the experiences of students from both programs. It is predicted that interviews will also allow for a more thorough exploration of the potential benefits of each method of delivery. Therefore, a mixed-methods design, incorporating both quantitative (measuring anxiety outcomes) and qualitative data (measuring student experience), allows for a more thorough comparison of universally and targeted delivery of the program.
Method
Study Design
A mixed-methods approach was employed by this study, which is a procedure for collecting and analysing both quantitative and qualitative data, then integrating (i.e., ‘mixing’) the data, all within a single study (Creswell, Reference Creswell2015). The rationale for using mixed methods was that neither quantitative nor qualitative measures alone were sufficient to understand the differences between universal and targeted approaches to social and emotional learning. When used in combination, quantitative and qualitative measures allowed for a more complete picture of the research problem.
The current study used a sequential explanatory mixed methods design (Creswell, Reference Creswell2015; Hanson, Creswell, Clark, Petska, & Creswell, Reference Hanson, Creswell, Clark, Petska and Creswell2005). This design involved two distinct phases of data collection. The first phase was to collect and analyse quantitative data, while the second phase involved collecting and analysing qualitative data to help to better understand and elaborate on the quantitative findings.
The quantitative phase of the study involved measuring anxiety symptoms pre- and post-program for both universally and targeted group approaches. These data were then analysed statistically and helped to inform the interview process for the qualitative component of the study. The second phase involved conducting interviews with participants from both the universal and targeted social and emotional learning programs to better understand their lived experience of the program and to investigate potential benefits of the program beyond anxiety outcomes.
It is common (and often recommended) in mixed-methods studies to prioritise one form of data (Creswell, Reference Creswell2015). In this study, emphasis is placed on the quantitative results, as reducing anxiety symptoms was one of the major aims of the social and emotional learning program evaluated by this study. The quantitative and qualitative elements of the study were connected when selecting participants to interview and designing interview questions based on the quantitative results. The results of the quantitative and qualitative phases were also integrated in the overall discussion of the study. Figure 1 depicts the mixed methods procedure employed by this study.
Visual model for explanatory sequential mixed methods design procedure.
Quantitative Analysis (Phase 1)
Participants
Data collection was conducted in accordance with the Declaration of Helsinki, and was approved by Monash University Human Research Ethics Committee (approval no. CF14/2118 – 2014001119). The first quantitative phase of the study involved 66 participants, all of whom were enrolled in Grades 5 to 6 (typically aged 9 to 12 years) at two schools in Melbourne, Australia. There were slightly more male participants (n = 39, 59.1%) than females (n = 27, 40.9%). Of these participants, 46 students in one school completed the program as part of their regular Grade 5 school curriculum; these students were part of the universal approach. The remainder of the participants (n = 20) were selected for the program from a second school, due to being deemed ‘at risk’ of social and emotional maladjustment; thus, these students participated in the targeted program. The ‘targeted’ students were referred to the program by classroom teachers or school wellbeing staff. Teachers were asked to identify students who had difficulties with anxiety as well as students who were at risk due to lack of support in the home environment, or experiencing adversity such as neglect and trauma.
Intervention Program
The Robust, Resilient, Ready-to-Go program (Triple R; Bannirchelvam, Bell, & Costello, Reference Bannirchelvam, Bell and Costello2017; Dove & Costello, Reference Dove and Costello2017; McCabe, Costello, & Roodenburg, Reference McCabe, Costello and Roodenburg2017) consisted of six sessions of a mindfulness-based program implemented over consecutive weeks, followed by two booster sessions of a similar nature in the subsequent school term. As such, there were eight weekly sessions of one hour in duration, implemented over approximately 16 weeks from initial session to final booster session. The program was delivered during school hours, on site, by qualified mental health professionals. The Triple R program was developed to address anxiety in students through explicit teaching of social and emotional competencies. The program was based on an extensive literature review and incorporated mindfulness (Greco, Blackledge, Coyne, & Ehrenreich, Reference Greco, Blackledge, Coyne, Ehrenreich, Orsillo and Roemer2005) and cognitive approaches to anxiety management (Kearney & Bensaheb, Reference Kearney and Bensaheb2006). The program used collaborative learning strategies such as small group problem-solving discussion, critical thinking tasks, skill development tasks, reflective tasks and themed games (Cahill et al., Reference Cahill, Coffey, Lester, Midford, Ramsden and Venning2014), which are integral to effective social and emotional learning programs (Herbert & Lohrmann, Reference Herbert and Lohrmann2011). An outline of the program can be found in Appendix A in the supplementary material.
Quantitative Measures
The Revised Childhood Anxiety and Depression Scale (RCADS; Chorpita, Yim, Moffitt, Umemoto, & Francis, Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000) was used to measure participant state anxiety. Previous research suggests that the RCADS is a valid and reliable measure (Chorpita et al., Reference Chorpita, Yim, Moffitt, Umemoto and Francis2000; Esbjørn, Sømhovd, Turnstedt, & Reinholdt-Dunne, Reference Esbjørn, Sømhovd, Turnstedt and Reinholdt-Dunne2012), appropriate for use in Australian research and clinical settings (de Ross, Gullone, & Chorpita, Reference de Ross, Gullone and Chorpita2012), and suitable for measuring changes in anxiety over time (Mathyssek et al., Reference Mathyssek, Olino, Hartman, Ormel, Verhulst and Van Oort2013). The RCADS is a 47-item youth self-report questionnaire. Items are rated on a 4-point Likert-scale from 0 (never) to 3 (always). The RCADS includes subscales for various anxiety and depressive disorders, and a total anxiety score (sum of the five anxiety subscales) that are converted to T scores based on participants’ age. The current study used the total anxiety score. T scores have a mean of 50, a standard deviation of 10, with T scores under 60 considered typical. T scores over 60 (one standard deviation above the mean) represent subclinical levels of anxiety. T scores of 70 and higher (two standard deviations above the mean) represent clinically significant anxiety. The RCADS was administered prior to the commencement of the program, and after the final session, under the supervision of the program facilitators and school staff.
Quantitative Data Analysis
Participants in the universal group were categorised to a ‘low-to-average anxiety group’ (those with RCADS total anxiety scores less than 60) and a ‘high anxiety group’ (those with RCADS total anxiety scores of 60 or greater). To measure the difference in pre-program anxiety scores between the three groups (whole-class average anxiety, whole-class high anxiety and targeted group), a one-way analysis of variance (ANOVA) was conducted. Given the small sample size, Welch's ANOVA was selected as it is robust for violations of homogeneity (Delacre, Lakens, & Leys, Reference Delacre, Lakens and Leys2017). Post hoc analyses were conducted using the Games-Howell test, which does not assume equal variances across groups. To measure the impact of the Triple R program, repeated measures t tests compared the mean difference of pre-program and post-program total anxiety scores for each group.
Qualitative Analysis (Phase 2)
Participant Selection
The quantitative results informed the selection of students for interview. Three interviewees were selected from the universal and targeted-group cohorts. In the universal cohort, one of these interviewees was selected due to having rated herself as anxious (greater than 1 standard deviation above the population mean) before the program. The remaining two interviewees were selected due to having average-to-low baseline levels of anxiety. In the targeted group, the three students selected for interview were of different ages and grades. The targeted-group participants were chosen due to having varied outcomes for the program; one student's anxiety increased after the program, one student's anxiety scores remained similar, and the final student showed decrease in anxiety post-program. This approach of using varied participants, referred to as the maximal variation principle (Creswell, Reference Creswell2015), allows for multiple perspectives from students who are likely to have varied experiences of the program.
Interview Protocol
Individual semi-structured interviews were conducted with participants after they had completed the Triple R program, to facilitate an exploration of what the participants had experienced during and after the program. The interviews were conducted in a quiet space free from distractions at the children's school and were audio recorded. The interviews were conducted by the first author, who had met the participants in the initial session but was not involved in delivering the program. One objective of the interview was to better understand the lived experience of the students who participated in the program, with a particular focus on discovering the differences in experience between the universal and targeted delivery methods. As such, the interview included specific questions about the delivery method of the program and questions about aspects of the program that the participants liked and disliked. The content of the interview protocol was also partially grounded in the quantitative results. One of the aims of the qualitative phase was to explore whether the non-anxious participants in the whole-grade condition benefitted from the program in a manner that was not captured by the quantitative analysis. The interview was intentionally kept brief with a relatively narrow focus, given that the qualitative measures were the minor focus of this mixed methods study. The interview schedule can be found in Appendix B in the supplementary material.
Qualitative Data Analysis
A thematic analysis guided by the principles outlined by Braun and Clark (Reference Braun and Clarke2006) was used to analyse the data from the semi-structured interviews. The thematic analysis was best described as ‘theoretical’ thematic analysis, as it was driven by particular area of interest and was highly focused and explicitly analyst driven (Braun & Clarke, Reference Braun and Clarke2006). Interviews were transcribed verbatim by the interviewer. Transcripts were read several times to highlight the emerging themes and notes were made by certain responses which appeared on a regular basis across participants (Guest, MacQueen, & Namey, Reference Guest, MacQueen and Namey2011). The process taken to conduct the thematic analysis is described in Table 1.
Process of Thematic Analysis of Semistructured Interviews
To complement the thorough review of transcripts and themes, further data validation took place via external consultation. The credibility of the data and themes was sought through peer debriefing with a psychologist who was not involved in the current study. Audits for dependability and confirmability of the data were obtained in a similar way whereby the external psychologist checked the transcripts and notes of the researcher to ensure all themes were present within the data itself and that appropriate extraction of themes had taken place (Braun & Clarke, Reference Braun and Clarke2006).
Results
Quantitative Phase
The mean pre- and post-program total anxiety scores for children in the universal and targeted groups are detailed in Table 2. In addition to the universal group mean anxiety score, mean scores are also reported for children within the universal subgroup who reported low-to-average anxiety pre-program (with total anxiety scores less than 60) and children within the universal subgroup who reported high anxiety pre-program (with total anxiety scores of 60 or higher). The table also provides the mean difference in anxiety T scores for each group.
Pre- and Post-Program RCADS Total Anxiety Scores by Group
A one-way Welch's ANOVA was conducted to determine whether baseline average anxiety scores differed between the targeted, universal high-anxiety and universal low-to-average anxiety groups. There was a significant difference in baseline anxiety between groups, with Welch's F(2, 27.09) = 60.16, p < .001. Games-Howell post hoc analysis revealed that the low-to-average anxiety group reported significantly lower mean anxiety at baseline than both the targeted group (−17.28, p < .001) and the universal high-anxiety group (−23.10, p < .001). There was no significant difference in mean baseline anxiety between the high-anxiety universal group and the targeted group (p = .736).
Repeated measures t tests were conducted to determine whether the difference in the anxiety reported by participants was significantly different following the Triple R program. For the students who were selected into the targeted group, there was a significant decrease in their self-reported anxiety levels post-program compared to pre-program, t(19) = 3.27 p = .004, d = 0.73.
For students in the whole-class program, the difference between pre- and post-test scores was non-significant, t(45) = 1.557 p = .126. However, for students in the whole class program who reported high anxiety pre-program, there was a significant decrease in their anxiety post-program, t(10) = 3.377 p = .007, d = 1.10. For participants in the universal delivery group who reported low-to-average anxiety levels pre-program, there was no significant change in their mean anxiety scores post-program, t(34) =.518, p = .608. Figure 2 represents the mean difference in anxiety scores before and after the Triple R program for the targeted group delivery and the high and average anxiety groups within the universal delivery group.
Mean total anxiety scores for groups before and after the Triple R program.
Qualitative Phase
The details of how the participants were selected using the maximal variation principle (Creswell, Reference Creswell2015) has been described in detail in the method section. Table 3 provides further information regarding the individuals selected for interview, including their age, pre- and post-program total anxiety scores, and whether they were in the whole class or targeted program. To ensure the anonymity of the individuals involved, pseudonyms were used. Participant descriptions of their experience in the program clustered around four main themes: mindfulness, social experience and social connectedness, helpful thinking, and recognition of individual differences
Summary Data of Participants Selected for Interview
Note: 1Pseudonyms used. 2T scores of have a mean of 50 and a SD of 10. T-scores below 60 are considered typical. T scores between 60 and 70 are considered subclinical, while T scores of 70 and above are clinically significant.
Mindfulness
An initial finding of the thematic analysis was that most of the participants found the mindfulness activities to be a particularly salient part of the Triple R program. Within this theme, it was apparent that almost all participants continued to use mindfulness. While participants reported having difficulties with attention at times during the mindfulness tasks, they were also quick to describe how the strategies had a calming effect on their mind and body. Suzie, from the targeted group, discussed how breathing and body awareness helped her to cope with strong emotions: ‘I liked how we would go in and lie on the floor to relax our bodies — releasing your anger and anxiety and put your hands on your stomach and feel it go up and down, calm you down.’ Within this theme, using breath and body awareness was a commonly described way of coping. Jackson said that a helpful strategy that he uses now is to ‘relax and forget about other things for a second and just focus on your body . . . just breathing, breathing in and breathing out to relax my body’. Some participants reported that they were not currently using mindfulness, but they recognised that it could be a helpful strategy that they could use if they ever experience stress. Dylan said he could use a body scan technique in the future if problems arose: ‘If I had a tough day I could use that thing where you tense different parts of your body and then relax them’.
Social Experience and Social Connectedness
Another important theme was how the participants felt about their peers being involved in the program. Many participants described a desire to be with their friends during the program. It appeared that the students felt more comfortable talking about their inner feelings and experiences with people they trusted and were familiar with. When asked if he would like to do the program with a smaller group Jackson said: ‘I wouldn't really like it because you don't have — or you might not have any friend or your best friends or something and I think you always want your friends around you so you can talk to them.’
In the case of the targeted group, the participants described feeling daunted and unsure about the process of meeting new people, and only had a few people they knew in the group. Participants described that initially attending the sessions was daunting, as they didn't know many people in the group. Max described how he was ‘nervous at the start’ because he only knew three people in his group. Despite this, students from the targeted group program also recognised the benefits of meeting other people from different grades and gaining friendships through the program. For example, Ella said: ‘I didn't know much of the kids, but yeah, I got to meet them. Catch up with them, got to know them a bit, make new friends.’ Max describes how even though it he was nervous at first, meeting new people was a positive experience: ‘At the end I knew pretty much everyone so I didn't really feel that nervous, only a bit, because I knew what they do and what their names were.’ Ella, however, did not feel connected to the younger students and said she would have preferred to do the program with her own class: ‘There were only little kids, I didn't know what was happening sometimes, and if it's your own class then you know people and you're comfortable with them.’ However, for the other students, it appears that although initially the targeted group format can lead to some experience of anxiety, by the end of the program, students felt socially connected to the other participants and felt that meeting new people in the group was a positive experience.
All of the participants in the whole-class condition expressed that they really enjoyed working in small groups with their peers. As Jackson explained: ‘I think I went better in groups because it just sort of works for me because you don't have to think about it just by yourself, you've got a whole group to communicate with.’ It seems student enjoyed sharing their experience with others in the group and felt the small group format was helpful as they could draw on the experiences and knowledge of their peers.
Helpful Thinking
A few participants described using cognitive strategies to change the way they thought about situations. Lulu, the anxious student from the whole-grade approach, explained that she used cognitive strategies to help her manage emotions: ‘It helped me to stop crying because I'm really sensitive. I just try and think, it doesn't matter, they just did sometime by mistake and I don't care about it — I can just ignore it.’ Here, Lulu describes a process of attempting to have more helpful thoughts about the situation and trying to let go of less helpful thoughts. This was also a theme that emerged in Ella's interview: ‘When you're worrying, like look at yourself and say “let's find a positive to the situation”.’ Even the participants in the average anxiety group talked about changing their thinking in the face of anxiety-provoking situations. For example, Jackson spoke about shifting his thinking from ‘worrying about letting the team down’ by changing his ‘headspace’ to think about ‘just trying my best’. Students seem to have grasped the idea that one way to change how one feels about an adverse situation is to change their own thought processes.
Recognising Individual Differences
Another theme was that interviewees in the whole-class condition spontaneously recognised that although some elements of the program were not helpful to them, they were important for others in the class. Dylan describes that mindfulness was not really helpful for him: ‘I don't really like get into that kind of stuff — like all that mindfulness.’ However, he said that he thought it was important for the whole class be included in the program, even if some parts were not relevant to him: ‘I think it's good that we were all included . . . I think, like a lot of the kids that did do it that haven't already done it actually got really into it and really like it. It's good for everyone to try it.’ In a similar vein, Jackson felt that activities and discussion around anger were not really relevant to him but would be for others: ‘I didn't see the point of that because I know that it doesn't happen to me and anger doesn't fill up inside me so . . . but other people have say, issues about it, and they need to know.’
Discussion
The aim of the current study was to use a mixed-methods design to compare the targeted and universal approaches of a social and emotional learning program (Triple R), with a focus on anxiety outcomes. It was expected that the targeted approach would yield greater effect sizes, but that both programs would lead to a reduction in anxiety symptoms. It was hypothesised that highly anxious children from the universal condition would see reductions in anxiety equivalent to the targeted group participants. Finally, it was anticipated that the qualitative measures (semi-structured interviews) would allow for a more thorough exploration of the potential benefits of each method of delivery, and thus the mixed-methods design would result in a more comprehensive comparison of the two approaches.
The first hypothesis that both programs would lead to reductions in anxiety was partially supported by the results. There was a significant decrease in the mean levels of anxiety experienced by the targeted group participants following the Triple R program. This is consistent with previous findings that targeted social and emotional learning programs lead to meaningful decreases in anxiety (Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008; Semple et al., Reference Semple, Reid and Miller2005; Teubert & Pinquart, Reference Teubert and Pinquart2011). However, there was no significant difference between the pre-program and post-program anxiety scores for the universal group. Although unanticipated, these results may be attributed to the fact that three-quarters of the universal group participants reported anxiety symptoms within the typical range, indicating minimal problems with anxiety. The low baseline rates of anxiety symptoms in these participants suggest that they had less room for improvement compared to their more anxious peers. Conversely, other studies have found that even well-adjusted students can experience reduced anxiety following social and emotional learning programs (Lowry-Webster et al., Reference Lowry-Webster, Barrett and Dadds2012; Seligman et al., Reference Seligman, Ernst, Gillham, Reivich and Linkins2009), although effect sizes for these reductions in universal groups have been relatively small (Jones & Bouffard, Reference Jones and Bouffard2012; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). Given the limited sample size of the current study, it is plausible that there was not enough statistical power to show these subtle changes.
The results obtained from stratifying the universal group into high anxiety and low-to-average anxiety are consistent with the theory that the low baseline of the majority students accounts for the lack of change in universal groups’ anxiety outcomes. Results revealed no significant change in the low-to-average anxiety students post-program. In comparison, children from the universal group who reported high anxiety pre-program did experience a significant decrease in mean levels anxiety symptoms post-program. These students were, on average, less anxious after the program. It was hypothesised that this decrease in anxiety for the high-anxious groups would be equivalent to the improvement in the targeted group. In fact, the effect sizes in the current study indicated the high-anxiety universal group participants reported an even greater decrease in anxiety scores than the targeted group.
As such, in terms of anxiety outcomes, the high-anxiety students in the universal delivery condition seemed to fare just as well, if not better, as those in the targeted group program. Notably, at baseline, there was no significant difference between the average anxiety levels of the targeted group and the high-anxiety universal group. It is likely that these high-anxiety children in the universal group would be selected for a targeted group intervention, if this was the chosen method of delivery. Yet despite doing the program alongside non-anxious peers who may not experience the same challenges as them, the results of this study suggest that these students fared at least as well as the targeted group.
An expectation of the current study was that qualitative measures (specifically semistructured interviews) would provide a more in-depth understanding of the difference between targeted and universal approaches to social and emotional learning programs. The insights provided by the students interviewed allowed for an understanding of their experiences of the program, and the qualitative differences between the participants’ experiences. In particular, it was apparent that helpful thinking and mindfulness strategies were particularly salient and useful elements of the program, according to all of the interviewees. Interviewees who reported themselves as anxious used these strategies to cope during times of emotional distress, such as times where they were feeling sad, nervous or angry. For example, Lulu describes how mindfulness and cognitive strategies ‘helped [her] to stop crying’ when she had been offended by her peers. Comparatively, the non-anxious interviewees, Dylan and Jackson, both talked about using breathing strategies and positive self-talk before big sporting events. All interviewees were also able to recall specific strategies they could use if they had problems in the future. In summary, it seemed that mindfulness and cognitive strategies were useful to both anxious and non-anxious children. Children who were experiencing distress employed these as coping strategies. In comparison, for the non-anxious interviewees, the mindfulness and cognitive strategies became tools of resilience; internal resources they could call upon in challenging situations (Fergus & Zimmerman, Reference Fergus and Zimmerman2005).
Social experience and social connectedness emerged as a poignant theme for all six participants. This is perhaps unsurprising, given the importance of peer relations during middle childhood (Kranke, Floersch, Townsend, & Munson, Reference Kranke, Floersch, Townsend and Munson2010; Parker & Asher, Reference Parker and Asher1993). It was noteworthy that students from the targeted group mentioned feelings of anxiety at the beginning of the program due to the experience of being in a group with children they did not know. However, two of the targeted group interviewees found that making new friends in the group was, overall, a positive experience. The exception to this experience was Ella, who lamented being in a group with younger children and expressed that she would have felt reassured by the presence of her peers and that the content might have been more age appropriate with her ‘own class’. This is a telling illustration of how, even in a small, targeted group, participant experiences can vary significantly. Ella's comments also highlight that within the targeted group there were some sources of heterogeneity, which may make it more difficult to tailor the program to the needs of the group. While the group may be more homogenous in terms of whether or not children experience anxiety, other factors, such as differences in age and maturity, may present challenges for facilitators, especially when participants are selected from different grades.
Social connectedness was also mentioned in the interviewees from the universal group. The participants expressed enjoying working with their friends and peers and how they found having friends in the group reassuring. The universal group interviewees expressed that they really enjoying working in small group formats. Jackson described how he benefitted from sharing ideas and hearing the perspectives of others in the group: ‘You don't have to think about it just by yourself, you've got a whole group to communicate with.’ This supports the use of collaborative learning strategies, which emphasises student-to-student interaction, and allows children to learn from their peers (Herbert & Lohrmann, Reference Herbert and Lohrmann2011).
In the universal group, interviewees indicated that not all content was relevant to them. Dylan reported that he did not really enjoy the mindfulness exercises, while Jackson said he could not relate to discussions around managing anger. However, both interviewees were insightful enough to recognise that these activities were included because they were relevant to other students in the class. Despite not finding every aspect of the program relevant, both interviewees were able to identify components of the program that they enjoyed or found highly relevant, and expressed that they were pleased to be included in the program.
Overall, the qualitative component of this study indicated that both programs were, for the most part, positively appraised by the six children interviewed. In seems that one benefit of the targeted approach, from the interviewees’ perspective, was the opportunity to make new friends and getting to know children from different year levels. In the universal condition, participants enjoyed learning alongside friends and familiar peers and were able to see the applicability of material, even if some components were less relevant than others. While there was no significant decrease in average anxiety post-program for the average-to-low anxiety participants, the information gathered from Dylan and Jackson indicates that the program was a positive experience that equipped them with strategies that they could use both now and in the future.
Implications
A key objective of the current study was to provide information on which method of delivery is most desirable, to inform the implementation of social and emotional programs in schools with limited resources. Ideally, research suggests that schools should implement both universal and targeted social and emotional learning programs (Greenberg et al., Reference Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnik and Elias2003; Offord, Kraemer, Kazdin, Jensen, & Harrington, Reference Offord, Kraemer, Kazdin, Jensen and Harrington1998). Evidence indicates that it is preferable for schools to support social and emotional development with a three-tier approach to supporting the social and emotional wellbeing of students that encompasses universally implemented social and emotional learning, targeted programs addressing specific concerns, and intensive individualised treatment (Osher et al., Reference Osher, Sprague, Weissberg, Axelrod, Keenan, Kendziora and Zins2008).
The results of the current study support a multilevel approach, with findings that both universal and targeted approaches to social and emotional learning result in benefits to students. However, in the face of limited time and finances, consideration for how to capitalise on resources is warranted. Given that highly anxious students in the universally delivered program saw even greater decreases in anxiety compared to those in the targeted group, this study suggests that the universal approach may be most appropriate. Furthermore, the qualitative results suggest even low-anxiety children could benefit from participating in the program. In particular, the program is likely to foster resilience in these students by equipping them with internal resources to manage future adversity.
In addition to these advantages, the universal approach embodies the values of inclusivity and equal opportunity. It avoids the risk of some children who are experiencing emotional distress by ‘flying under the radar’ and not being taught the skills to cope with their distress (Flett & Hewitt, Reference Flett and Hewitt2013). This is particularly important, given that school personnel who are often responsible for referring students to targeted programs have difficulties identifying children who are experiencing internalising difficulties, and may over-refer student with externalising problems (Papandrea & Winefield, Reference Papandrea and Winefield2011). In comparison, it seems that the targeted group approach did not result in larger effect sizes for anxious students, suggesting that participating in a universal program would not come at a cost for anxious students. Rather, they may experience even greater decreases in anxiety. The universal program may also provide the opportunity for anxious students to learn adaptive coping strategies from their well-adjusted peers.
Of course, the benefits of the targeted approach should not be overlooked. It is notable that anxiety reductions were evident in this group, and that interviewees reported that they enjoyed the opportunity to make friends with like-minded others from different grades. However, on balance it seems that the universal approach may result in a greater number of benefits to a greater number of students. Targeted approaches may be more appropriate as second-tier interventions that target specific concerns with students who are still experiencing distress after participating in universal social and emotional learning programs (Osher et al., Reference Osher, Sprague, Weissberg, Axelrod, Keenan, Kendziora and Zins2008). As second-tier interventions, targeted groups could be focused on addressing specific difficulties, which may allow for a more homogeneous group of participants.
Consistent with previous research, the current study supports the practice of integrating mindfulness and cognitive strategies into social and emotional programs (Lowry-Webster et al., Reference Lowry-Webster, Barrett and Dadds2012; Schonert-Reichl et al., Reference Schonert-Reichl, Oberle, Lawlor, Abbott, Thomson, Oberlander and Diamond2015). While this study cannot draw implications about the causal mechanisms for the decreases in anxiety symptoms, interviewees were able to recall mindfulness and cognitive strategies and articulate how they used these strategies in their day-to-day lives, suggesting these were useful components of the program.
A final implication of the current study is that it demonstrates the utility of using a mixed-methods design in evaluating program outcomes. While both the quantitatively measured anxiety outcomes and the qualitative data on the experiences of students in the program are useful information, the combination of both sources of information allows for a far more comprehensive understanding of the both delivery methods. The exploratory sequential design was particularly well suited to the evaluation of both methods of delivery, as it allows for some qualitative exploration of elements that were able to be adequately captured by the quantitative outcomes.
Limitations and Future Directions
Several limitations should be considered when interpreting the results from the current study. First, previous research suggest that school climate (such as values, norms, policy, interpersonal relationships, and leadership) moderates the effectiveness of social and emotional learning programs (Hughes, Cavell, Meehan, Zhang, & Collie, Reference Hughes, Cavell, Meehan, Zhang and Collie2005; McCormick, Cappella, O'Connor, & McClowry, Reference McCormick, Cappella, O'Connor and McClowry2015). In this study, different schools chose different delivery methods, so conditions were not randomly assigned. As such the climate of each school may have had an impact on which delivery method the schools chose to implement, and how well the program was received by the school community. Randomisation to delivery conditions and the inclusion of a control group would have strengthened the design of the current study.
A further limitation of the study was that the sample size was relatively small, especially when the participants were divided into high and low anxiety groups. However, significant effects were still evident with large effect sizes for high anxiety participants. As the analysis used repeated measures for each individual, there was enough statistical power despite the small sample. A greater sample size in future could reveal subtler changes in anxiety, and would be beneficial in increasing the generalisability of these results to the wider population.
In terms of the qualitative phase, the aim of this study was to gain an in-depth understanding on a limited number of issues, so interviews were intentionally kept brief and focused. A broader exploration of student experiences would likely be valuable; however, this was beyond the scope of the current study. This study provided a nuanced account of the experiences of six students. However, it does not seek to draw conclusions about how frequent these experiences are for students who participated in social and emotional learning programs. It is not assumed that these accounts are representative of all participants, and future research should focus on gaining insights from a larger cohort of students.
Finally, the focus of the current study was on anxiety outcomes. Future research might consider comparing other important outcomes such as impacts on depressive symptoms, prosocial behaviour, and academic achievement.
Conclusions
In summary, the current study found that anxious students in both the universal and targeted social and emotional learning programs were significantly less anxious post-program. Qualitative data revealed that the interviewees found the program to be a positive experience, and saw emerging themes of mindfulness, helpful thinking, social connectedness and experience, and recognising individual differences. Taken together, the results indicate that universally implemented programs are the most appropriate method of delivery for social and emotional learning programs, as they are likely to benefit the greatest number of students and are more inclusive. This study supports previous findings that mindfulness and cognitive strategies are useful components of social and emotional learning, and that collaborative learning strategies allow for peer-to-peer learning. Finally, this study provides evidence for the utility of mixed-methods designs in providing a more comprehensive evaluation of program outcomes.
Acknowledgments
No funding was received by the authors or university to support this project.
Author Contributions
V.E. developed the research question, conducted the analyses, and wrote the first draft. S.C. managed the project, providing feedback on editing and drafts, and finalised the draft for publication.
Conflicts of Interest
The authors declare no conflict of interest, and have no ownership or commercial interest in the Triple R project.
Supplementary material
To view supplementary material for this article, please visit https://doi.org/10.1017/jgc.2018.22
