A central function of schools is to equip students with knowledge and skills that will help them succeed in life. The most quantifiable method to determine whether this function has been achieved is through grades, assessments, and performance on standardised tests (Australian Curriculum Assessment and Reporting Authority [ACARA], 2008). As such, quantified measures of academic achievement are generally considered to be the gold standard indication of school success (Pollock & Winton, Reference Pollock and Winton2012; Stemler, Bebell, & Sonnabend, Reference Stemler, Bebell and Sonnabend2011). However, many schools also value student wellbeing, school belonging (Allen, Kern, Waters, & Vella-Brodrick, Reference Allen, Kern, Waters and Vella-Brodrick2016; Quinlan, Swain, Cameron, & Vella-Brodrick, Reference Quinlan, Swain, Cameron and Vella-Brodrick2014), and civic, emotional, and cognitive development (Stemler et al., Reference Stemler, Bebell and Sonnabend2011).
Most research investigating student wellbeing has focused on student level data (e.g., Reitzner, Reference Reitzner2014). Yet, numerous factors across multiple socioecological levels have an impact on student wellbeing, including peers, families, teachers, neighbourhoods, school climate, policies, laws, and cultural norms (Allen, Vella-Brodrick, & Waters, Reference Allen, Vella-Brodrick and Waters2016; Allen, Kern, Vella-Brodrick, Hattie, & Waters, Reference Allen, Kern, Vella-Brodrick, Hattie and Waters2016). Organisational features of a school, such as the structures, goals, policies, and norms, play an important role in creating a climate that either promotes or impedes wellbeing (Swearer et al., Reference Swearer, Peugh, Espelage, Siebecker, Kingsbury, Bevins, Jimerson and Furlong2006).
One such organisational feature is the vision or mission statement (VMS) of the school. Most schools have such a statement, which explicitly outlines the school's purpose, values, and priorities. The current study assesses whether there is a relationship between the degree to which schools prioritise academic achievement and wellbeing, as indicated by their VMSs, and the academic achievements of their students, as measured by National Assessment Program — Literacy and Numeracy (NAPLAN) results.
School Vision and Mission Statements and Achievement
It has been argued that VMSs provide schools with direction (Allen, Vella-Brodrick et al., Reference Allen, Vella-Brodrick and Waters2016). They ‘arise from a set of values that answer fundamental questions about the purpose of education and how the educational programs should be carried out’ (Boerema, Reference Boerema2006, p. 182) and typically define the physical, social, and political contexts that govern an institution. According to Fritz (Reference Fritz1996), VMSs exist to aid school improvement and provide insight into the daily operations of the school. Jones and Crochet (Reference Jones and Crochet2007) consider VMSs vital for school improvement, as they identify desired functional levels, provide a framework for decision making and actions, promote staff teamwork, and frame the goals that are set for the future.
School VMSs generally include both academic aspects and a range of other values including academic motivation, mental health promotion, school belongingness, teacher support, and personal character (Allen, Kern et al., Reference Allen, Kern, Waters and Vella-Brodrick2016; Stemler & Bebell, Reference Stemler and Bebell1999; Wilkerson, Reference Wilkerson2010). Only a handful of studies have subsequently linked these values to student outcomes. Brown, Choi, and Herman (Reference Brown, Choi and Herman2011) examined 34 independent schools in Texas that employed the Courageous Leadership Academy (CLA) — a school reform model that uses the development of core mission, vision, values, and goals as a central function. There was a strong relationship between the quality of implementation and student achievement. Davis, Ruhe, Lee, and Rajadhyaksha (Reference Davis, Ruhe, Lee and Rajadhyaksha2007) sampled students across 16 universities to investigate whether the use of ethical content in a VMS reflected students’ ethical orientation. Findings suggested that student character was affected by the traits that were reinforced in the VMSs, especially in relation to heart traits (associated with ethical behaviour), confidence, and learning. While the causality of these relationships cannot be determined (i.e., whether VMSs influence the character of the students within an institution, or students choose institutions that match their orientation, or a range of other confounding factors), these findings support the potential usefulness of VMSs within educational settings.
The use of VMSs by organisations is sometimes met with criticism. A central concern is their use in marketing and promotion campaigns, which raises questions regarding their applied and measurable impact in schools. For example, although a vision and mission statement may explicitly imply that a school prioritises the social and emotional development of students, that priority is not necessarily carried through in the implicit culture of the school. Still, some evidence suggests that when VMSs are well written, transparent, and void of fashionable statements, their values are often reflected in the actual practices of a school (Chapple, Reference Chapple2015; Manley & Hawkins, Reference Manley and Hawkins2009; Stemler et al., Reference Stemler, Bebell and Sonnabend2011). Moreover, a VMS, irrespective of whether or not it is applied to daily practices, can still indicate what the school may aspire to (Stemler et al., Reference Stemler, Bebell and Sonnabend2011). Thus, while it is beyond the scope of this article to evaluate whether or not schools actually implement the priorities noted in their statements, there still is value in considering the potential impact of VMSs on academic outcomes.
School Values and Academic Achievement
Allen, Kern et al. (Reference Allen, Kern, Waters and Vella-Brodrick2016) analysed the VMSs of secondary schools in Victoria, Australia, and identified 10 common themes: academic motivation, personal characteristics, school belonging, teacher support, other support, mental health promotion, the school environment, Christianity, future focus, and individual needs. While academic motivation was the most frequent theme, over half of the schools also included mental health promotion (MHP) and school belonging. In the current analysis, we examine the extent to which these three themes relate to academic achievement.
Academic motivation and academic achievement
Academic motivation has been conceptualised in multiple ways, but is generally concerned with the ‘extent to which students are motivated to learn and do well in school’ (Libbey, Reference Libbey2004, p. 278), Lee, McInerney, Liem, and Ortiga (Reference Lee, McInerney, Liem and Ortiga2010) define motivation as ‘an internal state that instigates, directs, and maintains behaviour’ (p. 264). The concept of motivation is best explained by one of the most dominant theories of motivation, self-determination theory (SDT; Ryan & Deci, Reference Ryan and Deci2000). SDT suggests that behaviours and cognitions are affected by one's source of motivation, which runs on a continuum from intrinsic motivation (internally driven; e.g., personal goals) to extrinsic motivation (externally driven; e.g., through rewards) to amotivation (absence of motivation). SDT explicitly includes both cognitive and behavioural components. Thus, academic motivation incorporates cognitions such as planning and setting goals, and behaviours such as the act of pursuing those goals and achieving higher levels of performance. For instance, DiPerna, Volpe, and Elliot (Reference DiPerna, Volpe and Elliott2002) found that academic skills and competence (cognitions) related to better study skills and greater school engagement (behaviours), which in turn resulted in higher academic achievement.
As motivation becomes increasingly internalised, better performance often occurs (e.g., DiPerna et al., Reference DiPerna, Volpe and Elliott2002; Robbins et al., Reference Robbins, Lauver, Le, Davis, Langley and Carlstrom2004). For instance, with 450 adolescents in Nigeria, academic motivation had a significant positive impact on secondary school students’ mathematic scores (Tella, Reference Tella2007). A meta-analysis of 109 studies, which defined academic motivation as being internal in nature (e.g., academic discipline, commitment, determination, and goal striving), found that motivation was highly correlated with high school and college grade point average, beyond all other factors investigated, including social support and socio-economic status (Robbins et al., Reference Robbins, Lauver, Le, Davis, Langley and Carlstrom2004).
In the current study, statements within the VMSs reflect the cognitive element, and academic performance reflects the behavioural component of academic motivation. VMSs often present an ideal vision for students and the school as a whole and thus reflect more internalised forms of motivation, with statements such as ‘We aim to develop imaginative, innovative thought and independent critical thinking’ (Allen, Kern et al., Reference Allen, Kern, Waters and Vella-Brodrick2016).
Mental health promotion and academic achievement
MHP is defined as proactively protecting and optimising mental health through enhancing strengths and capabilities, as opposed to reactively addressing mental-health disorder (Allen & McKenzie, Reference Allen and McKenzie2015; Donovan, James, Jalleh, & Sidebottom, Reference Donovan, James, Jalleh and Sidebottom2006; Kobau et al., Reference Kobau, Seligman, Peterson, Diener, Zack, Chapman and Thompson2011). At the school level, MHP may exist through mental health promotion campaigns, social-emotional learning programs, and the implementation of collaborative services that nurture holistic wellness (Rickwood, Reference Rickwood, O'Hanlon, Patterson and Parham2007).
It is well established in the literature that adolescents who experience mental health concerns during secondary school are at an educational disadvantage across a range of measures, including achievement (e.g., Greenberg et al., Reference Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnick and Elias2003; Zins, Weissberg, Wang, & Walberg, Reference Zins, Weissberg, Wang and Walberg2004). Durlak, Weissberg, Dymnicki, Taylor, and Schellinger (Reference Durlak, Weissberg, Dymnicki, Taylor and Schellinger2011) presented the strongest evidence for MHP, by conducting a meta-analysis of 213 social and emotional learning programs involving over 270,000 students, finding that such programs were significantly and positively associated with higher levels of achievement, potentially boosting test scores by as much as 11 percentile points. This finding has been supported by other large reviews and syntheses of the literature (e.g., the Collaborative for Academic Social and Emotional Learning [CASEL], 2007; Payton et al., Reference Payton, Weissberg, Durlak, Dymnicki, Taylor, Schellinger and Pachan2008). In Australia, Dix, Slee, Lawson, and Keeves (Reference Dix, Slee, Lawson and Keeves2012) investigated 260 schools employing the whole school mental health promotion initiative, KidsMatter. Across a 2-year period, schools that implemented KidsMatter improved their NAPLAN scores at an equivalent of 6 months’ additional schooling. Slee et al. (Reference Slee, Lawson, Russell, Askell-Williams, Dix, Owens and Spears2009) found that most teachers agreed that students who are socially and emotionally competent learn more.
School belonging and academic achievement
School belonging refers to an affective experience by students that involves a sense of connectedness or affiliation towards the school. A sense of school belonging is influenced by individual (e.g., academic motivation and self-efficacy), relational (e.g., perceived relationships with teachers and peers), and organisational (e.g., policies and practices concerned with safety) factors within a school setting (Allen, Vella-Brodrick et al., Reference Allen, Vella-Brodrick and Waters2016).
Studies suggest that a significant positive relationship exists between school belonging and academic achievement, both cross-sectionally and longitudinally (Benner, Graham, & Mistry, Reference Benner, Graham and Mistry2008; Bonny, Britto, Klostermann, Hornung, & Slap, Reference Bonny, Britto, Klostermann, Hornung and Slap2000; Mo & Singh, Reference Mo and Singh2008; Wilkinson-Lee, Zhang, Nuno, & Wilhelm, Reference Wilkinson-Lee, Zhang, Nuno and Wilhelm2011; Sirin & Rogers-Sirin, Reference Sirin and Rogers-Sirin2004, Reference Sirin and Rogers-Sirin2005). For example, Sirin and Rogers-Sirin (Reference Sirin and Rogers-Sirin2004) found school belonging to be one of the strongest predicators of academic performance in their sample of 336 African-American students. A second study found similar findings, even after controlling for gender, year level, cognitive functioning, and the educational background of parents (Sirin & Rogers-Sirin, Reference Sirin and Rogers-Sirin2005). Across 572 young people, Gillen-O'Neel and Fuligni (Reference Gillen-O'Neel and Fuligni2013) found that school belonging was associated with a higher level of academic motivation across a 4-year period. Moreover, the Wingspread Declaration on School Connections (2004) presented evidence that school belonging increases classroom engagement, which is considered to be an element of academic motivation (e.g., Connell & Wellborn, Reference Connell, Wellborn, Gunnar and Sroufe1991; Croninger & Lee, Reference Croninger and Lee2001; Klem & Connel, Reference Klem and Connell2003). Additional support for such findings appear throughout the literature (e.g., Battistich, Solomon, Kim, Watson, & Schaps, Reference Battistich, Solomon, Kim, Watson and Schaps1995; Roeser, Midgley, & Urdan, Reference Roeser, Midgley and Urdan1996).
Measuring Academic Outcomes
Although academic success can be conceptualised and measured in different ways, NAPLAN is currently the dominant outcome of interest in Australia (Guenther, Reference Guenther2013; Miller & Voon, Reference Miller and Voon2012; Polesel, Dulfer, & Turnbull, Reference Polesel, Dulfer and Turnbull2012; Quinell & Carter, Reference Quinell and Carter2011). NAPLAN is an annual assessment test undertaken by Australian students in Years 3, 5, 7, and 9. It is managed by ACARA, an independent body established by the Australian Government for the purpose of data collection, assessment, and curriculum implementation (Dix et al., Reference Dix, Slee, Lawson and Keeves2012). NAPLAN is administered nationally each year in government, independent, and Catholic schools. The test assesses four domains: reading, writing, language conventions (spelling, grammar, and punctuation), and numeracy, and provides standardised information on student performance across these areas. Average scores for each school are publicly available (http://www.myschool.edu.au), while the data for individual students is only released to the school (Miller & Voon, Reference Miller and Voon2012; Thompson, Reference Thompson2013).
NAPLAN and other standardised tests such as the Scholastic Aptitude Test (SAT) in the United States have been criticised for reducing the complexity of teaching practice to a set of numbers (Hardy & Boyle, Reference Hardy and Boyle2011). It imposes a one-size-fits-all model, which disadvantages minority groups (Guenther, Reference Guenther2013; Wigglesworth, Simpson, & Loakes (Reference Wigglesworth, Simpson and Loakes2011). Other criticisms of NAPLAN include possible narrowing of curriculum, manipulation of data, reduction of breadth subjects (e.g., arts, humanities, physical education), and undue stress in young people (Anderson, Reference Anderson2009; Comber, Reference Comber2012; Comber & Nixon, Reference Comber and Nixon2009; Lingard, Reference Lingard2010; Thompson & Cook, Reference Thompson and Cook2014). ‘Cheating and gaming’ can occur particularly when many stakeholders (e.g., government, parents, teachers) are interested in the results (Brown & Hattie, Reference Brown, Hattie, Suggate and Reese2012). Further, the validity of the numeracy tests have been questioned for using loaded language and vocabulary likely to confound students’ performance (Quinell & Carter, Reference Quinell and Carter2011).
Despite these criticisms, standardised tests continue to be considered the primary measures of academic performance (Volante, Reference Volante2004). When used properly, NAPLAN can aid educators in determining students’ strengths, achievement, growth, weaknesses, and decrements (Brown & Hattie, Reference Brown, Hattie, Suggate and Reese2012; Holmes, Reference Holmes2010; McMillan, Reference McMillan2000, Thompson, Reference Thompson2013). For better or worse, NAPLAN results are used to inform educators, parents, and guardians about a student's progress in the areas of literacy and numeracy (Quinell & Carter, Reference Quinell and Carter2011) and is also used for accountability purposes on a national level.
The Current Study
Most studies linking academic motivation, MHP, and school belonging with academic outcomes have focused on individual students. While motivation, mental health, and school belonging are internal characteristics of the students, the school environment plays an important role in creating a climate that is more or less conducive to wellbeing (Allen, Kern et al., Reference Allen, Kern, Waters and Vella-Brodrick2016). Although there will always be individual variation within the school, on average, academic performance might be better in more supportive environments. If VMSs do indeed guide the priorities, policies, and implicit and explicit norms of the school, then the question becomes the extent to which the values align with desired outcomes across the school as a whole.
In the current study, we examined associations between the presence of academic motivation, MHP, and school belonging themes within VMSs and academic achievement, operationalised as average NAPLAN scores. Based on the significant and positive relationship between academic achievement and these three variables found in past literature at the student level, we hypothesised that greater endorsement of academic motivation, MHP, or school belonging would be positively related to academic achievement at the school level. Further, due to the importance of social and emotional factors, we hypothesised that emphasising MHP or school belonging along with academic motivation (i.e., explicitly prioritising both the academic and socio-emotional characteristics of students) would relate to the highest NAPLAN scores, compared to schools that do not include these social and emotional factors in their VMSs.
Method
Participants
We used a set of VMSs from secondary schools in Victoria, Australia, compiled by Allen, Kern et al. (Reference Allen, Kern, Waters and Vella-Brodrick2016). Across 579 secondary schools, a sample of 308 schools, stratified by school type (government, independent, Catholic), gender composition (all boys, all girls, co-educational), and location (urban, rural) was randomly selected. In January 2012, Allen, Kern et al. (Reference Allen, Kern, Waters and Vella-Brodrick2016) sourced VMSs directly from each individual school's publicly available website. Where a VMS was not publicly available, schools were directly contacted (see Allen, Kern et al. for further details regarding data collection).
Of the 308 schools, Year 7 and/or Year 9 NAPLAN data were available for 287 schools. Missing NAPLAN data were primarily due to the school being newly founded, closed down, amalgamated with another school, or not having Years 7 or 9 at their school. The sample included government (n = 158), independent (n = 75), and Catholic (n = 54) schools from urban (n = 162) and rural (n = 125) settings, with co-educational (n = 245), boys (n = 13) and girls (n = 29) gender compositions.
Obtaining a general achievement score
NAPLAN data were chosen due to their public availability for most schools, which enabled comparisons to be made across as many schools as possible. NAPLAN data for Year 7 and Year 9 students were obtained on request from ACARA for 2012, matching the year that the school vision and mission statements were collected. The domains of Grammar and Punctuation, Numeracy, Persuasive Writing, Reading, and Spelling are highly correlated (e.g., r > .93), and when combined, create a reliable single achievement factor (α = .90) (Dix et al., Reference Dix, Slee, Lawson and Keeves2012). As such, in the present study, we combined the five NAPLAN domains to create a general achievement score (Year 7: α = .97, Year 9: α = .90). The number of students within each school with Year 7 and 9 NAPLAN data was also recorded (total N = 34,409; M = 120, SD = 91.46, range = 5–630).
Identifying Themes
We used two approaches to determine whether academic motivation, MHP, and school belonging were present within each statement: a manual coding approach, typically used in qualitative research, and a supervised word count approach, used in quantitative research.
First, through an emergent coding approach, Allen, Kern et al. (Reference Allen, Kern, Waters and Vella-Brodrick2016) rated the occurrence of each theme in the statements. Ten major themes and a number of subthemes within the VMSs were identified and coded by two independent researchers (interrater reliability: χ = .81). With emergent coding, categories are established following the preliminary examination of the data and then the researcher(s) seeks to examine these themes through an iterative process. Allen, Kern et al. used a coding rubric containing 10 major themes, each with a varying number of subthemes. Aligned with other studies (e.g., Stemler et al., Reference Stemler, Bebell and Sonnabend2011), themes were classified as major if they were mentioned by at least 20% of the schools, and variants classified by fewer schools were classified as subthemes. Based on these ratings, we used a dichotomous indicator of the overall academic motivation, MHP, and school belonging themes being present (coded 1) or absent (coded 0).
Second, we used a supervised lexical analysis approach. In lexical methods, a priori categories (or dictionaries) are developed based upon theory, and then the relative frequencies (i.e., total occurrence adjusted by the number of words) that those categories occur are automatically counted (see Kern et al., Reference Kern, Park, Eichstaedt, Schwartz, Sap, Smith and Ungar2016, for an overview of the methodology and consideration of this approach). The most commonly used lexica approach in the social sciences is the Linguistic Inquiry and Word Count (LIWC) program, developed by Pennebaker, Chung, Ireland, Gonzales, and Booth (Reference Pennebaker, Chung, Ireland, Gonzales and Booth2007), which includes over 100 dictionaries (e.g., positive emotion, family, pronouns).
As no existing lexica adequately captured the three themes, we developed three new lexica. Drawing on descriptions and definitions of each theme from the literature, we listed relevant words and two to three word phrases (e.g., teacher support). We added relevant words from the LIWC dictionaries, and consulted with other researchers to identify additional relevant terms. We next examined sample cases where the words from the lexicon occurred, to ensure we captured the correct context. For example, for Academic Motivation, the word ‘motivation’ was only used once and did reflect motivation for learning (i.e., ‘The structure of learning to foster motivation and engagement’), whereas the word ‘motivated’ referred to factors outside of academic motivation, such as ‘staff are passionate, highly motivated and professional’. We removed ‘motivated’ and other ambiguous words, resulting in a final list of 39, 40, and 24 words and phrases respectively in the academic motivation, mental health promotion, and school belonging lexica (see Supplement S1 for final lexica and the relative frequency that each occurred).
Using the resulting lexica, we processed the statements using NVivo 10 for Windows (2014), a Computer Aided Qualitative Data Analysis System (Richards, Reference Richards2005).Footnote 1 First, all words in the statements were indexed and tabulated with frequencies and synonyms and compared to the lexica to ensure no variation of relevant words had been overlooked. Next, computer-generated text searches found every word and phrase in the lexica and coded all mentions to nodes, which were organised into a thematic structure in line with the lexica themes. These nodes, which contained all references to key words and phrases, were then placed in a matrix intersecting them with all cases (schools). The results of these matrix-coding queries were then exported to an Excel table that contained the absolute frequencies, number of words, and relative frequencies (i.e., frequency adjusted by the number of words), separated by school. As frequencies had many zero values and were rightly skewed, we also computed a dichotomous indicator of each theme being present (coded 1) or absent (coded 0).
Data Analyses
The qualitative (emergent coding) approach produced dichotomous indicators of each theme being present, and the quantitative (supervised lexical) approach produced the continuous relative frequency and a dichotomous indicator of each theme being present. We first compared achievement scores across the six dichotomous variables using t tests. Second, we calculated correlations between the three continuous frequency variables and the mean NAPLAN scores, separately for Year 7 and Year 9 students, using Spearman's rank order correlations. We also regressed Year 7 or Year 9 achievement scores on the relative frequency of the three themes, testing the unique contribution of each them when controlling for the other two. Finally, to consider the potential value of including MHP and school belonging along with academic motivation, for schools that included academic motivation, t tests compared: (1) MHP present or absent, (2) school belonging present or absent, and (3) both MHP and school belonging present versus both being absent.
NAPLAN scores for a school represent the average scores of the students at the school. Unless student performance is homogenous, such estimates are more precise with larger sample sizes. School sizes ranged from 5 to 630, and thus unweighted correlations may overestimate effects in small schools and underestimate effects in large schools. To consider the robustness of estimates, analyses were first conducted at the school level (i.e., K = the number of schools), and then repeated weighting by the number of students (n) within the school.
T test, correlation, and regression analyses were performed using IBM SPSS Statistics (Version 22.0) software. We applied bootstrapping to the analyses, and include the 95% confidence intervals around the correlation coefficients or mean difference scores.Footnote 2
Results
Table 1 summarises descriptive information. Across methods, academic motivation was the most common theme, occurring in 87.5% and 73.9% of the statements for the two coding approaches respectively. School belonging was consistent across the two approaches (59.2% vs. 55.1%), appearing in just over half of the statements. MHP was less consistent across the two coding approaches, with the emergent coding approach identifying the presence of MHP in the statement almost twice as often as the lexical approach (67.2% vs. 35.2% of school statements).
Descriptive Statistics for Language and Achievement Data
Across both qualitative and quantitative coding approaches, achievement was negatively related to academic motivation and school belonging (see Table 2).Footnote 3 In contrast, for MHP, the qualitative approach indicated a negative association with achievement, Year 7: t(33,923) = −15.60, p < .001; Year 9: t(32,407) = −19.59, p < .001, whereas the quantitative approach indicated a positive association, Year 7: t(33,923) = 12.58, p < .001; Year 9: t(32,407) = 15.31, p < .001. Results were consistent across Year 7 and Year 9, with MHP having a stronger effect in Year 9 than in Year 7. Correlation and regression analyses with the continuous relative frequency variables followed the same pattern of results (see Supplement Tables S3 and S4), with academic motivation and school belonging negatively related to academic achievement and MHP positively related to academic achievement.
T Tests for Dichotomous Variables (Indicating Theme was Included or Not, Using Qualitative and Lexical Coding Approaches) and Achievement Scores (Weighted by the Number of Studies; see Supplement S2 for Unweighted Estimates)
Note. MD = mean difference, SE = standard error, CI = confidence interval. Mean difference and 95% confidence intervals based on 1000 bootstrapped samples.
Finally, we considered the potential added benefit of including MHP and school belonging (i.e., social and emotional themes) versus only academic motivation (i.e., a cognitive theme; see Supplemental Table S5). School belonging related to lower achievement scores in both the qualitative approach, Year 7: t(29,527) = −8.69, p < .001; Year 9: t(28,239) = −4.46, p < .001, and the lexical approach, Year 7: t(25,881) = −12.69, p < .001; Year 9: t(24,613) = −6.55, p < .001. Again, MHP depended on coding approach; the qualitative approach suggested a negative association between academic achievement and mental health promotion, Year 7: t(29,527) = −2.40, p = .02; Year 9: t(28,239) = −4.97, p < .001, whereas the lexical approach suggested a positive association, Year 7: t(25,881) = 8.98, p < .001; Year 9: t(24,613) = 9.11, p < .001.
Discussion
This study investigated the relationship between the extent to which schools explicitly prioritise academic motivation, mental health promotion, and school belonging in their values and mission statements and an objective measure of a school's average academic achievement, using two approaches for coding themes (emergent coding and supervised lexical analysis). Including academic motivation or school belonging related to lower academic performance. Results were less consistent for MHP, with the emergent coding approach suggesting a negative association between MHP and performance and the lexical analysis suggesting a positive association.
It is well established in the literature that the variables addressed in this study — academic motivation, MHP, and school belonging — are related to positive academic benefits for secondary school students at the individual level of analysis, but less is known at the collective level. School leaders play an important role in creating a school culture that is more or less supportive of achievement, belonging, and wellbeing, but it is challenging to determine the impact of different priorities and decisions. Although VMSs are often viewed as pivotal for guiding school policy and procedures (e.g., Allen, Vella-Brodrick et al., Reference Allen, Vella-Brodrick and Waters2016; Allen, Kern et al., Reference Allen, Kern, Waters and Vella-Brodrick2016 Stemler & Bebell, Reference Stemler and Bebell1999; Stemler et al., Reference Stemler, Bebell and Sonnabend2011), only a limited number of studies have compared these statements with academic outcomes (e.g., Brown et al., Reference Brown, Choi and Herman2011; Davis et al., Reference Davis, Ruhe, Lee and Rajadhyaksha2007).
Aspects of the implicit and explicit school culture are often hard to capture quantitatively and are thus ignored. The current study began with qualitative information — VMSs — and tested two approaches for quantitatively encoding the information. For academic motivation and school belonging, the two encoding approaches were comparable. Across methods, there was a negative association between academic scores and academic motivation and school belonging. This differs from other studies that found VMSs to be effective in influencing student character traits (Davis et al., Reference Davis, Ruhe, Lee and Rajadhyaksha2007). These results also differ from previous findings at the student level that have shown the importance of academic motivation and school belonging for achievement (Anderman, Reference Anderman2002; Booker, Reference Booker2007; DiPerna et al., Reference DiPerna, Volpe and Elliott2002; Robbins et al., Reference Robbins, Lauver, Le, Davis, Langley and Carlstrom2004; Sanchez, Col'on, & Esparza, Reference Sanchez, Col'on and Esparza2005; Walton & Cohen, Reference Walton and Cohen2011). It is possible that this is a true association, characteristic of Australian schools, compared to studies conducted in other countries. Alternatively, this association might be driven by the high percentage of schools that included academic motivation as a theme. It may be that not including the theme is problematic, but once included, a higher frequency does not add any additional benefit.
In contrast, the two approaches differed substantially in identifying the MHP theme, and this corresponded with different associations with academic achievement. The lexical approach suggested that MHP was related to higher NAPLAN scores. This finding supports past research, which has found mental health promotion to be important for student achievement (Greenberg et al., Reference Greenberg, Weissberg, O'Brien, Zins, Fredericks, Resnick and Elias2003; Zins et al., Reference Zins, Weissberg, Wang and Walberg2004). The correlation was slightly stronger in Year 9 than in Year 7, suggesting an increasing impact as students are in the school for a longer period of time. However, the emergent coding approach suggested an inverse association. The lexical approach was considerably more conservative in nature. It is unclear which approach is more accurate; the linguistic analyses add an alternative method for educational research, but they also add an additional layer of complexity. As technology continues to evolve, it will be increasingly possible to automate the coding of qualitative information. The results here suggest that the approach used can have an impact on the findings, and future research should continue to carefully evaluate the impact of different coding procedures.
At the collective level, it is also unclear what degree of aggregation should occur. Results were consistent across the weighted and unweighted estimates, but the weighted estimates offered greater precision. NAPLAN scores represent the average of the students at the school, and although it provides a simple aggregate measure of performance, it also obscures individual variation. The extent to which a school prioritises academic motivation, mental health promotion, and school belonging may have more of an impact for some students versus others. Further, the extent to which schools live out their explicit values in the day-to-day function of the school is unknown, as is the impact of socioeconomic status. Unfortunately, we did not have such data available, and future research will benefit from considering the moderating effect of socioeconomic status, gender, context, and other variables of interest.
Implications
The field of educational and developmental psychology focuses not only on the individuals within the school community, but also the broader systems and structures of schools. A goal therefore of educational and developmental psychologists is to advocate for the promotion of educational standards and goals, along with the policies and procedures needed to achieve them (Aslanargun, Reference Aslanargun2007). It is important for educational and developmental psychologists to consider all features of a school's system, including VMSs, when promoting wellbeing initiatives at a whole-school level. Designing a VMS for the purpose of improved student outcomes is an important topic of enquiry in educational settings. This study suggests that including MHP within statements may also be beneficial.
VMSs have been used as a social marketing tool for schools, which is also a central element of successful public health change programs (Donovan, Reference Donovan2011). VMSs may serve as a public declaration of valuing mental health. They may also be used to encourage individuals to adopt healthy behaviours. Regardless of the correlations with academic performance, they are an important part of the broader purposes of education. VMSs help create the climate of the school, and send a message to students, staff, and the broader school community that mental health and other aspects of the whole student are important to the school.
Limitations
The practical value of these findings should be interpreted with caution. Research that has investigated whether VMSs have a tangible impact in the school related to what they espouse is mixed (Gioia, Nag, & Corley, Reference Gioia, Nag and Corley2012). We could not determine the extent to which the schools were living up to their VMS. It may be that including such themes in the mission statement only matter to the extent to which curriculum, programs, and strategies actually promote these themes in the school. Further, it is unclear whether school VMSs result in actual cognitions and behaviours at the student level. Future research may wish to understand the extent to which the themes that are included in a school's VMS are put into practice.
We also could not determine how long the school had been using their current VMS. As it takes time for changing values to trickle down through the school, statements that have been around for longer most likely have a greater influence on how students function and feel than new statements, but the time it takes for changes in a statement to have an impact on the culture and practices of a school is unknown. Alternatively, a new vision statement could have a greater impact if it significantly disrupts traditions or is driven by a charismatic leader. Future studies might examine the extent to which VMSs drive stability or change in the school's culture.
In this study, we only focused on academic performance, operationalised through NAPLAN scores as the outcome of interest. While the current educational climate stresses the value of standardised test scores, many would argue that the goal of education goes well beyond academic performance, and includes developing the character and social and emotional health of young people (e.g., Seligman, Ernst, Gillham, Reivich, & Linkins, Reference Seligman, Ernst, Gillham, Reivich and Linkins2009). Future research might consider how different themes link to alternative outcomes, such as student wellbeing, psychosocial maturity, and behavioural measures (e.g., detentions, classroom behaviour, school dropout).
The small effect sizes must be acknowledged. Although the patterns across multiple analytic methods provide support for the benefits of focusing the school's mission on aspects separate from academics, a multitude of factors have an impact on academic performance. Still, effect sizes were similar to other studies examining associations between language and other characteristics (e.g., Kern et al., Reference Kern, Eichstaedt, Schwartz, Dziurzynski, Ungar, Stillwell and Seligman2014). Further, even small effect sizes can have an important impact when considered across tens of thousands of individuals (see Meyer et al., Reference Meyer, Finn, Eyde, Kay, Moreland, Dies and Reed2001, for comparison). For example, the impact of divorce on a child's wellbeing reports a very weak effect size of .09, yet most schools would ensure appropriate support and intervention is in place given the circumstances (Meyer et al., Reference Meyer, Finn, Eyde, Kay, Moreland, Dies and Reed2001).
Conclusion
This study aimed to assess the endorsement of academic motivation, MHP, or school belonging in school VMSs and the relationship with student academic achievement. Despite the cautions surrounding the correlations, this study provides a methodological contribution to future research by introducing the use of language analysis as a tool for future researchers to use when investigating multilevel constructs in schools. VMSs allow schools to express the goals and aspirations for which the school may aspire to and make a public statement of what the school values. Still, the values expressed in a statement alone are insufficient; efforts by school leaders and educational and developmental psychologists should occur at multiple socio-ecological levels to have the greatest positive impact on student wellbeing and achievement.
Financial Support
This work did not receive financial support.
Conflict of Interest
None.
Ethical Standards
This review did not involve human and/or animal experimentation.
Supplementary Material
To view supplementary material for this article, please visit https://doi.org/10.1017/edp.2017.5.
