On 19th March 2020, the Deputy First Minister of Scotland and Cabinet Secretary for Education John Swinney reported to the Scottish Parliament that, in light of the global coronavirus pandemic, schools across Scotland would close from 20th March, mirroring the policy of the UK government announced by the Secretary of State for Education Gavin Williamson the previous day. As part of this closure, Swinney announced that there would be no examinations set for the 2019-20 session, and that the Scottish Qualifications Authority (SQA) would instead enact a certification model employing coursework, teacher estimates of grades and evidence of prior achievement. In outlining the Scottish Government's plan to Holyrood, the Deputy First Minister declared: ‘It is a measure of the gravity of the challenge we now face that the exams will not go ahead this year. With the support of the wider education system, a credible certification model can be put in place that can command confidence in the absence of the exam diet – to ensure that young people in our schools and colleges who through no fault of their own are unable to sit exams, are not disadvantaged.’ (Scottish Government, 2020).
Ensuring rigour and equity in a grading system without examination is self-evidently a formidable task, especially since so much of pupils’ final grades in Scotland (and the rest of the UK) is informed by performance therein. Indeed, the system would need to be robust enough to justify final grades against the legitimate criticism that students were being given awards for courses that were technically incomplete, at least when compared to their predecessors. Like examining bodies serving other nations of the UK, SQA employed an algorithm to solve the lack of an exam, ostensibly taking the various factors outlined by Swinney into account before producing a final mark for every pupil.
Rather than producing a system that was dispassionate and credible, however, these algorithms proved highly controversial (Learmonth, Reference Learmonth2020). Nominally designed to prevent inaccurate grade inflation, in many cases they produced results which departed radically from teacher estimates, downgrading pupil results by one or two grade bands. More troublingly, the system appeared to apply deductions unevenly, with schools in some of Scotland's poorest communities reduced by an average of 15 percentile points, while those in more affluent areas observed an average deduction of fewer than ten (Carrell, Reference Carrell2020).
In all, some 124,564 results were downgraded following the moderation process, representing around 25% of the cohort nationally. As in other parts of the UK, this process attracted a high level of public and media attention, with many criticising SQA of exacerbating social inequalities. This pressure eventually led to the Chief Executive and Head of Qualifications at SQA appearing before the Scottish Parliament's Education Committee, but the change in policy was ultimately a political decision (McArthur, Reference McArthur2020). On 11th August, Swinney announced to Holyrood that, in light of the extraordinary circumstances prompted by coronavirus restrictions, final grades for every pupil in Scotland would instead be predicated wholly on teacher estimates provided to SQA. It was also announced that an independent review (under way at time of writing) would take place, examining a number of areas from the initial advice given to local authorities by SQA through to the impact of the crisis on pupils and families.
It was in the midst of this controversy that the Classical Association Teaching Board (CATB) collaborated with the Classical Association of Scotland (CAS) to survey teachers of Classics across the country.Footnote 1 This was done partly to compare results against those surveyed in England and Wales, but also to assess the reaction of teachers in Scotland, where Classics remains a niche subject, particularly in the state sector. CAS has been engaged in a long-term effort to revive Classics in Scotland's schools since 2017, and so its officers were eager to gauge what impact this kind of controversy might have on its still nascent programme (Imrie, Reference Imrie2019).
Method
In order to ensure a level of comparability between the Scottish and other UK datasets, a survey of seven questions was formulated. This tracked the questionnaire employed by CATB disseminated to schools in England and Wales, but required minor alteration to reflect the specifics of the Scottish curriculum, recognising, for example, that all senior certificated courses assessed by SQA are single-year modules.
The survey requested details of any relevant courses and class sizes taught by respondents, before seeking feedback on their satisfaction with the process and grading conducted by SQA in the 2019-20 session. The survey ended with a question gauging teacher satisfaction with SQA's consultation process for the 2020-21 session.
Our aim in constructing the questionnaire was to offer teachers for whom term had already begun the opportunity to complete the form as quickly or as volubly as suited them. Consequently, the survey offered a multiple-choice element in each question, with additional space for respondents to expand on their answers whenever they wished.
Once the survey was constructed, it was disseminated using a mixture of direct communication with schools that have collaborated with CAS recently, distribution through a mailing list of Classics teachers in Scotland, and publicised on social media (particularly Twitter), shared by the Classical Association and CAS, and assisted by Classics for All and Advocating Classics Education. This combined effort expanded the reach of the survey dramatically and ensured that there was a significant and rapid response over the course of the seven days in which the questionnaire was open, the survey being live between 2nd September and 9th September.
Findings
The CATB-CAS survey gathered 30 responses from teachers across Scotland. These were divided nearly evenly between local authority and independent schools (47% vs 53%) which is striking in itself, since there are relatively so few schools in the state sector offering Classical subjects compared to private institutions.Footnote 2 Respondents covered the entire breadth of senior level teaching in Classical subjects offered by SQA, covering National 5 (N5) Latin and Classical Studies examinations through to Advanced Higher Latin and Classical Studies examinations. Of these, the Higher Classical Studies qualification was the most widely taught, with 83% of teachers surveyed offering this course.Footnote 3
Owing to the minority nature of the subject in Scotland currently, class sizes were predictably low, compared to Classics provision in some other areas of the UK. At N5, classes of 11 pupils or more were found in both Latin (74%) and Classical Studies (73%).Footnote 4 This dropped significantly at Higher level (55% Latin; 68% Classical Studies).Footnote 5 At Advanced Higher, teachers reported class sizes of between 1-5 pupils in 86% of cases for Latin and 91% for Classical Studies. This was valuable data in the context of this year's controversy because it suggests that the grading algorithms were susceptible to an imbalance prompted by the size of cohort being assessed.
Regarding the process and guidance for the 2019-20 session and results initially awarded to pupils, the survey produced a rather more measured response from teachers than one might expect for so emotive a subject in the public consciousness, not to mention the inflammatory press coverage (Blackley, Reference Blackley2020). In fact, when asked about the advice given by SQA (and local authorities) concerning the creation of grade estimates, 75% of respondents rated the advice given by the examinations board as either ‘helpful’ or ‘very helpful’ (respondents were divided equally in these two categories). Of the remaining 25% who assessed SQA negatively here, the majority opted for ‘not so helpful’ rather than ‘not helpful at all.’
On the surface, this would appear to make encouraging reading for SQA; but in dissecting the additional commentary provided with teachers’ answers, it is apparent that there is a significant dichotomy in attitudes to the initial logistics of the process and its eventual results. Most who were satisfied with the guidance offered by the examining body commented that the process was, in reality, entirely unchanged from the process that teachers had been following for years prior: they were asked to formulate estimates on the basis of student performance as normal, with additional encouragement to consider pupil progress in the time lost between school closures and the final exam.Footnote 6 Where criticism appeared, it concerned SQA's subsequent apparent lack of consideration of these teacher estimates in guiding their final grade decisions during moderation. Responses here highlighted surprise that teachers were not consulted or asked for further evidence before a final decision to downgrade was reached, a factor that has led many of the respondents to question the value that SQA has placed on teacher estimates historically. One teacher summed up the wider issue: ‘It was an easy enough process to follow – that, of course, does not mean that there was integrity to it.’
A similar disconnect between the grading process mechanically and its wider implications was noted when teachers were asked to comment on the impact that the results and associated controversy has had on their pupils. The majority of respondents (53%) felt that the grading process had no discernibly positive or negative impact on the final grades awarded to their pupils, while a further 30% considered the impact to be positive in nature. This is perhaps reflective of smaller cohort sizes and the relatively recent introduction of Classics into some centres, meaning that the SQA algorithm either relied on a small sample size or had no real background information to call upon in altering results.Footnote 7 This being said, of course, there remain 17% of responses that considered the impact in highly negative terms, in one case notably in an area of high deprivation.
While the final results themselves might have been addressed this year through the intervention of the Scottish Government, it is clear that the wider implications thrown up by the exams crisis have had a discernible impact on both pupils and teachers in Scotland, one that may yet leave a lasting mark on the collective trust in SQA's assessment procedures. This was one area of the survey where respondents were particularly vocal, and it is thanks to their expansive testimony that we are able to identify the impact on their cohorts (reflecting pupil feedback to their teachers) as well as the teachers themselves.
Beginning with pupils, there was a consistent level of feedback suggesting that the majority of students desperately wanted the chance to sit a final exam in any format, and have felt short-changed by the lack of an opportunity to showcase their learning and to secure grades that they can be confident that they had deservedly earned. One teacher noted that it had caused a polarising reaction amongst their cohort's attitude towards coursework in general: ‘Some pupils now feel that even working very hard all year will not be as worth it, others now realise they cannot have an ‘easy’ year and then hammer the books pre-exam!’ While a warning against pupils applying less effort in their coursework, then ‘cramming’ for the exam alone, might be a welcome result of this crisis for some, the potential damage to faith in the assessment structure entirely is far more disturbing (especially given the fragility of Classical subjects in the Scottish curriculum).
Another respondent noted that their students felt let down by the grading algorithm in connection with courses that they had taken as ‘crash’ Highers, that is to say taking the Higher course without a background in the subject at N5. The teacher noted here that SQA's apparent rejection of teacher estimates was particularly damaging in this case, since they were individually best-placed to assess the student potential for improvement in the remaining session: ‘This is vital in the case of a crash higher where some young people who have achieved very little in the past really “pull it out of the hat” for their sixth year.’ Another teacher commented that this has similarly led to a bitterness among students who feel subjected to a ‘postcode lottery’ or as if their results have been determined by whether their parents could afford to send them to an independent school or not. The potential for SQA moderation to disadvantage pupils undertaking ‘crash’ options at local authority schools is particularly worrisome in the context of Classics, where many of these schools only have the staffing and resources to offer this type of Higher. Such a situation has disconcerting implications for the long-term equality of access to Classics in Scotland being pursued by CAS and other stakeholders.
Moving on to teachers’ concerns and frustrations, many respondents tempered their criticisms with an acknowledgement that SQA and other examining bodies were forced to operate in entirely unprecedented circumstances, and that it is unlikely that a single system could have been found to satisfy everyone. The most pronounced condemnation of the process made by nearly every respondent concerns the apparent gulf in SQA's system that initially emphasised the importance of diligent teacher estimates, only to then moderate over 100,000 results without further consultation. One teacher classed the process as a ‘fait accompli’ by SQA, separate from the teacher estimates; one felt that there was ‘a lot of misplaced faith in applying statistical models to living human beings’, while yet another accused SQA of laziness: ‘Fairness in human affairs does not come as the result of lazily calling up the statisticians.’ A corollary of this is that some have expressed a concern that 2020 exam results will inevitably be scrutinised excessively by universities and employers, regardless of how their final figure was reached, since the grades were assigned to courses that were ultimately incomplete, at least compared to preceding cohorts.
The prevailing viewpoint amongst teachers found in this survey, though, is confusion and anger that they were expected to devote time and effort into estimating their pupils’ progress, only for this information to be distrusted or rejected arbitrarily by the very organisation that asked for it. At its worst, this disconnect between teacher estimates and SQA awards, paired with continued vagueness on how and why deductions were decided, has damaged teacher faith in SQA as an organisation assessing their pupils and, by extension, their expertise made manifest. This was best seen in two responses, one labelling the procedure followed by SQA ‘an absolute bloody shambles’, while another accused SQA of having ‘little or no accountability to anyone and no empathy for young people, their teachers or the communities and schools they come from.’
This ill-feeling towards SQA appears to have translated into further dissatisfaction concerning the examining body's guidance and consultation concerning the 2020-21 session (which is itself likely to be affected by the ongoing cat-and-mouse pattern of coronavirus restrictions). While some teachers commended the information already disseminated by SQA, the majority (62%) were dissatisfied with the level of consultation offered hitherto. Most prevalent among the criticisms here were observations that the autumn term in Scotland is already underway, meaning that as further advice continues to be issued, this could mean that some have wasted time on elements of coursework or assessment that might be changed. This is particularly poignant in the case of the Classical Studies syllabus, where some changes are expected by teachers. Another self-evident theme permeating responses is a distrust that teachers’ input will be taken seriously following the controversy of the 2019-20 session, with one teacher labelling SQA's approach for consultation as ‘total tokenism’. From an external position, it is clear that information has spread to the teaching community rather unevenly, and that there is still considerable work to do in making sure all teachers of SQA courses feel fully informed and prepared for what will probably be another difficult session.
Conclusions
Like the rest of the UK, Scotland's educational framework has been rocked by an assessment and moderation system that appears to have discriminated against pupils from schools amongst the poorest 20% of society. The severity of the controversy is seen in the public apologies not only from John Swinney as Cabinet Secretary for Education, but also from Nicola Sturgeon as First Minister of Scotland, who stated in the midst of a coronavirus briefing: ‘We accept we didn't get this right and I'm sorry for that.’ (Davidson, G. Reference Davidson2020).
It is clear that the swift political move to restore results to the levels estimated initially by teachers prior to SQA intervention has been a popular move in the public mindset, particularly when compared to the relatively slow recourse to this action observed in our neighbours to the south, but it effectively changed the nature of the results from one in which a draconian moderation technique was applied to around 25% of results nationally, to one in which there was no external moderation process enacted at all (a ‘slipshod process’ according to one teacher). While undoubtedly the easiest response to a thorny problem, this has raised issues of its own and has potentially concerning implications for SQA and examinations in future years.
On an institutional level, questions have been raised already about the independence of SQA as an examining body from individuals and ministers within the Scottish Government. Indeed, one respondent to the CATB-CAS survey expressed concern that Swinney appeared to function as the ‘unappointed CEO’ of SQA, with power to reverse a framework (one that was, it must be stressed, formally agreed upon with the Scottish Government in March) owing to immediate concerns of ‘saving face’. In coming months, through its ongoing consultation with teachers, one of the challenges facing SQA will be to assert its position as an independent body that will not find itself at the mercy of shifting public or political currents.
While Swinney has recently stated that SQA intends for a full complement of examinations in the coming year (Vesty & O'Neill, Reference Vesty and O'Neill2020), if the 2020-21 session is affected in a similar fashion to this past academic term, then it is likely that teacher estimates will once again form a crucial baseline in formulating final results, especially if the exams are compromised. It is abundantly clear that SQA will need to consult and collaborate pro-actively with teachers as quickly as possible to agree upon a system of estimation and moderation that pupils and the wider public can have faith in: that it will not fall victim to the overly technocratic tendencies of 2020, but that will not be suspected of grade inflation by well-meaning teachers either.
Numerous respondents to the CATB-CAS survey expressed concern that while teachers are the individuals best placed to assess their students in these circumstances, ‘teachers will always see the best in their students’. This is potentially seen in the upward grade curve that this year's controversy has prompted. In his initial defence of SQA's moderation, Swinney remarked that the grade inflation in 2020 would have been extraordinary, were it not for SQA intervention, with N5 passes up 10.4%, Highers up 14% and Advanced Highers up 13.4% (Davidson, J. Reference Davidson2020). While we ought to avoid the temptation to base a cohort's prospective results purely on the one that preceded it, this increase is doubly impressive when marked against a pattern of marginally decreasing results across the preceding lustrum, with Higher results in 2019 attracting criticism for dropping to their lowest ebb in five years, dipping below 75% in A-C passes (Hepburn, Reference Hepburn2019). This is one area where the Scottish Government has been publicly inconsistent and must therefore be instrumental in finding a way forward, since its u-turn on SQA moderation in this case has made it only more likely that ever more pressure will be exerted on teachers themselves to be the arbiter of their pupils’ final results.
This is a probability that current teachers are mindful of. In their replies to this survey, teachers themselves expressed a desire for a system of checks to ensure that faith can be maintained in teacher estimates the length and breadth of the country. While it would be difficult (and arguably counter-productive) to attempt to add another layer of bureaucracy onto already strained teachers, one respondent suggested that a simple addition would be for teachers to sign an official declaration of commitment to best practice in estimates, with formal consequences if found to be reporting incorrectly, similar to the medical sponsio academica. From an external perspective, it seems that it would be difficult to police between genuinely optimistic predictions and the wilfully fictitious. It would, furthermore, likely create additional ill-feeling amongst teachers who have always striven, and continue to strive, for accuracy and fairness for their pupils.
Simply put, there is no ‘quick fix’ to this situation. With little sign of coronavirus retreating in the next 12 months at least, it is apparent that teachers, SQA and the Scottish Government will have to place their misgivings to one side in the hope of finding the ‘difficult fix’ that respects the expertise and experience of teachers while simultaneously removing the expectation that they should be responsible for moderating the same pupils that they have worked hard to nourish and develop.
In an environment where physical examination is not possible, efforts must be made either to exploit the facilities of digital technology (in a way that does not disadvantage the poorest in society who may lack sufficient access), or in a rebalancing of assessment altogether, with the weight placed on examination diminished in favour of other modes of assessment that can be carried out throughout the school year. Universities and employers in Scotland must also be sensitive to these efforts and should themselves be ready to adapt their admissions procedures and decisions to reflect the exceptional circumstances that the coronavirus pandemic has forced upon society at large. This is one area where subject associations and advocacy groups, such as CAS, Classics for All and Advocating Classics Education can and should be active, helping to bridge any disconnects between different constituencies involved, and able to speak to each party from a position that is invested but objective.
It will take considerable effort from multiple sources to ensure that the controversy of 2020 will be considered anomalous rather than the start of a ‘new normal.’ Regardless of how this is achieved, the significant damage wrought to the relationship between teachers (and pupils by extension) and SQA must be tackled head-on to clear the air, dispel any misconceptions and to rebuild trust in the educational framework as a whole in Scotland. This is doubly important in a currently small subject area like Classics, where a significant number of teachers also serve as SQA markers, assessors and upon the National Qualification Support Teams (NQST) that help guide the approach to examination in Scotland. This will likely be painful in the short term but, without this initial discomfort, the prognosis for a positive working relationship between teachers and the examining body is bleak.
However, this is one area where the intimate nature of the Classics community in Scotland may help us weather the rough seas ahead. Reflecting on my own time on the NQST for Classical Languages at SQA, I have only ever been struck by the warmth of personal friendship and professional collegiality that characterises its membership. Our great strength in Scottish Classics is that we have a community that has witnessed the near death-spiral of our discipline from the 1980s, has endured and is constantly looking for ways to develop and regrow the subject in all areas. It is this energy, together with academic honesty and transparency in planning, which will allow us to overcome this crisis more than anything else.
Open access
