3.1 Quantitative overview of EMI research in all phases of education

We now provide an overview of the geographical region of the studies and countries in which the studies took place (Table 1) and of the types of research designs used (Table 2). Asia and Europe dominate in all phases of education. Africa is particularly poorly represented at tertiary level as compared to secondary. However, we should note that our search strategy may not have located as many primary phase studies as we would have expected. We return to these geographical themes in our Discussion.

Regarding the five studies at the pre-primary school level, the topics explored were varied: whether a kindergarten English immersion programme leads learners to devalue their L1; how EMI programmes can help children develop socio-pragmatic awareness; how EMI programmes using stories can help children to develop skills in phonics and writing, and how environmental learning took place in an EMI Zoo-preschool.

At the primary level, based on the information available in the abstracts of the 41 studies, the research agendas were dominated by comparing immersion and non-immersion students’ performance in English and L1, content learning and students’ cognitive development. Tests were employed in most of these studies. Another prominent topic was exploring teachers’, students’ and parents’ perceptions of and experience with EMI, mainly through questionnaires and interviews (n = 8). Other studies analysed classroom observations to explore how EMI classrooms provide opportunities for students’ learning of English (n = 3) and content (n = 1), discursive practices in EMI classrooms (n = 1), codeswitching behaviours (n = 1), how multimedia can be used strategically (n = 1) and to compare EMI to EFL classes (n = 1). One study described an assessment tool for EMI classes, with another study presenting a successful example of an EMI teaching approach. Studies explored children's readiness for EMI education (n = 1) and the impact of EMI on students’ motivation of learning English (n = 1) through questionnaires, the text genres in EMI textbooks through genre analysis (n = 1), the social impact of EMI (n = 1), and the link between EMI policies and children's sociocultural ecology (n = 1). The focus of one study was unclear from the abstract.

At secondary level, most studies explored the impact of EMI on students’ English and content-subject achievements in EMI and non-EMI classrooms (n = 11). These studies involved measuring students’ receptive and productive vocabulary (n = 3), reading comprehension (n = 3), and actual L1 and L2 language use (n = 2). A significant number of studies also explored teachers’ and students’ attitudes towards EMI and the development of teaching approaches for combining both content and language. These studies employed semi-structured interviews (n = 25), questionnaires (n = 22), classroom observations (n = 20), discourse analysis of lesson transcripts, students’ writing, learning materials (n = 20) and document analysis (n = 6). Most studies were conducted between 1997 and 2015. Further details of the breakdown of studies for the tertiary phase are provided in the next section and we will return to our mapping of the other phases in our Discussion.

3.2 Quantitative overview of EMI research in HE

We now present a quantitative overview of the 83 HE studies that were reviewed in depth by the team (see Appendix 2 (Supplementary Materials) for the systematic map of EMI in HE).

Our analysis of year of publication shows that EMI is a relatively new but rapidly growing field of academic endeavour. The first study thrown up by the search strategy was a doctoral thesis from the Netherlands (Vinke Reference Vinke1995) perhaps reflecting the fact that Maastricht University in the Netherlands was one of the first institutions in Europe to extensively adopt EMI. Our search produced very few HE studies on EMI over the following ten years (only five studies from 1995 to 2005). Interest in EMI grew and academic output increased to 14 studies in a five-year period from 2006 to 2010. This was followed by a surge in academic research interest from 2011 to 2015. Sixty-three of the 83 studies were published in this five-year period of which 19 were published in 2014 alone.

The majority of studies (n =73) in the systematic review are journal articles. Four reports are included and four doctoral theses plus two studies reported in chapters in edited books. Investigations into EMI in HE were carried out in all geographical regions (see Table 3) with the exception of Africa. As well as one study which collected data at a global level (Dearden Reference Dearden2015) one at European level (Brenn-White & van Rest Reference Brenn-White and van Rest2012) and one of both Norway and Germany (Hellekjaer Reference Hellekjaer2010), the rest were at the single country level. In Europe the country most widely represented in the studies was Spain (n = 11 HE studies). In Asia, Korea was the country most represented. As well as Africa, we should note the relative absence of HE studies from South or Central America.

Of the 83 studies, 25 were classified by the review team as quantitative, 31 as qualitative and 27 as using mixed methods. There was a high proportion of investigations that we classified as ‘case studies of one institution using mixed methods’. Of the 83 studies it was surprising to notice that 37 (5 quantitative, 19 qualitative, 13 mixed methods) did not have a clearly stated sample size, or at least did not provide enough information of the sample size. Of the studies that used quantitative methods we can categorise sampling patterns as follows: 12 studies had a sample size of less than 30; 8 studies had a sample size of 31–100; 6 studies had a sample size of 101–200, and 18 studies had more than 200 (student) participants, (e.g. Evans & Morrison Reference Evans and Morrison2011; Bolton & Kuteeva Reference Bolton and Kuteeva2012; McMullen Reference McMullen2014; Lasagabaster & Doiz 2015).

3.3 Labelling and defining EMI in HE

We found that the labels given to the phenomenon of EMI and their definition are inconsistent and problematic. In 62 studies the label used was ‘English medium instruction’, with variations within that (English as a medium of instruction; English as the medium of instruction; English-medium education) with rarely an explanation of why such a label was being used. There were single cases of researchers using other terminology such as ‘parallel-language education’, ‘English as a lingua franca’, ‘partial English medium’, ‘English content-based instruction’, ‘English medium and Spanish as a first language’, and ‘English taught programmes’. The term ‘content and language integrated learning’ was used in 12 of the 83 HE studies. However, in none of these was there a clear justification for using this term based on evidence that the teachers/lecturers were trying to ‘integrate’ content and language or that there was an institutional policy to do so. There were no uses of the label ‘immersion’ in the titles of the 83 studies.

In the introduction we gave our own definition of EMI. The effect of this definition is to make a distinction between the use of English as an L2 to teach content in countries such as Australia, Canada, the UK and the USA, where the majority population has English as their L1, and those countries where the majority population does not. This distinction may be open to challenge (see, for example, Met (Reference Met1999) and more recently Fenton-Smith et al. (Reference Fenton-Smith, Humphries and Walkinshaw2017)) who in their edited volume include Australia as an EMI country on the basis that a substantial proportion of its HE population consists of international students whose L1 is not English. However, to support our own definition and exclusion criteria, we point to the clear difference in the environment immediately surrounding the two education settings/classrooms and the fact that international students have (presumably) chosen to study in Australia whereas, for example, Pakistani students in a state university in Pakistan may have no such choice available to them (see Mahboob Reference Mahboob2017).

Clearly then both the definition of EMI in HE and its practice appear to be fluid. It is unclear as to whether this fluidity is happening at: a macro level, internationally as in CLIL and integrating content and language in higher education (ICLHE), in Europe nationally as appears to be the case in Taiwanese research (now appearing to adopt CLIL instead of EMI); or even the micro level, i.e. where a university may decide to start calling the programme on offer a CLIL or ‘CLIL in HE’ or ICLHE programme instead of EMI.

We should bear in mind that within HE institutions which purport to offer EMI programmes there is great variation of provision (see Hino Reference Hino2017): are all programmes on offer EMI?; are they offered by some departments but not others (i.e. some sort of disciplinary distinction), if so on what basis?; are they only offered for what we might call ‘less traditional, internationally oriented’ subjects such as ‘international business’, ‘international programme of frontier biotechnology’?; are they programmes taught partly in English and partly in L1?; are they courses where the EMI offer is entirely dependent on the profile of the teacher in question (see Dearden & Macaro Reference Dearden and Macaro2016)?

3.4 The growth of EMI in HE around the world

We have found only one study (Dearden Reference Dearden2015) which has attempted to map the growth of EMI on a global scale. This study, using British Council staff as informed participants, reported a rapid expansion of EMI provision in the 54 countries they represented. Respondents reported more EMI at tertiary level than at secondary level due to universities rapidly becoming more international and EMI was more prevalent in the private than in the public sector. Whilst, overall, the numbers of private universities almost equalled that of public universities, respondents estimated that 90% of private universities taught through EMI compared to 78.2% of public universities. Sixty-seven % of the respondents reported a likely future increase in EMI provision in their countries.

Focusing now on various geographical areas, we start with Europe as this is the area which has received the most systematic measurements of EMI HE growth.

3.5 Implementation of EMI in HE

Given the rapid increase in the introduction and development of EMI programmes, it is not surprising to read, from a number of authors reviewed, references to the fact that the key actors in the process of teaching and learning through this new medium have rarely been consulted by policy makers at a national level or by university managers at the institutional level (Dearden & Macaro Reference Dearden and Macaro2016). Nevertheless, it is surprising that in none of the studies which have centred on teacher and/or student beliefs do we read, through the empirical data itself, of a process by which an institution has systematically engaged faculty, let alone students, in deciding how quickly and in what subjects to introduce EMI. We find some evidence of lack of consultation in Cho (Reference Cho2012), a Korean study where reference is made to the EMI policy being imposed: 52.9% of teacher respondents claimed that their motivation for teaching through English was because it had been ‘enforced by the school’. Kang & Park (Reference Kang and Park2005), in the same country, report that the main motivation for their student sample choosing an EMI course was that the subject they wanted to study was simply not offered in Korean. We have to bear in mind these factors of ‘volition and ownership’ as well as ‘availability’ when examining other motivations amongst these stakeholders. In Taiwan, additionally, motivation for enrolling on an EMI programme seemed to be associated more with the reputed expertise in the content subject of a particular lecturer rather than the MOI itself (Yeh Reference Yeh2014).

Numerous studies (Hamid, Jahan & Islam Reference Hamid, Jahan and Islam2013; Başıbek et al. Reference Başıbek, Dolmacı, Cengiz, Bürd, Dilek and Kara2014; Kim, Tatar & Choi Reference Kim, Tatar and Choi2014) have cited the existence of English language only resources and/or lack of resources in the L1 of their country, as a reason to adopt EMI. This appears to be the case particularly in technology and science. Indeed some student respondents also cited the need to match the language of exams with the language of the resources (Belhiah & Elhami Reference Belhiah and Elhami2014). In the Saudi context, 45% of Al-Masheikhi, Al-Mahrooqi & Denman's (Reference Al-Masheikhi, Al-Mahrooqi and Denman2014) respondents linked the English language with the language of science and technology and therefore considered it important to learn science through EMI (although 30% disagreed).

3.6 Teacher and student beliefs about EMI

Despite the above mentioned negative aspects of EMI implementation, in many studies, more positive motivations for implementing EMI can be found. The cultural, as well as the financial benefits of internationalisation of HE, by attracting more international students, feature prominently in a number of studies asking teachers and students the motivation-type question. Many of Jensen & Thøgersen's (Reference Jensen and Thøgersen2011) Danish lecturers believed that the number of EMI courses should be increased in order to attract non-Danish students. The key motivator for teaching and learning through EMI was that English as an international language was an essential constituent of the internationalisation process. The majority of student respondents in Khan's (Reference Khan2013) study in Pakistan held the belief that EMI was essential in HE and that this in turn could contribute to modernisation and a progressive national outlook. In Germany, Earls (Reference Earls2016) showed that both lecturers and students were convinced that, in the context of globalisation of goods and services, it was inevitable that EMI would be the status quo and that, at least for some subjects such as International Accounting, it would not ‘make sense teaching it in German’ (p. 108), a view similarly held by business school lecturers in Korea (Byun et al. Reference Byun, Chu, Kim, Park, Kim and Jung2010). All 15 of Bozdoğan & Karlıdağ’s (Reference Bozdoğan and Karlıdağ2013) Turkish student interviewees believed there were instrumental advantages in studying through English (improving English, leading to job opportunities) and in Chapple's (Reference Chapple2015) study of Japanese students’ reasons for enrolling on EMI programmes, ‘improving English ability’ (p. 4) was cited much more frequently as a reason for enrolment than it being an institutional requirement.

Despite concerns (outlined below), two further studies report positive attitudes to EMI: In Bangladesh, Hamid et al.’s (Reference Hamid, Jahan and Islam2013) lecturers in a private university believed that English was essential for communicating with different people across the world. In Qatar, Ellili-Cherif & Alkhateeb's (Reference Ellili-Cherif and Alkhateeb2015) students expressed the view that English had a higher status than Arabic and therefore it was needed to improve their career prospects.

In China, Hu, Li & Lei (Reference Hu, Li and Lei2014) compared institutional policy documents and the beliefs of lecturers and found convergence of motivational factors for increasing EMI provision. Because of the high value and prestige accorded to the English language, lecturers believed that EMI was capable of ushering in benefits both at the institutional and national level such as social mobility and career prospects. Less convergence between policy makers and faculty was found in Ali's (Reference Ali2013) study of a Malaysian university where lecturers did not share the same understanding of the meaning of internationalisation but rather held what the author calls ‘a silent understanding’ of it.

Positive motivations were found in Dearden & Macaro's (Reference Dearden and Macaro2016) study of lecturers in Austria, Italy and Poland where the desire to give students the same opportunities of studying abroad as they had had was frequently cited as being an advantage of EMI. In Earls’ (Reference Earls2016) study there were lecturer motivations which went even beyond instrumentality or the necessity of knowing English, but rather oriented towards intercultural understanding, learning from different cultures and ‘mental flexibility’ (p. 124) with regard to teaching and learning. In Gao's (Reference Gao2008) longitudinal study, mainland Chinese students studying in Hong Kong shifted their motivation from extrinsic, context-mediated orientations towards more self-determined and intrinsic ones.

Virtually no studies in our review have asked an outright ‘for or against EMI in your country’ question to lecturers or students. A possible exception would be that of Başıbek et al. (Reference Başıbek, Dolmacı, Cengiz, Bürd, Dilek and Kara2014) who asked 63 Turkish lecturers whether they supported adopting EMI at their institution and obtained a generally affirmative answer. However, both sample size and lack of generalisability to a population of (say) all lecturers in Turkey, could not lead one to infer overall support for EMI from this result and therefore be able to inform policy makers. In Turkey, researchers also found that students claim they would learn content more easily through the L1 than through English (Kirkgöz Reference Kirkgöz2014; West et al. Reference West, Güven, Parry and Ergenekon2015) but that, once again, does not provide a definitive answer as to whether EMI should be increased, kept at current levels, or reduced.

Despite these propelling forces towards EMI (expressed as constraints or affordances), a number of deep concerns have been expressed by lecturers and students and in virtually all studies consulted. Probably, and unsurprisingly, the concern most often expressed is that of student English language proficiency, lecturer proficiency, or both. We should pause briefly here to note a lack of definition or specification regarding the construct of proficiency which in any case is referred to in various ways by different authors and even within-study. For example, Ghorbani & Alavi (Reference Ghorbani and Alavi2014) refer to ‘English skills’, Huang (Reference Huang2015) refers to ‘English ability’, Islam (Reference Islam2013) to ‘English proficiency’, Francomacaro (Reference Francomacaro2011) to ‘language competence’, and so on. This variety of terminology and lack of specification is not a trivial observation on our part if we consider it in reference to the seminal model proposed by Cummins (Reference Cummins1979) where a distinction is made between academic and non-academic language. In the context of international students in Anglophone countries which often impose language requirements there is still a debate about whether general English proficiency (as measured even by a test such as IELTS which contains some academic language) and/or academic language proficiency (as provided by a general EAP programme) are sufficient. We propose that the issue is even more in need of debate in the context of EMI. In this paper we will stick to the term ‘proficiency’ whilst acknowledging that it is only being used in a general sense to describe participant beliefs as reported in studies consulted.

Putting aside the question of lack of specificity with regard to proficiency, we find lecturers deeply concerned about their students’ inability to survive, or better still thrive, when taught through English. This is contrary to what European ‘programme directors’ in Wächter & Maiworm's (Reference Wächter and Maiworm2014) comprehensive study claimed was the case. In Turkey, the collective research picture is one of deep concern in terms of level of English in general and vocabulary knowledge in particular (Kirkgöz Reference Kirkgöz2009; Başıbek et al. Reference Başıbek, Dolmacı, Cengiz, Bürd, Dilek and Kara2014; Macaro, Akincioglu & Dearden Reference Macaro, Akincioglu and Dearden2016).This is matched by student evaluations of their own English proficiency (Bozdoğan & Karlıdağ Reference Bozdoğan and Karlıdağ2013) as being insufficient to enable adequate learning of content. In Spain (Doiz, Lasagabaster & Sierra Reference Doiz, Lasagabaster and Sierra2011) and in France (Napoli & Sourisseau Reference Napoli and Sourisseau2013) we find evidence that lecturers not only see student proficiency as inadequate but even as a possible barrier to being able to access an EMI programme. In the UAE, Rogier's (Reference Rogier2012) respondents claimed that students were weak in the skills of listening (presumably to lectures) and in academic writing.

Amongst Borg's (Reference Borg2016) sample of lecturers in Iraqi Kurdistan, as many as three quarters believed their students’ proficiency was barely above elementary level. In Korea, Choi's (Reference Choi2013) lecturers considered students’ English proficiency levels to be the single greatest barrier to effective content learning and Kim & Shin's (Reference Kim and Shin2014) sample considered that nearly a third were ill-equipped linguistically to benefit from an EMI programme. Lecturer beliefs about student's linguistic deficiencies also exist in countries such as Sweden (Airey Reference Airey2011) where EMI is established and there is a widespread belief that everyone speaks English well.

Students’ perceptions of their own linguistic abilities seem to concur with those of their lecturers. In Thailand, Hengsadeekul, Koul & Kaewkuekool (Reference Hengsadeekul, Koul and Kaewkuekool2014) found a negative correlation between instrumental motivation and speaking anxiety, fear of negative evaluation, and fear of social comparison suggesting that only confident English speakers were sufficiently motivated. Notable exceptions are in Taiwan (Yeh Reference Yeh2014) and Hong Kong (Evans & Morrison Reference Evans and Morrison2011). In Taiwan, students expressed confidence, possibly due to teaching strategies adopted by the lecturers in this specific institutional context rather than proficiency. In Hong Kong, despite early (undergraduate) student feelings of inadequacy, especially in productive skills, the authors report that students generally overcame their difficulties as their course progressed.

Overwhelmingly though, students report insufficient levels of proficiency (Cho Reference Cho2012; Khan Reference Khan2013; Ellili-Cherif & Alkhateeb Reference Ellili-Cherif and Alkhateeb2015). Cho's (Reference Cho2012) Korean students complained of very limited listening skills with some claiming to understand only 60% of their lectures. Ellili-Cherif & Alkhateeb's (Reference Ellili-Cherif and Alkhateeb2015) Qatari students, even in ‘internationally oriented subjects’ (e.g. Business Studies) believed they were learning considerably less well than if they had been studying through Arabic. Khan's (Reference Khan2013) Pakistani students had problems understanding both lectures and written materials and were very hesitant to speak in English.

Similar findings are reported by Bangladeshi students (Islam Reference Islam2013). Kang & Park's (Reference Kang and Park2005) large-scale Korean study (n = 366) of students with different proficiency levels shows quite clearly, through inferential statistics, that the level of proficiency in English impacts on students’ ability to understand lectures and whether or not they needed to resort to content materials in translation. In this study, low proficiency was also linked to higher levels of anxiety.

Sultana (Reference Sultana2014) found that first-year Bangladeshi students from public Bangla-medium schools faced more problems than students from private EMI schools when they entered English medium universities. With little or no English, students from public schools were at a disadvantage academically and socially and their lack of English language proficiency in EMI universities negatively affected their socialisation on campus, self-image, identity and chances of learning. They also felt that they were not noticed by teachers, and found it difficult to participate in any discussion or debates in English or to interact with English-speaking students. Thus English played a significant role in marginalising the students who did not have adequate competence in English.

Having considered the beliefs about students’ English proficiency, we now turn to evidence as to whether lecturers believe they have the necessary language requirements themselves to teach through English and then whether students share this belief about their teachers. Here the picture is less clearly discernible for a number of reasons. First, there is less homogeneity amongst lecturers in terms of age groups than there is with students. Jensen & Thøgerson's large study (n = 1131) of Danish lecturers suggested that younger faculty are more confident with their level of English. However, even within this type of finding may lurk confounding variables such as willingness to adopt more interactive methodologies, leading to better student understanding, leading to lecturers therefore feeling that their English is sufficient for the task.

There is no HE research, to our knowledge, that matches some kind of language proficiency test with actual practice in order to determine a minimum level needed to teach. Apart from a relatively superficial evaluation of teachers’ linguistic competence to teach EMI (e.g. Kling & Stæhr Reference Kling and Stæhr2012), there is no definitive benchmark for the level a teacher needs to be able to teach through English at the national level and certainly not at the international level. The Dearden & Macaro (Reference Dearden and Macaro2016) study showed that none of the interviewed lecturers from Austria, Italy or Poland had a clear idea of what an appropriate level might be and that some believed the selection was made based on whether they had a doctorate from an Anglophone country, had taught abroad or were just considered by managers to speak English well. The meaning of ‘ability to teach’, without careful specification, can also mean different things to different people. Dearden's (Reference Dearden2015) survey showed that 83% of ‘informed observers’ in 54 countries thought there were not enough qualified teachers in their country even though the term ‘qualified’ was left undefined so that it could be interpreted purely in language terms, or in terms of certification. Lastly, it is clear from a number of research sources that teachers in different academic subjects consider their subject to be more or less dependent on language (see Lo Reference Lo2014 for evidence of this at the secondary level). For example, Turkish lecturers in Physics considered the language needed to put across the subject to be quite minimal (Macaro et al. Reference Macaro, Akincioglu and Dearden2016), and Dearden & Macaro (Reference Dearden and Macaro2016) cite one lecturer believing that in mathematics one is ‘saved by the formulae’ meaning that little or no language is needed beyond the mathematical code itself. Nevertheless, despite the lack of benchmarking of teacher proficiency, it is clear that the vast majority of respondents do consider that a high level of English is required (Choi Reference Choi2013).

Leaving aside complications regarding lecturer (self-assessed) proficiency, more studies reported lecturers as identifying that they had linguistic problems than those that did not. For example, Italian university lecturers expressed a belief that their English was inadequate, and that students would find it difficult to understand them, leading to possible incorrect language learning on the part of the students (Pulcini & Campagna Reference Pulcini, Campagna, Dimova, Hultgren and Jensen2015; Campagna Reference Campagna, Campagna, Ochse, Pulcini and Solly2016). Also in Italy, at least ten out of 53 participants in Guarda & Helm's (Reference Guarda and Helm2016) study considered their own language skills as one of the major sources of difficulty in teaching on EMI programmes. Less concern was expressed by Italian Engineering lecturers who all reported feeling quite confident with their English and did not consider they faced problems interacting with students, or evaluating students’ progress (Francomacaro Reference Francomacaro2011). However, Francomacaro (Reference Francomacaro2011) does provide a telling comment that: ‘The discussion revealed how the discipline lecturers are unaware of the linguistic implications of their teaching and of their students’ (p. 67).

Considerable concerns have been expressed by lecturers in Iraqi Kurdistan (Borg Reference Borg2016), in Vietnam (Vu & Burns Reference Vu and Burns2014) and in Denmark (Werther et al. Reference Werther, Denver, Jensen and Mees2014). However, conflating such disparate educational settings should be done with caution. It may well be that the level of English causing concern to Danish lecturers was more to do with not ‘being able to speak off-the-cuff . . . and playing with the language’ (Werther et al., p. 455) than not having the vocabulary and grammatical structures to provide even adequate content explanations, as can be inferred from other studies.

We should also give voice to students’ opinions of their lecturers. An interesting divergence of beliefs was found in a Swedish study by Bolton & Kuteeva (Reference Bolton and Kuteeva2012) where only a minority of teachers considered they had language problems whereas a sizeable proportion of students were less enthusiastic about their teachers’ level of English.

Concerns, if not negative attitudes, towards EMI are not confined to beliefs about insufficient levels of English proficiency although some are indirectly linked to this aspect. Nearly all studies of teacher beliefs allude to the extra work involved in teaching through English and the generally laborious nature of EMI (Hellekjaer Reference Hellekjaer2010; Başıbek et al. Reference Başıbek, Dolmacı, Cengiz, Bürd, Dilek and Kara2014) which could be interpreted to include language proficiency but also the question of redesigning and if necessary reconceptualising lectures and material that lecturers use in their L1. Vinke (Reference Vinke1995) found that agricultural science and economics lecturers believed that EMI entailed a reduced number of different types of pedagogical tasks or activities that could be offered to the students and Airey's (Reference Airey2011) Swedish lecturers alluded to the inability to introduce humour and provide the local context through EMI especially, one assumes, when the students come from different cultural backgrounds. Taking the notion of local context one step further, Kilickaya's (Reference Kilickaya2006) study of 100 Turkish lecturers found that participants demonstrated a fear of the negative impact of EMI on the Turkish language and culture and, indeed, wished the government to be more forceful in ensuring that L1 education was protected.

Again we should treat this amalgamation of findings from different geographical settings with some caution because at the secondary level of interpretation comparisons are difficult without greater insider knowledge that could be achieved through primary comparison studies.

3.7 Variables in stakeholder beliefs

One final area concerning stakeholder beliefs is connected with all the possible variables in the field of EMI in HE. We would argue that, because of the broad sweep of the loci of research, several variables need to be researched before attempting any kind of overall conclusion of where the EMI phenomenon is going: These might be the different:

• • Beliefs held by female and male students as both L2 learning and certain academic subjects tend to be gendered

• • Beliefs held by those in different academic disciplines

• • Attitudes held by students (and lecturers) in private and state universities

• • Motivations for enrolling on undergraduate or postgraduate courses

• • Beliefs expressed by international and home students in EMI programmes and, possibly, in relation to different levels of L2 proficiency.

Although few studies set out to explore these variables directly, the tendency for investigations into students’ beliefs having, not surprisingly, larger samples, does allow for some evidence of variability into these student-types. In terms of gender differences, McMullen (Reference McMullen2014) found no significant differences with regard to whether the Saudi students in his sample thought a preparatory year was necessary before embarking on EMI. However, females were less likely to believe that they needed the 20 hours of English that were being required compared to males suggesting a possible greater confidence with their current proficiency level (although no inferential statistics appear to have been carried out). Hengsadeekul, Koul & Kaewkuekool (Reference Hengsadeekul, Koul and Kaewkuekool2014) in Thailand did carry out inferential statistics and were able to demonstrate that females had significantly higher goals that could be described as instrumental but no differences in terms of self-estimated proficiency.

With regard to different programme levels, postgraduate students in Sweden (Bolton & Kuteeva Reference Bolton and Kuteeva2012) reported higher levels of confidence and satisfaction to learn through EMI than undergraduate students. Humanities students appeared to be less favourable to a current or putatively increased level of EMI provision compared to science students (although there were also high levels of ‘neutral’ students). Finally, with regard to the home and international students’ variation, Kim et al. (Reference Kim, Tatar and Choi2014) found that international students believed their level of English was adequate whereas Korean (home) students perceived their level of English to negatively affect their understanding of content (Bartram Reference Bartram2008). Chapple (Reference Chapple2015) found that the Japanese (home) students in his sample struggled quite considerably with EMI whereas at least some of the international students felt their time was being wasted to accommodate the relatively low proficiency of the Japanese students.

3.8 Professional development of EMI teachers

Another concern expressed by lecturers is the lack of preparation to teach, and lack of professional development opportunities. We have already discussed the absence of a benchmark of teacher English proficiency in HE. However, and in relation to the phrase ‘ability to teach through English’, there is also virtually no research data available on types of teacher preparation programmes in EMI in HE. As far as we are aware they simply do not exist, and even in an educational setting such as Hong Kong where at tertiary level English has been the language of instruction for generations, there are no pre-service teacher programmes required or even optionally on offer for HE lecturers. Some studies reporting in-service training have been carried out but we have no notion of the level of take up or the overall success of programmes in any country or jurisdiction. Given that it is generally recognised from secondary education EMI/CLIL studies (Hoare Reference Hoare2003) that teaching subject content is not just a simple matter of translating what one was going to say or present visually into English, this research gap can only be due to two factors: either the pre-service programmes do not exist or researchers have not been interested in exploring them; we tend to the former. According to Wächter & Maiworm's (Reference Wächter and Maiworm2014) European overview, faculty are ‘encouraged’ to improve their level of English: even this is not mandatory. They do not provide evidence that EMI teachers undertake development in methodology, whether optional or required.

In Sweden, Airey (Reference Airey2011) investigated a professional development course in EMI involving 18 teachers from various disciplines with limited or no experience of EMI. We should note that one of the reported reasons for opting-in to the course was a belief it might contribute to getting a promotion. Some of these participants divulged that they had suddenly been asked to switch to EMI without sufficient time to prepare. In Spain, Aguilar & Rodríguez (Reference Aguilar and Rodríguez2012) report the provision of some limited development in the form of an EMI (CLIL) taster course lasting 15 weeks. The authors report that the main motivation for taking part was the lecturers’ desire to up-skill in English language rather than consider changing methodology. In Italy, Guarda & Helm (Reference Guarda and Helm2016) report on a seemingly successful development programme which involved over 50 lecturers in presenting lectures, and seminars followed by group discussions. Pre- and post-analysis of views expressed by participants shows some promising change of perceptions and greater willingness to interact with students in English, as well as to consider student needs.

3.9 Impact of EMI on English language learning

We now turn to research on the impact of EMI courses on English language proficiency in HE. The crucial research question is, ‘what is the impact on learning of this dramatic change from L1 medium of instruction to EMI?’ As Jiménez-Muñoz (Reference Jiménez-Muñoz2014: 30) argues with regard to impact: ‘until conclusive proof is found, the controversy over the usefulness of CLIL and EMI to promote excellence in both content and language learning will continue’. We will first consider the evidence of impact on (English) language learning.

From our total of 83 HE studies that we have included in the systematic map, only seven have in some way measured the impact, via objective language tests, that EMI programmes have had on English language learning or English language proficiency. Of these, five have used national or international standardised testing instruments (Lin & Morrison Reference Lin and Morrison2010; Rogier Reference Rogier2012; Hu & Lei Reference Hu and Lei2014; Aguilar & Muñoz Reference Aguilar and Muñoz2014; Yang Reference Yang2015); and two have used instruments created by the researchers specifically for the purposes of their study (e.g. Pessoa, Miller & Kaufer Reference Pessoa, Miller and Kaufer2014). This paucity of language impact studies, coupled with the variability in test instrument types makes any kind of overview assessment of impact extremely difficult. Moreover, there is variability even within the test-types with some testing potential improvement in single skills, others testing more general abilities.

There is also the issue of what kind of language test is most appropriate for measuring the impact of EMI on language learning: should it be ‘general’ or ‘academic English’ improvement? This is especially tricky when the study design involves a comparison of EMI and non-EMI classes where students have been taught in the L1 but may have followed contemporaneous EFL classes. The problem here is one of comparing like with like: the expectation might be that EMI students improve their academic English more than non-EMI students and that the latter might improve their general English more than the former. Potentially, therefore, one might be looking for research studies which tested both types of English when such comparisons were being made. In other words there is a potential difference between general English proficiency and the competence to learn effectively in an EMI context. This also relates to the broader question of whether EMI programmes are intended or indeed should be intended as language proficiency development programmes.

The studies described in this section did not provide, in our view, sufficient theoretical discussion of what type of English language proficiency or competence EMI programmes might be expected to develop and, as a corollary, the kind of language testing instrument that would be appropriate when a comparison was being made between EMI and non-EMI students’ progress.

A study measuring general English proficiency and using standardized English tests (IELTS) was carried out by Rogier (Reference Rogier2012). Set in the context of one university in the UAE, she specifically set out to measure longitudinally the extent to which students’ English language skills improved after studying via EMI in a four-year undergraduate programme. We should note that at this institution students were expected to have IELTS 5.0 before starting the course and this resulted in only 10% of students being pre-selected for EMI courses. They were also expected to reach 5.5 IELTS after two years and to graduate with at least 6.0. Rogier found that there was a statistically significant score gain in all four of the English-language skill areas (writing, reading, listening and speaking). Students’ largest gains were in the areas of speaking where, on average, they made half a band gain, followed by reading where they made nearly that level of gain. Although Rogier did not have a non-EMI comparison group she does compare these gains with studies of General English programmes elsewhere (e.g. Elder & O'Loughlin Reference Elder and O'Loughlin2003) which have shown that, to move up one half of a level, students need between 200 and 240 hours of general English tuition (or, say, eight weeks on a fairly intensive course). Thus, if the comparison is valid, to move up on average only one half band score in four years would not seem to be a great validation of the beneficial effect on English in this particular institution.

Also measuring four skills, but this time with a national General English Proficiency test, was a study carried out by Yang (Reference Yang2015) in Taiwan. Results showed that students, after two years of EMI instruction, had made significant improvements in their receptive skills. Again, there was no comparison group within the institution but Yang compares them favourably to national level measurements from non-EMI universities. However, the EMI students were already scoring higher than the non-EMI students at baseline and most had previously been studying Tourism through English in their vocational high schools; it is not possible to ascertain the high school medium of the non-EMI students. The EMI sample size was also quite small (n = 29) compared to the national sample and the significant improvement was only noted in listening comprehension, which would seem to be an expected consequence of lectures delivered in English. Lastly, we are not told whether the EMI students had access to additional EFL courses.

In contrast, a study with a direct comparison group was carried out by Hu et al. (Reference Hu, Li and Lei2014) in China. The researchers used two national tests: the National Matriculation English Test (NMET) and the College English Test (CET). Findings are somewhat inconclusive because the EMI students scored no better than the Chinese MOI students on the NMET but significantly better on the CET. However, even on the latter they did not score better than the CMI students once the pre-test differences were partialled out.

We now turn to studies which have directly sought to measure specific skills or areas of language competence. The first of these, Aguilar & Muñoz (Reference Aguilar and Muñoz2014), was carried out in Spain examining the impact of EMI on postgraduate Engineering students’ listening proficiency and grammatical knowledge using the Oxford Placement Test (OPT). The results show students improved in listening skills but not in grammar after a semester (60 teaching hours). Students that gained significantly more in listening and grammar were those with the lowest initial level of proficiency (elementary level, OPT band 3). This suggested that they benefited more from EMI instruction. The higher proficiency students actually performed worse in the grammar post-test than they did in the pre-test. We may note that: there was no comparison group so the significant gains were within-group only; the sample underwent very high levels of attrition (205 initial participants was reduced to 63), which may have affected the result.

In Taiwan, a study by Tai (Reference Tai2015) sought to assess the progress over one semester made in writing accuracy, complexity and fluency. The participants appear to be linguistics students who had elected to take the course. Findings suggest that there was a significant improvement in fluency with large effect sizes but no significant improvement for complexity and accuracy, perhaps reflecting the focus of the programme regarding writing.

Although strictly speaking a study about the impact of EMI in secondary schools, the study by Lin & Morrison (Reference Lin and Morrison2010) in Hong Kong is included here because it is relevant in terms of the transition to tertiary education. To examine the vocabulary size of first-year undergraduate students who had been instructed via EMI or Chinese MOI at secondary schools, 762 first-year undergraduate students were administered two tests on passive and active vocabulary. Of these students, 413 students’ essays were analysed to examine the lexical richness and the lexical appropriateness. The results show that only a very small number of EMI students (about 5%) lacked a satisfactory level of receptive academic vocabulary, compared to almost 40% of the L1MOI students. This suggests that Chinese MOI was not preparing students well for their (compulsory) EMI tertiary education. In the productive vocabulary test, only a small percentage of EMI students (23.9%) and an even smaller number of L1MOI students (1.5%) could produce a satisfactory level of academic vocabulary. However, due to their larger academic vocabulary knowledge, EMI students produced better quality essays than the L1MOI students.

Another example of a transition study is Pessoa et al.'s (Reference Pessoa, Miller and Kaufer2014) four-year longitudinal study. This study examined the challenges that multilingual high school students face when transitioning into college. The authors recorded the development of Qatari students’ literacy skills at an EMI university. They analysed samples of writing of 86 students on two academic English courses, representing a total of 7 texts from each student, over a two-semester period (602 texts in total). Results indicated that students had clear difficulties with reading comprehension. This was due to a lack of background knowledge, inadequate vocabulary knowledge and a dearth of reading stamina. Students also had difficulty with English academic writing. Nevertheless, once students had been taught strategies to apply to their writing, they understood better the expectations of academic writing, and their writing appeared to improve.

The above mentioned studies are the only HE studies which used objective tests to measure the impact of EMI on English language learning or English proficiency. A cluster of others have measured it via students’ self-assessment of their progress in English but these findings are usually embedded in much wider investigations of student beliefs. In Korea, Byun et al.’s (Reference Byun, Chu, Kim, Park, Kim and Jung2010) students generally believed EMI had led to an improvement in their English proficiency. In Taiwan, in Yeh's (Reference Yeh2014) study, 75% of students claimed via self-assessment that EMI had a positive influence on their English and in particular their listening comprehension. In contrast, Bozdoğan & Karlıdağ (Reference Bozdoğan and Karlıdağ2013) report that all students (self-assessed) felt they had made no progress or even regressed in their productive skills. No explanation can be arrived at for these different self-assessed findings. Bozdoğan & Karlıdağ propose that their results may have been due to very low levels of participation in class, but there is no evidence that there was high participation in either the Korean or the Taiwanese study. Once again this demonstrates a need for comparative studies which drill down into reasons for findings within each context.

3.10 Impact of EMI on content comprehension and learning

A relatively smaller number of studies (Vinke Reference Vinke1995; Hellekjaer Reference Hellekjaer2010; Joe & Lee Reference Joe and Lee2013; Dafouz, Camacho & Urquia Reference Dafouz, Camacho and Urquia2014) have measured students’ comprehension of lectures, and although their design and measurement instruments vary considerably we will nonetheless compare them here.

Vinke's (Reference Vinke1995) Ph.D. research in the Netherlands consisted of three separate studies of which one used an experimental design to compare the content understanding and learning of second-year students, from various majors in EMI and non-EMI classes taught by the same lecturer. For one lesson an experimental group of 50 students were taught through English and a control group of 50 students were taught through Dutch. The topic of the lecture was new to both groups, as checked via a questionnaire prior to the lecture. English proficiency differences of the two groups were taken into account in the analysis in order for it not to be a confounding variable. Whether these students could represent the average level of their peers was also checked based on their grades in English, Dutch, Maths and Physics. The EMI and non-EMI lessons were both video-taped thus ensuring fidelity to condition. In the post-test the students in the L1MOI group performed significantly better than the EMI group in terms of lecture comprehension.

Hellekjaer's (Reference Hellekjaer2010) study adopted students’ self-reported questionnaires to compare EMI students’ overall listening comprehension level of their EMI lectures and L1MOI lectures. The sample (n = 411) consisted of 364 Norwegian undergraduate and postgraduate students majoring in Business and Engineering from one university in Norway and 47 German graduate students from two universities in Germany. The findings suggest that the students’ listening comprehension of EMI lectures were lower than the L1MOI lectures, (based on descriptive statistics only – i.e. with no inferential statistical tests run). The main difficulty seemed to be unfamiliar and unintelligible words and expressions. By contrast, Joe & Lee's (Reference Joe and Lee2013) study in Korea found no obvious differences in lecture comprehension between EMI and L1MOI students. Although in Dafouz et al.’s (Reference Dafouz, Camacho and Urquia2014) study students’ self-reported comprehension of lectures was lower in EMI classes than in non-EMI classes, the researchers found no obvious differences when it came to the two groups of students’ official grades. So how might we account for these different findings?

Joe & Lee's (Reference Joe and Lee2013) study in Korea compared a group of 61 first- and second-year medical students’ learning of content knowledge through English and Korean in three lectures. The first half of each lecture was delivered in English and the second half in Korean, all taught by one Korean professor. However, unlike Vinke (Reference Vinke1995) the content of the two halves of the lectures was different. A pre-test in Korean was administered before each lecture to test students’ existing knowledge of the content of that lecture. A post-test was conducted in English for the EMI half of the lecture and in Korean for the Korean MOI half. In this study, the students’ English proficiency level was clearly specified with a mean score of 590.38 from PB (paper-based) TOEFL, which was considered high by the author. However, this finding should be interpreted with caution as there may be several confounding variables. First, there was no information given regarding fidelity to condition as it was unknown how much English was actually used in the EMI half of the lecture. Second, the pre-test was only in Korean, which means the students’ previous knowledge in English was not tested. Lastly, it was unclear whether the content difficulty levels of the EMI and non-EMI parts of the lectures were matched to ensure the MOI was the only independent variable.

Dafouz et al.’s (Reference Dafouz, Camacho and Urquia2014) study, located in Spain, compared EMI and non-EMI students’ official grades, including 106 students in Accounting, 115 students in Finance and 95 students in History from one university, also using a self-reported questionnaire. However, insufficient information is available about the design of the EMI and non-EMI tests and how much English was used in the EMI classes was not controlled for (or at least not reported).

Moving on from lecture comprehension, we should note a study which examined the issue of the impact of using English in written exams (Tatzl & Messnarz Reference Tatzl and Messnarz2013). Two exams with questions about Physics were administered to two groups of Engineering major students from four year groups at one institute in Austria. The results show no significant difference on students’ scores in the test in German and in English. As the author argued, this implied that in written EMI exams for Engineering undergraduate students there may not be a need for extra time allocated because of English. However, looking at the research design, we may question this conclusion. For example, there was no pre-test to ensure the students in the two groups had equivalent understanding in Physics, nor any explicit information to show the two groups had a similar starting point. In addition, the students’ low score on the test, which was due to the lack of preparation as claimed by the researcher, is in contradiction with the fact that the test was deliberately designed in a way that was easy for the students.

These are the only HE studies which have used some kind of objective measure to examine the relationship between studying a subject through the medium of English and achievement in content. Again a handful of studies embed self-assessed performance. For example, Kang & Park (Reference Kang and Park2005), in a Korean university, found a correlation between students’ self-assessed English and their performance on EMI courses. Kim et al. (Reference Kim, Tatar and Choi2014), also in Korea, found correlations between self-assessed language competence and confidence with academic subject learning. These findings are not surprising.

3.11 Focus on pedagogical strategies in the EMI classroom

Studies of interaction in EMI in HE, as retrieved by our systematic search, are relatively rare when compared to studies of classroom discourse and interaction in the secondary education phase (e.g. Dalton-Puffer Reference Dalton-Puffer2007).

Two features of lecturer discourse were investigated by Dafouz, Núñez & Sancho (Reference Dafouz, Núñez and Sancho2007), in particular the use of the pronoun ‘we’ and in relation to modal verbs. Three lecturers with Spanish L1 were recorded teaching a group of Aeronautics Engineering students through the medium of English on a summer school in Spain. As these students were drawn from 14 different nationalities and had just come together for a relatively short course this makes the EMI setting somewhat special and even more so given that the authors describe it as a ‘CLIL University Context’ in their title, although there is no evidence provided that the three lecturers were aware of CLIL pedagogical approaches. The findings show that the use of ‘we’ was used as a device to create solidarity with the audience: ‘We will see later’; ‘here we have’. . . ‘we tried to solve yesterday’ (p. 655). These were often collocated with the modal ‘can’: ‘we can calculate’. The authors propose that the heavy presence of these discourse features ‘open up a space for negotiation’ (p. 659) not normally present in the lecture situation. The paper does not report whether the lecturers themselves were aware of using these features frequently for the benefit of the students nor whether the students believed they benefited – something that future research of this kind in an EMI setting might usefully consider.

Operating in the Saudi context, but this time with smaller classes, Jawhar (Reference Jawhar2012) explored the extent to which classroom interaction is conducive to both content and language learning. The corpus consisted of teachers and students in an all-female institute studying Physics, Chemistry, and Early Child Education. Sixteen hours of EMI classes were recorded. The key finding was that teachers and students used an albeit limited number of linguistic devices (for example, the researcher focused on the use of ‘yes’ both as an acknowledgement of an assertion and as a backchannel by participants to encourage the speaker to continue) to facilitate the interaction in which the understanding of content was a key aim. Nonetheless, Jawhar wonders if, despite the presence of these features, it is enough to label even these small classrooms ones where language and content are being integrated. It should be noted that both Dafouz, Núñez & Sancho (Reference Dafouz, Núñez and Sancho2007) and Jawhar (Reference Jawhar2012) label their classroom settings as ‘CLIL’. However, the latter researcher is at pains to point out that – in the Saudi HE context – CLIL is not a term that everyone is aware of, let alone understands (p. 2).

Does the speed of delivery change because of switching to EMI from L1MOI? This was part of the topic of a study carried out by Thøgersen & Airey (Reference Thøgersen and Airey2011). Using ‘syllables per second’ and ‘mean length of run’ (the average number of syllables between pauses) they found that one lecturer's delivery did indeed slow down when he taught the lesson in English as opposed to Danish (L1). The researchers controlled for lesson topic and other factors. However, when later asked about his speech, the lecturer denied intentionally reducing the speech rate or even noticing that his speech rate had decreased. As he is described as ‘a highly experienced lecturer who teaches in English daily’ (p. 210), one can only assume that he was subconsciously accommodating for the learners’ lower command of the L2. However, the researchers also analysed his rhetorical style and found that in L2 (EMI) he was using a much more formal style perhaps more akin to written English which may have contributed to the slower speech rate. Also, of primary interest, is that the researchers found that despite the reduction in speech rate there was no obvious reduction in the amount of content delivered.

An aspect of interaction which has received little attention at HE level in the EMI context is the use of the L1 by teachers. Codeswitching behaviour was documented by Tarnopolsky & Goodman (Reference Tarnopolsky and Goodman2014) in a Ukrainian university by comparing codeswitching between EFL and EMI classes using a corpus of 52 lectures. EMI teachers switched for similar reasons to those in EFL classes, for example, explaining the meaning of words, and checking students’ comprehension. However, EMI teachers, unlike their EFL counterparts, did not switch for grammatical explanations, suggesting that the teaching of language forms was not high on their agenda. One reason for codeswitching that did appear to be specific to the EMI classes was explaining in L1 subject-specific English terminology encountered by students. A second different reason for switching (p. 388) was due to the (somewhat rare) occasions when the teacher did not know or had forgotten a word in English and therefore was obliged to slip back to L1. The latter is a reason for codeswitching not often found (or at least admitted to) in the EFL literature. Costa (Reference Costa2012) also found evidence of codeswitching to L1 Italian in her corpus of 16 hours of lectures. An interesting finding was that the lecturers switched even though there were non-native speakers of Italian in the audience; there were as many as 25% non-natives in one of the classes (Architecture). Nonetheless, in one episode the lecturer appears to be using an Italian expression ‘fabbrica del duomo’ and then translating it into English presumably, as Costa suggests, for the benefit of the international students.

4.1 Summary of findings

1. 1. Throughout the educational phases, Asia and Europe dominate the body of research carried out on EMI. Africa is particularly poorly represented in HE although better represented in other phases.

2. 2. In all phases of education, no particular design type stands out as being the one most adopted by researchers in order to explore the research questions.

3. 3. In primary and secondary education, there appears to have been a much greater focus than in HE on comparing the nature of classes taught through L1 and classes taught through English.

4. 4. In primary and secondary education, there appears to have been a much greater focus than in HE on comparing the outcomes of L2-taught programmes with L1-taught programmes.

Specifically in HE:

1. 1. There is a lack of consensus in the research field as to what label should be attributed to the phenomenon under observation and there is a lack of definition and specification as to what that label actually represents. Is EMI the ‘umbrella term’ which then encompasses CLIL, ICLHE, and so on? This lack of consensus is only just beginning to be tackled through theoretical analysis and discussion (see, for example, Dafouz & Smit Reference Dafouz and Smit2014).

2. 2. Growth of EMI in HE is evident in all geographical areas of the world although there are a few individual countries where it is in a state of contestation or even decline (e.g. Malaysia and Qatar). Europe is the only area which has received repeated and rigorous measurement of that growth.

3. 3. Growth appears to be top-down policy driven, rather than bottom-up and promoted by enthusiastic key stakeholders (teachers and students).

4. 4. The research field has expanded rapidly post-2005 in line with the growth of EMI.

5. 5. EMI in HE research is dominated by research questions relating to teacher and/or student beliefs, perceptions and attitudes towards its introduction and practice.

6. 6. There is a dearth of research, using objective tests rather than self-report, on the impact of EMI on improving students’ English proficiency. There is also a dearth of research on the impact of EMI on content learning. This means that any cost-benefit evaluation of EMI is inconclusive at best and impossible at worst.

7. 7. We need more studies which document transition from secondary education to HE.

8. 8. The preponderance of research based on case studies of single institutions further exacerbates the problem of not being able to ascertain the impact of EMI on either English proficiency or on content learning. The almost total absence of any comparative studies amongst institutions and/or amongst countries (except for the issue of EMI growth) means that the rigour offered by comparative education methodology (Bray, Adamson & Mason Reference Bray, Adamson and Mason2014) is largely absent.

9. 9. In terms of teacher and/or student beliefs, the positive motivations are: attracting international/foreign students; compensating for lack of resources in L1; instrumental advantages for home students (improving English and opportunities to study abroad); high value placed on international English.

10. 10. The negative motivations amongst teachers and students can be summarised as levels of English proficiency being too low and its potentially negative impact on content. Additional concerns include: the creation or consolidation of socio-economic elites and anti-egalitarian outcomes for students; additional workload for teachers switching to EMI; lack of teacher professional development and support.

11. 11. In terms of student beliefs and attitudes, variables such as gender, private-versus public, different academic subjects, different year levels, have not been sufficiently explored.

12. 12. The concept of ‘proficiency needed to teach through EMI’ is underspecified either through empirical research or by institutional requirements.

13. 13. Compared to the secondary sector there is a lack of research on classroom interaction in HE and what there is provides only a fragmented picture.

4.2 Weight of evidence measure

Each of the 83 studies was read in depth and information extracted from it ‘blind’ by two members of the review team who independently filled in a grid (see Appendix 4, Supplementary Materials) before they came together to discuss different interpretations and evaluations of what they had read. This procedure follows the systematic review guidelines to which we have previously referred. The ‘weight of evidence’ was measured on four criteria: the relevance of the research topic(s) of each study to the current review; the appropriateness of the research design to answer our review questions; the quality of the way the individual study was conducted and the trustworthiness of the research findings and conclusions; and the contribution of the research paper in general to this systematic review.

We must acknowledge that, even with two independent raters following guidelines (having received training based on the Evidence for Policy and Practice Information (EPPI) educational guidelines, 2017), a degree of subjectivity remains. Nevertheless, we would argue that Table 4 represents a fairly accurate picture of the overall state of research of EMI in HE. The table shows that of the 83 studies, 86% were judged to have either high or medium relevance to what we were trying to find out. Put differently, the topic or topics that the studies addressed were indeed of interest to us in trying to answer the review questions. Just 22% of the studies employed research designs which, prima facia, we judged as highly appropriate and 56% as fairly appropriate in answering the review questions. Only 16% of the 83 studies were judged to have a high level of trustworthiness (51% medium level). Thus, our overall conclusion about what we had read suggested that in all only 16% of studies were able to make a powerful contribution to answering our review questions and 55% a fairly strong contribution.

This suggests that research in EMI, despite the widespread and growing interest in the phenomenon, is still in an adolescent stage. The research agenda for the moment appears to be ad hoc, or at least without a consensus from the research community of what it should be. The level of quality of the research we would imagine could be improved considerably if there was greater discussion of what that agenda might be.

4.4 Limitations of this review

Papers written in languages other than English, listed on the electronic databases searched, were not included because of resource and space implications (but see the Supplementary Materials in Appendix 5 for the Chinese search procedure.) In addition, our search strategy was not able to locate several chapters in books which more recently we have become aware of through non-electronic search means.

As the search of electronic databases was completed in November 2015, literature published since then has not been included in the in-depth review. In addition, ‘grey literature’Footnote ¹ was not included in this search, nor blog posts, nor magazine or newspaper articles which have reported the changing EMI situation in countries such as Tanzania (see, for example, Mohammed Reference Mohammed2015) or ongoing debates happening in Fiji (see Maharaj Reference Maharaj2014). The reliability of the exclusion of studies based on titles and abstracts may be uncertain due to ambiguity, vagueness and lack of clarity, which made the relevance of studies challenging to assess. Finally, although numerous sources were searched and located, inherent publication bias cannot be ignored, which could lead to this review reinforcing the ‘file drawer’ effect.