Interdependence of research, practice, and technology

How research and practice relate to one another has long been an issue of discussion in CALL. As early as 1997, the European Association for Computer Assisted Language Learning (EUROCALL) adopted the theme of “Where research and practice meet” for their annual conference held in Dublin, and it was at this conference where Garrett (Reference Garrett1998) pointedly argued that research in CALL must maintain a close interrelationship with and play a key role in the development of practice in order to ensure the survival of CALL as a discipline.

As was discussed in Chapter 1, research in CALL is diverse in nature, as reflected in Egbert’s (Reference Egbert, Egbert and Petrie2005) description as “studies that take an analytic approach by looking at one or more … variables [of context, task, tool, language, and people] in any number of ways or studies that look at the system of which these variables are part, at their interactions and complexities and their effects on each other” (p. 5). Practice in CALL is equally diverse, referring to how technology is actually implemented in achieving language learning goals, and includes the tasks and activities that are used, often related to an underlying pedagogical view. Given that practice is “dependent on the individual language-learning environment, the tools that are available … and the expertise of teachers and learners” (Levy and Stockwell, Reference Levy and Stockwell2006, p. 178), the complexities are clearly obvious.

The goals of research in CALL are highly varied and, depending on how the research is carried out, we might see situations where practice feeds into research or, alternatively, cases where research feeds back into practice. For instance, where the goal is primarily to investigate a technology or a technique or pedagogy that uses technology, the starting point would generally be some element of the learning environment. In this case, it is the practice in the environment which forms the foundation of the research. In contrast, when the goal is to replicate or refine previous work described in the literature, the starting point may be questions or hypotheses that have been formulated from previous research, and the learning environment acts as a platform through which this research may be conducted. Thus, in the first instance, research is shaped around the learning environment, whereas in the second instance, the research design takes precedence, meaning that the learning environment may be shaped in order to achieve the goals of the research. It is of course possible for a single study to exhibit qualities of both, and a study that examines a new technology in a learning environment may well start with questions that have been based on previous research, just as a replication study may be conducted within an intact learning environment with a view to better understand practice. It should be kept in mind that the role of technology in any kind of research will depend very much on what was being examined and for what reason. Technology is obviously present in all CALL research but whether it is the focus of the research or rather a means of facilitating the research is highly dependent upon the objectives of the research. When considering these factors, it is possible to see the complexity of the interplay that exists between research, practice, and technology. Exactly what effect each of these factors has on the others, however, requires a deeper investigation of the CALL research, as has been done in the following section.

Research and practice in CALL

Investigations into the nature of research in CALL are not new, and periodically researchers have stopped to take stock of where the field is, where it has been, and where it is heading. These have considered aspects such as research paradigms (Chapelle, Reference Chapelle1997), the scope, goals, and methods of research (Levy, Reference Levy2000), criteria for effective research (Huh and Hu, Reference Huh, Hu, Egbert and Petrie2005), the technologies used for teaching language skills and areas (Stockwell, Reference Stockwell2007a), and so forth. A few investigations have been rather critical of some of the research conducted in CALL, such as Hubbard (Reference Hubbard2005) who, among other things, argues that it is not uncommon for insufficient information to be collected about subjects, and Felix (Reference Felix2008), who points out that there is an element of CALL research which is poorly conceived and conducted. In saying, this, however, the majority of research in CALL is approached in a systematic manner, and has contributed to our understanding of both the processes and the outcomes of learning a language through technology. CALL research is dependent upon the technology that is selected for investigation, the way in which the technology is used, and the research design which is implemented. The affordances of technology can give us insights into learning processes in ways not possible pre-CALL (e.g. see Blake, Reference Blake2000) and, as Garrett (Reference Garrett1998, p. 8) suggests, CALL technology can not only “inform language learning and teaching but [also] actively shape it.” For this to happen, however, we must be keenly aware of how CALL research is conducted in order to validate the findings.

In order to get a clearer idea of the way in which research is framed and carried out in CALL, a sample of articles in the CALL literature were examined and categorized. The sample included all articles (excepting software reviews, editorials, and commentaries) from 2001 through 2010 in four major English language CALL journals, CALICO Journal, Computer Assisted Language Learning, Language Learning and Technology, and ReCALL, coming to a total of 808 articles. These journals were selected as they focus specifically on computer-assisted language learning and because they have all been published for over ten years, hence were thought to provide the most consistent representation of CALL over this period. The aim was not to make definitive conclusions about research and practice in CALL, but rather to get a broad perspective of the nature of CALL research as seen through trends in the literature.

Table 9.1 provides an overview of the most commonly encountered types of research in the CALL literature according to this investigation. The descriptions of the research approaches were developed formatively during examination of the literature, and as such concrete counts were not recorded, but approximate figures were maintained. It should be pointed out that the research approaches in Table 9.1 do not cover all possibilities, and one notable omission from this table is that of technologies that were specifically designed for language learning purposes, such as research into commercial courseware. The reason for this omission is that, from the corpus of research that was investigated, this type of research has appeared with lower frequency in recent years, which is likely to be a reflection of a recent trend to use technologies that learners either already have access to and/or experience with outside the language learning environment rather than introducing new commercially available software. There are obviously cost reasons for using something that is already established elsewhere in the institution, that learners have already purchased themselves, or that can be used without cost, and while there are still examples of institutions purchasing commercial software for specific learning purposes, these are becoming less common, at least as far as the CALL literature is concerned. There is also a small amount of software designed for language learning which is made freely available to language teachers such as BETSY (e.g. Coniam, Reference Coniam2009) and FlashMeeting (Hopkins, Reference Hopkins2010), but again, these do not appear frequently in the literature.

Table 9.1 Approaches to research in CALL

It should also be noted that the research approaches are not necessarily discrete or mutually exclusive, and in some cases studies contained elements that were representative of two or more approaches. In saying this, however, Table 9.1 gives a general overview of the kind of research that is being undertaken in CALL, which may be useful to get a broad picture of what the field is about. The first four approaches focus quite specifically on the interrelationship between technologies and practice, while the final three approaches are more concerned with the interplay between research and practice in CALL. Research, practice, and technology are of course present in each of the approaches, but they take on a more or less central role depending on the research approach being adopted.

In the first approach – research that investigates the use of a new or existing generic technology for its applicability to language learning – the technology serves as the starting point for the research. The technology used is described as being “generic” as its original purpose was not for language learning purposes, but rather for use by either the larger community in general or other more specific non-language learning purposes. It is a technology which is new in its use within CALL, however, and in this sense is close to what Levy and Stockwell (Reference Levy and Stockwell2006) term as “emergent CALL,” in that it is still not considered as being in the mainstream. As technologies are rarely classified as being “new” for any significant period of time, research of this nature is generally very time-sensitive, and what was considered as a new technology at one point will either move into the mainstream or essentially disappear from significant use in CALL contexts. Due to the fact that there is generally very little previous research into the use of the technology for language learning purposes, descriptions will generally consist of a detailed overview of what the technology is and how it may be used, and in some cases may also include a description of its actual usage. Examples of this include Peterson’s (Reference Peterson2010a) overview of Second Life, Rosell-Aguilar’s (Reference Rosell-Aguilar2007b) description of podcasting, and Brett’s (Reference Brett2004) account of the speech synthesis and analysis software, PRAAT. In Peterson’s study, there is a description of the actual use of the technology in order to investigate how learners interacted using this new medium, while Rosell-Aguilar and Brett provide a detailed overview of the technologies without including examples of actual usage. The focus of this kind of research remains very heavily on the actual technology itself, and discussion of its potential for future use in language learning often remains at a rather hypothetical level.

The next research approach looks at the characteristics of new or existing technologies in a given language learning environment. It differs from the first approach in that while the focus still remains on the technology to an extent, it moves beyond the hypothetical level that makes up much of the research in the previous category to focus rather on actual usage in language learning environments. The technology may still be emergent, but is more likely to be what Levy and Stockwell (Reference Levy and Stockwell2006) describe as “established” in that it is already in use in CALL, although how widely it is accepted or used will depend very much on the technology. The research is “more concerned with what is rather than what might be technologically” (Levy and Stockwell, Reference Levy and Stockwell2006, p. 246), and the types of technologies that appear here include a variety of generic technologies that were already well established and used both in CALL and in the wider community at the time of publication. Examples might range from commonly used technologies such as email (e.g. Stockwell and Harrington, Reference Stockwell and Harrington2003) and word-processors (Rimrott and Heift, Reference Rimrott and Heift2005), through to generic technologies that were gaining in popularity at the time of writing, such as instant messenger (Li and Erben, Reference Li and Erben2007) or NetMeeting (Wang, Reference Wang2004). Rather than considering the technology from a general language learning perspective, studies of this nature generally embed the technology within a given language learning context and examine the learning process and/or outcomes.

The third approach investigates the development of what Levy (Reference Levy2000) refers to as new CALL artifacts, which are, as the name suggests, the software or hardware that are developed for language learning purposes. These artifacts may be developed primarily by the researcher(s) individually (e.g. Lee et al., Reference Lee, Wong, Cheung and Lee2009), at an institutional level (e.g. Hampel and Hauck, Reference Hampel and Hauck2004), or even at a cross-institutional level (e.g. Corda and Jager, Reference Corda and Jager2004). Such artifacts vary greatly in their functionality and complexity, and in the majority of cases are designed for specific learning environments. Some artifacts can be very sophisticated – such as the ICALL system developed by Tokuda and Chen (Reference Tokuda and Chen2004), which automatically diagnoses learner translations and provides feedback based on their errors – or, in contrast, we may see artifacts that are created through existing authoring software, such as Hot Potatoes (e.g. Allum, Reference Allum2004), where the basic template is already in place, but the content is completely created by the teacher. Research of this type may remain at a relatively hypothetical level (i.e. describe what is possible) or it may also provide examples of the artifact in actual practice with learners. In the majority of cases, however, the focus tends to remain on the technology itself in terms of its effectiveness.

The following research approach is concerned with identifying a problem in practice and finding a solution using technology. It is different from the earlier approaches in that this type of research does not start with a particular technology, but rather the choice of technology is made after identifying a specific learning goal or lack within a given learning environment, and matching this with the affordances that a technology provides. This approach is in line with what Colpaert (Reference Colpaert2006) refers to as a pedagogy-based approach, where design of a CALL environment begins from a “detailed specification of what is needed for language-teaching and learning purposes in a specific context” (p. 479). As Colpaert alludes to, this type of approach appears in the literature with surprisingly little frequency in favor of studies that focus on a pre-determined technology accompanied by a rationale to justify the usage and/or development of this technology. The problem with this research approach is the difficulty in viewing a learning context without some idea in advance of the technologies that are available at the institutional or the learner level. If, for instance, an institution has already incorporated an LMS such as WebCT, adopting or creating new technologies can be very risky, both in terms of institutional support and the potential load that learning a new system may place on learners (see Levy and Stockwell, Reference Levy and Stockwell2006, for a discussion of integration of CALL). Similarly, deciding to use technologies already owned by learners such as mobile phones, for example, without being aware of the percentage of learners who possess them, has the potential to isolate those who do not, unless it is possible to provide phones for all learners as necessary. It is not surprising, then, that the example study given in Table 9.1 by Torlavić and Deugo (Reference Torlavić and Deugo2004) does indicate that it was the intention of the authors in advance to create the Adverbial Analyzer artifact that is described in the study. What makes this different from the studies included in the third approach above is that rather than using the technology as the point of departure, they instead indicate the difficulties in teaching adverbs in their context, and describe the development of the system from a second language acquisition (SLA) perspective. Thus, the primary difference between this approach and the one that precedes it is the way in which the study is framed, although the line that defines them is not a clear-cut one.

In the fifth research approach, while of course the technology still plays a largely central role, the main goal is to build upon previous research. Stockwell (Reference Stockwell2010), for example, describes earlier studies carried out using mobile phones for language learning, but data were insufficient to identify the reasons why learners chose to use desktop computers in preference to mobile phones for completing vocabulary activities. As a result, the research was undertaken to identify possible reasons, specifically the effect of the platform used by learners to carry out the activities on scores and time taken to see if any causative relationship could be found. The study in this case built upon research that was specific to the technology being used, but there are also cases where the background research includes little or minimal reference to research incorporating technology. Also looking at vocabulary acquisition through technology, Yoshii and Flaitz (Reference Yoshii and Flaitz2002) investigated the effect of annotations on incidental vocabulary retention. Their description of the previous research focuses largely on the learning process itself, and the technology takes a more backseat position in both the previous research and the research design of their study. Despite the different angles of attack that were adopted, the intention of both studies is to achieve wider-reaching goals, which go beyond the specific environment being examined, in order to make a contribution to knowledge of affordances of a technology or language learning processes.

Meta-analyses, the sixth approach, are seen periodically in the research and are conducted by making a survey of certain literature, generally defined by the sources and the dates of the articles being investigated. Researchers will sometimes have concrete criteria in mind in advance of undertaking the meta-analysis, but it is also possible for criteria to be formed continually throughout the analysis (as has been done in the meta-analysis undertaken in this section). The topics and scope of meta-analyses are very broad. Hubbard (Reference Hubbard2008), for instance, looked at the range of theories used in CALL in a single journal, the CALICO Journal, over a twenty-five-year period, to investigate which theories were used in CALL research (see Chapter 1 for more details of this study). Levy (Reference Levy2000) examined research within a single year, 1999, looking at five journals and four books that dealt specifically with CALL, to determine the range and objectives of the research that took place in that year. The scope does generally tend to be somewhat wider than these two studies, as in Felix’s (2005a) examination of twenty-two different journals from 2000 through 2004 to investigate how the effectiveness of CALL is portrayed in research, Hubbard’s (Reference Hubbard2005) investigation of four major CALL journals from 2000 to 2003 for characteristics of subjects described in the CALL literature, and Stockwell’s (Reference Stockwell2007a) study of four major CALL journals from 2001 to 2005 to determine the choice of technology for teaching specific language skills and areas. Meta-analyses can serve to either identify streams of research that can be followed by others (e.g. Levy, Reference Levy2000), or to point out areas that provoke researchers to think about how their research is conducted (e.g. Felix, Reference Felix2005b). In this sense, they have a more obvious relationship with research, but there are analyses that can also relate to improving practice in terms of being a benchmark of our current understanding of work done in the field.

The final approach includes discussions that are not about nor specific to a particular technology, but rather about concepts such as culture (Levy, Reference Levy2007); research paradigms (Chapelle, Reference Chapelle1997); theories such as constructivism (Felix, Reference Felix2002); interactionism (Harrington and Levy, Reference Harrington and Levy2001; Chapelle, Reference Chapelle, Egbert and Petrie2005), or socioculturalism (Warschauer, Reference Warschauer, Egbert and Petrie2005); or even pedagogical approaches, such as Doughty and Long’s (Reference Doughty and Long2003) overview of methodological principles and pedagogical procedures for optimizing a distance learning environment. Discussions in this approach will generally not include empirical data, but there are some exceptions where the discussion is accompanied by first-hand examples of actual usage (e.g. Levy and Kennedy, Reference Levy and Kennedy2004). The general purpose of this approach to research is to make readers aware of the concept being discussed in order to improve research or practice in the field.

Looking at these research approaches, it is evident that the term “diversity” is directly relevant to research and practice in CALL in many regards. There was a good deal of variation in the frequencies of each of the approaches in the literature, however, with technology-centered approaches outweighing more research-centered ones. A large proportion of research described the characteristics of generic technologies, and while these technologies varied considerably, synchronous computer-mediated communication (CMC) technologies including chat, audio-, and video-conferencing were very common. Other approaches that featured heavily in the literature were the development and/or use of a technology and the investigation of the use of a generic technology for its applicability to language learning. As pointed out above, research that identified problems in practice was very rare and, in many cases, descriptions appeared to justify use of a technology that had already been selected. Research that discussed concepts, theories, or pedagogies related to CALL appeared periodically in the literature, as did studies that identified gaps in research. Meta-analyses were generally reserved for more “experienced” researchers, and those who conducted one meta-analysis sometimes followed this up with other meta-analyses on a similar or different theme at a later date.

One trend that was evident in research over the ten-year period is a slow decline in studies about technologies that have been developed by the researcher or institution in favor of generic technologies. It is not clear whether this is a reflection of the lack of time and/or expertise to develop materials that would meet learner expectations, or of a greater interest in emerging generic technologies, but it is a trend that needs following more closely over the next few years. Another tendency that could be seen in the literature is a marked increase in the number of studies that investigate CMC technologies. This is probably not surprising given the increased accessibility to faster Internet services and improved functionality, making it possible to carry out interactions not only through text but also through audio and video, but it is perhaps also related to the fact that these technologies are readily available and studies can be carried out with little extra cost or time. There has also been an increase in the number of studies that include empirical data, where studies that included data collection and analysis comprised less than half the total articles in the early 2000s, these increased to around three-quarters of the total articles published in the journals over the past three years. This might be considered an indication of the increased expectations on researchers in CALL to examine their individual learning contexts rather than simply describing the possibilities of the technology. Care should be taken, however, not to fall into the trap of undertaking only outcomes-based CALL research (see Burston, Reference Burston2003; Chapelle, Reference Chapelle2010), which attempts to justify the existence of using technology for language teaching. Rather, there is a need to explore the complexities involved in the interplay between research, practice, and technology, and to link this to further research and development in the future.

Example 1: investigating applicability of a new technology

Example 2: describing the development and use of a technology

Example 3: identifying a gap in research

Selecting a starting point for research

Each of the three studies listed above has a very different point of departure. In the first study, the starting point is clearly the technology with a view to showing how it can be used in language teaching and learning. Rosell-Aguilar (Reference Rosell-Aguilar2007b) argues that the technology is applicable to language learning with regard to its affordances, available resources that may be used in language teaching, and theories of language learning. The starting point of the second study by Baturay et al. (Reference Baturay, Dalaglu and Yildirim2010) is also a technology, but in this case it is a technology that has been developed by the authors themselves, and as a result the authors’ aim appears to be to show that the technology is appropriate for language learning and that it was viewed positively by the learners. The final study has research questions as its starting point, where Yoshii (Reference Yoshii2006) identifies an area of inquiry from the literature and seeks to investigate this in a thorough and methodical manner. These three very different starting points – a generic technology, a self-developed technology, and research questions – have very much dictated the way in which the study was framed and presented. The reasons behind these different starting points are, however, considerably more difficult to determine.

Without being familiar with each of the researchers who carried out these studies it is impossible to tell what their individual motivations might have been. It may be possible, however, to consider some of the possible reasons for carrying out research in CALL and the factors that may contribute to the types of research that appear in the CALL literature. Just as learners who use CALL are varied in their backgrounds, training, and experience with technology, motivation, goals, and their individual personalities, we could say that the same applies to those who carry out research into CALL. Teachers who have previously learned through CALL as a student will have the benefit of seeing it through the learners’ eyes (Kolaitis et al., Reference Kolaitis, Mahoney, Pomann, Hubbard, Hubbard and Levy2006), just as formal training in CALL can give teachers a wider perspective and grounding that will likely have a large effect on how CALL materials are designed and/or implemented (see Hubbard and Levy, Reference Hubbard, Levy, Hubbard and Levy2006, for details on teaching education in CALL). Similarly, attempting to learn how to implement CALL without sufficient support can result in teachers using very simple materials or even opting to not use it at all (Stockwell, Reference Stockwell2009). Of course, experience as a researcher is another essential factor, and this will have an effect on how research is approached. A background in research into SLA, for example, might lead to a stronger focus on the learning process rather than the technology, which again will affect both what is researched and how the research is conducted.

The motivation for using CALL in the first place will also greatly affect the research approach. There will likely be a difference between a teacher who wants to try technology out of interest, a teacher who identifies a learning need in their given learning context and decides to adopt technology to fill this need, and a teacher who is forced to use technology because of institutional pressures. Goals with technology may be very small, such as using authoring software like Hot Potatoes to create quizzes in a single class, or they may be somewhat grander, such as creating a system that will be used by several institutions by learners of different languages. As with any endeavor, the individual personality of the teacher will also impact what is researched and how it is undertaken, even if all other factors are more or less equal. Someone with a desire to try new things may be more adventurous in the technologies they select, whereas someone who is more conservative may choose to stick with something more mainstream. Similarly, the ability to view an environment holistically or discretely will likely affect not only how technologies are implemented, but also how the research methodology is designed.

An important underlying factor behind CALL research is the environment in which the learning takes place. There will be differences in the technologies that are appropriate for use in, for instance, a face-to-face and a distance environment (see Chapter 5). Even within the same face-to-face environment, other differences will emerge dependent upon the technologies that are available, both at an institutional and individual level. Approaches to undertaking research may also vary depending on whether there are pressures to research, either self-imposed or external, and the expected outlets for publication if research is to be published. More “prestigious” journals will generally require a more rigid research framework to be put in place, which in turn can directly affect what happens in the language learning environment. It becomes evident, then, that it is all of these factors – personal, environmental, and institutional – which come together to guide the way in which CALL practitioners make decisions about how to undertake research.

Diverse outcomes of research

If the starting points of the research are different, then it follows naturally that the outcomes of the research will also be very different. If the goal of research is to describe a new technology for use in language learning, then the outcomes of such research might be, as was the case with Rosell-Aguilar (Reference Rosell-Aguilar2007b), a list of suggestions of how this technology might be used in teaching, and perhaps also thoughts for research for others who are considering using the technology in the future. Where the starting point is to see how a self-designed artifact such as the WEBGRAM system created by Baturay et al. (Reference Baturay, Dalaglu and Yildirim2010) is viewed by learners, the outcomes may be more localized. While Baturay and her colleagues provide design principles for those who may plan to develop similar systems, a secondary outcome of this type of research might be to show that a project was successful, which would be of particular importance if there are financial stakeholders associated with it. Finally, when the point of departure is to investigate a particular aspect based on research, the outcomes will, as Yoshii (Reference Yoshii2006) proposes, suggest how the results fit in with previous research and what further research is still required. Research of this kind has the potential to inform further research, not only in CALL but potentially in other aspects as well. Thus, the outcomes of the research will be determined by the starting point of the research in terms of the applicability to practice and further research, and how much these results can be of use to a wider audience.

This brings us to a rather pointed comment by Pica, which is of some relevance to the CALL community as well, when she suggests that “many teachers believe that research on L2 learning has little to say to their everyday classroom needs and decisions” (Reference Pica1994, p. 49). It is not difficult to see how CALL practitioners may also feel that research on CALL has little to say to their everyday classroom needs and decisions, in particular when it describes a particular technology that cannot readily be obtained or a teaching approach that does not suit the goals or the environment they are teaching in. Without being critical of the study by Baturay et al. (Reference Baturay, Dalaglu and Yildirim2010), it is all too easy for outcomes from research to have a very limited audience to whom it can be useful, and this is a trend that is often seen in research surrounding self-developed artifacts. This problem can be alleviated somewhat by taking the time to include details of how the implications can be of relevance to others outside the scope of the specific environment being described when disseminating the research. As the conductor of the research, the researcher is in the best position to examine all aspects of the environment along with any limitations and shortcomings that may have arisen. Armed with this knowledge, the researcher has the ability to frame the outcomes of the study such that it can be of relevance for others undertaking similar work, even in the case of self-developed artifacts. Bearing in mind the difference between research for evaluative purposes (i.e. for in-house justification or improvement) compared with research for dissemination purposes (i.e. for the wider CALL community), can make it easier to frame the results in a way that can be of benefit for both research and practice at a local and wider level.

Considering research outcomes is of course not only limited to self-developed artifacts, but is also relevant to other approaches to research as well. When examining a generic technology for its applicability for language learning, moving beyond the hypothetical to the actual can provide a valuable foundation for others who are also investigating the possibilities of using this technology. While of course the suggestions provided by Rosell-Aguilar (Reference Rosell-Aguilar2007b) on podcasting are very useful and have the potential to shape practice and further research, conclusions based on empirical research can give deeper insights into the applicability of a technology and its pedagogical applications. Moreover, well-thought-out suggestions for teaching based on personal experiences can make a difference, not only to future research that is undertaken, but also as a guide to teachers who may decide to use the technology without undertaking formal research on it, thus directly impacting actual practice as well.