If the desire of most practitioners to professionalise language teaching were not motivation enough, increasing demands for accountability in foreign and second language education – both from those who finance such programmes and from students, the consumers – make programme evaluation of growing importance for applied linguists, administrators and language teachers. To be satisfactory, evaluations will need to be rigorous, theoretically motivated and data-based. As such, the field experiences and developing expertise among second language programme evaluators are suitable topics for inclusion in the Cambridge Applied Linguistics Series.

The more sophisticated study of second language acquisition and second language classroom processes of the 1970s and 1980s has placed second language programme evaluators in a better position to meet the demands for accountability. So has the increasing availability in masters level programmes of basic training in research methods and statistics. What is still often lacking, however, is much appreciation of the potential of qualitative methods in evaluation work, and of how useful a combination of second language research findings and quantitative and qualitative methods can often be.

Drs Alderson and Beretta are both seasoned programme evaluators and very aware of these issues. In Evaluating Second Language Education, they have provided a series of case studies which illustrate the strengths and limitations of qualitative and quantitative approaches and the frequent value of combining them. In addition, in this book, they and the other contributors have sought to document evaluation studies from the inside, in all their gory, private detail, rather than to provide more polished, but less informative, versions for the shop window.

Introduction

The purpose of the present chapter is to provide an account of the evaluation of a program, not so much so that the program may be judged but in order to document the shifting perspectives, false starts, design alterations, obstacles, and difficulties of interpretation that often characterise evaluations.

Evaluation reports that give the impression of a smooth, professional operation that went off without a hitch are far from the mark. Current thinking in evaluation would deny the validity of such impressions. Discussion with other evaluators, my own experience, and established perspectives in the evaluation literature (e.g. Cronbach et al. 1980) suggest that a more realistic picture would present a messy, chaotic series of compromises, where classic research designs disintegrate, where vain hopes of contributing to certain kinds of learning theory are soon dispelled, and where all that lies between the pragmatic evaluator and scholarly perdition is a sense of disciplined inquiry, whether it relates to quasi-experimental research, to a host of naturalistic approaches, or to policy analysis.

The intention of this chapter is to provide food for thought not only for would-be evaluators, but for L2 program developers too, who can assist substantially in the evaluation process in ways that will hopefully become evident in the course of the present account.

In the mid-1980s, I conducted an evaluation of an English language project in Bangalore, India. In this chapter, I will use this evaluation as a focus for discussion as it features many of the issues that are of interest to evaluators.

This book examines prominent theoretical and methodological issues in the evaluation of second language education. It offers a range of state-of-the-art case studies, a review of the literature showing how the field got to where it is, and practical advice on how to design and execute evaluations in the field.

As a cursory glance at publishers' lists and the contents of the major journals reveals, the discipline of second language education evaluation has been afforded relatively little attention. By contrast, in general social and educational spheres, provision is routinely made for evaluation, and this emphasis is reflected in their respective literatures. One of the most eminent behavioural scientists of the last 50 years has devoted two recent books to the subject (Cronbach 1982, Cronbach et al. 1980) and has called evaluation ‘the liveliest frontier of American social science’ (1980:13).

In second language education, there has long been dissatisfaction with evaluation which has been methodologically inflexible and uncertain or misguided as to its role. If the published literature is at all representative, it has not been a lively frontier.

This edited collection is designed as a first step in bringing the evaluation of second language education to the cutting edge.

Specifically, this book offers commentary and data intended to foster a self-reflecting attitude in researchers already involved in evaluation and to provide useful input to teacher-training programmes.

In designing the book, it was judged that what an evaluator really needs to know is how to deal with ad hoc investigation that appears to have no tradition that can be appealed to.

Introduction

This chapter has two principal aims. The first is to provide a review of previous evaluation studies in foreign language teaching, so that future evaluations may be informed by past experience in the field; the second, to set the evaluation of language education within the broader framework of educational evaluation.

An overview of the history and development of the evaluation of language education since the early 1960s is presented. Although occasional studies were carried out before the 1960s, the past quarter of a century has seen a notable growth in such studies and therefore the period from the 1960s to the present will be the focus of the review. In addition, reference is also made, in the second part of the chapter, to evaluation studies and theory in education more generally. The aim is not to be exhaustive, but to characterise trends and developments in language education evaluation.

To date, very few books have appeared on the evaluation of language teaching programs in general. This compares unfavorably with the general field of educational evaluation, where dozens of titles appear annually in one publishing house alone (Sage). In the language teaching journals, evaluation studies are rarely published which do not focus on the seemingly never-ending ‘methods’ debate (which had wearied Sweet as long ago as 1899) or on the highly politicised bilingual programs. Yet in the educational and social spheres, specialist evaluation journals proliferate.