Blunt instruments

By contrast, I seemed to have arrived at my relationship with my vacuum cleaner cluelessly. And perhaps as a consequence, to this day, I remain oblivious to its transcendental, transformative or hedonistic potential. Beyond the confines of that questionnaire – apart from the period when a member of my household was an enormous shaggy dog – I have simply never thought in depth about vacuum cleaners. Some relationships just remain unexciting and unexamined. And it's better that way.

Nonetheless, struggling to fill out that questionnaire eventually influenced my worldview.

As applied linguists, we routinely apply a priori umbrella labels to cover all of the diverse individuals who happen to participate in our research. But in fact, our subjects may not define themselves primarily, or at all, by the categories in which we frame them (e.g., as learners, let alone as learners who are variously motivated or willing-to-communicate). Once you have been asked, as a university professor, to characterize the extent to which you like to think, and seen your sum individual life experience miniaturized to fit within a Likert scale, the extent to which one-size-fits-all instruments necessarily obscure the orientations and dispositions of their varied subjects becomes clear.

That is why, long ago, I renounced all such research. Seeking the ‘particularization’ foregrounded by one of my academic heroes, Leo van Lier (e.g., Reference Van Lier and Hinkel2005), I turned my back on the extensive training in statistics of my grad school years. I relegated the use of quantitative methods to a minimal supporting role in case studies and other primarily qualitative inquiry. In all of my post-dissertation years, I had not once agreed to be involved in anything to do with surveys. Almost all of my published research involves theory-driven and otherwise elaborated storytelling about the lives and activities of particular people in their unique circumstances.

Yet today I am the Project Director for an unprecedented massive nationwide survey of college alumni who have studied languages abroad. In what follows, I endeavor to explain how and why that happened.

Whence an aversion to surveys?

In academia, there are few better ways to grasp the significance of turf wars (acrimonious disputes over a particular sphere of influence) than to enter a field as poorly defined as ‘Applied Linguistics’ in the 1990s. Various factions remained at odds over who and what should be entitled to claim dominance. Members of my generation came of age as scholars during the ‘SLA Theory Wars’ (Atkinson, Reference Atkinson2019, p. 113). They pitted those who believed second language acquisition (SLA) should be an exact, positivist cognitive science aiming for systematic reduction of theory (e.g., Gregg, Reference Gregg1993, Reference Gregg2000, Reference Gregg2001; Long, Reference Long1993) against early champions of the ‘social turn,’ who claimed space for multiple epistemological and ontological frameworks (e.g., Lantolf, Reference Lantolf1996, Reference Lantolf2001) and for the applicability of SLA findings to language education (Van Lier, Reference Van Lier1994).

My first attempt to enter this field followed a then-typical pathway: I transitioned from a foreign language major to graduate study in nineteenth century French literature. That career ended with the realization that I chose to focus exclusively on authors whose work I intensely disliked, whose message I was only too delighted to suffocate and obscure in a cloud of analytic gloom. When I caught myself plowing through Les Contemplations by setting Victor Hugo's alexandrines to the tune of The Yellow Rose of Texas, it was clearly time to quit. That left me free to privately, freely, openly, simply and uncritically steep myself in writing I actually adored.

I then landed quite unprepared into the workforce. Eventually I was hired by a contractor to the Foreign Service Institute. My job was to salvage ‘special’ or ‘problem’ learners of French. The stakes for them could not have been higher: They were for the most part mid-career military personnel or international aid workers whose careers and livelihoods had stalled. They were unable to advance because they lacked language competence to discuss the complex situations upon which countless everyday lives balanced: deforestation in the Sahel, porcine epidemics in Haiti, and the like.

In fact, my classroom was a perfect set-up for failure all around. I was the last resort for so-called fossilized learners, students who had already failed to learn and had hit a wall in the entrenched Foreign Service program based on audiolingual methodology. As for me, I really needed the job and would almost certainly be fired for the slightest deviation from the lesson plan. So we huddled together in desperation, dutifully chanting phrases drawn from two massive, unillustrated, brownish, mustard colored tomes first published in 1960. The sounds my students struggled to perfect ultimately built up to form inane dialogues about shoe sizes or finding a maid who was good with small children or bookish, genteel girl watching using demonstrative pronouns in everyday ‘situations’:

They might as well have explored one's relationship with one's vacuum cleaner.

So I did what perhaps countless sincere and desperate language instructors in my position have always done: I cheated. I began illicitly researching, xeroxing and smuggling into the classroom contraband readings, prominently featuring materials that were directly relevant to my students’ work. And they suddenly began to progress. Still, nobody ever had anything to say during breaks. Eventually, I began encouraging them to talk about their lives and their work, however haltingly.

Finding ways to improve language instruction became my calling.

Armed with a background in literary study and a newfound commitment, I plunged into the confusing and conflictual world of Ph.D. studies in Second Language Acquisition. Most of the training I received was in the mainstream cognitive approaches of that era. It prepared me to study input processing while half-heartedly upholding Krashen's model in teacher education roles. Other courses, however, were in sociolinguistics and the ethnography of communication; connections between these topics and contemporary SLA were difficult to grasp. My advisor, Sandra Savignon, was among the most visible advocates for a model of communicative competence including pragmatic and social dimensions of language use. Although they seemed essential, these aspects of language remained peripheral at best, far from the mainstream of SLA research of that era. I completed my studies. Yet the better way I sought remained elusive.

Over time, I would pledge allegiance to one of the newly arrived, socially-oriented theoretical frameworks: Vygotskian sociocultural theory (SCT) (Lantolf & Thorne, Reference Lantolf and Thorne2006).

Logical positivism dominated SLA at the time. Pursuing a theory that had emerged from a distinct and very different intellectual tradition meant engaging in a constant struggle to overcome much of my prior education. The first step was to understand that SCT is not a theory of language acquisition but a theory of mind, within which language plays a privileged role and for which praxis (i.e., finding validity in the usefulness of findings) is key. It is a theory of human development, cultural change, social history and evolutionary processes.

Understanding language competence as a fully social phenomenon instantiated in activity in turn meant dismissing the conception of language competence as a property of the individual mind. It was essential to reconceive language not as the grammar of the written sentence but as a cognitive tool used to mediate activity in a time and a space that were fully contextualized. It required me to stop viewing learners as independent processing devices, but instead as ‘unified, self-interpreting cultural agents’ (Dunn & Lantolf, Reference Dunn and Lantolf1998) whose subjectivity and agency are engaged in the activities they carry out. Above all, it was important to grasp Vygotsky's dialectical process ontology (Reference Vygotsky1978): While nature plays a role in human development, this development also influences nature, thereby creating novel developmental conditions that can only be understood, and studied, as historical processes.

Once this was understood, it was not difficult to find further support for rejecting positivism as the unique gold standard for inquiry in the human sciences. Bruner's essay on Two modes of thought (Reference Bruner and Brunner1987) opened my mind to the value of narrative as a key mode of cognition, alongside logico-scientific thinking. Gould's The mismeasure of man (Reference Gould1981), a history of psychometrics as an instantiation of scientific racism, disabused me of the notion that human science is or can be timeless in its pursuit of universal truths. Danzinger's Naming the mind (Reference Danzinger1997) demonstrated the historicity and cultural specificity of timely constructs such as motivation, attitude, and intelligence and the underlying reasons for their contemporary relevance to SLA. Ratner's scathing critique of positivist methodology in Cultural psychology and qualitative methodology (Reference Ratner1997), convinced me that, with its emphasis on fragmented variables, quantification of behavior, and context-insensitive operationalization:

Which brings us back to surveys: because they are assumed to measure relatively stable underlying traits or attitudes, surveys reveal little or nothing about historical ontology. Instead, as Ratner points out, they elicit ‘simple, discrete responses that fall far short of expressing the full significance of a psychological phenomenon’ (Reference Ratner1997, p. 41). They are essentially useless for research in the SCT tradition and might even be misleading or harmful in cases where there is a fundamental mismatch between survey constructs and features of local cultural practice.

Otherwise put, surveys run the risk of measuring phenomena that are just not relevant to participants. It's like asking soldiers in the trenches about the depth and quality of their love of vacuum cleaners.

SCT energized my scholarship and collaboration with colleagues in my home department of Applied Linguistics. It provided a strong rationale in support of qualitative approaches once considered lacking in rigor (e.g., Davis, Reference Davis1995) and a reliable analytic framework for a wide variety of research and teaching projects. These have included analyses of contextualized language practices as sites for microgenetic development in telecollaborative arrangements, in classrooms including study abroad returnees, and at study abroad homestay dinner tables. My case studies reconstructing the history of language learning have drawn on the sociocultural concept of ‘narrative templates’ (Wertsch, Reference Wertsch2002) to interpret participants’ active framing of their experiences in narrative. Following the insights of my former doctoral student Rémi A. Van Compernolle, I have relied on SCT as a basis for devising Concept-Based Instruction in the pragmatics of French which is, in my view, truly a better way. I am still learning about SCT and see no reason to set it aside.

Meanwhile…

During the 2020 coronavirus crisis and legitimate, widespread protests I will be the first to admit to having become discouraged, particularly as the gravity of the situation became clear. Prior to the crisis, public education in my country was already under long-term siege, as state funding was gradually reduced while neo-liberal corporatizing tyrannies cast their looming shadow over every aspect of university life. The prospects for foreign language education were particularly dire, with steep declines in enrollment, decreases in federal investment, proprietary tests of intercultural development excluding consideration of language proficiency (e.g., Hammer, Reference Hammer, Vande Berg, Paige and Lou2012) and the loss of over 650 college-level programs in the past few years. In my annual visits to the national foreign language educators’ convention, the effects of long-term neglect and defunding were apparent in the way the profession's sources of inspiration appeared to have stagnated sometime in the 1980s, with eager young teachers snapping cellphone photos of slide shows introducing communicative competence and the Input Hypothesis. Never mind sociocultural theory, or any other socially-oriented approach to teaching and learning: We are still listening to Stephen Krashen. Next we might go back to chanting sexist dialogues about women's bodies.

What will become of these endeavors now? On bad days I am distracted from research by scans of news about urban violence, shortages and supply chain breakdowns as well as university personnel furloughs, salary reductions and firings across the country, wondering not only about the survival of my profession but also of my own well being. What is the point of continuing to focus on language learning research when everything is broken or about to break? Is face-to-face teaching obsolete? How will I teach? How will I teach about teaching? What will become of my doctoral students’ thesis projects? Should I stuff my money into a mattress? Should I try to find more vegetable seeds? Should I order more canning supplies before the newly committed gardeners who bought all the seeds realize what happens in gardens? Should I try to find my ancient copy of Euell Gibbons’ Stalking the wild asparagus and brush up on my skill at foraging for edible weeds? Should I talk to my evangelical, pro-life neighbors about setting up an exchange of my dandelions for eggs from their chicken coop?

On better days I am reminded of why I originally decided to stop worrying about my exquisitely principled and rigorously defended academic stance, and why I embraced a survey project that did not mesh seamlessly with everything I have stood for as a scholar in the past. My country desperately needs education advocacy, whenever and however it arises, and the need for language education advocacy is especially acute in these dire times. When I assumed Project Directorship of what is presumably the largest-ever survey of former language learners abroad, it was to seize an opportunity to direct meaningful efforts that might contribute, however modestly, to the survival of public language education. Help can be found in the uplifting wisdom of colleagues, such as these words published in an earlier First Person Singular essay by Diane Larsen-Freeman:

To remain inspired and aware that foreign languages are valuable, I can consider, as my study's interview participants constantly remind me, that it doesn't matter much to them which language teaching guru had captured their teachers’ allegiance. They remember other things: the discovery of new ways of thinking, creative lessons, wacky classroom antics, the pride of self-expression, being funny, dreaming in another language, and above all whether or not others, including teachers, seemed to care about them. I can also recall being a young foreign language teacher myself, a minor rebel bucking the system in exchange for the pleasure of watching my students actually learn to talk about things they needed to say – and seeing the light in their eyes as they made meaningful, if clandestine, progress.

How I learned to love the survey

The survey in question represents one part of a sequential mixed-methods investigation into the lifelong influences of language study abroad. The project took shape in discussions with my colleagues Kevin McManus and Robert Schrauf as we contemplated responding to a call for research proposals on post-secondary international education from the United States Department of Education. Prior research in applied linguistics had neglected to study the aftermath of overseas language programs, to the point where Plews had identified post-study abroad as a new ‘frontier’ (Reference Plews2016). Our review of the literature had revealed a number of larger-scale studies of non-linguistic outcomes from academic focus (Hadis, Reference Hadis2005) and preparedness for international careers (Franklin, Reference Franklin2010) to global engagement (Paige, Fry, Stallman, Josic, & Jon, Reference Paige, Fry, Stallman, Josic and Jon2009), but painfully few studies with an explicit focus on language learning. Our proposal, entitled ‘The Careers of Language Study Abroad Alumni: A Comprehensive Investigation’ entailed a mixed-methods approach with a large-scale survey followed by life history interviews with selected participants.

Initially, I saw the survey primarily as a pathway toward access to abundant qualitative data. I had in mind particular people whose life stories I wished to document with permission from my university's ethics board. Over time, however, after funding was granted and the project got underway, my orientation began to shift. Our partner research team at the American Councils for International Education possesses significant experience and genuine expertise in survey design. As our partnership progressed, I began to appreciate the subtleties involved in generating an instrument capable of eliciting key information without overtaxing participants’ patience. In collaboration with Amelia Dietrich of The Forum on Education Abroad, who would facilitate our recruitment process, I became aware of the extent to which a project of broad scope would be valued throughout the international education enterprise. In particular, it became clear that while there is strong demand for research on the outcomes of education abroad at the national level, few study abroad provider organizations or university offices of global education normally have the necessary resources to carry out such research on their own. A best-case scenario would therefore involve collaboration between these institutions and a team of researchers with support from the federal government – precisely what we proposed.

Many doors have opened, including some on my own campus, in response to this project.

Previously I had cast survey designers’ efforts at standardization and interpretation of responses as uniform across participants as fundamentally absurd and as a violation of human self-expression and agency. But in working closely with my colleague Robert Schrauf and engaging his work on mixed-methods research (Reference Schrauf2016), I discovered a refreshingly different outlook on the fundamental nature and meaning of surveys, per se. Schrauf argues convincingly for the social-interactive nature of survey-taking as a practice in which participants ‘activate and utilize the everyday meaning-making strategies of normal conversation in construing their responses’ (Reference Schrauf2016, p. 77). This is evident, for example, in the perennial problem of ‘social desirability’ (Reference Schrauf2016, p. 81) for survey design: Respondents skirt the truth to provide answers that will either please the researcher or show themselves in a good light.

For Schrauf, the very existence of such ‘desirability’ issues demonstrates that survey participants see themselves as involved in a social encounter. As in everyday conversation, survey takers assess and interpret the intentions of survey authors and tailor their responses accordingly. Schrauf offers an example in which a survey about statistical literacy among applied linguists asked him to check boxes corresponding to the statistics packages he regularly uses: In fact, he uses three statistics packages that were not even listed among the options. Having considered the survey authors’ intention to assess basic statistical literacy, rather than attempting to convey a completely true response corresponding to his high level of literacy in that area, he checked the one he uses with students and wrote in one additional package under ‘Other.’

In retrospect, the vacuum cleaner episode should have taught me that survey takers are not necessarily victimized or silenced by surveys. They interact with these instruments and in so doing exercise their agency in a range of ways, including furious scribblings on margins in the case of paper surveys, or yelling at screens while filling out online questionnaires.Footnote ²

The project

Addressed to language study abroad alumni of all ages, the survey was launched in February of 2019 and closed in June of that year with 11,090 initial responses, 4,899 complete responses and 2,741 individuals who agreed to be contacted for an interview. We had recruited more than twice the number of respondents (2,000) named as a goal in our grant proposal. I was recently shocked and embarrassed to catch myself bragging about the extraordinary size of my N.

Having 2,741 people who had agreed to be interviewed for a research project was certainly another first. The question was how to devise a fully suitable and rigorous approach to designating whom to contact? After initial cleaning of the 4,899 complete responses, Robert Schrauf, our resident specialist in mixed-methods research, together with Jingyuan Zhuang, our research assistant, created a typology of life paths to guide selection of interviewees based on Multiple Correspondence Analysis, an exploratory statistical technique.

In this phase of the project, I observed the power of a process involving creative use of quantitative approaches to derive meaningful categories from a large data set. It provided a robust, structural link between the quantitative and qualitative phases of a sequential, truly mixed-method study. It also guaranteed principled selection of participants, freeing us from any concerns about cherry-picking − choosing only the most ideal candidates. As of this writing, using the typology as a guide, we have conducted 50 hour-long life history interviews with participants in a broad range of career paths, from sociology, engineering, banking, law, entrepreneurship, marketing, international aid, government service, literary translation and arts management to journalism, health care, sports management and, of course, education (which in our data subsumes high school teaching, executive roles within universities and study abroad administration). Many of our participants have found language competence to be directly relevant to their work, and some cite awareness of language as contributing to their ability, for example, to master computer programming or to interact harmoniously in international groups. Still others describe how the joy of language learning has enriched their life satisfaction in general.