Hostname: page-component-76fb5796d-x4r87 Total loading time: 0 Render date: 2024-04-28T19:37:39.728Z Has data issue: false hasContentIssue false

Things Are Not Always What They Seem: French-English Differences and the Problem of Measurement Equivalence*

Published online by Cambridge University Press:  10 November 2009

André Blais
Affiliation:
Université de Montréal
Elisabeth Gidengil
Affiliation:
McGill University

Abstract

This study examines the problem of achieving linguistic equivalence in Canada-wide surveys that rely, as they must, on two different questionnaires, one in English and one in French. It is argued that this is a crucial problem which needs to be dealt with squarely. Since language does not coincide neatly with territorial boundaries in Canada, there will always be some francophones outside Quebec and some non-francophones within Quebec who are not interviewed in their first language. A comparison of responses given in both languages by members of the same language group enables the investigator to distinguish true cultural differences from mere artifacts of measurement. That methodology is applied to one specific issue, support for electoral democracy.

Résumé

L'étude examine la difficulté de satisfaire le critère de l'équivalence linguistique dans les sondages pan-canadiens qui doivent nécessairement avoir recours à deux questionnaires, un en français et un en anglais. C'est là un problème fondamental qui doit être confronté directement. Le texte prèsente une méthodologie pour évaluer l'équivalence linguistique de questions de sondage. La méthodologie tire profit de l'existence de minorités linguistiques au Canada. Il en résulte qu'un bon nombre de francophones hors Québec et de non-francophones au Québec sont interviewés dans une langue qui n'est pas leur première langue. En consequence, il est possible de comparer les réponses fournies par les membres du même groupe linguistique selon que ses membres sont interviewés en français ou en anglais. Une telle comparaison permet de distinguer les différences entre groupes linguistiques qui sont réelles de celles qui sont artificielles, découlant d'une non-équivalence des questions françhises et anglaises. La méthodologie est appliqué à une question spécifique, l'appui à la démocratie électorale.

Type
Research Article
Copyright
Copyright © Canadian Political Science Association (l'Association canadienne de science politique) and/et la Société québécoise de science politique 1993

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1 See Davis, Steven, “Richler Was Wrong,” The Montreal Gazette, October 2, 1991Google Scholar; “On ne joue pas avec ce mot-là,” Le Devoir, October 2, 1991Google Scholar; and “L'accusation d'antisémitisme est une des pires à lancer à un peuple,” La Presse, October 2, 1991Google Scholar. For an extended rejoinder, see Sniderman, Paul M. et al. , “Psychological and Cultural Foundations of Prejudice: The Case of Anti-Semitism in Quebec,”; Canadian Review of Sociology and Anthropology 30 (1993), 242–70.CrossRefGoogle Scholar

2 See, for example, Almond, Gabriel and Verba, Sidney, “Some Methodological Problems in Cross-National Research,” in Etzioni, Amitai and Dubow, Frederic L., eds., Comparative Perspectives: Theories and Methods (Boston: Little, Brown, 1970), 349–64Google Scholar; Deutscher, Irwin, “Asking Questions Cross-Culturally: Some Problems of Linguistic Comparability,” in Warwick, Donald P. and Osherson, Samuel, eds., Comparative Research Methods (Englewood Cliffs: Prentice Hall, 1973), 163–86Google Scholar; Frey, Frederick, “Cross-Cultural Survey Research in Political Science,” in Holt, Robert T. and Turner, John E., eds., The Methodology of Comparative Research (New York: The Free Press, 1970), 173294Google Scholar; Hymes, Dell, “Linguistic Aspects of Comparative Research,” in Holt and Turner, eds., The Methodology of Comparative Research, 295342Google Scholar; and Verba, Sidney, “The Uses of Survey Research in the Study of Comparative Politics: Issues and Strategies,” in Rokkan, Stein and Verba, Sidney, eds., Comparative Survey Analysis (The Hague: Mouton, 1969), 56106Google Scholar. On the general problem of measurement equivalence in comparative inquiry, see also Przeworski, Adam and Teune, Henry, The Logic of Comparative Social Inquiry (New York: John Wiley, 1970).Google Scholar

3 An exception is Iyengar, Shanto, “Assessing Linguistic Equivalence in Multilingual Surveys,” Comparative Politics 8 (1976), 577–88.CrossRefGoogle Scholar

4 On this point, see Phillips, Herbert, “Problems of Translation and Meaning in Field Work,” in Etzioni and Dubow, eds., Comparative Perspectives, 385404.Google Scholar

5 Frey, , “Cross-Cultural Survey Research,” 275.Google Scholar

6 Deutscher, , “Asking Questions Cross-Culturally,” 167.Google Scholar

7 Cloutier, Edouard, “Les conceptions américaine, canadienne-anglaise et canadienne-française de l'idée d'égalité,” this Journal 9 (1976), 581604.Google Scholar

8 Verba proposes parallel blind translations from the original into the second language as being more efficient and more to the point than back translation. See “The Uses of Survey Research.” While this technique is useful when a word or phrase in the original language has multiple meanings in the second language, it does not solve the problem of achieving conceptual equivalence.

9 Verba, , “The Uses of Survey Research,” 64.Google Scholar

10 The survey can be pre-tested with bilinguals in both of its translated forms. Each bilingual respondent answers both versions and responses are compared to assess equivalence. Unless there is a considerable time lapse between the two administrations, however, there is the risk that respondents' recollection of the first-language survey will colour their responses to the second. Response-equivalence, moreover, is no guarantee of measurement equivalence. This is essentially a test of reliability across languages, but what is required is a demonstration of validity within and across languages. See Frey, , “Cross-Cultural Survey Research,”Google Scholar and Iyengar, , “Assessing Linguistic Equivalence.”Google Scholar

11 Hymes, . “Linguistic Aspects,” 327.Google Scholar

12 See the discussion in Verba, , “The Uses of Survey Research.”Google Scholar

13 Trudeau, Pierre Elliott, “Some Obstacles to Democracy in Quebec,” Canadian Journal of Economic and Political Science 24 (1958), 297CrossRefGoogle Scholar. While the word “French Canadians” is systematically used in the text, it is clear, given the content and the structure of the analysis and the reference to Quebec in the essay's title, that it was French-speaking Quebeckers that Trudeau had in mind. On the same theme, see Scott, Frank, “Canada et Canada Français,” Esprit (0809 1952), 185–86.Google Scholar

14 Trudeau, , “Some Obstacles to Democracy,” 299.Google Scholar

15 Ibid., 300.

16 See, for example, Pennock, J. Roland, Democratic Political Theory (Princeton: Princeton University Press, 1979)Google Scholar. In fact, representation was initially deemed to be incompatible with democracy and it evolved within the framework of monarchical and aristocratic institutions. Only much later was representation “widely accepted by democrats and republicans as a solution that eliminated the ancient limits on the size of democratic states.” See Dahl, Robert A., Democracy and Its Critics (New Haven: Yale University Press, 1989), 29.Google Scholar

17 See, for example, Pateman, Carole, Participation and Democratic Theory (Cambridge: Cambridge University Press, 1970)CrossRefGoogle Scholar, and Barber, Benjamin, Strong Democracy: Participatory Politics for a New Age (Berkeley: University of California Press, 1984).Google Scholar

18 Having one good basic question seems preferable to the sort of approach used, for example, by Dennis, where support for elections is tapped through a number of questions dealing with voting duty, attitudes about parties and sense of political efficacy. See Dennis, Jack, “Support for the Institution of Elections by the Mass Public,” American Political Science Review 64 (1970), 819–35CrossRefGoogle Scholar. The problem with that approach is that none of the questions directly gets at whether respondents agree with the idea of choosing our decision-makers through elections. At best, they tap possible correlates of support for elections. We believe that a question which asks respondents to compare the situation with elections with what might happen without elections more directly measures support for elections as an institutional device.

19 For technical details and a general analysis of the results, see Blais, André and Gidengil, Elisabeth, Making Representative Democracy Work: The Views of Canadians (Toronto: Dundurn Press, 1993).Google Scholar

20 A francophone is defined here as someone whose first language is French.

21 The results in Table 2 were obtained through an ordinary least square (OLS) regression where the dependent variable, support for electoral democracy, was dichotomized, taking the value of I for those who responded “worse” or “much worse” and 0 for those who said “better,” “much better,” or “about the same.” We obtained very similar results when we ran PROBIT regressions, and also when we used scores ranging from 0 (much better) to 1 (much worse) to measure the degree of support for elections. For the sake of simplicity, we report only the OLS results with the dichotomized dependent variable, which tell the basic story. For a description of the variables see the Appendix. The coefficients in Table 2 indicate the percentage point difference in support for electoral democracy that remains when all other variables are controlled for. For example, the –.20 coefficient in Table 1, equation 1, indicates that, everything else being equal, support for electoral democracy is 20 percentage points lower among francophone Quebeckers than among other respondents across Canada.

22 See Pateman, Carole, “The Civic Culture Revisited: A Philosophical Critique,” in Almond, Gabriel A. and Verba, Sydney, eds., The Civic Culture Revisited (Boston: Little, Brown, 1980), 57102.Google Scholar

23 See Cairns, Alan C., “Constitutional Minoritarianism in Canada,” in Watts, Ronald L. and Brown, Douglas M., eds., Canada: The State of the Federation, 1990 (Kingston: Institute of Intergovernmental Relations, 1990), 7196.Google Scholar

24 Lijphart, Arend, Democracies: Patterns of Majoritarian and Consensus Government in Twenty-One Countries (New Haven: Yale University Press, 1984), 22.CrossRefGoogle Scholar

25 We tested this possibility by creating a dummy variable equal to 1 for all those who said they would vote for the Parti québécois or the Bloc québécois if there were a provincial or federal election. Not only did this variable prove to be nonsignificant, but it had the wrong sign (that is, positive).

26 Northrup, David and Oram, Ann, A Survey of Attitudes about Electoral Reform: A Study for the Royal Commission on Electoral Reform and Party Financing. Technical Documentation (Toronto: York University, Institute for Social Research, 1991), 6.Google Scholar

27 As Table 3 indicates, there were too few francophone Quebeckers interviewed in English and no non-francophones interviewed in French outside Quebec to make the other two logical comparisons with any degree of confidence.

28 More precisely, the probability that the differences between those interviewed in French and those interviewed in English is due to sampling error is .03 for francophones in Quebec, .25 for francophones outside Quebec and .07 for non-francophones in Quebec. When the two key groups—non-francophones in Quebec and francophones outside Quebec—are combined, the probability that the difference between those interviewed in French and those interviewed in English is due to sampling error is only .02.

29 The same pattern emerges if we also include the two attitudinal variables in equation 2 of Table 2 (see Blais, André and Gidengil, Elisabeth, “What Would Happen if We Stopped Having Elections in Canada? Support for Electoral Democracy in Canada,” paper presented at the annual meeting of the Public Choice Society, 1992).Google Scholar

30 We thank Alain Noël for suggesting this interpretation.

31 This becomes clear if we consider replacing the offending “la situation” with the exact translation “les choses” (which, ironically, we had). Since “les choses seraient bien meilleures, meilleures…” would read oddly, the rest of the question would have to change to “les choses iraient bien mieux, mieux…” potentially changing the meaning yet again.

32 We have, of course, examined only one aspect of the democratic creed, but it is a basic aspect: there can hardly be a democracy in a large polity without there being elections. While we cannot rule out the possibility that belief in other dimensions of democracy may be weaker in Quebec, studies that have dealt with other aspects of the democratic ideal, such as tolerance and support for civil liberties, support our conclusion that francophone Quebeckers believe as strongly in democracy as other Canadians. See Johnston, Richard and Blais, André, “Meech Lake and Mass Politics: The ‘Distinct Society’ Clause,” Canadian Public Policy 14 (1988), 2543CrossRefGoogle Scholar, and Sniderman, Paul M., Fletcher, Joseph F., Russell, Peter H. and Tetlock, Philip E., “Political Culture and the Problem of Double Standards: Mass and Elite Attitudes toward Language Rights in the Canadian Charter of Rights and Freedoms,” this Journal 22 (1989), 259–85Google Scholar. There remains the possibility that the hypothesis was valid at the time Trudeau wrote his essay, and that francophone Quebeckers have become more democratic since the Quiet Revolution.

33 Verba, , “The Uses of Survey Research,” 69.Google Scholar