Skip to main content Accessibility help
×
Hostname: page-component-7c8c6479df-ws8qp Total loading time: 0 Render date: 2024-03-28T15:14:09.055Z Has data issue: false hasContentIssue false

2 - Equivalence and Bias: A Review of Concepts, Models, and Data Analytic Procedures

Published online by Cambridge University Press:  05 June 2012

David Matsumoto
Affiliation:
San Francisco State University
Fons J. R. van de Vijver
Affiliation:
Universiteit van Tilburg, The Netherlands
Get access

Summary

Introduction

This chapter addresses the methodological issues associated with equivalence and bias in cross-cultural research. These issues are a consequence of the nonexperimental nature of the research designs of cross-cultural studies. True experiments are based on the random assignment of participants to different experimental conditions, which presumably ensures that confounding variables are equated across experimental conditions. However, participants cannot be randomly assigned to cultures, and groups that are compared in cross-cultural studies can hardly ever be seen as matched on all background variables that are relevant for the constructs of interest. Cross-cultural psychology is not unique in the impossibility of matching groups; many studies in clinical and educational psychology involve situations in which intact groups are studied, and the assumption of the similarity of background characteristics across groups is unrealistic. The inability to conduct true experiments to address essential questions in cross-cultural psychology implies that we have to be careful in conducting our studies, being cognizant of relevant methodological knowledge and tools. It also implies that cross-cultural studies are always threatened by bias and inequivalence when cultural groups are being compared.

This chapter reviews the extant knowledge of these methodological issues. Our main message is that maximizing the validity of inferences should be the main concern of cross-cultural research and that methodological rigor in terms of establishing cross-cultural equivalence and suppressing bias across cultures plays a crucial role in this endeavor. We first describe a taxonomy of equivalence and bias, how they can be assessed, and the measures that can be taken to increase the validity of cross-cultural inferences. The second part gives an overview of procedures for adapting tests and survey questionnaires across cultures. Conclusions are presented in the final section.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2010

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aquilino, W. S. 1994 Interviewer mode effects in surveys of drug and alcohol usePublic Opinion Quarterly 58 210CrossRefGoogle Scholar
Azhar, M. Z.Varma, S. L. 2000 Mental illness and its treatment in MalaysiaAl-Issa, I.Al-Junun: Mental illness in the Islamic world163Madison, CTInternational Universities PressGoogle Scholar
Azuma, H.Kashiwagi, K. 1987 Descriptors for an intelligent person: A Japanese studyJapanese Psychological Research 29 17CrossRefGoogle Scholar
Baddeley, A. D.Thomson, N.Buchanan, M. 1975 Word length and the structure of short-term memoryJournal of Verbal Learning and Verbal Behavior 14 575CrossRefGoogle Scholar
Barrett, P. T.Petrides, K. V.Eysenck, S. B. G.Eysenck, H. J. 1998 The Eysenck Personality Questionnaire: An examination of the factorial similarity of P, E, N, and L across 34 countriesPersonality and Individual Differences 25 805CrossRefGoogle Scholar
Borg, I.Groenen, P. 1997 Modern multidimensional scaling: Theory and applicationsNew YorkSpringerCrossRefGoogle Scholar
Brislin, R. W. 1980 Translation and content analysis of oral and written materialsTriandis, H. C.Berry, J. W.Handbook of cross-cultural psychology389BostonAllyn and BaconGoogle Scholar
Brown, P.Levinson, S. C. 1987 Politeness: Some universals in language usageNew YorkCambridge University PressCrossRefGoogle Scholar
Camilli, G.Shepard, L. A. 1994 Methods for identifying biased test itemsThousand Oaks, CASageGoogle Scholar
Campbell, D. T. 1986 Science's social system of validity-enhancing collective belief change and the problems of the social sciencesFiske, D. W.Shweder, R. A.Metatheory in social science108ChicagoUniversity of Chicago PressGoogle Scholar
Chan, W.Ho, R. M.Leung, K.Chan, D. K. S.Yung, Y. F. 1999 An alternative method for evaluating congruence coefficients with Procrustes rotation: A bootstrap procedurePsychological Methods 4 378CrossRefGoogle Scholar
Cheung, M. W. L.Leung, K.Au, K. 2006 Evaluating multilevel models in cross-cultural research: An illustration with social axiomsJournal of Cross-Cultural Psychology 37 522CrossRefGoogle Scholar
Commandeur, J. J. F. 1991 Matching configurationsLeiden, the NetherlandsDSWO Press LeidenGoogle Scholar
Epstein, J. F.Barker, P. R.Kroutil, L. A. 2001 Mode effects in self-reported mental health dataPublic Opinion Quarterly 65 529CrossRefGoogle Scholar
Fernández, A. L.Marcopulos, B. A. 2008 A comparison of normative data for the Trail Making Test from several countries: Equivalence of norms and considerations for interpretationScandinavian Journal of Psychology 49 239CrossRefGoogle ScholarPubMed
Fidalgo, A. M.Ferreres, D.Muniz, J. 2004 Utility of the Mantel–Haenszel procedure for detecting differential item functioning in small samplesEducational and Psychological Measurement 64 925CrossRefGoogle Scholar
Fidalgo, A. M.Hashimoto, K.Bartram, D.Muniz, J. 2007 Empirical Bayes versus standard Mantel–Haenszel statistics for detecting differential item functioning under small sample conditionsJournal of Experimental Education 75 293CrossRefGoogle Scholar
Fontaine, J. R. J.Poortinga, Y. H.Delbeke, L.Schwartz, S. H. 2008 Structural equivalence of the values domain across cultures: Distinguishing sampling fluctuations from meaningful variationJournal of Cross-Cultural Psychology 39 345CrossRefGoogle Scholar
Formann, A. K.Piswanger, K. 1979 Wiener Matrizen-Test. Ein Rasch-skalierter sprachfreier IntelligenztestWeinheim, GermanyBeltz TestGoogle Scholar
Gomez, R.Burns, G. L.Walsh, J. A. 2008 Parent ratings of the oppositional defiant disorder symptoms: Item response theory analyses of cross-national and cross-racial invarianceJournal of Psychopathology and Behavioral Assessment 30 10CrossRefGoogle Scholar
Goodwin, R.Lee, I. 1994 Taboo topics among Chinese and English friends: A cross-cultural comparisonJournal of Cross-Cultural Psychology 25 325CrossRefGoogle Scholar
Grigorenko, E. L.Geissler, P. W.Prince, R.Okatcha, F.Nokes, C.Kenny, D. A. 2001 The organisation of Luo conceptions of intelligence: A study of implicit theories in a Kenyan villageInternational Journal of Behavioral Development 25 367CrossRefGoogle Scholar
Grill, J. J.Bartel, N. R. 1977 Language bias in tests: ITPA Grammatic ClosureJournal of Learning Disabilities 10 229CrossRefGoogle Scholar
Groves, R. M. 1989 Survey errors and survey costsNew YorkWileyCrossRefGoogle Scholar
Hambleton, R. K.Merenda, P. F.Spielberger, C. D. 2005 Adapting educational and psychological tests for cross-cultural assessmentMahwah, NJErlbaumGoogle Scholar
Harkness, J. A.Van de Vijver, F. J. R. 2010 Developing instruments for cross-cultural researchManuscript in preparationGoogle Scholar
Harkness, J. A.Van de Vijver, F. J. R.Johnson, T. P. 2003 Questionnaire design in comparative researchHarkness, J. A.Van de Vijver, F. J. R.Ph. Mohler, P.Cross-cultural survey methods19New YorkWileyGoogle Scholar
Harzing, A. 2006 Response styles in cross-national survey research: A 26-country studyJournal of Cross Cultural Management 6 243CrossRefGoogle Scholar
Ho, D. Y. F. 1996 Filial piety and its psychological consequencesBond, M. H.Handbook of Chinese psychology155Hong KongOxford University PressGoogle Scholar
Hofer, J.Chasiotis, A.Friedlmeier, W.Busch, H.Campos, D. 2005 The measurement of implicit motives in three cultures: Power and affiliation in Cameroon, Costa Rica, and GermanyJournal of Cross-Cultural Psychology 36 689CrossRefGoogle Scholar
Holland, P. W.Wainer, H. 1993 Differential item functioningHillsdale, NJErlbaumGoogle Scholar
Hui, C.Triandis, H. C. 1989 Effects of culture and response format on extreme response styleJournal of Cross-Cultural Psychology 20 296CrossRefGoogle Scholar
Jensen, A. R. 1980 Bias in mental testingNew YorkFree PressGoogle Scholar
Johnson, T. 1998 1
Leung, K.Bond, M. H. 2004 Social Axioms: A model for social beliefs in multi-cultural perspectiveAdvances in Experimental Social Psychology 36 119CrossRefGoogle Scholar
Leung, K.Van de Vijver, F. J. R. 2008 Strategies for strengthening causal inferences in cross-cultural research: The consilience approachJournal of Cross Cultural Management 8 145CrossRefGoogle Scholar
Lyberg, L.Biemer, P.Collins, M.De Leeuw, E.Dippo, C.Schwarz, N.Trewin, D. 1997 Survey measurement and process qualityNew YorkWileyCrossRefGoogle Scholar
Malda, M.Van de Vijver, F. J. R.Srinivasan, K.Transler, C.Sukumar, P.Rao, K. 2008 Adapting a cognitive test for a different culture: An illustration of qualitative proceduresPsychology Science Quarterly 50 451Google Scholar
McCrae, R. R.Costa, P. T. 1997 Personality trait structure as human universalAmerican Psychologist 52 509CrossRefGoogle ScholarPubMed
Muniz, J.Hambleton, R. K.Xing, D. 2001 Small sample studies to detect flaws in item translationsInternational Journal of Testing 1 115CrossRefGoogle Scholar
Piswanger, K. 1975 Interkulturelle Vergleiche mit dem Matrizentest von FormannUniversity of Vienna, AustriaGoogle Scholar
Poortinga, Y. H. 1989 Equivalence of cross-cultural data: An overview of basic issuesInternational Journal of Psychology 24 737CrossRefGoogle ScholarPubMed
Poortinga, Y. H.Van de Vijver, F. J. R. 1987 Explaining cross-cultural differences: Bias analysis and beyondJournal of Cross-Cultural Psychology 18 259CrossRefGoogle Scholar
Spini, D. 2003 Measurement equivalence of 10 value types from the Schwartz Value Survey across 21 countriesJournal of Cross-Cultural Psychology 34 3CrossRefGoogle Scholar
Suzuki, K.Takei, N.Kawai, M.Minabe, Y.Mori, N. 2003 Is Taijin Kyofusho a culture-bound syndrome?American Journal of Psychiatry 160 1358CrossRefGoogle ScholarPubMed
Tanaka-Matsumi, J.Draguns, J. G. 1997 Culture and psychotherapyBerry, J. W.Segall, M. H.Kagitcibasi, C.Handbook of cross-cultural psychology449Needham Heights, MAAllyn and BaconGoogle Scholar
Tanzer, N. K.Sim, C. Q. E. 1991 Test anxiety in primary school students: An empirical study in SingaporeDepartment of Psychology, University of GrazGrazGoogle Scholar
Vandenberg, R. J. 2002 Toward a further understanding of and improvement in measurement invariance methods and proceduresOrganizational Research Methods 5 139CrossRefGoogle Scholar
Vandenberg, R. J.Lance, C. E. 2000 A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational researchOrganizational Research Methods 2 4CrossRefGoogle Scholar
Van de Vijver, F. J. R. 2003 Bias and equivalence: Cross-cultural perspectivesHarkness, J. A.Van de Vijver, F. J. R.Ph. Mohler, P.Cross-cultural survey methods143New YorkWileyGoogle Scholar
Van de Vijver, F. J. R. 2008 On the meaning of cross-cultural differences in simple cognitive measuresEducational Research and Evaluation 14 215CrossRefGoogle Scholar
Van de Vijver, F. J. R.Daal, M.Van Zonneveld, R. 1986 The trainability of abstract reasoning: A cross-cultural comparisonInternational Journal of Psychology 21 589 615CrossRefGoogle Scholar
Van de Vijver, F. J. R.Leung, K. 1997 Methods and data analysis for cross-cultural researchNewbury Park, CASageGoogle Scholar
Van de Vijver, F. J. R.Valchev, V.Suanet, I. 2009 Structural equivalence and differential item functioning in the Social Axioms SurveyLeung, K.Bond, M. H.Advances in research on social axioms51New YorkSpringerGoogle Scholar
Van Hemert, D. A.Van de Vijver, F. J. R.Poortinga, Y. H.Georgas, J. 2002 Structural and functional equivalence of the Eysenck Personality Questionnaire within and between countriesPersonality and Individual Differences 33 1229CrossRefGoogle Scholar
Van Leest, P. F. 1997 Bias and equivalence research in the NetherlandsEuropean Review of Applied Psychology 47 319Google Scholar
Werner, O.Campbell, D. T. 1970 Translating, working through interpreters, and the problem of decenteringNaroll, R.Cohen, R.A handbook of cultural anthropology398New YorkAmerican Museum of Natural HistoryGoogle Scholar

Save book to Kindle

To save this book to your Kindle, first ensure coreplatform@cambridge.org is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×