Skip to main content Accessibility help
×
Hostname: page-component-848d4c4894-jbqgn Total loading time: 0 Render date: 2024-07-05T03:33:17.559Z Has data issue: false hasContentIssue false

14 - The Evolution of Multigroup Comparison Testing across Culture: Past, Present, and Future Perspectives

from Part 3 - Culture and Assessment

Published online by Cambridge University Press:  21 January 2021

Michael Bender
Affiliation:
Universiteit van Tilburg, The Netherlands
Byron G. Adams
Affiliation:
Universiteit van Tilburg, The Netherlands
Get access

Summary

Although multi-group comparisons have been of substantial interest to cross-cultural psychologists for the past 6 decades, the critical importance of measurement invariance was neither recognized nor made possible until the seminal CFA work of Jöreskog (1971) in developing a method capable of such analyses. Further complications have been the assumption that any difference findings derive from the cultures involved, albeit with no identification or justification for such differences (Matsumoto & Yoo, 2006). Indeed, the 1990’s saw a substantial increase in testing for measurement and latent mean equivalencies across numerous groups that commonly included countries. It was during this 20-year period (1990-2010) that the impracticality of the CFA approach to these analyses became known and the development of a new generation of alternative methodological procedures that allow for approximate, rather than exact measurement invariance across groups became known. In this chapter, we trace the early difficulties in testing for measurement equivalence across cultural groups, identify common quandaries encountered in applying these analyses across multiple groups, and outline the pros and cons pertinent to the more recent statistical procedures used in testing for approximate measurement equivalence.

Type
Chapter
Information
Publisher: Cambridge University Press
Print publication year: 2021

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Allik, J., & Ralo, A. (2019). Culture and personality. In Matsumoto, D. & Hwang, H. C. (eds.), Oxford Handbook of Culture and Psychology. New York: Oxford University Press.Google Scholar
Asparouhov, T., & Muthén, B. (2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16, 397438.CrossRefGoogle Scholar
Asparouhov, T., & Muthén, B. (2014). Multiple-group factor analysis alignment. Structural Equation Modeling: A Multidisciplinary Journal, 21, 495508.CrossRefGoogle Scholar
Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: Conceptual, strategic, and statistical considerations. Journal of Personality and Social Psychology, 51, 11731882.Google Scholar
Boer, D., Hanke, K., & He, J. (2018). On detecting systematic measurement error in cross-cultural research: A review and critical reflection on equivalence and invariance tests. Journal of Cross-Cultural Psychology 49, 713734.CrossRefGoogle Scholar
Bond, M. H. (1988). Social psychology across cultures: Two ways forward. In Adair, J. G., Belanger, D., & Dion, K. L. (eds.), Advances in Psychological Science: Social, Personal, and Cultural Aspects (vol. 1, pp. 137150). Sussex: Psychology Press, Ltd.Google Scholar
Bond, M. H. (2019). Traveling from the past into the future of cross-cultural psychology: A personal-scientific journey. In Matsumoto, D. & Hwang, H. C. (eds.), Oxford Handbook of Culture and Psychology. New York: Oxford University Press.Google Scholar
Bond, M. H., Leung, K., Au, A., Tong, K. K., Reimel de Carrasquel, S., Murakami, F., & Lewis, J. R. (2004). Culture-level dimensions of social axioms and their correlates across 41 cultures. Journal of Cross-Cultural Psychology, 35, 548570.CrossRefGoogle Scholar
Bowden, S. C., Saklofske, D. H., van de Vijver, F. J. R., Sudarshan, N. J., & Eysenck, S. B. G. (2016). Cross-cultural measurement invariance of the Eysenck Personality Questionnaire across 33 countries. Personality and Individual Differences, 103, 5360.CrossRefGoogle Scholar
Bryk, A. S., & Raudenbush, S. W. (1992). Hierarchical Linear Models: Applications and Data Analysis Methods. Thousand Oaks, CA: Sage.Google Scholar
Byrne, B. M. (1989). Multigroup comparisons and the assumption of equivalent construct validity across groups: Methodological and substantive issues. Multivariate Behavioral Research, 24, 503523.Google Scholar
Byrne, B. M. (1991). The Maslach Burnout Inventory: Validating factorial structure and invariance across intermediate, secondary, and university educators. Multivariate Behavioral Research, 26, 583605.CrossRefGoogle ScholarPubMed
Byrne, B. M. (2006). Structural Equation Modeling with EQS: Basic Concepts, Applications, and Programming (2nd ed.). Mahwah, NJ: Erlbaum.Google Scholar
Byrne, B. M. (2008). Testing for multigroup equivalence of a measuring instrument: A walk through the process. Psicothema, 20, 872882.Google Scholar
Byrne, B. M. (2012). Structural Equation Modeling with Mplus: Basic Concepts, Applications, and Programming. New York: Taylor & Francis/Routledge.Google Scholar
Byrne, B. M. (2016a). Structural Equation Modeling with Amos: Basic Concepts, Applications, and Programming (3rd ed.). New York: Taylor & Francis/Routledge.CrossRefGoogle Scholar
Byrne, B. M. (2016b). Adaptation of assessment scales in cross-national research: Issues, guidelines, and caveats. International Perspectives in Psychology: Research, Practice, Consultation, 5, 5165.Google Scholar
Byrne, B. M., & Campbell, T.L. (1999). Cross-cultural comparisons and the presumption of equivalent measurement and theoretical structure: A look beneath the surface. Journal of Cross-Cultural Psychology, 30, 557576.CrossRefGoogle Scholar
Byrne, B. M., & Shavelson, R. J. (1987). Adolescent self-concept: Testing the assumption of equivalent structure across gender. American Educational Research Journal, 24, 365385.CrossRefGoogle Scholar
Byrne, B. M., Shavelson, R. J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456466.Google Scholar
Byrne, B. M., & van de Vijver, F. J. R. (2010). Testing for measurement and structural equivalence in large-scale cross-cultural studies: Addressing the issue of nonequivalence. International Journal of Testing, 10, 107132.CrossRefGoogle Scholar
Byrne, B. M., & van de Vijver, F. J. R. (2014). Factorial structure of the Family Values Scale from a multilevel-multicultural perspective. International Journal of Testing, 14, 168192.Google Scholar
Byrne, B. M., & van de Vijver, F. J. R. (2017). The maximum likelihood alignment approach to testing for approximate measurement invariance: A paradigmatic cross-cultural application. Psicothema, 29, 539551.Google Scholar
Campbell, D. T. (1961). The mutual methodological relevance of anthropology and psychology. In Hsu, F. L. (ed.), Psychological Anthropology (pp. 333352). Homewood, IL: Dorsey.Google Scholar
Cheung, M. W-L., Leung, K., & Au, K. (2006). Evaluating multilevel models in cross cultural research: An illustration with social axioms. Journal of Cross-Cultural Psychology, 37, 522541.CrossRefGoogle Scholar
Chow, S-M. (2017). Bayesian estimation and modeling: Editorial to the second special issue on Bayesian data analysis. Psychological Methods, 22, 609615.Google Scholar
Cieciuch, J., Davidov, E., & Schmidt, P. (2019). Alignment optimization: Estimation of the most trustworthy means in cross-cultural studies even in the presence of noninvariance. In Davidov, E., Schmidt, P., Billiet, J., & Meuleman, B. (eds.), Cross-Cultural Analysis: Methods and Applications. New York: Routledge.Google Scholar
Clark, L. A. (1987). Mutual relevance of mainstream and cross-cultural psychology. Journal of Consulting & Clinical Psychology, 55, 461470.Google Scholar
Cole, M., & Scribner, S. (1974). Culture and thought: A psychological introduction. New York: Wiley.Google Scholar
Cooperrider, K., Marghetis, T., & Núñez, R. (2017). Where does the ordered line come from? Evidence from a culture of papua new guinea. Psychological Science, 28, 599608.CrossRefGoogle ScholarPubMed
Davidov, E. (2008). A cross-country and cross-time comparison of human values measurements with the second round of the European Social Survey. Survey Research Methods, 2, 3346.Google Scholar
Davidov, E., Cieciuch, J., Mueleman, B., Schmidt, P., Algesheimer, R., & Hausherr, M. (2015). The comparability of measurements of attitudes towards immigration in the European Social Survey: Exact versus approximate measurement equivalence. Public Opinion Quarterly, 19, 244266.CrossRefGoogle Scholar
Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual Review of Sociology, 40, 5575.Google Scholar
De Bondt, N., & van Petegem, P. (2015). Psychometric evaluation of the Overexcitability Questionnaire-Two applying Bayesian Structural Equation Modeling (BSEM) and multiple-group BSEM-based alignment with approximate measurement invariance. Frontiers in Psychology, 6, 19631980.Google Scholar
Drasgow, F., & Kanfer, R. (1985). Equivalence of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662680.CrossRefGoogle ScholarPubMed
Ekman, P. (1972). Universal and cultural differences in facial expression of emotion. In Cole, J. R. (ed.), Nebraska Symposium on Motivation, 1971 (vol. 19, pp. 207283). Lincoln, NE: Nebraska University Press.Google Scholar
Ellefson, M. R., Ng, F. F.-Y., Wang, Q., & Hughes, C. (2017). Efficiency of executive function: A two-generation cross-cultural comparison of samples from Hong Kong and the United Kingdom. Psychological Science, 28(5), 555566.Google Scholar
Fang, X., Sauter, D. A., & Kleef, G. A. V. (2017). Seeing mixed emotions: The specificity of emotion perception from static and dynamic facial expressions across cultures. Journal of Cross-Cultural Psychology, 49, 130148.CrossRefGoogle ScholarPubMed
Fischer, R., & Poortinga, Y. H. (2018). Addressing methodological challenges in culture-comparative research. Journal of Cross-Cultural Psychology, 49, 691712Google Scholar
Hayes, A. F. (2013). Introduction to Mediation, Moderation, and Conditional Process Analysis: A Regression Based Approach. New York: Guilford Press.Google Scholar
Hofstede, G. H. (1980). Culture’s Consequences: International Differences in Work-Related Values. Beverly Hills, CA: Sage.Google Scholar
Hofstede, G. H. (2001). Culture’s Consequences: Comparing Values, Behaviors, Institutions and Organizations across Nations (2nd ed.). Thousand Oaks, CA: Sage.Google Scholar
Hoijtink, H., & Chow, S-M. (2017). Bayesian hypothesis testing: Editorial to the special issue on Bayesian data analysis. Psychological Methods, 22, 211216.CrossRefGoogle Scholar
Horn, J. L., & McArdle, J. J. (1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research, 18, 117144.Google Scholar
International Test Commission (2017). ITC guidelines for translating and adapting tests (2nd ed.). Retrieved from www.InTestCom.org.Google Scholar
Ion, A., Iliescu, D., Aldhafri, S., Rana, N., Ratanadilok, K., Widyanti, A., & Nedelcea, C. (2017). A cross-cultural analysis of personality structure through the lens of the HEXACO model. Journal of Personality Assessment, 99, 2534.Google Scholar
Ispas, D., Iliescu, D., Ilie, A., & Johnson, R.E. (2014). Exploring the cross-cultural generalizability of the Five-Factor Model of Personality: The Romanian NEO PI-R. Journal of Cross-Cultural Psychology, 45, 10741088.CrossRefGoogle Scholar
Jöreskog, K. G. (1971). Simultaneous factor analysis in several populations. Psychometrika, 36, 409426.CrossRefGoogle Scholar
Kashima, Y. (2001). Culture and social cognition: Toward a social psychology of cultural dynamics. In Matsumoto, D. (ed.), Handbook of Culture and Psychology (pp. 325–260). New York: Oxford University Press.Google Scholar
Kim, E. S., Cao, C., Wang, Y., & Nguyen, D.T. (2017). Measurement invariance testing with many groups: A comparison of five approaches. Structural Equation Modeling: A Multidisciplinary Journal, 24, 524544.CrossRefGoogle Scholar
Leung, K., Bond, M. H., de Carrasquel, S. R., Muñoz, C., Hernández, M., Murakami, F.,… Singelis, T. M. (2002). Social axioms: The search for universal dimensions of general beliefs about how the world functions. Journal of Cross-Cultural Psychology, 33, 286302.Google Scholar
Lloyd, E. P., Hugenberg, K., McConnell, A. R., Kunstman, J. W., & Deska, J. C. (2017). Black and White lies: Race-based biases in deception judgments. Psychological Science, 28, 11251136.Google Scholar
Lonner, W., Keith, K., & Matsumoto, D. (2019). Culture and the psychology curriculum: Foundations and resources. In Matsumoto, D. & Hwang, H. C. (eds.), Oxford Handbook of Culture and Psychology. New York: Oxford University Press.Google Scholar
Lun, V. M.-C., & Bond, M. H. (2016). Achieving subjective well-being around the world: The moderating influence of gender, age and national goals for socializing children. Journal of Happiness Studies, 17, 587608.Google Scholar
MacKinnon, D. P., Lockwood, C. M., Hoffman, J. M., West, S. G., & Sheets, V. (2002). A comparison of methods to test mediation and other intervening variable effects. Psychological Methods, 7, 83104.Google Scholar
Marsh, H. W., Abduljabbar, A. S., Parker, P. D., Morin, A. J. S., Abdelfattah, F., & Nagengast, B. (2014). The big fish-little-pond effect in mathematics: A cross-cultural comparison of U.S. and Saudi Arabian TIMSS responses. Journal of Cross-Cultural Psychology, 45, 777804.CrossRefGoogle Scholar
Marsh, H.W., & Hocevar, D. (1985). Application of confirmatory factor analysis to the study of self-concept: First and higher-order factor models and their invariance across groups. Psychological Bulletin, 97, 562582.Google Scholar
Marsh, H. W., Lüdtke, O., Muthén, B., Asparouhov, T., Morin, A. J. S., Trautwein, U., & Nagengast, B. (2010). A new look at the Big Five factor structure through exploratory structural equation modeling. Psychological Assessment, 22, 471491.Google Scholar
Matsumoto, D. (2006). Are cultural differences in emotion regulation mediated by personality traits? Journal of Cross-Cultural Psychology, 37, 421437.Google Scholar
Matsumoto, D., Grissom, R., & Dinnel, D. (2001). Do between-culture differences really mean that people are different? A look at some measures of cultural effect size. Journal of Cross-Cultural Psychology, 32, 478490.CrossRefGoogle Scholar
Matsumoto, D. & Hwang, H. C. (eds.) (2019). Oxford Handbook of Culture and Psychology. New York: Oxford University Press.Google Scholar
Matsumoto, D., & Juang, L. (2016). Culture and Psychology (6th ed.). Belmont, CA: Cengage.Google Scholar
Matsumoto, D., Nezlek, J. B., & Koopmann, B. (2007). Evidence for universality in phenomenological emotion response system coherence. Emotion, 7, 5767.Google Scholar
Matsumoto, D., & Yoo, S. H. (2006). Toward a new generation of cross-cultural research. Perspectives on Psychological Science, 1, 234250.Google Scholar
Mishra, R. C. (2001). Cognition across cultures. In Matsumoto, D. (ed.), Handbook of Culture and Psychology (pp. 119136). New York: Oxford University Press.Google Scholar
Munck, I., Barber, C., & Torney-Purta, J. (2017). Measurement invariance in comparing attitudes towards immigrants among youth across Europe in 1999 and 2009: The alignment method applied to IEA, CIVED, and ICCS. Survey Methods Research, 47, 687728.CrossRefGoogle Scholar
Munet-Vilaró, F., Gregorich, S. E., & Folkman, S. (2002). Factor structure of the Spanish version of the ways of coping questionnaire. Journal of Applied Social Psychology, 32, 19381954.CrossRefGoogle Scholar
Muthén, B., & Asparouhov, T. (2012). Bayesian structural equation modeling: A more flexible representation of substantive theory. Psychological Methods, 17, 313335.Google Scholar
Muthén, B., & Asparouhov, T. (2014). IRT studies of many groups: The alignment method. Frontiers in Psychology, 5, 978985.Google ScholarPubMed
Neff, K. D., Tóth-Király, I., Yarnell, L. M., Arimitsu, K., Castilho, P., Ghorbani, N., & Mantzios, M. (2019). Examining the factor structure of the Self-Compassion scale in 20 diverse samples: Support for use of a total score and six subscale scores. Psychological Assessment, 31, 2745.Google Scholar
Poortinga, Y. H. (1989). Equivalence of cross-cultural data: An overview of basic issues. International Journal of Psychology, 24, 737756.Google Scholar
Poortinga, Y. H., van de Vijver, F. J. R., Joe, R. C., & van de Koppel, J. M. H. (1987). Peeling the onion called culture: A synopsis. In Kagitcibasi, C. (ed.), Growth and Progress in Cross-Cultural Psychology (pp. 2234). Berwyn, PA: Swets North America.Google Scholar
Raudenbush, S. W., Bryk, A. S., Cheong, Y. F., & Congdon, R. (2000). HLM 5. Chicago: Scientific software.Google Scholar
Rivers, W. H. R. (1905). Observations on the senses of the Todas. British Journal of Psychology, 1, 321396.Google Scholar
Robert, C., Lee, W. C., & Chan, K. Y. (2006). An empirical analysis of measurement equivalence with the INDCOL measure of individualism and collectivism: Implications for valid cross-cultural inference. Personnel Psychology, 59, 6599.Google Scholar
Rutkowski, L., & Svetina, D. (2014). Assessing the hypothesis of measurement invariance in the context of large-scale surveys. Educational and Psychological Measurement, 74, 3157.CrossRefGoogle Scholar
Rutkowski, L., & Svetina, D. (2017). Multidimensional invariance in international surveys: Categorical indicators and fit measure performance. Applied Measurement in Psychology, 30, 3951.Google Scholar
Santos, H. C., Varnum, M. E. W., & Grossmann, I. (2017). Global increases in individualism. Psychological Science, 28, 12281239.Google Scholar
Schwartz, S. H. (1992). Universals in the content and structure of values: Theoretical advances and empirical tests in 20 countries. In Zanna, M. (ed.), Advances in Experimental Social Psychology (vol. 25, pp. 165). New York: Academic Press.Google Scholar
Schwartz, S. H. (1994). Beyond individualism/collectivism: New cultural dimensions of values. In Kim, U. E. and Triandis, H. C. et al. (eds.), Individualism and Collectivism: Theory, Method, and Applications (vol. 18, pp. 85119). Newbury Park, CA: Sage.Google Scholar
Schwartz, S. H., Melech, G., Lehmann, A., Harris, M., & Owens, V. (2001). Extending the cross-cultural validity of the theory of basic human values with a different method of measurement. Journal of Cross-Cultural Psychology, 32, 519542.Google Scholar
Segall, M. H., Campbell, D. T., & Herskovits, J. (1963). Cultural differences in the perception of geometric illusions. Science, 193, 769771.Google Scholar
Singelis, T. M., Bond, M., Sharkey, W. F., & Lai, C. S. Y. (1999). Unpackaging culture’s influence on self-esteem and embarrassability. Journal of Cross-Cultural Psychology, 30, 315341.Google Scholar
Sörbom, D. (1974). A general method for studying differences in factor means and factor structures between groups. British Journal of Mathematical and Statistical Psychology, 27, 229239.Google Scholar
Svetina, D., & Rutkowski, L. (2017). Multidimensional measurement invariance in an international context: Fit measure performance with many groups. Journal of Cross-Cultural Psychology, 48, 9911008.Google Scholar
Tanaka-Matsumi, J. (2019). Abnormal psychology and culture. In Matsumoto, D. & Hwang, H. C. (eds.), Oxford Handbook of Culture and Psychology (pp. 431467). New York: Oxford University Press.CrossRefGoogle Scholar
Thalmayer, A. G., & Saucier, G. (2014). The Questionnaire Big Six in 26 nations: Developing cross-culturally applicable Big Six, Big Five, and Big Two inventories. European Journal of Personality, 28, 482496.Google Scholar
Trafimow, D., Triandis, H. C., & Goto, S. G. (1991). Some tests of the distinction between the private self and the collective self. Journal of Personality and Social Psychology, 60, 649655.Google Scholar
Van de Schoot, R., Kluytmans, A., Tummers, L., Lugtig, P., Hox, J., & Muthén, B. (2013). Facing off with Scylla and Charybdis: A comparison of scalar, partial, and the novel possibility of approximate measurement invariance. Frontiers in Psychology, 4, 700714.Google Scholar
Van de Schoot, R., Sijbrandij, M., Depaoli, S., Winter, S. D., Olff, M., & van Loey, N. E. (2018). Bayesian PTSD-trajectory analysis with informed priors based on a systematic literature search and expert elicitation. Multivariate Behavioral Research, 53, 267291.Google Scholar
Van de Schoot, R., Winter, S.D., Ryan, O., & Zondervan-Zwijnenburg, M. (2017). A systematic review of Bayesian articles in psychology: The last 25 years. Psychological Methods, 22, 217239.Google Scholar
Van de Vijver, F. J. R. (2001). Evolution of research methods. In Matsumoto, D. (ed.), Handbook of Culture and Psychology (pp. 7797). New York: Oxford University Press.Google Scholar
Van de Vijver, F. J. R., & Leung, K. (1997). Methods and Data Analysis for Cross-Cultural Research. Newbury Park, CA: Sage.Google Scholar
Van de Vijver, F. J. R., & Poortinga, Y. H. (2004). Structural equivalence in multilevel research. Journal of Cross-Cultural Psychology, 35, 141156.Google Scholar
Van Erp, S., Mulder, J., & Oberski, D. L. (2017). Prior sensitivity analysis in default Bayesian structural equation modeling, Psychological Methods, 23, 363388.Google Scholar
Whiting, B. B., & Whiting, J. W. (1975). Children of Six Cultures: A Psycho-Cultural Analysis. Oxford: Harvard University Press.CrossRefGoogle Scholar
Woehr, D. J., Arciniega, L. M., & Lim, D. H. (2007). Examining work ethic across populations: A comparison of the Multidimensional Work Ethic Profile across three diverse cultures. Educational and Psychological Measurement, 67, 154168.Google Scholar
Yamagishi, T. (1986). The provisioin of a sanctioning system as a public good. Journal of Personality and Social Psychology, 51(1), 110116.Google Scholar
Yamagishi, T. (1988a). Exit from the group as an individualistic solution to the free rider problem in the United States and Japan. Journal of Experimental Social Psychology, 24, 530542.Google Scholar
Yamagishi, T. (1988b). The provision of a sanctioning system in the United States and Japan. Social Psychology Quarterly, 51, 265271.Google Scholar
Yoo, J., Miyamoto, Y., Rigotti, A., & Ryff, C. D. (2017). Linking positive affect to blood lipids: A cultural perspective. Psychological Science, 28(10), 14681477.Google Scholar
Zercher, F., Schmidt, P., Cieciuch, J., & Davidov, E. (2015). The comparability of the universalism value over time and across countries in the European Social Survey: Exact versus approximate measurement invariance. Frontiers of Psychology, 6, 733748.Google Scholar

Save book to Kindle

To save this book to your Kindle, first ensure [email protected] is added to your Approved Personal Document E-mail List under your Personal Document Settings on the Manage Your Content and Devices page of your Amazon account. Then enter the ‘name’ part of your Kindle email address below. Find out more about saving to your Kindle.

Note you can select to save to either the @free.kindle.com or @kindle.com variations. ‘@free.kindle.com’ emails are free but can only be saved to your device when it is connected to wi-fi. ‘@kindle.com’ emails can be delivered even when you are not connected to wi-fi, but note that service fees apply.

Find out more about the Kindle Personal Document Service.

Available formats
×

Save book to Dropbox

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Dropbox.

Available formats
×

Save book to Google Drive

To save content items to your account, please confirm that you agree to abide by our usage policies. If this is the first time you use this feature, you will be asked to authorise Cambridge Core to connect with your account. Find out more about saving content to Google Drive.

Available formats
×