Hostname: page-component-78c5997874-lj6df Total loading time: 0 Render date: 2024-11-05T05:27:29.043Z Has data issue: false hasContentIssue false
  • English
  • Français

Evaluating Sensitivity of Parameters of Interest to Measurement Invariance in Latent Variable Models

Published online by Cambridge University Press:  04 January 2017

Daniel L. Oberski
Daniel L. Oberski*
Affiliation:
Department of Methodology and Statistics, Tilburg University, Room P 1105, PO Box 90153, 5000 LE Tilburg, The Netherlands e-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Latent variable models can only be compared across groups when these groups exhibit measurement equivalence or “invariance,” since otherwise substantive differences may be confounded with measurement differences. This article suggests examining directly whether measurement differences present could confound substantive analyses, by examining the expected parameter change (EPC)-interest. The EPC-interest approximates the change in parameters of interest that can be expected when freeing cross-group invariance restrictions. Monte Carlo simulations suggest that the EPC-interest approximates these changes well. Three empirical applications show that the EPC-interest can help avoid two undesirable situations: first, it can prevent unnecessarily concluding that groups are incomparable, and second, it alerts the user when comparisons of interest may still be invalidated even when the invariance model appears to fit the data. R code and data for the examples discussed in this article are provided in the electronic appendix (http://hdl.handle.net/1902.1/21816).

Type
Research Article
Information
Political Analysis , Volume 22 , Issue 1 , Winter 2014 , pp. 45 - 60
Copyright
Copyright © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Armstrong, D. A. 2011. Stability and change in the Freedom House political rights and civil liberties measures. Journal of Peace Research 48: 653–62.CrossRefGoogle Scholar
Bartholomew, D. J., Knott, M., and Moustaki, I. 2011. Latent variable models and factor analysis: A unified approach. New York: John Wiley & Sons.Google Scholar
Bentler, P. M., and Chou, C. P. 1992. Some new covariance structure model improvement statistics. Sociological Methods & Research 21: 259–82.Google Scholar
Bollen, K. A. 1989. Structural equations with latent variables. New York: John Wiley & Sons.Google Scholar
Byrne, B. M., Shavelson, R. J., and Muthén, Bengt. 1989. Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin 105: 456.Google Scholar
Chen, F. F. 2007. Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling 14: 464–504.CrossRefGoogle Scholar
Cheung, G. W., and Rensvold, R. B. 2002. Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling 9: 233–55.Google Scholar
Chou, C. P., and Bentler, P. M. 1993. Invariant standardized estimated parameter change for model modification in covariance structure analysis. Multivariate Behavioral Research 28: 97–110.CrossRefGoogle ScholarPubMed
Clinton, J., Jackman, S., and Rivers, D. 2004. The statistical analysis of roll call data. American Political Science Review 98: 355–70.Google Scholar
Davidov, E. 2009. Measurement equivalence of nationalism and constructive patriotism in the ISSP: 34 countries in a comparative perspective. Political Analysis 17: 64–82.CrossRefGoogle Scholar
Davidov, E., Meuleman, B., Billiet, J., and Schmidt, P. 2008. Values and support for immigration: A cross-country comparison. European Sociological Review 24: 583–99.Google Scholar
Fan, Xitao, Thompson, Bruce, and Wang, Lin. 1999. Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Structural Equation Modeling: A Multidisciplinary Journal 6: 56–83.Google Scholar
Fan, Weihua, and Hancock, Gregory R. 2006. Impact of post hoc measurement model overspecification on structural parameter integrity. Educational and Psychological Measurement 66: 748–64.Google Scholar
French, B. F., and Finch, W. H. 2006. Confirmatory factor analytic procedures for the determination of measurement invariance. Structural Equation Modeling 13: 378–402.Google Scholar
Hancock, Gregory R. 1999. A sequential Scheffé-type respecification procedure for controlling type I error in exploratory structural equation model modification. Structural Equation Modeling: A Multidisciplinary Journal 6: 158–68.Google Scholar
Hancock, Gregory R., Stapleton, Laura M., and Arnold-Berkovits, Ilona. 2009. The tenuousness of invariance tests within multisample covariance and mean structure models. In structural equation modeling in educational research: Concepts and applications, eds. Teo, T. and Khine, M. S., 137–74. Rotterdam, The Netherlands: Sense Publishers.Google Scholar
Hausman, J. A. 1978. Specification tests in econometrics. Econometrica: Journal of the Econometric Society 46(6): 1251–71.CrossRefGoogle Scholar
Hu, L., and Bentler, P. M. 1998. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification. Psychological Methods 3: 424.CrossRefGoogle Scholar
Imai, K., and Yamamoto, T. 2010. Causal inference with differential measurement error: Nonparametric identification and sensitivity analysis. American Journal of Political Science 54: 543–60.Google Scholar
Jackman, S. 2001. Multidimensional analysis of roll call data via Bayesian simulation: Identification, estimation, inference, and model checking. Political Analysis 9: 227–41.Google Scholar
Jowell, Roger, Roberts, Caroline, Fitzgerald, Rory, and Eva, Gillian. 2007. Measuring attitudes cross-nationally: Lessons from the European Social Survey. Thousand Oaks, CA: SAGE.Google Scholar
Kaplan, D. 1989. Model modification in covariance structure analysis: Application of the expected parameter change statistic. Multivariate Behavioral Research 24: 285–305.CrossRefGoogle ScholarPubMed
King, B. L. 2011. Unbiased measurement of health-related quality-of-life. Unpublished PhD dissertation, University of Amsterdam. http://dare.uva.nl/document/342053 (accessed June 1, 2013).Google Scholar
Kolenikov, S. 2009. Biases of parameter estimates in misspecified structural equation models. Sociological Methodology 41: 119–57.Google Scholar
Kwok, Oi-Man, Luo, Wen, and West, Stephen G. 2010. Using modification indexes to detect turning points in longitudinal data: A Monte Carlo study. Structural Equation Modeling 17: 216–40.CrossRefGoogle Scholar
Lord, F. M., and Novick, M. R. 1968. Statistical theories of mental scores. Reading, MA: Addison-Wesley.Google Scholar
MacCallum, R. C., Roznowski, M., and Necowitz, L. B. 1992. Model modifications in covariance structure analysis: The problem of capitalization on chance. Psychological Bulletin; Psychological Bulletin 111: 490.Google Scholar
Magnus, J. R., and Vasnev, A. L. 2007. Local sensitivity and diagnostic tests. Econometrics Journal 10: 166–92.Google Scholar
Meredith, W. 1993. Measurement invariance, factor analysis and factorial invariance. Psychometrika 58: 525–43.CrossRefGoogle Scholar
Meuleman, B. 2012. When are item intercept differences substantively relevant in measurement invariance testing? In Methods, theories, and empirical applications in the social sciences: Festschrift for Peter Schmidt, eds. Salzborn, S., Davidov, E., and Reinecke, J., 97–104. Wiesbaden, Germany: Springer-Verlag.Google Scholar
Millsap, R. E. 1997. Invariance in measurement and prediction: Their relationship in the single-factor case. Psychological Methods 2: 248.Google Scholar
Millsap, R. E. 2007. Invariance in measurement and prediction revisited. Psychometrika 72: 461–73.Google Scholar
Millsap, R. E., and Everson, H. T. 1993. Methodology review: Statistical approaches for assessing measurement bias. Applied Psychological Measurement 17: 297–334.Google Scholar
Millsap, R. E., and Kwok, O. M. 2004. Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods 9: 93.Google Scholar
Neudecker, H., and Satorra, A. 1991. Linear structural relations: Gradient and Hessian of the fitting function. Statistics and Probability Letters 11: 57–61.Google Scholar
Oberski, D. L. 2012. Comparability of survey measurements. In Handbook of survey methodology for the social sciences, ed. Gideon, Lior, 477–98. New York: Springer-Verlag.Google Scholar
R Core Team. 2012. R: A language and environment for statistical computing . Vienna, Austria: R Foundation for Statistical Computing. http://www.R-project.org/ (accessed June 1, 2013).Google Scholar
Rosseel, Y. 2012. lavaan: An R package for structural equation modeling. Journal of Statistical Software 48: 1–36.CrossRefGoogle Scholar
Saris, W. E., Satorra, A., and Sörbom, D. 1987. The detection and correction of specification errors in structural equation models. Sociological Methodology 17: 105–29.CrossRefGoogle Scholar
Saris, W. E., Satorra, A., and Van der Veld, W. M. 2009. Testing structural equation models or detection of misspecifications? Structural Equation Modeling 16: 561–82.Google Scholar
Satorra, A. 1989. Alternative test criteria in covariance structure analysis: A unified approach. Psychometrika 54: 131–51.CrossRefGoogle Scholar
Schmitt, N., and Kuljanin, G. 2008. Measurement invariance: Review of practice and implications. Human Resource Management Review 18: 210–22.Google Scholar
Schwartz, S. H., and Bilsky, W. 1987. Toward a universal psychological structure of human values. Journal of Personality and Social Psychology 53: 550.Google Scholar
Schwartz, S. H., and Rubel, T. 2005. Sex differences in value priorities: Cross-cultural and multimethod studies. Journal of Personality and Social Psychology 89: 1010–28.Google Scholar
Steenkamp, J. B. E. M., and Baumgartner, H. 1998. Assessing measurement invariance in cross-national consumer research. Journal of Consumer Research 25: 78–107.Google Scholar
Treier, S., and Jackman, S. 2008. Democracy as a latent variable. American Journal of Political Science 52: 201–17.Google Scholar
Vandenberg, R. J., and Lance, C. E. 2000. A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods 3: 4–70.CrossRefGoogle Scholar
Whittaker, T. A. 2012. Using the modification index and standardized expected parameter change for model modification. Journal of Experimental Education 80: 26–44.Google Scholar
Yoon, M., and Millsap, R. E. 2007. Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling 14: 435–63.Google Scholar
Yuan, K. H., Marshall, L. L., and Bentler, P. M. 2003. Assessing the effect of model misspecifications on parameter estimates in structural equation models. Sociological Methodology 33: 241–65.Google Scholar
Supplementary material: PDF

Oberski supplementary material

Appendix

Download Oberski supplementary material(PDF)
PDF 117.2 KB