Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-08T09:30:04.472Z Has data issue: false hasContentIssue false
  • English
  • Français

Design and Analysis of Incomplete Multitrait-Multimethod Studies from a Multiplicative Perspective

Published online by Cambridge University Press:  01 January 2025

Guangjian Zhang  and
Michael W. Browne
Guangjian Zhang*
Affiliation:
The University of Notre Dame
Michael W. Browne
Affiliation:
The Ohio State University
*
Requests for reprints should be sent to Guangjian Zhang, The University of Notre Dame, Psychology Department, Haggar Hall, Notre Dame, IN 46556, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The composite direct product (CDP) model is a multiplicative model for multitrait-multimethod (MTMM) designs. It is extended to incomplete MTMM correlation matrices where some trait-method combinations are not available. Rules for omitting trait-method combinations without resulting in an indeterminate model are also suggested. Maximum likelihood estimation and the log absolute correlation procedure are used to fit the model, and are found to yield similar results. The balanced incomplete MTMM design tends to yield more accurate estimates than the randomly missing design.

Keywords

Multitrait-multimethod matricesdirect product modelplanned missingness
Type
Original Paper
Information
Psychometrika , Volume 72 , Issue 3 , September 2007 , pp. 361 - 375
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank Filip Lievens for bringing the problem of incomplete MTMM matrices to our attention and for providing us with his data, Herbert Marsh for providing us with information about MTMM data sets, and Kris Preacher for reading the manuscript and suggesting changes. We thank the Associate Editor and two anonymous reviewers for extensive and constructive suggestions. In particular, we are indebted to a reviewer for suggesting the use of Mx for maximum likelihood estimation and sending us Mx code (that we adapted to suit our formulation of the model).

References

Arthur, W.A. Jr., Woehr, D.J., Maldegen, R. (2000). Convergent and discriminant validity of assessment center dimensions: A conceptual and empirical re-examination of the assessment center construct-related validity paradox. Journal of Management, 26, 813–815.Google Scholar
Bobrow, W., Leonards, J.S. (1997). Development and validation of an assessment center during organizational change. Journal of Social Behavior and Personality, 12, 217–236.Google Scholar
Browne, M.W. (1982). Covariance structures. In Hawkins, D.M. (Ed.), Topics in applied multivariate analysis (pp. 72–141). Cambridge, England: Cambridge University Press.CrossRefGoogle Scholar
Browne, M.W. (1984). The decomposition of multitrait-multimethod matrices. British Journal of Mathematical and Statistical Psychology, 37, 1–21.CrossRefGoogle ScholarPubMed
Browne, M.W. (2000). Cross-validation methods. Journal of Mathematical Psychology, 44, 108–132.CrossRefGoogle ScholarPubMed
Browne, M.W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In Bollen, K.A. & Long, J.S. (Eds.), Testing structural equation models (pp. 136–162). Newbury Park, CA: Sage.Google Scholar
Browne, M.W., & Strydom, H.F. (1997). Non-iterative fitting of the direct product model for multitrait-multimethod matrices. In Berkane, M. (Ed.), Latent variable modeling with applications to causality (pp. 229–245). New York: Springer-Verlag.CrossRefGoogle Scholar
Campbell, D.T., Fiske, D.W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56, 81–105.CrossRefGoogle ScholarPubMed
Campbell, D.T., O’Connell, E.J. (1967). Method factors in multitrait-multimethod matrices: Multiplicative rather than additive. Multivariate Behavioral Research, 2, 409–426.CrossRefGoogle ScholarPubMed
Cudeck, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, 105, 317–327.CrossRefGoogle Scholar
Dean, A., & Voss, D. (1999). Design and analysis of experiments. New York: Springer-Verlag.CrossRefGoogle Scholar
Flamer, S. (1983). Assessment of the multitrait-multimethod matrix validity of Likert scales via confirmatory factor analysis. Multivariate Behavioral Research, 18, 275–308.CrossRefGoogle ScholarPubMed
Harris, M.M., Becker, A.S., Smith, D.E. (1993). Does the assessment center scoring method affect the cross-situational consistency of ratings?. Journal of Applied Psychology, 78, 675–678.CrossRefGoogle Scholar
Jennrich, R.I. (1970). An asymptotic chi-square test for the equality of two correlation matrices. Journal of the American Statistical Association, 65, 904–915.Google Scholar
Jöreskog, K.G. (1974). Analysing psychological data by structural analysis of covariance matrices. In Atkinson, R.C. et al. (Eds.), Contemporary developments in mathematical psychology (pp. 1–16). San Francisco: Freeman.Google Scholar
Jöreskog, K.G., Söbom, D. (1996). LISREL 8: User’s reference guide, Chicago: Scientific Software.Google Scholar
Joyce, L.W., Thayer, P.W., Pond, S.B. (1994). Managerial functions: An alternative to traditional assessment center dimensions. Personnel Psychology, 47, 109–121.CrossRefGoogle Scholar
Kudisch, J.D., Ladd, R.T., Dobbins, G.H. (1997). New evidence on the construct validity of diagnostic assessment centers: The findings may not be so troubling after all. Journal of Social Behavior and Personality, 12, 129–144.Google Scholar
Lance, C.E., Foster, M.R., Gentry, W.A., Thoresen, J.D. (2004). Assessor cognitive processes in an operational assessment center. Journal of Applied Psychology, 89, 22–35.CrossRefGoogle Scholar
Lievens, P., van Keer, E. (2001). The construct validity of a Belgian assessment center: A comparison of different models. Journal of Occupational and Organizational Psychology, 74, 373–378.CrossRefGoogle Scholar
Marsh, H.W. (1990). Confirmatory factor analysis of multitrait-multimethod data: The construct validation of multidimensional self-concept responses. Journal of Personality, 58, 661–692.CrossRefGoogle Scholar
Marsh, H.W., Hocevar, D. (1984). The factorial invariance of students’ evaluations of college teaching. American Educational Research Journal, 21, 341–366.CrossRefGoogle Scholar
Neale, M.C., Miller, M.B. (1997). The use of likelihood-based confidence intervals in genetic models. Behavior Genetics, 27, 113–120.CrossRefGoogle ScholarPubMed
Neale, M.C., Boker, S.M., Xie, G., & Maes, H.H. (2004). Mx: Statistical modeling (6th ed., last revised on April 15, 2004). Richmond, VA: Author. (http://www.vcu.edu/mx)Google Scholar
Saris, W.E., Satorra, A., Coenders, G. (2004). A new approach to evaluating the quality of measurement instruments: The split-ballot MTMM design. Sociological Methodology, 34, 311–347.CrossRefGoogle Scholar
Seber, G.A.F. (1977). Linear regression analysis, New York: Wiley.Google Scholar
Steiger, J.H. (1989). EzPATH: A supplementary module for SYSTAT and SYGRAPH, Evanston, IL: SYSTAT.Google Scholar
Swain, A.J. (1975). Analysis of parametric structures for variance matrices. Unpublished doctoral dissertation, University of Adelaide.Google Scholar
Wothke, W. (1996). Models for multitrait-multimethod analysis. In Marcoulides, G.A. & Schumacker, R.E. (Eds.), Advanced structural equation modeling: Issues and techniques. Hillsdale, NJ: Erlbaum.Google Scholar
Wothke, W., Browne, M.W. (1990). The direct product model for the MTMM matrix parametrized as a second-order factor analysis model. Psychometrika, 55, 255–262.CrossRefGoogle Scholar
Zhang, G., & Browne, M.W. (2004). LINLOGM: User guide. Department of Psychology, The Ohio State University.Google Scholar