Hostname: page-component-599cfd5f84-2stsh Total loading time: 0 Render date: 2025-01-07T05:25:56.572Z Has data issue: false hasContentIssue false
  • English
  • Français

Raw Data Maximum Likelihood Estimation for Common Principal Component Models: A State Space Approach

Published online by Cambridge University Press:  01 January 2025

Fei Gu  and
Hao Wu
Fei Gu*
Affiliation:
McGill University
Hao Wu
Affiliation:
Boston College
*
Correspondence should bemade to Fei Gu, McGill University,Montreal, Quebec Canada. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The specifications of state space model for some principal component-related models are described, including the independent-group common principal component (CPC) model, the dependent-group CPC model, and principal component-based multivariate analysis of variance. Some derivations are provided to show the equivalence of the state space approach and the existing Wishart-likelihood approach. For each model, a numeric example is used to illustrate the state space approach. In addition, a simulation study is conducted to evaluate the standard error estimates under the normality and nonnormality conditions. In order to cope with the nonnormality conditions, the robust standard errors are also computed. Finally, other possible applications of the state space approach are discussed at the end.

Keywords

state space modelprincipal component analysiscommon principal component model
Type
Original Paper
Information
Psychometrika , Volume 81 , Issue 3 , September 2016 , pp. 751 - 773
Copyright
Copyright © 2016 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Anderson, T. W. (2003). An introduction to multivariate statistical analysis, 3New York, USA: WileyGoogle Scholar
Bauer, D. J. (2003). Estimating multilevel linear models as structural equation models. Journal of Educational and Behavioral Statistics, 28, 135–167.CrossRefGoogle Scholar
Bechger, T. M., Blanca, M. J., & Maris, G. (2014). The analysis of multivariate group differences using common principal components. Structural Equation Modeling, 21, 577–587.CrossRefGoogle Scholar
Browne, M. W., & Shapiro, A. (1988). Robustness of normal theory methods in the analysis of linear latent variate models. British Journal of Mathematical and Statistical Psychology, 41, 193–208.CrossRefGoogle Scholar
Caines, P. E., & Rissanen, J. (1974). Maximum likelihood estimation of parameters in multivariate Gaussian stochastic processes. IEEE Transactions on Information Theory, 20, 102–104.CrossRefGoogle Scholar
Curran, P. J. (2003). Have multilevel models been structural equation models all along?. Multivariate Behavioral Research, 38, 529–569.CrossRefGoogle ScholarPubMed
Dolan, C. V. (1996). Principal component analysis using LISREL 8. Structural Equation Modeling, 2, 307–322.CrossRefGoogle Scholar
Dolan, C. V., Bechger, T. M., & Molenaar, P. C. M. (1999).Using structural equation modeling to fit models incorporating principal components. Structural Equation Modeling, 6, 233–261.CrossRefGoogle Scholar
Fleishman, A. I. (1978). A method for simulating non-normal distributions. Psychometrika, 43, 521–532.CrossRefGoogle Scholar
Flury, B. N. (1984). Common principal components in k groups. Journal of the American Statistical Association, 79, 892–898.Google Scholar
Flury, B. N. (1986). Asymptotic theory for common principal component analysis. The Annals of Statistics, 14, 418–430.CrossRefGoogle Scholar
Flury, B. N. (1987). Two generalizations of the common principal component model. Biometrika, 74, 59–69.CrossRefGoogle Scholar
Flury, B. N. (1988). Common principal components and related multivariate models, New York, USA: WileyGoogle Scholar
Flury, B. N., & Constantine, G. (1985). Algorithm AS 211: The F-G algorithm. Journal of the Royal Statistical Society Series C (Applied Statistics), 34, 177–183.Google Scholar
Flury, B. N., & Gautschi, W. (1986). An algorithm for simultaneous orthogonal transformation of several positive definite symmetric matrices to nearly diagonal form. SIAM Journal on Scientific and Statistical Computing, 7, 169–184.CrossRefGoogle Scholar
Flury, B. D., Neuenschwander, B. E., & Zahar, R. V. M. (1994). Simultaneous diagonalization algorithms with applications in multivariate statistics. Approximation and computation: A Festschrift in honor of Walter Gautschi, Boston: Birkhäuser 179–205.CrossRefGoogle Scholar
Flury, B. D., Neuenschwander, B. E., & Krzanowski, W. J. (1995). Principal component models for patterned covariance matrices with application to canonical correlation analysis of several sets of variables. Recent advances in descriptive multivariate analysis, New York, USA: Oxford University Press 90–112.CrossRefGoogle Scholar
Goria, M. N., & Flury, B. D. (1996). Common canonical variates in k\documentclass[12pt]{minimal} \usepackage{amsmath} \usepackage{wasysym} \usepackage{amsfonts} \usepackage{amssymb} \usepackage{amsbsy} \usepackage{mathrsfs} \usepackage{upgreek} \setlength{\oddsidemargin}{-69pt} \begin{document}$$k$$\end{document} independent groups. Journal of the American Statistical Association, 91, 1735–1742.Google Scholar
Gu, F., Preacher, K. J., Wu, W., & Yung, Y. -F. (2014).A computationally efficient state space approach to estimating multilevel regression models and multilevel confirmatory factor models. Multivariate Behavioral Research, 49, 119–129.CrossRefGoogle ScholarPubMed
Hotelling, H. (1933). Analysis of a complex of statistical variables into principal components. Journal of Educational Psychology, 24, 417–441.CrossRefGoogle Scholar
Huber, P. J. (1967). The behavior of maximum likelihood estimates under nonstandard conditions. Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, 1, 221–233.Google Scholar
Jones, R. H. (1993). Longitudinal data with serial correlation: A state-space approach, London: Chapman & HallCrossRefGoogle Scholar
Kalman, R. E. (1960). A new approach to linear filtering and prediction problems. Transactions of the ASME-Journal of Basic Engineering (Series D), 82, 35–45.CrossRefGoogle Scholar
Kiers, H. A. L., & ten Berg, J. M. F. (1989).Alternating least squares algorithms for simultaneous components analysis with equal component weight matrices in two or more populations. Psychometrika, 54, 467–473.CrossRefGoogle Scholar
Kiers, H. A. L., & ten Berg, J. M. F. (1994).Hierarchical relations between methods for simultaneous component analysis and a technique for rotation to a simple simultaneous structure. British Journal of Mathematical and Statistical Psychology, 47, 109–126.CrossRefGoogle Scholar
Neale, M. C., Hunter, M. D., Pritikin, J. N., Zahery, M., Brick, T. R., Kirkpatrick, R. M., & Boker, S. M. (2015). OpenMx 2.0: Extended structural equation and statistical modeling. Psychometrika. doi:10.1007/s11336-014-9435-8.CrossRefGoogle Scholar
Neuenschwander, B. E. (1991). Common principal components for dependent random vectors. Unpublished doctoral dissertation. Switzerland: Department of Statistics, University of Berne.Google Scholar
Neuenschwander, B. E., & Flury, B. D. (1995). Common canonical variates. Biometrika, 82, 553–560.CrossRefGoogle Scholar
Neuenschwander, B. E., & Flury, B. D. (2000). Common principal components for dependent random vectors. Journal of Multivariate Analysis, 75, 163–183.CrossRefGoogle Scholar
Nevitt, J., & Hancock, G. R. (2001). Performance of bootstrapping approaches to model test statistics and parameter standard error estimation in structural equation modeling. Structural Equation Modeling, 8, 353–377.CrossRefGoogle Scholar
Pearson, K. (1901). On lines and planes of closest fit to systems of points in space. Philosophical Magazine, 2, 559–572.Google Scholar
R Development Core Team. (2014). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. http://www.R-project.org.Google Scholar
Schweppe, F. (1965). Evaluation of likelihood functions for Gaussian signals. IEEE Transactions on Information Theory, 11, 61–70.CrossRefGoogle Scholar
Timmerman, M. E., & Kiers, H. A. L. (2003). Four simultaneous component models for the analysis of multivariate time series from more than one subject to model intraindividual and interindividual differences. Psychometrika, 68, 105–121.CrossRefGoogle Scholar
Trendafilov, N. T. (2010). Stepwise estimation of common principal components. Computational Statistics and Data Analysis, 54, 3446–3457.CrossRefGoogle Scholar
Yamada, T. (2008). The efficiency of the asymptotic expansion of the distribution of the canonical vector under nonnormality. Journal of The Japan Statistical Society, 38, 451–474.CrossRefGoogle Scholar
White, H. (1980). A heteroscedasticity-consistent covariance matrix estimator and a direct test for heteroscedasticity. Econometrica, 48, 817–838.CrossRefGoogle Scholar