Hostname: page-component-7b9c58cd5d-wdhn8 Total loading time: 0 Render date: 2025-03-14T05:20:55.212Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Estimation and Testing of Structural Equation Models

Published online by Cambridge University Press:  01 January 2025

Richard Scheines ,
Herbert Hoijtink  and
Anne Boomsma
Richard Scheines*
Affiliation:
Department of Philosophy, Carnegie Mellon University
Herbert Hoijtink
Affiliation:
Department of Methodology and Statistics, University of Utrecht, The Netherlands
Anne Boomsma
Affiliation:
Department of Statistics, Measurement Theory, and Information Technology, University of Groningen, The Netherlands
*
Requests for reprints should be sent to Richard Scheines at the Department of Philosophy, Carnegie Mellon University, Pittsburgh, PA, 15213. Email: [email protected].
Get access Rights & Permissions [Opens in a new window]

Abstract

The Gibbs sampler can be used to obtain samples of arbitrary size from the posterior distribution over the parameters of a structural equation model (SEM) given covariance data and a prior distribution over the parameters. Point estimates, standard deviations and interval estimates for the parameters can be computed from these samples. If the prior distribution over the parameters is uninformative, the posterior is proportional to the likelihood, and asymptotically the inferences based on the Gibbs sample are the same as those based on the maximum likelihood solution, for example, output from LISREL or EQS. In small samples, however, the likelihood surface is not Gaussian and in some cases contains local maxima. Nevertheless, the Gibbs sample comes from the correct posterior distribution over the parameters regardless of the sample size and the shape of the likelihood surface. With an informative prior distribution over the parameters, the posterior can be used to make inferences about the parameters underidentified models, as we illustrate on a simple errors-in-variables model.

Keywords

Bayesian inferenceGibbs samplerposterior predictive p-valuesstructural equation models
Type
Original Paper
Information
Psychometrika , Volume 64 , Issue 1 , March 1999 , pp. 37 - 52
Copyright
Copyright © 1999 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

We thank David Spiegelhalter for suggesting applying the Gibbs sampler to structural equation models to the first author at a 1994 workshop in Wiesbaden. We thank Ulf Böckenholt, Chris Meek, Marijtje van Duijn, Clark Glymour, Ivo Molenaar, Steve Klepper, Thomas Richardson, Teddy Seidenfeld, and Tom Snijders for helpful discussions, mathematical advice, and critiques of earlier drafts of this paper.

References

Baldwin, B.O. (1986). The effects of structural model misspecification and sample size on the robustness of LISREL maximum likelihood parameter estimates. Unpublished doctoral dissertation, Department of Administrative and Foundational Services, Lousiana State University.Google Scholar
Bearden, W.O., Sharma, S., & Teel, J.E. (1982). Sample size effects on chi-square and other statistics used in evaluating causal models. Journal of Marketing Research, 19, 425–430.CrossRefGoogle Scholar
Bentler, P.M. (1995). EQS: Structural equations program manual, Encino, CA: Multivariate Software.Google Scholar
Bentler, P.M., & Tanaka, J.S. (1983). Problems with EM algorithms for ML factor analysis. Psychometrika, 48, 247–251.CrossRefGoogle Scholar
Bollen, K. (1989). Structural equations with latent variables, New York: Wiley.CrossRefGoogle Scholar
Bollen, K. (1995). An alternative two stage least squares (2SLS) estimator for latent variable equations. Psychometrika, 61, 109–121.CrossRefGoogle Scholar
Boomsma, A. (1982). The robustness of LISREL against small sample sizes in factor analysis models. In Jöreskog, K.G., & Wold, H. (Eds.), Systems under indirect observation: Causality, structure, prediction (pp. 149–173). Amsterdam: North-Holland.Google Scholar
Boomsma, A. (1983). On the robustness of LISREL (maximum likelihood estimation) against small sample size and nonnormality, Amsterdam: Sociometric Research Foundation.Google Scholar
Boomsma, A. (1996). De adequaatheid van covariantiestructuurmodellen: een overzicht van maten en indexen. Kwantitatieve Methoden, 52, 7–52. [The adequacy of structural equation models: An overview of statistics and indices]Google Scholar
Box, G.E.P., & Tiao, G.C. (1973). Bayesian inference in statistical analysis, Reading, MA: Addison-Wesley.Google Scholar
Casella, G., & George, E.I. (1992). Explaining the Gibbs sampler. The American Statistician, 46, 167–174.CrossRefGoogle Scholar
Chib, S., & Greenberg, E. (1995). Understanding the Metropolis-Hastings algorithm. The American Statistician, 49, 327–335.CrossRefGoogle Scholar
Chou, C.-P., Bentler, P.M., & Satorra, A. (1991). Scaled test statistics and robust standard errors for non-normal data in covariance structure analysis: A Monte Carlo study. British Journal of Mathematical and Statistical Psychology, 44, 347–357.CrossRefGoogle ScholarPubMed
Geiger, D., Heckerman, D., & Meek, C. (1996). Asymptotic model selection for directed networks with hidden variables, Seatle, WA: Microsoft Research.Google Scholar
Gelfand, A.E., & Smith, A.M.F. (1990). Sampling based approaches to calculating marginal densities. Journal of the American Statistical Association, 85, 398–409.CrossRefGoogle Scholar
Gelman, A., Carlin, J.B., Stern, H.S., & Rubin, D.B. (1995). Bayesian data analysis, London: Chapman & Hall.CrossRefGoogle Scholar
Gelman, A., Meng, X.-L., & Stern, H.S. (1996). Posterior predictive assessment of model fitness via realized discrepancies (with discussion). Statistica Sinica.Google Scholar
Gelman, A., & Rubin, D.B. (1992). Inference from iterative simulation using multiple sequences. Statistical Science, 7, 457–511.CrossRefGoogle Scholar
Geman, S., & Geman, D. (1984). Stochastic relaxation, Gibbs distributions, and the Bayesian restoration of images. IEEE Transactions on. Pattern Analysis and Machine Intelligence, 6, 721–741.CrossRefGoogle ScholarPubMed
Gilks, W.R., Richardson, S., & Spiegelhalter, D. (1996). Markov chain Monte Carlo in practice (pp. 163–187). London: Chapman & Hall.Google Scholar
Greene, T., & Ernhart, C. (1993). Dentine lead and intelligence prior to school entry: A statistical sensitivity analysis. Journal of Clinical Epidemiology, 46, 323–329.CrossRefGoogle ScholarPubMed
Heckerman, D., & Geiger, D. (1995). Likelihoods and parameter priors for Bayesian networks, Seattle, WA: Microsoft Research.Google Scholar
Hoogland, J.J., & Boomsma, A. (1998). Robustness studies in covariance structure modeling: An overview and a meta-analysis. Sociological Methods & Research, 26, 329–368.CrossRefGoogle Scholar
Hu, L.-T., & Bentler, P.M. (1995). Evaluating model fit. In Hoyle, R.H. (Eds.), Structural equation modeling: Concepts, issues, and applications (pp. 76–99). Thousand Oaks, CA: Sage.Google Scholar
Hu, L.-T., Bentler, P.M., & Kano, Y. (1992). Can test statistics in covariance structure analysis be trusted?. Psychological Bulletin, 112, 351–362.CrossRefGoogle ScholarPubMed
Jensen, F.V. (1996). An introduction to Bayesian networks, New York: Springer Verlag.Google Scholar
Jöreskog, K.G., & Sörbom, D. (1993). LISREL 8: User's reference guide, Chicago, IL: Scientific Software International.Google Scholar
Klepper, S. (1988). Regressor diagnostics for the classical errors-in-variables model. Journal of Econometrics, 37, 225–250.CrossRefGoogle Scholar
Klepper, S., & Leamer, E. (1984). Consistent sets of estimates for regressions with errors in all variables. Econometrica, 52, 163–183.CrossRefGoogle Scholar
MacEachern, S.N., & Berliner, L.M. (1994). Subsampling the Gibbs sampler. The American Statistician, 48, 188–190.CrossRefGoogle Scholar
Madigan, D., & Raftery, A. E. (1991). Model selection and accounting for model uncertainty in graphical models using Occam's window, Washington, DC: University of Washington, Department of Statistics.Google Scholar
Meng, X.L. (1994). Posterior predictive p-values. The Annals of Statistics, 22, 1142–1160.CrossRefGoogle Scholar
Muthén, B. (1984). A general structural equation model with dichotomous, ordered categorical and continuous latent variable indicators. Psychometrika, 49, 115–132.CrossRefGoogle Scholar
Needleman, H., Geiger, S., & Frank, R. (1985). Lead and IQ Scores: A Reanalysis. Science, 227, 701–704.CrossRefGoogle ScholarPubMed
Pearl, J. (1988). Probabilistic reasoning in intelligent systems, San Mateo, CA: Morgan and Kaufman.Google Scholar
Press, S.J. (1989). Bayesian statistics: Principles, models, and applications, New York: Wiley.Google Scholar
Press, W.H., Teukolsky, S.A., Vetterling, W.T., & Flannery, B.P. (1992). Numerical recipes in Fortran, Cambridge: Cambridge University Press.Google Scholar
Raftery, A.E. (1993). Bayesian model selection in structural equation models. In Bollen, K.A., & Long, J.S. (Eds.), Testing structural equation models (pp. 163–180). Newbury Park, CA: Sage.Google Scholar
Raftery, A. E. (1994). Bayesian model selection in social research, Washington, DC: University of Washington, Center for Studies in Demography and Ecology.Google Scholar
Raftery, A. E. (1996). Hypothesis testing and model selection. In Gilks, W.R., Richardson, S., & Spiegelhalter, D. (Eds.), Markov chain Monte Carlo in practice (pp. 163–187). London: Chapman & Hall.Google Scholar
Rubin, D.B. (1984). Bayesian justifiable and relevant frequency calculations for the applied statistician. The Annals of Statistics, 12, 1151–1172.CrossRefGoogle Scholar
Rubin, D.B., Stern, H.S. (1994). Testing in latent class models using a posterior predictive check distribution. In von Eye, A., & Clogg, C.C. (Eds.), Latent variables analysis: Applications for developmental research (pp. 420–438). Thousand Oaks, CA: Sage.Google Scholar
Scheines, R. (1997, January). Estimating Latent Causal Influence: TETRAD II Model Selection and Bayesian Parameter Estimation. In Madigan, D. (Ed.) Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics.Google Scholar
Scheines, R., Boomsma, A., & Hoijtink, H. (1997). The multimodality of the likelihood function in structural equation models, Pittsburgh, PA: Carnegie Mellon University, Department of Philosophy.Google Scholar
Scheines, R., Spirtes, P., Glymour, C., & Meek, C. (1994). TETRAD II: Tools for causal modeling. User's manual, Hillsdale, NJ: Erlbaum.Google Scholar
Smith, A.F.M., & Roberts, G.O. (1993). Bayesian computation via the Gibbs sampler and related Markov chain Monte Carlo methods. JOurnal of the Royal Statistical Society, Series B, 55, 3–23.CrossRefGoogle Scholar
Spirtes, P., Glymour, C., & Scheines, R. (1993). Causation, prediction, and search, New York: Springer.CrossRefGoogle Scholar
Tanner, M.A. (1993). Tools for statistical inference: Methods for the exploration of posterior distributions and likelihood functions 2nd ed.,, New York: Springer.CrossRefGoogle Scholar
Tierney, L. (1994). Markov chains for exploring posterior distributions (with discussion). Annals of Statistics, 22, 1701–1762.Google Scholar
Wheaton, B., Muthén, B., Alwin, D., & Summers, G. (1977). Assessing reliability and stability in panel models. In Heise, D.R. (Eds.), Sociological Methodology 1977 (pp. 84–136). San Francisco: Jossey-Bass.Google Scholar
Yung, Y.-F., & Bentler, P.M. (1994). Bootstrap-corrected ADF test statistics. British Journal of Mathematical and Statistical Psychology, 47, 63–84.CrossRefGoogle ScholarPubMed
Zeger, S.L., & Karim, M.R. (1991). Generalized linear models with random effect; a Gibbs sampling approach. Journal of the American Statistical Association, 86, 79–86.CrossRefGoogle Scholar