Hostname: page-component-745bb68f8f-cphqk Total loading time: 0 Render date: 2025-01-08T08:07:20.907Z Has data issue: false hasContentIssue false
  • English
  • Français

Bayesian Hierarchical Multivariate Formulation with Factor Analysis for Nested Ordinal Data

Published online by Cambridge University Press:  01 January 2025

Terrance D. Savitsky  and
Daniel F. McCaffrey
Terrance D. Savitsky*
Affiliation:
RAND Corporation
Daniel F. McCaffrey
Affiliation:
RAND Corporation
*
Requests for reprints should be sent to Terrance D. Savitsky, RAND Corporation, 1776 Main Street, Box 2138, Santa Monica, CA 90401-2138, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

This article devises a Bayesian multivariate formulation for analysis of ordinal data that records teacher classroom performance along multiple dimensions to assess aspects characterizing good instruction. Study designs for scoring teachers seek to measure instructional performance over multiple classroom measurement event sessions at varied occasions using disjoint intervals within each session and employment of multiple ratings on intervals scored by different raters; a design which instantiates a nesting structure with each level contributing a source of variation in recorded scores. We generally possess little a priori knowledge of the existence or form of a sparse generating structure for the multivariate dimensions at any level in the nesting that would permit collapsing over dimensions as is done under univariate modeling. Our approach composes a Bayesian data augmentation scheme that introduces a latent continuous multivariate response linked to the observed ordinal scores with the latent response mean constructed as an additive multivariate decomposition of nested level means that permits the extraction of de-noised continuous teacher-level scores and the associated correlation matrix. A semi-parametric extension facilitates inference for teacher-level dependence among the dimensions of classroom performance under multi-modality induced by sub-groupings of rater perspectives. We next replace an inverse Wishart prior specified for the teacher covariance matrix over dimensions of instruction with a factor analytic structure to allow the simultaneous assessment of an underlying sparse generating structure. Our formulation for Bayesian factor analysis employs parameter expansion with an accompanying post-processing sign re-labeling step of factor loadings that together reduce posterior correlations among sampled parameters to improve parameter mixing in our Markov chain Monte Carlo (MCMC) scheme. We evaluate the performance of our formulation on simulated data and make an application for the assessment of the teacher covariance structure with a dataset derived from a study of middle and high school algebra teachers.

Keywords

Bayesian hierarchical modelsordinal datalatent modelsMarkov chain Monte Carlodata augmentationDirichlet process
Type
Original Paper
Information
Psychometrika , Volume 79 , Issue 2 , April 2014 , pp. 275 - 302
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Aaronson, D., Barrow, L., Sander, W. (2007). Teachers and student achievement in the Chicago public high schools. Journal of Labor Economics, 25(1), 95135CrossRefGoogle Scholar
Albert, J.H., Chib, S. (1993). Bayesian analysis of binary and polychotomous response data. Journal of the American Statistical Association, 88, 669–679CrossRefGoogle Scholar
Blackwell, D., MacQueen, J.B. (1973). Ferguson distributions via Pólya urn schemes. The Annals of Statistics, 1, 353–355CrossRefGoogle Scholar
Casabianca, J., McCaffrey, D.F., Gitomer, D., Bell, C., & Hamre, B.K. (2012). Effect of observation mode on measures of secondary mathematics teaching (Working Paper, pp. 1–51). Google Scholar
Congdon, P. (2005). Bayesian models for categorical data, New York: WileyCrossRefGoogle Scholar
Cowles, M.K., Carlin, B.P., Connett, J.E. (1996). Bayesian Tobit modeling of longitudinal ordinal clinical trial compliance data with nonignorable missingness. Journal of the American Statistical Association, 91, 86–98CrossRefGoogle Scholar
Curtis, S.M., & Erosheva, E.A. (2011). Specification of rotational constraints in Bayesian confirmatory factor analysis (Working Paper). Google Scholar
Escobar, M.D., West, M. (1995). Bayesian density estimation and inference using mixtures. Journal of the American Statistical Association, 90, 577–588CrossRefGoogle Scholar
Geisser, S., Eddy, W.F. (1979). A predictive approach to model selection. Journal of the American Statistical Association, 74, 153–160 Corr: Vol. 75 p. 765CrossRefGoogle Scholar
Gelfand, A.E., Dey, D.K. (1994). Bayesian model choice: asymptotics and exact calculations. Journal of the Royal Statistical Society. Series B. Methodological, 56, 501–514CrossRefGoogle Scholar
Gelman, A. (2006). Comment on “A comparison of Bayesian and likehood-based methods for fitting mutilevel models (pp. P473–P550)”. Bayesian Analysis, 1(3), 515534Google Scholar
Geweke, J., Zhou, G. (1996). Measuring the pricing error of the arbitrage pricing theory. The Review of Financial Studies, 9, 557–587 STMA Vol. 38, p. 2589CrossRefGoogle Scholar
Ghahramani, Z., Griffiths, T.L., Sollich, P. (2007). Bayesian nonparametric latent feature models. In Bernardo, J.M., Bayarri, M.J., Berger, J.O., Dawid, A.P., Heckerman, D., Smith, A.F.M., West, M. (Eds.), Proceedings of the 8th Valencia international meeting held in Benidorm, London: Oxford University Press 201226Google Scholar
Ghosh, J., Dunson, D.B. (2009). Default prior distributions and efficient posterior computation in Bayesian factor analysis. Journal of Computational and Graphical Statistics, 18(2), 306320CrossRefGoogle ScholarPubMed
Goldhaber, D., & Hansen, M. (2010). Is it just a bad class? Assessing the stability of measured teacher performance. Google Scholar
Hamre, B.K., Hafen, C., Pianta, R.C., Bell, C., Gitomer, D.H., & McCaffrey, D.F. (2012a). Teaching through interactions in middle and high school classrooms: validating the factor structure of the classrooom assessment scoring system—secondary (Working Paper). Curry School of Education, University of Virginia. Google Scholar
Hamre, B.K., Pianta, R.C., Burchinal, M., Field, S., LoCasale-Crouch, J., Downer, J.T., Howes, C., LoParo, K., Scott-Little, C. (2012). A course on effective teacher-child interactions: effects on teacher beliefs, knowledge, and observed practice. American Educational Research Journal, 49, 88–123CrossRefGoogle Scholar
Hanushek, E.A. (1992). The trade-off between child quantity and quality. Journal of Political Economy, 100(1), 84117CrossRefGoogle Scholar
Hausman, J.A., Wise, D.A. (1978). A conditional probit model for qualitative choice: discrete decisions recognizing interdependence and heterogeneous preferences. Econometrica, 46, 403–426CrossRefGoogle Scholar
Hogan, J.W., Tchernis, R. (2004). Bayesian factor analysis for spatially correlated data, with application to summarizing area-level material deprivation from census data. Journal of the American Statistical Association, 99(466), 314324CrossRefGoogle Scholar
Holzinger, K.J., Swineford, F. (1937). The bi-factor method. Psychometrika, 2, 41–54CrossRefGoogle Scholar
Hung, L.-F., Wang, W.-C. (2011). The generalized multilevel facets model for longitudinal data. Journal of Educational and Behavioral Statistics, 37(2), 231255CrossRefGoogle Scholar
Ishwaran, H. (2000). Univariate and multirater ordinal cumulative link regression with covariate specific cutpoints. The Canadian Journal of Statistics/La Revue Canadienne de Statistique, 28(4), 715730CrossRefGoogle Scholar
Johnson, V.E., Albert, J.H. (1999). Ordinal data modeling, Berlin: SpringerCrossRefGoogle Scholar
Jones, G.L., Haran, M., Caffo, B.S., Neath, R. (2006). Fixed-width output analysis for Markov chain Monte Carlo. Journal of the American Statistical Association, 101(476), 15371547CrossRefGoogle Scholar
Kottas, A., Müller, P., Quintana, F. (2005). Nonparametric Bayesian modeling for multivariate ordinal data. Journal of Computational and Graphical Statistics, 14(3), 610625CrossRefGoogle Scholar
Muirhead, R.J. (1982). Aspects of multivariate statistical theory, New York: WileyCrossRefGoogle Scholar
Neal, R.M. (2000). Markov chain sampling methods for Dirichlet process mixture models. Journal of Computational and Graphical Statistics, 9(2), 249265CrossRefGoogle Scholar
Qiu, Z., Song, P.X.K., Tan, M. (2002). Bayesian hierarchical models for multilevel repeated ordinal data using winbugs. Journal of Biopharmaceutical Statistics, 12(2), 121135CrossRefGoogle ScholarPubMed
Rivkin, S.G., Hanushek, E.A., Kain, J.F. (2005). Teachers, schools and academic achievement. Econometrica, 73(2), 417458CrossRefGoogle Scholar
Rogers, S., Girolami, M., Polajnar, T. (2009). Semi-parametric analysis of multi-rater data. Statistics and Computing, 20(3), 317334CrossRefGoogle Scholar
Sethuraman, J. (1994). A constructive definition of Dirichlet priors. Statistica Sinica, 4(2), 639650Google Scholar
Spiegelhalter, D.J., Best, N.G., Carlin, B.P., van der Linde, A. (2002). Bayesian measures of model complexity and fit. Journal of the Royal Statistical Society. Series B, 64(4), 583616CrossRefGoogle Scholar
Stephens, M. (2000). Dealing with label switching in mixture models. Journal of the Royal Statistical Society. Series B. Statistical Methodology, 62(4), 795809CrossRefGoogle Scholar
U.S. DOE (2011). Also available as http://www2.ed.gov/programs/racetothetop/executive-summary.pdf. Google Scholar