Hostname: page-component-5f745c7db-q8b2h Total loading time: 0 Render date: 2025-01-07T05:02:29.320Z Has data issue: true hasContentIssue false
  • English
  • Français

Consistency of Cluster Analysis for Cognitive Diagnosis: The Reduced Reparameterized Unified Model and the General Diagnostic Model

Published online by Cambridge University Press:  01 January 2025

Chia-Yi Chiu  and
Hans-Friedrich Köhn
Chia-Yi Chiu*
Affiliation:
Rutgers, The State University of New Jersey
Hans-Friedrich Köhn
Affiliation:
University of Illinois at Urbana-Champaign
*
Correspondence should be made to Chia-Yi Chiu, Rutgers, The State University of New Jersey, New Brunswick, NJ, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

The asymptotic classification theory of cognitive diagnosis (ACTCD) provided the theoretical foundation for using clustering methods that do not rely on a parametric statistical model for assigning examinees to proficiency classes. Like general diagnostic classification models, clustering methods can be useful in situations where the true diagnostic classification model (DCM) underlying the data is unknown and possibly misspecified, or the items of a test conform to a mix of multiple DCMs. Clustering methods can also be an option when fitting advanced and complex DCMs encounters computational difficulties. These can range from the use of excessive CPU times to plain computational infeasibility. However, the propositions of the ACTCD have only been proven for the Deterministic Input Noisy Output “AND” gate (DINA) model and the Deterministic Input Noisy Output “OR” gate (DINO) model. For other DCMs, there does not exist a theoretical justification to use clustering for assigning examinees to proficiency classes. But if clustering is to be used legitimately, then the ACTCD must cover a larger number of DCMs than just the DINA model and the DINO model. Thus, the purpose of this article is to prove the theoretical propositions of the ACTCD for two other important DCMs, the Reduced Reparameterized Unified Model and the General Diagnostic Model.

Keywords

cognitive diagnosisdiagnostic classification modelReduced Reparameterized Unified Model (Reduced RUM)general diagnostic model (GDM)general diagnostic classification modelscluster analysisconsistency
Type
Original Paper
Information
Psychometrika , Volume 81 , Issue 3 , September 2016 , pp. 585 - 610
Copyright
Copyright © 2016 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ayers, E., Nugent, R., & Dean, N. (2008). Skill set profile clustering based on student capability vectors computed from online tutoring data. In R. S. J. de Baker, T. Barnes, & J. E. Beck (Eds.), Educational data mining 2008: Proceedings of the 1st International conference on educational data mining, Montreal, QC, Canada (pp. 210–217). Retrieved from http://www.educationaldatamining.org/EDM2008/uploads/proc/full%20proceedings.Google Scholar
Chiu, C.-Y., Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.CrossRefGoogle Scholar
Chiu, C.-Y., & Köhn, H.-F. (2015a). Consistency of cluster analysis for cognitive diagnosis: The DINO model and the DINA model revisited. Applied Psychological Measurement, 39, 465–479.CrossRefGoogle ScholarPubMed
Chiu, C.-Y., & Köhn, H.-F. (2015b). The Reduced RUM as a logit model: Parameterization and constraints. Psychometrika. doi:10.1007/s11336-015-9460-2.CrossRefGoogle Scholar
Chiu, C.-Y., & Ma, W. (2013). ACTCD: Asymptotic classification theory for cognitive diagnosis. R package version 1.0-0. Retrieved from the Comprehensive R Archive Network [CRAN] website http://cran.r-project.org/web/packages/ACTCD/.Google Scholar
de la Torre, J. The generalized DINA model framework. Psychometrika, 2011 76, 179–199.CrossRefGoogle Scholar
DiBello, L. V., Roussos, L. A., Stout, W. F. Rao, C. R., & Sinharay, S. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of statistics: Vol. 26. Psychometrics, Amsterdam: Elsevier 979–1030.Google Scholar
DiBello, L. V., Stout, W. F., Roussos, L. A. Nichols, P. D., Chipman, S. F., & Brennan, R. L. Unified cognitive/psychometric diagnostic assessment likelihood-based classification techniques. Cognitively diagnostic assessment, 1995 Mahwah, NJ: Erlbaum 361–389.Google Scholar
DiBello, L., Stout, W., Roussos, L., Templin, J., Chen, H., Zapata, D., & Hartz, S. Arpeggio documentation and analysis manual, 2010 Chicago, IL: Applied Informative Assessment Research Enterprise (AIARE)-LLCGoogle Scholar
Feng, Y., Habing, B. T., & Huebner, A. Parameter estimation of the Reduced RUM using the EM algorithm. Applied Psychological Measurement, 2014 38, 137–150.CrossRefGoogle Scholar
Haberman, S. J., von Davier, M. Rao, C. R., & Sinharay, S. (2007). Some notes on models for cognitively based skills diagnosis. Handbook of statistics: Vol. 26. Psychometrics, Amsterdam: Elsevier 1031–1038.Google Scholar
Hartigan, J. A. Clustering algorithms, 1975 New York, USA: WileyGoogle Scholar
Hartz, S. M. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3044108)Google Scholar
Hartz, S. M., & Roussos, L. A. (October 2008). The Fusion Model for skill diagnosis: Blending theory with practicality. (Research report No. RR-08-71). Princeton, NJ: Educational Testing Service.CrossRefGoogle Scholar
Hastie, T., Tibshirani, R., & Friedman, J. (2009). The elements of statistical learning, 2New York, USA: SpringerCrossRefGoogle Scholar
Henson, R. A., Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.CrossRefGoogle Scholar
Hubert, L., & Arabie, P. Comparing partitions. (1985). Journal of Classification, 2, 193–218.CrossRefGoogle Scholar
Johnson, S. C. (1967). Hierarchical clustering schemes. Psychometrika, 32, 241–254.CrossRefGoogle ScholarPubMed
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
Leighton, J., & Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and applications, Cambridge, UK: Cambridge University PressCrossRefGoogle Scholar
Lunn, D., Spiegelhalter, D., Thomas, A., & Best, N. The BUGS project: Evolution, critique, and future directions. Statistics in Medicine, 2009 28, 3049–3067.CrossRefGoogle ScholarPubMed
Macready, G. B., & Dayton, C. M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99–120.CrossRefGoogle Scholar
Muthén, L. K., & Muthén, B. O. (1998–2012). MplusUser’s guide (7th ed.). Los Angeles: Muthén & Muthén.Google Scholar
Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2015). CDM: Cognitive diagnosis modeling. R package version 3.1-14. Retrieved from the Comprehensive R Archive Network [CRAN] website http://CRAN.R-project.org/package=CDMGoogle Scholar
Rupp, A. A., Templin, J. L., & Henson, R. A. Diagnostic measurement. Theory, methods, and applications, 2010 New York, USA: GuilfordGoogle Scholar
Steinley, D. (2004). Properties of the Hubert–Arabie Adjusted Rand Index. Psychological Methods, 9, 386–396.CrossRefGoogle ScholarPubMed
Tatsuoka, K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational Statistics, 12, 55–73.CrossRefGoogle Scholar
Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.CrossRefGoogle ScholarPubMed
Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.CrossRefGoogle ScholarPubMed
Vermunt, J. K., & Magidson, J. (2000). Latent GOLD’s users’s guide, Boston: Statistical Innovations IncGoogle Scholar
von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.Google Scholar
von Davier, M. (2011). Equivalency of the DINA model and a constrained general diagnostic model (Research report No. RR-11-37). Princeton, NJ: Educational Testing Service.Google Scholar
von Davier, M. (2006). Multidimensional latent trait modelling (MDLTM) [Software program], Princeton, NJ: Educational Testing ServiceGoogle Scholar
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.CrossRefGoogle ScholarPubMed
von Davier, M. (2014). The DINA model as a constrained general diagnostic model: Two variants of a model equivalency. British Journal of Mathematical and Statistical Psychology, 67, 49–71.CrossRefGoogle Scholar
von Davier, M., Cheng, C., Cheng, C. A. Yan, D., von Davier, A. A., & Lewis, C. (2014). Multistage testing using diagnostic models. Computerized multistage testing, Boca Raton, FL: CRC Press Taylor & Francis 219–227.Google Scholar
Ward, J. H. Hierarchical grouping to optimize an objective function. (1963). Journal of the American Statistical Association, 58, 236–244.CrossRefGoogle Scholar
Willse, J., Henson, R., & Templin, J. (2007). Using sum scores or IRT in place of cognitive diagnosis models: Can existing or more familiar models do the job? Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar