Hostname: page-component-cd9895bd7-lnqnp Total loading time: 0 Render date: 2025-01-05T15:34:35.107Z Has data issue: false hasContentIssue false
  • English
  • Français

Consistency Theory for the General Nonparametric Classification Method

Published online by Cambridge University Press:  01 January 2025

Chia-Yi Chiu  and
Hans-Friedrich Köhn
Chia-Yi Chiu*
Affiliation:
Rutgers, The State University of New Jersey
Hans-Friedrich Köhn
Affiliation:
University of Illinois at Urbana-Champaign
*
Correspondence should be made to Chia-Yi Chiu, Rutgers, The State University of New Jersey, New Brunswick, NJ, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Parametric likelihood estimation is the prevailing method for fitting cognitive diagnosis models—also called diagnostic classification models (DCMs). Nonparametric concepts and methods that do not rely on a parametric statistical model have been proposed for cognitive diagnosis. These methods are particularly useful when sample sizes are small. The general nonparametric classification (GNPC) method for assigning examinees to proficiency classes can accommodate assessment data conforming to any diagnostic classification model that describes the probability of a correct item response as an increasing function of the number of required attributes mastered by an examinee (known as the “monotonicity assumption”). Hence, the GNPC method can be used with any model that can be represented as a general DCM. However, the statistical properties of the estimator of examinees’ proficiency class are currently unknown. In this article, the consistency theory of the GNPC proficiency-class estimator is developed and its statistical consistency is proven.

Keywords

cognitive diagnosisQ-matrixDINA modelDINO modelgeneral DCMG-DINA modelnonparametric classificationgeneral nonparametric classification method
Type
Original Paper
Information
Psychometrika , Volume 84 , Issue 3 , September 2019 , pp. 830 - 845
Copyright
Copyright © 2019 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ayers, E. Nugent, R., & Dean, N. (2008). Skill set profile clustering based on student capability vectors computed from online tutoring data. In de Baker, R. S. J. Barnes, T., & Beck, J. E. (Eds.), Educational data mining 2008: Proceedings of the 1st international conference on educational data mining, Montreal, Quebec, Canada (pp. 210–217). Retrieved from http://www.educationaldatamining.org/EDM2008/uploads/proc/full%20proceedings.pdf.Google Scholar
Chiu, C.-Y. (2008). Cluster analysis for cognitive diagnosis: Theory and applications (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No. 3337778).Google Scholar
Chiu, C.-Y., & Douglas, J. A. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response profiles. Journal of Classification, 30, 225–250.CrossRefGoogle Scholar
Chiu, C.-Y.Douglas, J. A., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.CrossRefGoogle Scholar
Chiu, C.-Y.Sun, Y., & Bian, Y. (2018). Cognitive diagnosis for small educational programs: The general nonparametric classification method. Psychometrika, 83, 355–375.CrossRefGoogle ScholarPubMed
de la Torre, J. (2009). DINA model and parameter estimation: A didactic. Journal of Educational and Behavioral Statistics, 34, 115–130.CrossRefGoogle Scholar
de la Torre, J. The generalized DINA model framework. Psychometrika, (2011). 76, 179–199.CrossRefGoogle Scholar
de la Torre, J., & Douglas, J. A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.CrossRefGoogle Scholar
DiBello, L. V.Roussos, L. A., & Stout, W. F.Rao, C. R.Sinharay, S. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. Handbook of statistics: Volume 26. Psychometrics, 979–1030. Amsterdam: Elsevier.Google Scholar
Haberman, S. J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. Rao, C. R., & Sinharay, S. Handbook of Statistics: Volume 26. Psychometrics, Amsterdam: Elsevier. 1031–1038.Google Scholar
Haertel, E. H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.CrossRefGoogle Scholar
Henson, R. A.Templin, J. L., & Willse, J. T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.CrossRefGoogle Scholar
Hoeffding, W. (1963). Probability inequalities for sums of bounded random variables. Journal of the American Statistical Association, 58 (301), 13–30.CrossRefGoogle Scholar
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
Köhn, H.-F., & Chiu, C.-Y. (2016a). Conditions of completeness of the Q-matrix of tests for cognitive diagnosis. In A. L. van der Ark, D. M. Bolt, W.-C. Wang, J. A. Douglas, & M. Wiberg (Eds.), Quantitative psychology research: The 80th annual meeting of the psychometric society, Beijing, 2015 (pp. 255–264). New York: Springer.Google Scholar
Köhn, H.-F., & Chiu, C.-Y. (2016b). A proof of the duality of the DINA model and the DINO model. Journal of Classification, 33, 171–184.CrossRefGoogle Scholar
Köhn, H.-F., & Chiu, C.-Y. (2017). A procedure for assessing completeness of the Q-matrix of cognitive diagnosis models. Psychometrika, 82, 112–132.CrossRefGoogle Scholar
Leighton, J., & Gierl, M. Cognitive diagnostic assessment for education: Theory and applications, (2007). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
Ma, W., & de la Torre, J. (2019). GDINA: The generalized DINA model framework. R package version 2.3.2. Retrieved from the Comprehensive R Archive Network [CRAN] website https://cran.r-project.org/web/packages/GDINA/.Google Scholar
Macready, G. B., & Dayton, C. M. The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, (1977). 33, 379–416.Google Scholar
Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.CrossRefGoogle Scholar
Nichols, P. D.Chipman, S. E., & Brennan, R. L. (1995). Cognitively diagnostic assessment, Mahwah, NJ: Erlbaum.Google Scholar
Park, Y. S., & Lee, Y.-S. (2011). Diagnostic cluster analysis of mathematics skills. IERI Monograph Series: Issues and Methodologies in Large-Scale Assessments, 4, 75–107.Google Scholar
Robitzsch, A., Kiefer, T., George, A. C., & Uenlue, A. (2018). CDM: Cognitive diagnosis modeling. R package version 7.1-20. Retrieved from the Comprehensive R Archive Network [CRAN] website https://cran.r-project.org/web/packages/CDM/.Google Scholar
Rupp, A. A.Templin, J. L., & Henson, R. A. Diagnostic Measurement. Theory, Methods, and Applications, (2010). New York: Guilford.Google Scholar
Stout, W. (2002). Psychometrics: From practice to theory and back. Psychometrika, 67, 485–518.CrossRefGoogle Scholar
Tatsuoka, K. K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational and Behavioral Statistics, 12, 55–73.CrossRefGoogle Scholar
Tatsuoka, K. K. (2009). Cognitive assessment. An introduction to the rule space method, New York: Routledge/Taylor & Francis.CrossRefGoogle Scholar
Templin, J. L., & Henson, R. A. (2006). Measurement of psychological disorders using cognitive diagnosis models. Psychological Methods, 11, 287–305.CrossRefGoogle ScholarPubMed
von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.Google Scholar
von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.CrossRefGoogle ScholarPubMed
Wang, S., & Douglas, J. (2015). Consistency of nonparametric classification in cognitive diagnosis. Psychometrika, 80, 85–100.CrossRefGoogle ScholarPubMed
Willse, J., Henson, R., & Templin, J. (2007). Using sum scores or IRT in place of cognitive diagnosis models: Can existing or more familiar models do the job? Paper presented at the Annual Meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar
Zheng, Y., & Chiu, C.-Y. (2014). NPCD: Nonparametric methods for cognitive diagnosis. R package version 1.0-5. Retrieved from the Comprehensive R Archive Network [CRAN] website http://CRAN.R-project.org/package=NPCD.Google Scholar
Supplementary material: PDF

Supplementary Information

The online version contains supplementary material available at https://doi.org/10.1007/s11336-024-09978-1.

Download Supplementary Information(PDF)
PDF 1.5 MB