Hostname: page-component-5cf477f64f-xc2pj Total loading time: 0 Render date: 2025-03-29T07:20:52.913Z Has data issue: false hasContentIssue false
  • English
  • Français

Consistency of Nonparametric Classification in Cognitive Diagnosis

Published online by Cambridge University Press:  01 January 2025

Shiyu Wang  and
Jeff Douglas
Shiyu Wang
Affiliation:
University of Illinois at Urbana-Champaign
Jeff Douglas*
Affiliation:
University of Illinois at Urbana-Champaign
*
Requests for reprints should be sent to Jeff Douglas, University of Illinois at Urbana-Champaign, Champaign, IL, USA. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Latent class models for cognitive diagnosis have been developed to classify examinees into one of the 2K attribute profiles arising from a K-dimensional vector of binary skill indicators. These models recognize that response patterns tend to deviate from the ideal responses that would arise if skills and items generated item responses through a purely deterministic conjunctive process. An alternative to employing these latent class models is to minimize the distance between observed item response patterns and ideal response patterns, in a nonparametric fashion that utilizes no stochastic terms for these deviations. Theorems are presented that show the consistency of this approach, when the true model is one of several common latent class models for cognitive diagnosis. Consistency of classification is independent of sample size, because no model parameters need to be estimated. Simultaneous consistency for a large group of subjects can also be shown given some conditions on how sample size and test length grow with one another.

Keywords

cognitive diagnosisnonparametric classificationlarge sample theory
Type
Original Paper
Information
Psychometrika , Volume 80 , Issue 1 , March 2015 , pp. 85 - 100
Copyright
Copyright © 2013 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Ayers, E., Nugent, R., & Dean, N. (2008). Skill set profile clustering based on student capability vectors computed from online tutoring data. In Baker, R.S.J.d., Barnes, T., & Beck, J.E. (Eds.), Proceedings of the 1st international conference on education data mining, Montreal (pp. 218–225).Google Scholar
Barnes, T. (2010). Novel derivation and application of skill matrices: the q-matrix method. Handbook on educational data mining. Boca Raton: CRC Press (pp. 159–172.Google Scholar
Chiu, C., & Douglas, J. (2013). A nonparametric approach to cognitive diagnosis by proximity to ideal response patterns. Journal of Classification, 30, 225–250.CrossRefGoogle Scholar
Chiu, C., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnostic theory and applications. Psychometrika, 74, 633–665.CrossRefGoogle Scholar
de la Torre, J. (2011). The generalized DINA model framework. Psychometrika, 76, 179–199.CrossRefGoogle Scholar
Hartz, S., Roussos, L., Henson, R., & Templin, J. (2005). The Fusion Model for skill diagnosis: blending theory with practicality. Unpublished manuscript.Google Scholar
Henson, R., Templin, J., & Willse, J. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.CrossRefGoogle Scholar
Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
Macready, G.B., & Dayton, C.M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 2, 99–120.CrossRefGoogle Scholar
Maris, E. (1999). Estimating multiple classification latent class models. Psychometrika, 64, 187–212.CrossRefGoogle Scholar
Park, Y., & Lee, Y. (2011). Diagnostic cluster analysis of mathematics skills.Google Scholar
Roussos, L., Templin, J., & Henson, R. (2007). Skills diagnosis using IRT-based latent class models. Journal of Educational Measurement, 44(4), 293–311.CrossRefGoogle Scholar
Rupp, A.A., & Templin, J.L. (2007). Unique characteristics of cognitive diagnosis models. The annual meeting of the National Council for Measurement in Education, Chicago, April 2007.Google Scholar
Tatsuoka, K. (1983). Rule-space: an approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 34–38.CrossRefGoogle Scholar
Tatsuoka, K. (1985). A probabilistic model for diagnosing misconceptions in the pattern classification approach. Journal of Educational Statistics, 12, 55–73.CrossRefGoogle Scholar
Tatsuoka, K. (1990). Architecture of knowledge structures and cognitive diagnosis: a statistical pattern classification approach. In Nichols, P., Chipman, S., & Brennan, R. Cognitively diagnostic assessments (pp. 327–359). Hillsdale: Erlbaum.Google Scholar
Tatsuoka, K., & Tatsuoka, M. (1987). Bug distribution and pattern classification. Psychometrika, 52, 193–206.CrossRefGoogle Scholar
von Davier, M. (2005). A general diagnostic model applied to language testing data (ETS Research Report RR-05-16). Princeton: Educational Testing Service.Google Scholar
Willse, J., Henson, R., & Templin, J. (2007). Using sum scores or IRT in place of cognitive diagnosis models: can existing or more familiar models do the job? Paper presented at the Annual Meeting of the National Council on Measurement in Education. Chicago, Illinois.Google Scholar