Hostname: page-component-cd9895bd7-p9bg8 Total loading time: 0 Render date: 2025-01-05T14:57:16.142Z Has data issue: false hasContentIssue false
  • English
  • Français

Conditional Covariance Structure of Generalized Compensatory Multidimensional Items

Published online by Cambridge University Press:  01 January 2025

Jinming Zhang  and
William Stout
Jinming Zhang*
Affiliation:
Educational Testing Service
William Stout
Affiliation:
University of Illinois at Urbana-Champaign
*
Requests for reprints should be sent to Jinming Zhang, Educational Testing Service, MS 02-T, Rosedale Road, Princeton NJ 08541. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Some nonparametric dimensionality assessment procedures, such as DIMTEST and DETECT, use nonparametric estimates of item pair conditional covariances given an appropriately chosen subtest score as their basic building blocks. Such conditional covariances given some subtest score can be regarded as an approximation to the conditional covariances given an appropriately chosen unidimensional latent composite, where the composite is oriented in the multidimensional test space direction in which the subtest score measures best. In this paper, the structure and properties of such item pair conditional covariances given a unidimensional latent composite are thoroughly investigated, assuming a semiparametric IRT modeling framework called a generalized compensatory model. It is shown that such conditional covariances are highly informative about the multidimensionality structure of a test. The theory developed here is very useful in establishing properties of dimensionality assessment procedures, current and yet to be developed, that are based upon estimating such conditional covariances.

In particular, the new theory is used to justify the DIMTEST procedure. Because of the importance of conditional covariance estimation, a new bias reducing approach is presented. A byproduct of likely independent importance beyond the study of conditional covariances is a rigorous score information based definition of an item's and a score's direction of best measurement in the multidimensional test space.

Keywords

item response theorydimensionalitymultidimensionalitygeneralized compensatory modelconditional covarianceinformationmultidimensional critical ratio functiontest spaceDETECTDIMTEST
Type
Original Paper
Information
Psychometrika , Volume 64 , Issue 2 , June 1999 , pp. 129 - 152
Copyright
Copyright © 1999 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This paper is based on a chapter of the first author's doctoral dissertation, written at the University of Illinois and supervised by the second author. Part of this research has been presented at the annual meeting of the National Council on Measurement in Education, San Francisco, April 1995.

The authors would like to thank Jeff Douglas, Xuming He and Ming-mei Wang for their comments and suggestions. The research of the first author was partially supported by an ETS/GREB Psychometric Fellowship, and by Educational Testing Service Research Allocation Project 884-01. The research of the second author was partially supported by NSF grant DMS 97-04474.

References

Ackerman, T. A. (1994). Using multidimensional item response theory to understand what items and tests are measuring. Applied Measurement in Education, 7, 255–278.CrossRefGoogle Scholar
Bock, R. D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.CrossRefGoogle Scholar
Bock, R. D., Gibbons, R., & Muraki, E. (1988). Full information item factor analysis. Applied Psychological Measurement, 12, 261–270.CrossRefGoogle Scholar
Bolt, D., Roussos, L. & Stout, W. F. (1997, March). Conditional covariance based representation of multidimensional test structure. Paper presented at the annual meeting of the National Council on Measurement in Education, Chicago, IL.Google Scholar
Chen, W. H., & Thissen, D. (1997). Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.CrossRefGoogle Scholar
Douglas, J., Kim, H. R., & Stout, W. F. (1994, April). Exploring and explaining the lack of local independence through conditional covariance functions. Paper presented at the 1994 annual meeting of the American Educational Research Association, New Orleans, LA.Google Scholar
Fraser, C. (1988). NOHARM: A computer program for fitting both unidimensional and multidimensional normal ogive models of latent trait theory, Armidale, New South Wales, Australia: The University of New England.Google Scholar
Fraser, C., & McDonald, R. P. (1988). NOHARM: Least squares item factor analysis. Multivariate Behavioral Research, 23, 267–269.CrossRefGoogle ScholarPubMed
Hattie, J., Krakowski, K., Roger, H. J., & Swaminathan, H. (1996). An assessment of Stout's index of essential dimensionality. Applied Psychological Measurement, 20, 1–14.CrossRefGoogle Scholar
Holland, P. W., & Rosenbaum, P. R. (1986). Conditional association and unidimensionality in monotone latent variable models. The Annals of Statistics, 14, 1523–1543.CrossRefGoogle Scholar
Joag-Dev, K., & Proschan, F. (1983). Negative association of random variables, with applications. The Annals of Statistics, 11, 286–295.CrossRefGoogle Scholar
Johnson, R. A., & Wichern, D. W. (1992). Applied multivariate statistical analysis 3rd ed.,, Upper Saddle River, NJ: Prentice Hall.Google Scholar
Junker, B. (1993). Conditional association, essential independence and monotone unidimensional item response models. The Annals of Statistics, 21, 1359–1378.CrossRefGoogle Scholar
Kim, H. R. (1994). New technqiues for the dimensionality assessment of standardized test data. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.Google Scholar
Knol, D. L., & Berger, P. F. (1991). Empirical comparison between factor analysis and multidimensional item response models. Multivariate Behavioral Research, 26, 457–477.CrossRefGoogle ScholarPubMed
Lehmann, E.L. (1966). Some concepts of dependence. The Annals of Mathematical Statistics, 37, 1137–1153.CrossRefGoogle Scholar
Lord, F. M. (1980). Application of item response theory to practical testing problems, Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar
Lord, F. M., & Novick, M. (1968). Statistical theories of mental test scores, Reading, MA: Addison Wesley.Google Scholar
McDonald, R. P. (1967). Nonlinear factor analysis. Psychometric Monograph, No. 15.Google Scholar
McDonald, R. P., & Ahlawat, K. S. (1974). Difficulty factors in binary data. British Journal of Mathematical and Statistical Psychology, 27, 82–99.CrossRefGoogle Scholar
Mislevy, R. J. (1986). Recent developments in the factor analysis of categorical variables. Journal of Educational Statistics, 11, 3–31.CrossRefGoogle Scholar
Nandakumar, R., & Stout, W. F. (1993). Refinement of Stout's procedure for assessing latent trait essential unidimensionality. Journal of Educational Statistics, 18, 41–68.Google Scholar
Plackett, R.L. (1954). A reduction formula for normal multivariate integrals. Biometrika, 41, 351–360.CrossRefGoogle Scholar
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.CrossRefGoogle Scholar
Reckase, M. D., & McKinley, R. L. (1991). The discriminating power of items that measure more than one dimension. Applied Psychological Measurement, 15, 361–373.CrossRefGoogle Scholar
Rosenbaum, P. (1984). Testing the conditional independence and monotonicity assumptions of item response theory. Psychometrika, 49, 425–435.CrossRefGoogle Scholar
Roussos, L.A., Stout, W.F., & Marden, J. (1998). Using new proximity measures with hierarchical cluster analysis to detect multidimensionality. Journal of Educational Measurement, 35, 1–30.CrossRefGoogle Scholar
Stout, W. F. (1987). A nonparametric approach for assessing latent trait dimensionality. Psychometrika, 52, 589–617.CrossRefGoogle Scholar
Stout, W. F. (1990). A new item response theory modeling approach with applications to unidimensional assessment and ability estimation. Psychometrika, 55, 293–326.CrossRefGoogle Scholar
Stout, W. F., Douglas, J., Junker, B. W., & Roussos, L. A. (1993). DIMTEST Manual. Program for dimensionality testing. Unpublished manuscript available from W. F. Stout, University of Illinois at Urbana-Champaign.Google Scholar
Stout, W.F., Habing, B., Douglas, J., Kim, H. R., Roussos, L.A., & Zhang, J. (1996). Conditional covariance based nonparametric multidimensionality assessment. Applied Psychological Measurement, 20, 331–354.CrossRefGoogle Scholar
Yen, W. M. (1984). Effects of local item dependence on the fit and equating performance of the three-parameter logistic model. Applied Psychological Measurement, 8, 125–145.CrossRefGoogle Scholar
Zhang, J. (1996). Some fundamental issues in item response theory with applications. Unpublished doctoral dissertation, University of Illinois at Urbana-Champaign, Department of Statistics.Google Scholar
Zhang, J. & Stout, W.F. (1995, April). Theoretical Results Concerning DETECT. Paper presented at the annual meeting of the National Council on Measurement in Education, San Francisco, CA.Google Scholar
Zhang, J., & Stout, W. F. (1996, April). A theoretical index of dimensionality and its estimation. Paper presented at the annual meeting of the National Council on Measurement in Education, NY.Google Scholar
Zhang, J., & Stout, W. F. (1999). The theoretical DETECT index of dimensionality and its application to approximate simple structure. Psychometrika, 64, 213–249.CrossRefGoogle Scholar
Zhang, J., & Wang, M. (1998, April). Relating reported scores to latent traits in a multidimensional test. Paper presented at the annual meeting of American Educational Research Association, San Diego, CA.Google Scholar