Hostname: page-component-745bb68f8f-grxwn Total loading time: 0 Render date: 2025-01-08T10:03:31.307Z Has data issue: false hasContentIssue false
  • English
  • Français

The Effect of Uncertainty of Item Parameter Estimation on Ability Estimates

Published online by Cambridge University Press:  01 January 2025

Robert K. Tsutakawa  and
Jane C. Johnson
Robert K. Tsutakawa*
Affiliation:
University of Missouri
Jane C. Johnson
Affiliation:
University of Missouri
*
Requests for reprints should be sent to Robert K. Tsutakawa, Department of Statistics, University of Missouri, 222 Math Sciences, Columbia, MO 65211.
Get access Rights & Permissions [Opens in a new window]

Abstract

The conventional method of measuring ability, which is based on items with assumed true parameter values obtained from a pretest, is compared to a Bayesian method that deals with the uncertainties of such items. Computational expressions are presented for approximating the posterior mean and variance of ability under the three-parameter logistic (3PL) model. A 1987 American College Testing Program (ACT) math test is used to demonstrate that the standard practice of using maximum likelihood or empirical Bayes techniques may seriously underestimate the uncertainty in estimated ability when the pretest sample is only moderately large.

Keywords

ability estimationBayesian IRTcalibrationpretestthree-parameter logistic
Type
Original Paper
Information
Psychometrika , Volume 55 , Issue 2 , June 1990 , pp. 371 - 390
Copyright
Copyright © 1990 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This work was partially supported under contract No. N00014-85-K-0113, NR150-535, from the Cognitive Science Program, Office of Naval Research. The authors wish to thank Mark D. Reckase for providing the ACT data used in the illustration and two referees, Associate Editor and Editor for helpful suggestions.

References

Birnbaum, A. (1968). Some latent trait models and their use in inferring an examinee's ability. In Lord, F. M. & Novick, M. R. (Eds.), Statistical theories of mental test scores (pp. 395–479). Reading, MA: Addison-Wesley.Google Scholar
Birnbaum, A. (1969). Statistical theory for logistic mental test models with prior distribution of ability. Journal of Mathematical Psychology, 6, 258–276.CrossRefGoogle Scholar
Bock, R. D., & Aitken, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm. Psychometrika, 46, 443–459.CrossRefGoogle Scholar
Deeley, J. J., & Lindley, D. V. (1981). Bayes empirical Bayes. Journal of the American Statistical Association, 76, 833–841.CrossRefGoogle Scholar
DeGroot, M. H. (1986). Probability and statistics 2nd ed.,, Reading, MA: Addison-Wesley.Google Scholar
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion). Journal of the Royal Statistical Society, 39, 1–38.CrossRefGoogle Scholar
Lindley, D. V. (1980). Approximate Bayesian methods. Trabajos Estadistica, 31, 223–237.CrossRefGoogle Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems, Hillsdale, NJ: Erlbaum.Google Scholar
Lord, F. M. (1986). Maximum likelihood and Bayesian parameter estimation in item response theory. Journal of Educational Measurement, 23, 157–162.CrossRefGoogle Scholar
Mislevy, R. J., & Bock, R. D. (1984). BILOG: Item analysis and test scoring with binary logistic models, Mooresville, IN: Scientific Software.Google Scholar
Tsutakawa, R. K. (1984). Estimation of two-parameter logistic item response curves. Journal of Educational Statistics, 9, 263–276.CrossRefGoogle Scholar
Tsutakawa, R. K. (1988). Dirichlet prior in Bayesian estimation of item response curves, Columbia, MO: University of Missouri, Department of Statistics.Google Scholar
Tsutakawa, R. K., & Soltys, M. J. (1988). Approximations for Bayesian ability estimation. Journal of Educational Statistics, 13, 117–130.CrossRefGoogle Scholar
Wingersky, M. S., Barton, M. A., & Lord, F. M. (1982). LOGIST user's guide, Princeton, NJ: Educational Testing Service.Google Scholar