Hostname: page-component-6bf8c574d5-b4m5d Total loading time: 0 Render date: 2025-02-22T05:55:25.887Z Has data issue: false hasContentIssue false
  • English
  • Français

Multiple-Answer Multiple-Choice Test Items: Responding and Scoring Through Bayes and Minimax Strategies

Published online by Cambridge University Press:  01 January 2025

George T. Duncan  and
E. O. Milton
George T. Duncan*
Affiliation:
Carnegie-Mellon Univérsity
E. O. Milton
Affiliation:
University of California, Davis
*
Requests for reprints should be sent to George T. Duncan, Department of Statistics, Carnegie-Mellon University, Pittsburgh, Pennsylvania 15213.
Get access Rights & Permissions [Opens in a new window]

Abstract

A multiple-answer multiple-choice test item has a certain number of alternatives, any number of which might be keyed. The examinee is also allowed to mark any number of alternatives. This increased flexibility over the one keyed alternative case is useful in practice but raises questions about appropriate scoring rules. In this article a certain class of item scoring rules called the binary class is considered. The concepts of standard scoring rules and equivalence among these scoring rules are introduced in the “misinformation” model for which the traditional “knowledge” model is a special case. The examinee’s strategy with respect to a scoring rule is examined. The critical role of a quantity called the scoring ratio is emphasized. In the case of examinee uncertainty about the number of correct alternatives on an item, a Bayes and a minimax strategy for the examinee are developed. Also an appropriate response for the examiner to the minimax strategy is outlined.

Keywords

multiple-choice testtest scoringBayesminimaxmultiple-answer test items
Type
Original Paper
Information
Psychometrika , Volume 43 , Issue 1 , March 1978 , pp. 43 - 57
Copyright
Copyright © 1978 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Research partially supported under Grants N00014-67-A-0314-0022 from the Office of Naval Research and GS-32514 and MPS 75-07539 from the National Science Foundation.

References

Boas, R. P. Jr. A primer of real functions. Carus Mathematical Monographs, 1960, 13.Google Scholar
Collet, L. S. Elimination scoring: An empirical evaluation. Journal of Educational Measurement, 1971, 8, 209–214.CrossRefGoogle Scholar
Coombs, C. H. On the use of objective examinations. Educational and Psychological Measurement, 1953, 13, 308–10.CrossRefGoogle Scholar
Coombs, C. H., Milholland, J. E. & Womer, J. F. B. The assessment of partial knowledge. Educational and Psychological Measurement, 1956, 16, 13–37.CrossRefGoogle Scholar
Cronbach, L. J. An experimental comparison of the multiple true-false and multiple multiple-choice tests. Journal of Educational Psychology, 1941, 32, 533–543.CrossRefGoogle Scholar
Cronbach, L. J. Note on the multiple true-false test exercise. Journal of Educational Psychology, 1939, 30, 628–631.CrossRefGoogle Scholar
Cureton, E. E. The correction for guessing. The Journal of Experimental Education, 1966, 34, 44–47.CrossRefGoogle Scholar
Dressel, P. L. & Schmid, J. Some modifications of the multiple-choice item. Educational and Psychological Measurement, 1953, 13, 574–595.CrossRefGoogle Scholar
Duncan, G. T. An empirical Bayes approach to scoring multiple-choice tests in the misinformation model. Journal of the American Statistical Association, 1974, 69, 50–57.CrossRefGoogle Scholar
Ebel, R. L. Measuring educational achievement, 1965, Englewood Cliffs, New Jersey: Prentice-Hall.Google Scholar
Haven, S. E. & Copeland, H. A. A note on the “multiple-choice” test. Journal of Applied Psychology, 1932, 26, 219–221.CrossRefGoogle Scholar
Lord, F. M. & Novick, M. R. Statistical theories of mental test scores, 1968, Reading, Mass.: Addison-Wesley.Google Scholar
Orleans, J. S. & Sealy, G. A. Objective tests, 1928, Yonkers, New York: World Book Company.Google Scholar
Scheidemann, N. V. Multiplying the possibilities of the multiple-choice form of objective question. Journal of Applied Psychology, 1933, 27, 337–340.CrossRefGoogle Scholar
Shuford, E. H., Albert, A. & Massengill, H. Admissible probability measurement procedures. Psychometrika, 1966, 31, 125–145.CrossRefGoogle ScholarPubMed
Weitzman, R. A. Ideal multiple-choice items. Journal of the American Statistical Association, 1970, 65, 71–89.CrossRefGoogle Scholar
Zinger, A. A note on multiple-choice items. Journal of the American Statistical Association, 1972, 65, 340–1.CrossRefGoogle Scholar