Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-08T10:33:48.505Z Has data issue: false hasContentIssue false
  • English
  • Français

Test Theory without an Answer Key

Published online by Cambridge University Press:  01 January 2025

William H. Batchelder  and
A. Kimball Romney
William H. Batchelder*
Affiliation:
University of California, Irvine
A. Kimball Romney
Affiliation:
University of California, Irvine
*
Requests for reprints should be sent to W. H. Batehelder, School of Social Sciences, University of California, Irvine, CA 92717.
Get access Rights & Permissions [Opens in a new window]

Abstract

A general model is presented for homogeneous, dichotomous items when the answer key is not known a priori. The model is structurally related to the two-class latent structure model with the roles of respondents and items interchanged. For very small sets of respondents, iterative maximum likelihood estimates of the parameters can be obtained by existing methods. For other situations, new estimation methods are developed and assessed with Monte Carlo data. The answer key can be accurately reconstructed with relatively small sets of respondents. The model is useful when a researcher wants to study objectively the knowledge possessed by members of a culturally coherent group that the researcher is not a member of.

Keywords

test theorydichotomous responsessignal detectionlatent class modelparameter estimationinformation pooling
Type
Original Paper
Information
Psychometrika , Volume 53 , Issue 1 , March 1988 , pp. 71 - 92
Copyright
Copyright © 1988 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This research was supported by NSF Grant No. SES-8320173 to the authors. We gratefully acknowledge comments and suggestions from John Boyd, Tarow Indow, and Kathy Maher as well as the editor and several anonymous referees.

References

Batchelder, W. H., Maher, K. M., & Romney, A. K. (1986). Extensions of a model for test theory without an answer key. Manuscript submitted for publication. (Prepublication draft available upon request).Google Scholar
Batchelder, W. H., Romney, A. K. (1986). The statistical analysis of a general Condorcet model for dichotomous choice situations. In Grofman, B., Owen, G. (Eds.), Information pooling and group decision making (pp. 103–112). Greenwich, CT: JAI Press.Google Scholar
Blackwell, H. R. (1963). Neural theories of simple visual discriminations. Journal of the Optical Society of America, 53, 129–160.CrossRefGoogle ScholarPubMed
Bliss, C. I. (1967). Statistics in biology (Vol. I), New York: McGraw-Hill.Google Scholar
Carré, B. (1979). Graphs and networks, Oxford, England: Oxford University Press.Google Scholar
Clogg, Clifford C. (1981). New developments in latent structure analysis. In Jackson, D. M., Borgatta, E. F. (Eds.), Factor analysis and measurement in sociological research, Beverly Hills, CA: Sage.Google Scholar
Comrey, A. L. (1962). The minimum residual method of factor analysis. Psychological Reports, 11, 15–18.CrossRefGoogle Scholar
Goodman, L. A. (1974). Exploratory latent structure analysis using both identifiable and unidentifiable models. Biometrika, 61, 219–231.CrossRefGoogle Scholar
Green, D. M., Swets, J. A. (1966). Signal detection theory and psychophysics, New York: Wiley.Google Scholar
Grofman, B., Feld, S. L., Owen, G. (1984). Group size and the performance of a composite group majority: Statistical truths and empirical results. Organizational Behavior and Human Performance, 33, 350–359.CrossRefGoogle Scholar
Guilford, J. P. (1954). Psychometric methods 2nd ed.,, New York: McGraw-Hill.Google Scholar
Hogg, R. V., Craig, A. T. (1978). Introduction to mathematical statistics 4th ed.,, New York: Macmillan.Google Scholar
Lazarsfeld, P. F., Henry, N. W. (1968). Latent structure analysis, New York: Houghton Mifflin.Google Scholar
Loéve, M. (1963). Probability theory, Princeton: Van Nostrand.Google Scholar
Lord, F. M. (1974). Individualized testing and item characteristic curve theory. In Krantz, D. H., Atkinson, R. C., Luce, R. D., Suppes, P. (Eds.), Contemporary developments in mathematical psychology (Vol. 2), San Francisco: Freeman.Google Scholar
Lord, F. M., Novick, M. R. (1968). Statistical theories of mental test scores, Reading, Mass.: Addison-Wesley.Google Scholar
Maher, K. M. (1987). A multiple choice model for aggregating group knowledge and estimating individual competencies. Unfinished doctoral dissertation, University of California, Irvine. (Available upon request.)Google Scholar
Nie, N. H., Hull, J. G., Jenkens, J. G., Steinbrenner, K., Bent, D. M. (1975). Statistical package for the social sciences, New York: McGraw Hill.Google Scholar
Nitzan, S., Paroush, J. (1982). Optimal decision rules in uncertain dichotomous choice situations. International Economics Review, 23, 289–297.CrossRefGoogle Scholar
Rasch, G. (1960). Probabilistic models for some intelligence and attainment tests, Copenhagen: Danmarks Paedagogiske Institut.Google Scholar
Romney, A. K., Weller, S. C., Batchelder, W. H. (1986). Culture as consensus: A theory of culture and informant accuracy. American Anthropologist, 88, 313–338.CrossRefGoogle Scholar