Hostname: page-component-745bb68f8f-mzp66 Total loading time: 0 Render date: 2025-01-08T09:31:32.210Z Has data issue: false hasContentIssue false
  • English
  • Français

Optimal Sequential Designs for On-line Item Estimation

Published online by Cambridge University Press:  01 January 2025

Douglas H. Jones  and
Zhiying Jin
Douglas H. Jones*
Affiliation:
Graduate School of Management, Rutgers, The State University of New Jersey
Zhiying Jin
Affiliation:
Graduate School of Management, Rutgers, The State University of New Jersey
*
Requests for reprints should be sent to Douglas H. Jones; Graduate School of Management; Rutgers, The State University; 92 New Street; Newark, NJ 07102.
Get access Rights & Permissions [Opens in a new window]

Abstract

Replenishing item pools for on-line ability testing requires innovative and efficient data collection designs. By generating local D-optimal designs for selecting individual examinees, and consistently estimating item parameters in the presence of error in the design points, sequential procedures are efficient for on-line item calibration. The estimating error in the on-line ability values is accounted for with an item parameter estimate studied by Stefanski and Carroll. Locally D-optimal n-point designs are derived using the branch-and-bound algorithm of Welch. In simulations, the overall sequential designs appear to be considerably more efficient than random seeding of items.

Keywords

branch-and-boundcomputerized adaptive testexact n-point D-optimalinteger programmingitem response theorymeasurement errors modelon-line testingsequential design
Type
Original Paper
Information
Psychometrika , Volume 59 , Issue 1 , March 1994 , pp. 59 - 75
Copyright
Copyright © 1994 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

This report was prepared under the Navy Manpower, Personnel, and Training R&D Program of the Office of the Chief of Naval Research under Contract N00014-87-C-0696. The authors wish to acknowledge the valuable advice and consultation given by Ronald Armstrong, Charles Davis, Bradford Sympson, Zhaobo Wang, Ing-Long Wu and three anonymous reviewers.

References

Abramowitz, M., Stegun, I. A. (1970). Handbook of mathematical functions, New York: Dover.Google Scholar
Berger, Martijn P. F. (1991). On the efficiency of IRT models when applied to different sampling designs. Applied Psychological Measurement, 15, 293–306.CrossRefGoogle Scholar
Berger, Martijn P. F., van der Linden, W. J. (1991). Optimality of sampling designs in item response theory models. In Wilson, M. (Eds.), Objective measurement: Theory into practice, Norwood NJ: Ablex Publishing.Google Scholar
Box, G. E. P., Draper, N. R. (1959). A basis for the selection of a response surface design. Journal of the American Statistical Association, 54, 622–653.CrossRefGoogle Scholar
Donev, A. N., Atkinson, A. C. (1988). An adjustment algorithm for the construction of exact D-optimum experimental designs. Technometrics, 30, 429–433.Google Scholar
Federov, V. V. (1972). Theory of optimal experiments, New York: Academic Press.Google Scholar
Ford, I. (1976). Optimal static and sequential design: A critical review. Unpublished doctoral dissertation, University of Glasgow.Google Scholar
Ford, I., Kitsos, C. P., Titterington, D. M. (1989). Recent advances in nonlinear experimental design. Technometrics, 31, 49–60.CrossRefGoogle Scholar
Ford, I., Silvey, S. D. (1980). A sequentially constructed design for estimating a nonlinear parametric function. Biometrika, 67, 381–388.CrossRefGoogle Scholar
Ford, I., Titterington, D. M., Wu, C. F. J. (1985). Inference and sequential design. Biometrika, 72, 545–551.CrossRefGoogle Scholar
Fuller, W. A. (1987). Measurement error models, New York: John Wiley & Sons.CrossRefGoogle Scholar
Haines, L. M. (1987). The application of the annealing algorithm to the construction of exact optimal designs for linear-regression models. Technometrics, 29, 439–447.Google Scholar
Lord, F. M. (1971). Tailored testing, an application of stochastic approximation. Journal of the American Statistical Association, 66, 707–711.CrossRefGoogle Scholar
Lord, F. M. (1980). Applications of item response theory to practical testing problems, Hillside, NJ: Erlbaum.Google Scholar
Mitchell, T. J. (1974). An algorithm for the construction of D-optimal experimental designs. Technometrics, 16, 203–210.Google Scholar
Silvey, S. D. (1980). Optimal design, New York: Chapman and Hall.CrossRefGoogle Scholar
Stefanski, L. A., Carroll, R. J. (1985). Covariate measurement error in logistic regression. Annals Statistics, 13, 1335–1351.CrossRefGoogle Scholar
Steinberg, D. M., Hunter, W. G. (1984). Experimental design: Review and comment. Technometrics, 26, 71–130.CrossRefGoogle Scholar
Stocking, M. L. (1990). Specifying optimum examinees for item parameter estimation in item response theory. Psychometrika, 55, 461–475.CrossRefGoogle Scholar
Vale, C. D. (1986). Linking item parameters onto a common scale. Applied Psychological Measurement, 10, 333–344.CrossRefGoogle Scholar
van der Linden, W. J. (1988). Optimizing incomplete sampling designs for item response model parameters, Enschede, The Netherlands: University of Twente.Google Scholar
Welch, W. J. (1982). Branch-and-bound search for experimental designs based on D optimality and other criteria. Technometrics, 24, 41–48.Google Scholar
Wingersky, M. S., Lord, F. M. (1984). An investigation of methods for reducing sampling error in certain IRT procedures. Applied Psychological Measurement, 8, 347–364.CrossRefGoogle Scholar
Wu, C. F. J. (1985). Asymptotic inference from sequential design in a nonlinear situation. Biometrika, 72, 553–558.CrossRefGoogle Scholar