Hostname: page-component-745bb68f8f-lrblm Total loading time: 0 Render date: 2025-01-08T09:13:33.982Z Has data issue: false hasContentIssue false
  • English
  • Français

Psychometric Modeling of Response Speed and Accuracy with Mixed and Conditional Regression

Published online by Cambridge University Press:  01 January 2025

Gerard J. P. Van Breukelen
Gerard J. P. Van Breukelen*
Affiliation:
Maastricht University, The Netherlands
*
Requests for reprints should be sent to Gerard J.P. Van Breukelen, Department of Methodology and Statistics, Maastricht University, P.O. Box 616, 6200 MD Maastricht, The Netherlands. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Human performance in cognitive testing and experimental psychology is expressed in terms of response speed and accuracy. Data analysis is often limited to either speed or accuracy, and/or to crude summary measures like mean response time (RT) or the percentage correct responses. This paper proposes the use of mixed regression for the psychometric modeling of response speed and accuracy in testing and experiments. Mixed logistic regression of response accuracy extends logistic item response theory modeling to multidimensional models with covariates and interactions. Mixed linear regression of response time extends mixed ANOVA to unbalanced designs with covariates and heterogeneity of variance. Related to mixed regression is conditional regression, which requires no normality assumption, but is limited to unidimensional models. Mixed and conditional methods are both applied to an experimental study of mental rotation. Univariate and bivariate analyzes show how within-subject correlation between response and RT can be distinguished from between-subject correlation, and how latent traits can be detected, given careful item design or content analysis. It is concluded that both response and RT must be recorded in cognitive testing, and that mixed regression is a versatile method for analyzing test data.

Keywords

time limit testsconditional accuracy functionspeed-accuracy tradeoffconditional logistic regressionCox regressionmixed regressionmultilevel analysislatent traitmental rotation
Type
Original Paper
Information
Psychometrika , Volume 70 , Issue 2 , June 2005 , pp. 359 - 376
Copyright
Copyright © 2005 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

I am grateful to Rogier Donders for putting his data at my disposal.

References

Bloxom, B. (1985). Considerations in psychometric modeling of response time. Psychometrika, 50, 383–397.CrossRefGoogle Scholar
Cox, D.R. (1972). Regression models and life tables (with discussion). Journal of the Royal Statistical Society B, 34, 187–220.CrossRefGoogle Scholar
Cox, D.R., & Oakes, D. (1984). Analysis of Survival Data. London: Chapman and Hall.Google Scholar
Donders, R. (1997). The Validity of Basic Assumptions Underlying Models for Time Limit Tests. PhD thesis, Nijmegen University, The NetherlandsGoogle Scholar
Fischer, G.H. (1974). Einführung in die theorie psychologischer tests [Introduction to the theory of psychological tests]. Bern: Huber.Google Scholar
Gao, S. (2004). A shared random effect parameter approach for longitudinal dementia data with non-ignorable missing data. Statistics in Medicine, 23, 211–219.CrossRefGoogle ScholarPubMed
Goldstein, H. (1995). Multilevel Statistical Models (2nd ed.). London: Edward Arnold.Google Scholar
Hambleton, R.K., & Swaminathan, H. (1985). Item Response Theory: Principles and Applications. Boston (MA): Kluwer Academic Publishers.CrossRefGoogle Scholar
Hedeker, D., & Gibbons, R.D. (1996). MIXOR: a computer program for mixed-effects ordinal regression. Computer Methods and Programs in Biomedicine, 49, 157–176.CrossRefGoogle ScholarPubMed
Hedeker, D., & Gibbons, R.D. (1996). MIXREG: a computer program for mixed-effects regression with autocorrelated errors. Computer Methods and Programs in Biomedicine, 49, 229–252.CrossRefGoogle ScholarPubMed
Hosmer, D.W., & Lemeshow, S. (1989). Applied Logistic Regression. New York: Wiley.Google Scholar
Kahane, M., & Loftus, G. (1999). Response time versus accuracy in human memory. In Sternberg, R.J. (Eds.), The Nature of Cognition (pp. 323–384). Cambridge (MA): MIT.Google Scholar
Lord, F.M., & Novick, M.R. (1968). Statistical Theories of Mental Test Scores. Reading (MA): Addison-Wesley.Google Scholar
Luce, R.D. (1986). Response Times: Their Role in Inferring Elementary Mental Organization. New York: Oxford University Press.Google Scholar
Maris, E. (1993). Additive and multiplicative models for gamma distributed variables, and their application as models for response times. Psychometrika, 58, 445–469.CrossRefGoogle Scholar
Marley, A.A.J., & Colonius, H. (1992). The “horse race” random utility model for choice probabilities and reaction times, and its competing risks interpretation. Journal of Mathematical Psychology, 36, 1–20.CrossRefGoogle Scholar
Metzler, J., & Shepard, R.N. (1974). Tranformational studies of the internal representation of three-dimensional objects. In Solso, R.L. (Eds.), Theories in Cognitive Psychology: The Loyola Symposium (pp. 147–201). Potomac (MD): Erlbaum.Google Scholar
Moerbeek, M., Van Breukelen, G., & Berger, M. (2001). Optimal experimental design for multilevel logistic models. The Statistician, 50, 17–30.CrossRefGoogle Scholar
Moerbeek, M., Van Breukelen, G., & Berger, M. (2003). A comparison of estimation methods for multilevel logistic models. Computational Statistics, 18, 19–38.CrossRefGoogle Scholar
Pachella, R.G. (1974). The interpretation of reaction time in information-processing research. In Kantowitz, B.H. (Eds.), Human Information Processing: Tutorials in Performance and Cognition (pp. 41–82). Hillsdale (NJ): Erlbaum.Google Scholar
Rasbash, J., Browne, W., Goldstein, H., Yang, M., Plewis, I., Healy, M., Woodhouse, G., Draper, D., Langford, I., & Lewis, T. (2000). A User’s Guide to MLwiN. Multilevel Models Project, Institute of Education, University of London, Version 2.1Google Scholar
Ratcliff, R. (1988). Continuous versus discrete information processing: modeling accumulation of partial information. Psychological Review, 95, 238–255.CrossRefGoogle ScholarPubMed
Ratcliff, R., & Smith, P.L. (2004). A comparison of sequential sampling models for two-choice reaction time. Psychological Review, 111, 333–367.CrossRefGoogle ScholarPubMed
Rijmen, F., & DeBoeck, P. (2002). The random weights linear logistic test model. Applied Psychological Measurement, 26, 271–285.CrossRefGoogle Scholar
Shepard, R.N., & Metzler, J. (1971). Mental rotation of three-dimensional objects. Science, 171, 701–703.CrossRefGoogle ScholarPubMed
Snijders, T.A.B., & Bosker, R.J. (1999). Multilevel Analysis: An Introduction to Basic and Advanced Multilevel Modeling. London: Sage Publications.Google Scholar
Sternberg, S. (1969). The discovery of processing stages: extensions of Donders’ method. Acta Psychologica, 30, 276–315.CrossRefGoogle Scholar
Storms, G., & Delbeke, L. (1992). The irrelevance of distributional assumptions on RTs in in multidimensional scaling of same/different tasks. Psychometrika, 57, 599–614.CrossRefGoogle Scholar
Therneau, T.M., & Grambsch, P.M. (2000). Modeling Survival Data: Extending the Cox Model. New York: Springer.CrossRefGoogle Scholar
Thissen, D. (1983). Timed testing: an approach using item response theory. In Weiss, D.J. (Eds.), New Horizons in Testing: Latent Trait Theory and Computerized Adaptive Testing (pp. 179–203). New York: Academic Press.Google Scholar
Thurstone, L.L. (1937). Ability, motivation and speed. Psychometrika, 2, 249–254.CrossRefGoogle Scholar
Townsend, J.T., Ashby, F.G. (1983). The Stochastic Modeling of Elementary Psychological Processes. Cambridge: University Press.Google Scholar
Townsend, J.T., & Nozawa, G. (1995). Spatio-temporal properties of elementary perception: an investigation of parallel, serial, and coactive theories. Journal of Mathematical psychology, 39, 321–359.CrossRefGoogle Scholar
Ulrich, R., & Miller, J. (1993). Information processing models generating lognormally distributed reaction times. Journal of Mathematical Psychology, 37, 513–525.CrossRefGoogle Scholar
Van Breukelen, G.J.P. (1989). Concentration, Speed and Precision in Mental Tests: a Psychonometric Approach. PhD thesis, The Netherlands, Nijmegen UniversityGoogle Scholar
Van Breukelen, G.J.P. (1995a). Psychometric and information processing properties of selected response time models. Psychometrika, 60, 95–113.CrossRefGoogle Scholar
Van Breukelen, G.J.P. (1995b). Parallel processing models compatible with lognormally distributed response times. Journal of Mathematical Psychology, 39, 396–399.CrossRefGoogle Scholar
Van Breukelen, G.J.P. (1997). Separability of item and person parameters in response time models. Psychometrika, 62, 525–544.CrossRefGoogle Scholar
Van Breukelen, G.J.P., & Roskam, E.E.Ch.I. (1991). A Rasch model for the speed-accuracy tradeoff in time limit tests. In Doignon, J.P., & Falmagne, J.C. (Eds.), Mathematical Psychology: Current Developments (pp. 251–271). New York: Springer.CrossRefGoogle Scholar
Van der Linden, W.J., & Hambleton, R.K. (1997). Handbook of Modern Item Response Theory. New York: Springer.CrossRefGoogle Scholar
Vander Linden, W.J., Scrams, D.J., & Schnipke, D.L. (1999). Using response-time constraints to control for differential speededness in computerized adaptive testing. Applied Psychological Measurement, 23, 195–210.CrossRefGoogle Scholar
Verbeke, G., & Molenberghs, G. (2000). Linear Mixed Models for Longitudinal Data. New York: Springer.Google Scholar
Verhelst, N.D., Verstralen, H.H.F.M., & Jansen, M.G.H. (1997). A logistic model for time limit tests. In Van der Linden, W.J., & Hambleton, R.K. (Eds.), Handbook of Modern Item Response Theory (pp. 169–186). New York: Springer.CrossRefGoogle Scholar
Vorberg, D., & Ulrich, R. (1987). Random search with unequal rates: serial and parallel generalizations of McGill’s model. Journal of Mathematical Psychology, 31, 1–23.CrossRefGoogle Scholar
Wenger, M.J., & Gibson, B.S. (2004). Using hazard functions to assess changes in processing capacity in an attentional cueing paradigm. Journal of Experimental Psychology: Human Perception and Performance, 30, 708–719.Google Scholar
Zwinderman, A.H. (1991). A generalized Rasch model for manifest predictors. Psychometrika, 56, 589–600.CrossRefGoogle Scholar