Hostname: page-component-599cfd5f84-5kswg Total loading time: 0 Render date: 2025-01-07T07:01:48.011Z Has data issue: false hasContentIssue false
  • English
  • Français

Incorporating Functional Response Time Effects into a Signal Detection Theory Model

Published online by Cambridge University Press:  01 January 2025

Sun-Joo Cho ,
Sarah Brown-Schmidt ,
De Boeck Paul ,
Matthew Naveiras ,
Si On Yoon  and
Aaron Benjamin
Sun-Joo Cho*
Affiliation:
Vanderbilt University
Sarah Brown-Schmidt
Affiliation:
Vanderbilt University
De Boeck Paul
Affiliation:
The Ohio State University and KU Leuven
Matthew Naveiras
Affiliation:
Vanderbilt University
Si On Yoon
Affiliation:
University of Iowa
Aaron Benjamin
Affiliation:
University of Illinois at Urbana-Champaign
*
Correspondence should be made to Sun-Joo Cho, Vanderbilt University, Nashville, USA. Email: [email protected]; URL: http://www.vanderbilt.edu/psychological_sciences/bio/sun-joo-cho
Get access Rights & Permissions [Opens in a new window]

Abstract

Signal detection theory (SDT; Tanner & Swets in Psychological Review 61:401–409, 1954) is a dominant modeling framework used for evaluating the accuracy of diagnostic systems that seek to distinguish signal from noise in psychology. Although the use of response time data in psychometric models has increased in recent years, the incorporation of response time data into SDT models remains a relatively underexplored approach to distinguishing signal from noise. Functional response time effects are hypothesized in SDT models, based on findings from other related psychometric models with response time data. In this study, an SDT model is extended to incorporate functional response time effects using smooth functions and to include all sources of variability in SDT model parameters across trials, participants, and items in the experimental data. The extended SDT model with smooth functions is formulated as a generalized linear mixed-effects model and implemented in the gamm4R package. The extended model is illustrated using recognition memory data to understand how conversational language is remembered. Accuracy of parameter estimates and the importance of modeling variability in detecting the experimental condition effects and functional response time effects are shown in conditions similar to the empirical data set via a simulation study. In addition, the type 1 error rate of the test for a smooth function of response time is evaluated.

Keywords

generalized additive mixed modelgeneralized linear mixed-effects modelresponse timesignal detection theorysmooth function
Type
Theory & Methods
Information
Psychometrika , Volume 88 , Issue 3 , September 2023 , pp. 1056 - 1086
Copyright
Copyright © 2023 The Author(s) under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Agresti, A.(2002). Categorical data analysis, 2Wiley.CrossRefGoogle Scholar
Baayen, R. H., Davidson, D. J., &Bates, D. M.(2008). Mixed-effects modeling with crossed random effects for subjects and items. Journal of Memory and Language, 59, 390–412.CrossRefGoogle Scholar
Baayen, R. H., Vasishth, S., Kliegl, R., &Bates, D.(2017). The cave of shadows: Addressing the human factor with generalized additive mixed models. Journal of Memory and Language, 94, 206–234.CrossRefGoogle Scholar
Bates, D., Maechler, M., Bolker, B., &Walker, S.(2015). Fitting linear mixed-effects models using lme4. Journal of Statistical Software, 67, 1–48.CrossRefGoogle Scholar
Benjamin, A. S.(2001). On the dual effects of repetition on false recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 27, 941–947.Google ScholarPubMed
Benjamin, A. S.(2013). Where is the criterion noise in recognition? (Almost) everyplace you look: Comment on Kellen, Klauer, and Singmann (2012). Psychological Review, 120, 720–726.CrossRefGoogle Scholar
Benjamin, A. S., Diaz, M., &Wee, S.(2009). Signal detection with criterion noise: Applications to recognition memory. Psychological Review, 116, 84–115.CrossRefGoogle ScholarPubMed
Besson, G., Ceccaldi, M., Didic, M., &Barbeau, E. J.(2012). The speed of visual recognition memory. Visual Cognition, 20, 1131–1152.CrossRefGoogle Scholar
Bolsinova, M., De Boeck, P., &Tijmstra, J.(2017). Modeling conditional dependence between response time and accuracy. Psychometrika, 82, 1126–1148.CrossRefGoogle ScholarPubMed
Bolsinova, M., &Molenaar, D.(2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9, 1525.CrossRefGoogle ScholarPubMed
Brown, S., &Heathcote, A.(2008). The simplest complete model of choice response time: Linear ballistic accumulation. Cognitive Psychology, 57, 153–178.CrossRefGoogle ScholarPubMed
Chen, H., De Boeck, P., Grady, M., Yang, C. -L., &Waldschmidt, D.(2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16–23.CrossRefGoogle Scholar
Cho, S. -J., Partchev, I., & De Boeck, P.(2012). Parameter estimation of multiple item profiles models. British Journal of Mathematical and Statistical Psychology, 65, 438–466.CrossRefGoogle Scholar
De Boeck, P., &Jeon, M.(2019). An overview of models for response times and processes in cognitive tests. Frontiers in Psychology, 10, 102.CrossRefGoogle ScholarPubMed
DeCarlo, L. T.(1998). Signal detection theory and generalized linear models. Psychological Methods, 3, 186–205.CrossRefGoogle Scholar
DeCarlo, L. T.(2010). On the statistical and theoretical basis of signal detection theory and extensions: Unequal variance, random coefficient, and mixture models. Journal of Mathematical Psychology, 54, 304–313.CrossRefGoogle Scholar
DeCarlo, L. T.(2011). Signal detection theory with item effects. Journal of Mathematical Psychology, 55, 229–239.CrossRefGoogle Scholar
DeCarlo, L. T.(2021). On joining a signal detection choice model with response time models. Journal of Educational Measurement,CrossRefGoogle Scholar
Finch, W. H., & Finch, M. H.(2018). A simulation study evaluating the generalized additive model for assessing intervention effects with small samples. Journal of Experimental Education, 86(4),652–670.CrossRefGoogle Scholar
Frenkel, T. I., Lamy, D., Algom, D., &Bar-Haim, Y.(2009). Individual differences in perceptual sensitivity and response bias in anxiety: Evidence from emotional faces. Cognition and Emotion, 23, 688–700.CrossRefGoogle Scholar
Goldhammer, F., Steinwascher, M. A., Kroehne, U., &Naumann, J.(2017). Modeling individual response time effects between and within experimental speed conditions: A GLMM approach for speeded tests. British Journal of Mathematical and Statistical Psychology, 70, 238–256.CrossRefGoogle ScholarPubMed
Jaeger, T. F.(2008). Categorical data analysis: Away from ANOVAs (transformation or not) and towards logit mixed models. Journal of Memory and Language, 59, 434–446.CrossRefGoogle ScholarPubMed
James, A. N., Fraundorf, S. H., Lee, E. K., &Watson, D. G.(2018). Individual differences in syntactic processing: Is there evidence for reader-text interactions?. Journal of Memory and Language, 102, 155–181.CrossRefGoogle ScholarPubMed
Kang, I., De Boeck, P., &Partchev, I.(2022). A randomness perspective on intelligence processes. Intelligence, 91.CrossRefGoogle Scholar
Kang, I., De Boeck, P., &Ratcliff, R.(2022). Modeling conditional dependence of response accuracy and response time with the diffusion item response theory model. Psychometrika, .CrossRefGoogle ScholarPubMed
Krauss, R. M., &Weinheimer, S.(1964). Changes in reference phrases as a function of frequency of usage in social interaction: A preliminary study. Psychonomic Science, 1, 113–114.CrossRefGoogle Scholar
MacLeod, C. M., Gopie, N., Hourihan, K. L., Neary, K. R., &Ozubko, J. D.(2010). The production effect: Delineation of a phenomenon. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36, 671–685.Google ScholarPubMed
Marra, G., &Wood, S. N.(2012). Coverage properties of confidence intervals for generalized additive model components. Scandinavian Journal of Statistics, 39(1),53–74.CrossRefGoogle Scholar
Martínez, E., Mollica, F., &Gibson, E.(2022). Poor writing, not specialized concepts, drives processing difficulty in legal language. Cognition, 224.CrossRefGoogle Scholar
Parasuraman, R., Masalonis, A. J., &Hancock, P. A.(2000). Fuzzy signal detection: Basic postulates and formulas for analyzing human and machine performance. Human Factors, 42, 636–659.CrossRefGoogle ScholarPubMed
R Core Team. (2019). R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. https://www.R-project.org/.Google Scholar
Ramsay, J. O., &Silverman, B. W.(2005). Functional data analysis, 2Springer.CrossRefGoogle Scholar
Ratcliff, R.(1978). A theory of memory retrieval. Psychological Review, 85, 59–108.CrossRefGoogle Scholar
Ratcliff, R., Smith, P. L., & McKoon, G.(2015). Modeling regularities in response time and accuracy data with the diffusion model. Current Directions in Psychological Science, 24, 458–470.CrossRefGoogle ScholarPubMed
Ratcliff, R., Thapar, A., &McKoon, G.(2006). Aging and individual differences in rapid two-choice decisions. Psychonomic Bulletin & Review, 13, 626–635.CrossRefGoogle ScholarPubMed
Rodríguez, G. (2007). Lecture notes on generalized linear models. Downloaded from https://data.princeton.edu/wws509/notes/.Google Scholar
Rouder, J. N., &Lu, J.(2005). An introduction to Bayesian hierarchical models with an application in the theory of signal detection. Psychonomic Bulletin and Review, 12, 573–604.CrossRefGoogle ScholarPubMed
Rouder, J. N., Lu, J., Sun, D., Morey, R., &Naveh-Benjamin, M.(2007). Signal detection models with random participant and item effects. Psychometrika, 72, 621–642.CrossRefGoogle Scholar
Ryskin, R., Benjamin, A. S., Tullis, J., & Brown-Schmidt, S.(2015). Perspective-taking in comprehension, production, and memory: An individual differences approach. Journal of Experimental Psychology: General, 144, 898–915.CrossRefGoogle ScholarPubMed
Schnipke, D. L., &Scrams, D. J.(2002). Exploring issues of examinee behavior: Insights gained from response-time analyses.Mills, C. N., Potenza, M. T., Fremer, J. J., &Ward, W. C. Computer-based testing: Building the foundation for future assessments, Lawrence Erlbaum Associates Publishers.237–266.Google Scholar
Schwarz, G.(1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.CrossRefGoogle Scholar
Silverman, B. W.(1985). Some aspects of the spline smoothing approach to nonparametric regression curve fitting. Journal of the Royal Statistical Society, Series B, 47(1),1–21.CrossRefGoogle Scholar
Slamecka, N. J., &Graf, P.(1978). The generation effect: Delineation of a phenomenon. Journal of Experimental Psychology: Human Learning and Memory, 4, 592–604.Google Scholar
Tanner, W. P., &Swets, J. A.(1954). A decision-making theory of visual detection. Psychological Review, 61, 401–409.CrossRefGoogle ScholarPubMed
Vaida, F., &Blanchard, S.(2005). Conditional Akaike information for mixed-effects models. Biometrika, 92, 351–370.CrossRefGoogle Scholar
van der Linden, W. J.(2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72, 287–308.CrossRefGoogle Scholar
van der Linden, W. J.(2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46, 247–272.CrossRefGoogle Scholar
Wickelgren, W. A.(1972). Trace resistance and the decay of long-term memory. Journal of Mathematical Psychology, 9, 418–455.CrossRefGoogle Scholar
Wickelgren, W. A.(1977). Speed-accuracy tradeoff and information processing dynamics. Acta Psychologica, 41, 67–85.CrossRefGoogle Scholar
Wieling, M.(2018). Analyzing dynamic phonetic data using generalized additive mixed modeling: A tutorial focusing on articulatory differences between L1 and L2 speakers of English. Journal of Phonetics, 70, 86–116.CrossRefGoogle Scholar
Wood, S. N. (2019). Package ‘mgcv’: Mixed GAM computation vehicle with automatic smoothness estimation. https://cran.r-project.org/web/packages/mgcv/mgcv.pdf.Google Scholar
Wood, S. N., & Scheipl, F. (2020). gamm4: Generalized additive mixed models using ‘mgcv’ and ‘lme4’. R package version 0.2-6. https://CRAN.R-project.org/package=gamm4.Google Scholar
Wood, S. N.(2004). Stable and efficient multiple smoothing parameter estimation for generalized additive models. Journal of the American Statistical Association, 99(467),673–686.CrossRefGoogle Scholar
Wood, S. N.(2006). Low-rank scale-invariant tensor product smooths for generalized additive mixed models. Biometrics, 62(4),1025–1036.CrossRefGoogle ScholarPubMed
Wood, S. N.(2013). On p-values for smooth components of an extended generalized additive model. Biometrika, 100(1),221–229.CrossRefGoogle Scholar
Wood, S. N.(2017). Generalized additive models: An introduction with R, 2CRC Press.CrossRefGoogle Scholar
Wood, S. N., Pya, N., &Säfken, B.(2016). Smoothing parameter and model selection for general smooth models. Journal of the American Statistical Association, 111(516),1548–1563.CrossRefGoogle Scholar
Wright, D. B., Horry, R., &Skagerberg, E. M.(2009). Functions for traditional and multilevel approaches to signal detection theory. Behavior Research Methods, 41, 257–267.CrossRefGoogle ScholarPubMed
Yoon, S. O., Benjamin, A. S., &Brown-Schmidt, S.(2016). The historical context in conversation: Lexical differentiation and memory for the discourse history. Cognition, 154, 102–117.CrossRefGoogle ScholarPubMed
Yoon, S. O., Benjamin, A. S., &Brown-Schmidt, S.(2021). Referential form and memory for the discourse history. Cognitive Science, .CrossRefGoogle ScholarPubMed