Hostname: page-component-599cfd5f84-wh4qq Total loading time: 0 Render date: 2025-01-07T07:15:42.688Z Has data issue: false hasContentIssue false
  • English
  • Français

A Latent Space Diffusion Item Response Theory Model to Explore Conditional Dependence between Responses and Response Times

Published online by Cambridge University Press:  01 January 2025

Inhan Kang Open the ORCID record for Inhan Kang [Opens in a new window] ,
Minjeong Jeon  and
Ivailo Partchev
Inhan Kang*
Affiliation:
Yonsei University
Minjeong Jeon
Affiliation:
University of California, Los Angeles
Ivailo Partchev
Affiliation:
Cito
*
Correspondence should be made to Inhan Kang, Yonsei University, 403 Widang Hall, 50 Yonsei-ro, Seodaemungu, Seoul 03722, Republic of Korea. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Traditional measurement models assume that all item responses correlate with each other only through their underlying latent variables. This conditional independence assumption has been extended in joint models of responses and response times (RTs), implying that an item has the same item characteristics fors all respondents regardless of levels of latent ability/trait and speed. However, previous studies have shown that this assumption is violated in various types of tests and questionnaires and there are substantial interactions between respondents and items that cannot be captured by person- and item-effect parameters in psychometric models with the conditional independence assumption. To study the existence and potential cognitive sources of conditional dependence and utilize it to extract diagnostic information for respondents and items, we propose a diffusion item response theory model integrated with the latent space of variations in information processing rate of within-individual measurement processes. Respondents and items are mapped onto the latent space, and their distances represent conditional dependence and unexplained interactions. We provide three empirical applications to illustrate (1) how to use an estimated latent space to inform conditional dependence and its relation to person and item measures, (2) how to derive diagnostic feedback personalized for respondents, and (3) how to validate estimated results with an external measure. We also provide a simulation study to support that the proposed approach can accurately recover its parameters and detect conditional dependence underlying data.

Keywords

conditional dependencelatent space modeldiffusion processitem response theoryresponse timeinteraction
Type
Theory & Methods
Information
Psychometrika , Volume 88 , Issue 3 , September 2023 , pp. 830 - 864
Copyright
Copyright © 2023 The Author(s) under exclusive licence to The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

Supplementary Information The online version contains supplementary material available at https://doi.org/10.1007/s11336-023-09920-x.

References

Batchelder, W. H., &Bershad, N. J.(1979). The statistical analysis of a thurstonian model for rating chess players. Journal of Mathematical Psychology, 19(1),39–60CrossRefGoogle Scholar
Bolsinova, M., De Boeck, P., & Tijmstra, J.(2017). Modelling conditional dependence between response and accuracy. Psychometrika, 82(4),1126–114827738955CrossRefGoogle ScholarPubMed
Bolsinova, M., &Maris, G.(2016). A test for conditional independence between response time and accuracy. British Journal of Mathematical and Statistical Psychology, 69(1),62–79CrossRefGoogle ScholarPubMed
Bolsinova, M., &Molenaar, D.(2018). Modeling nonlinear conditional dependence between response time and accuracy. Frontiers in Psychology, 9(1525),1–12CrossRefGoogle ScholarPubMed
Bolsinova, M., &Tijmstra, J.(2018). Improving precision of ability estimation: Getting more from response times. British Journal of Mathematical and Statistical Psychology, 71(1),13–38CrossRefGoogle ScholarPubMed
Bolsinova, M., Tijmstra, J., & Molenaar, D.(2017). Response moderation models for conditional dependence between response time and response accuracy. British Journal of Mathematical and Statistical Psychology, 70, 257–279CrossRefGoogle ScholarPubMed
Bolsinova, M., Tijmstra, J., Molenaar, D., &De Boeck, P.(2017). Conditional dependence between response time and accuracy: An overview of its possible sources and directions for distinguishing between them. Frontiers in Psychology, 8, 202CrossRefGoogle ScholarPubMed
Borg, I., &Gorenen, P.(2005). Modern multidimensional scaling: Theory and applications (2, ndNew York: Springer.Google Scholar
Chen, H., De Boeck, P., Grady, M., Yang, C. -L., &Waldschmidt, D.(2018). Curvilinear dependency of response accuracy on response time in cognitive tests. Intelligence, 69, 16–23CrossRefGoogle Scholar
Coombs, C. H. (1964). A theory of data. Wiley.Google Scholar
De Boeck, P., Chen, H., &Davison, M.(2017). Spontaneous and imposed speed of cognitive test responses. British Journal of Mathematical and Statistical Psychology, 70(2),225–237CrossRefGoogle ScholarPubMed
DiTrapani, J., Jeon, M., De Boeck, P., &Partchev, I.(2016). Attempting to differentiate fast and slow intelligence: Using generalized item response trees to examine the role of speed on intelligence tests. Intelligence, 56, 82–92CrossRefGoogle Scholar
Elo, A.(1978). The rating of chess players, past and present, London: Batsford.Google Scholar
Friel, N., Rastelli, R., Wyse, J., &Raftery, A. E.(2016). Interlocking directorates in irish companies using a latent space model for bipartite networks. Proceedings of the National Academy of Sciences, 113(24),6629–6634CrossRefGoogle ScholarPubMed
Gelman, A.(1996). Inference and monitoring convergence.Gilks, W. R., Richardson, S., &Spiegelhalter, D. J. Markov chain monte carlo in practice, CRC Press.131–143Google Scholar
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A., & Rubin, D. B. (2013).Google Scholar
Goldhammer, F., Naumann, J., &Greiff, S.(2015). More is not always better: The relation between item response and item response time in raven’s matrices. Journal of Intelligence, 3(1),21–40CrossRefGoogle Scholar
Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rólke, H., &Klieme, E.(2014). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106(3),608–626CrossRefGoogle Scholar
Handcock, M. S., Raftery, A. E., &Tantrum, J. M.(2007). Model-based clustering for social networks. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(2),301–354CrossRefGoogle Scholar
Hoff, P. D., Raftery, A. E., &Handcock, M. S.(2002). Latent space approaches to social network analysis. Journal of the American Statistical Association, 97(460),1090–1098CrossRefGoogle Scholar
Ishwaran, H., &Rao, J. S.(2005). Spike and slab variable selection: Frequentist and Bayesian strategies. The Annals of Statistics, 33(2),730–773CrossRefGoogle Scholar
Jeon, M., Jin, I., Schweinberger, M., &Baugh, S.(2021). Mapping unobserved item-respondent interactions: A latent space item response model with interaction map. Psychometrika, 86, 378–403CrossRefGoogle ScholarPubMed
Jin, I., &Jeon, M.(2019). A doubly latent space joint model for local item and person dependence in the analysis of item response data. Psychometrika, 84, 236–260CrossRefGoogle ScholarPubMed
Kang, I., De Boeck, P., &Partchev, I.(2022). A randomness perspective on intelligence processes. Intelligence, 91CrossRefGoogle Scholar
Kang, I., De Boeck, P., & Ratcliff, R. (2022). Modeling conditional dependence of response accuracy and response time with the diffusion item response theory model. Psychometrika, Advance Online Publication. https://doi.org/10.1007/s11336-021-09819-5CrossRefGoogle Scholar
Lewandowski, D., Kurowicka, D., &Joe, H.(2009). Generating random correlation matrices based on vines and extended onion method. Journal of Multivariate Analysis, 100(9),1989–2001CrossRefGoogle Scholar
Lu, J., &Wang, C.(2020). A response time process model for not-reached and omitted items. Journal of Educational Measurement, 57(4),584–620CrossRefGoogle Scholar
Lu, J., Wang, C., & Shi, N. (2021). A mixture response time process model for aberrant behaviors and item nonresponses. Multivariate Behavioral Research, Ahead-of-Print. https://doi.org/10.1080/00273171.2021.1948815CrossRefGoogle Scholar
Man, K., Harring, J. R., Jiao, H., &Zhan, P.(2019). Joint modeling of compensatory multidimensional item responses and response times. Applied Psychological Measurement, 43(8),639–654CrossRefGoogle ScholarPubMed
Meng, X. -B., Tao, J., &Chang, H. -H.(2015). A conditional joint modeling approach for locally dependent item responses and response times. Journal of Educational Measurement, 52(1),1–27CrossRefGoogle Scholar
Mitchell, T. J., &Beauchamp, J. J.(1988). Bayesian variable selection in linear regression. Journal of the American Statistical Association, 83(404),1023–1032CrossRefGoogle Scholar
Molenaar, D. , Tuerlinckx, F. , & van der Maas, H. L. J. (2015). Fitting diffusion item response theory models for responses and response times using the r package diffirt. Journal of Statistical Software, 66(4), 1–34. https://doi.org/10.18637/jss.v066.i04CrossRefGoogle Scholar
Partchev, I., &De Boeck, P.(2012). Can fast and slow intelligence be differentiated?. Intelligence, 40(1),23–32CrossRefGoogle Scholar
Ratcliff, R.(1978). A theory of memory retrieval. Psychological Review, 85(2),59–108CrossRefGoogle Scholar
Ratcliff, R., &McKoon, G.(2008). The diffusion decision model: Theory and data for two-choice decision tasks. Neural Computation, 20(4),873–922CrossRefGoogle ScholarPubMed
Roberts, J. S., Donoghue, J. R., &Laughlin, J. E.(2000). A general item response theory model for unfolding unidimensional polytomous responses. Applied Psychological Measurement, 24(1),3–32CrossRefGoogle Scholar
Roberts, J. S., &Laughlin, J. E.(1996). A unidimensional item response model for unfolding responses from a graded disagree-agree response scale. Applied Psychological Measurement, 20(3),231–255CrossRefGoogle Scholar
Smith, A. L., Asta, D. M., &Calder, C. A.(2019). The Geometry of Continuous Latent Space Models for Network Data. Statistical Science, 34(3),428–453CrossRefGoogle ScholarPubMed
Stan Development Team. (2021). Stan Modeling Language User’s Guide and Reference Manual. URL:http://mc-stan.org/Google Scholar
Thurstone, L. L.(1927). A law of comparative judgment. Psychological Review, 34(4),273–286CrossRefGoogle Scholar
Tuerlinckx, F.(2004). The efficient computation of the cumulative distribution and probability density functions in the diffusion model. Behavior Research Methods, Instruments, and Computers, 36(4),702–716CrossRefGoogle ScholarPubMed
Tuerlinckx, F., &De Boeck, P.(2005). Two interpretations of the discrimination parameter. Psychometrika, 70(4),629–650CrossRefGoogle Scholar
van der Linden, W. J.(2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3),287–308CrossRefGoogle Scholar
van der Linden, W. J.(2009). Conceptual issues in response-time modeling. Journal of Educational Measurement, 46(3),247–272CrossRefGoogle Scholar
van der Linden, W. J., & Glas, CAW(2010). Statistical tests of conditional independence between responses and/or response times on test items. Psychometrika, 75, 120–139CrossRefGoogle Scholar
van der Maas, H. L. J., Molenaar, D., Maris, G., Kievit, R. A., &Borsboom, D.(2011). Cognitive psychology meets psychometric theory: On the relation between process models for decision making and latent variable models for individual differences. Psychological Review, 118(2),339–356CrossRefGoogle ScholarPubMed
van der Maas, H. L. J., &Wagenmakers, E. -J.(2005). A psychometric analysis of chess expertise. The American journal of psychology, 118, 29–60CrossRefGoogle ScholarPubMed
van Rijn, P. W., & Ali, U. S.(2017). A comparison of item response models for accuracy and speed of item responses with applications to adaptive testing. British Journal of Mathematical and Statistical Psychology, 70(2),317–345CrossRefGoogle ScholarPubMed
Wang, C., &Xu, G.(2015). A mixture hierarchical model for response times and response accuracy. British Journal of Mathematical and Statistical Psychology, 68(3),456–477CrossRefGoogle ScholarPubMed
Wang, C., Xu, G., & Shang, Z.(2018). A two-stage approach to differentiating normal and aberrant behavior in computer based testing. Psychometrika, 83(1),223–254CrossRefGoogle ScholarPubMed
Zhan, P., Jiao, H., &Liao, D.(2018). Cognitive diagnosis modelling incorporating item response times. British Journal of Mathematical and Statistical Psychology, 71(2),262–286CrossRefGoogle ScholarPubMed
Zhan, P. , Jiao, H. , Wang, W. C. , & Man, K. (2018). A multidimensional hierarchical framework for modeling speed and ability in computer-based multidimensional tests. Available online at: arXiv:1807.04003Google Scholar
Supplementary material: File

Kang et al. supplementary material

Kang et al. supplementary material
Download Kang et al. supplementary material(File)
File 5 MB