Hostname: page-component-745bb68f8f-g4j75 Total loading time: 0 Render date: 2025-01-08T09:55:08.254Z Has data issue: false hasContentIssue false
  • English
  • Français

Assessing the Accuracy of Errors of Measurement. Implications for Assessing Reliable Change in Clinical settings

Published online by Cambridge University Press:  01 January 2025

Alberto Maydeu-Olivares Open the ORCID record for Alberto Maydeu-Olivares [Opens in a new window]
Alberto Maydeu-Olivares*
Affiliation:
University of South Carolina University of Barcelona
*
Correspondence should be made to Alberto Maydeu-Olivares, Department of Psychology, University of South Carolina, Barnwell College, 1512 Pendleton St., Columbia, SC29208, USA. Email: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

Item response theory (IRT) models are non-linear latent variable models for discrete measures, whereas factor analysis (FA) is a latent variable model for continuous measures. In FA, the standard error (SE) of individuals’ scores is common for all individuals. In IRT, the SE depends on the individual’s score, and the SE function is to be provided. The empirical standard deviation of the scores across discrete ranges should also be computed to inform the extent to which IRT SEs overestimate or underestimate the variability of the scores. Within the target range of scores the test was designed to measure, one should expect IRT SEs to be smaller and more precise than FA SEs, and therefore preferable to assess clinical change. Outside the target range, IRT SEs may be too large and more imprecise than FA SEs, and FA more precise to assess change. As a result, whether FA or IRT characterize reliable change more accurately in a sample will depend on the proportion of individuals within or outside the IRT target score range. An application is provided to illustrate these concepts.

Keywords

Classical test theory
Type
Application Reviews and Case Studies
Information
Psychometrika , Volume 86 , Issue 3: Special Section: Advancing Methods to Assess Patient-Reported Outcomes: Lessons Learned from the Patient-Reported Outcomes Measurement Information System® (PROMIS®) Initiative , September 2021 , pp. 793 - 799
Copyright
Copyright © 2021 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bock, R. D., Aitkin, M. Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, (1981). 46 (4 443–459CrossRefGoogle Scholar
Christensen, L., Mendoza, J. L. A method of assessing change in a single subject: An alteration of the RC index. Behavior Therapy, (1986). 17 (3 305–308CrossRefGoogle Scholar
Forero, C. G., Maydeu-Olivares, A. Estimation of IRT graded response models: Limited versus full information methods. Psychological Methods, (2009). 14 (3 275–299CrossRefGoogle ScholarPubMed
Hayakawa, K. Corrected goodness-of-fit test in covariance structure analysis. Psychological Methods, (2019). 24 (3 371–389CrossRefGoogle ScholarPubMed
Hays, R. D., Spritzer, K. L., Reise, S. P. Using item response theory to identify responders to treatment: Examples with the patient-reported outcomes measurement information system (PROMIS®) physical function scale and emotional distress composite. Psychometrika, (2021).CrossRefGoogle ScholarPubMed
Jacobson, N. S., Follette, W. C., Revenstorf, D. Psychotherapy outcome research: Methods for reporting variability and evaluating clinical significance. Behavior Therapy, (1984). 15 (4 336–352CrossRefGoogle Scholar
Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), 12–19. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/2002127.CrossRefGoogle ScholarPubMed
Lord, F. M. (1980). Applications of item response theory to practical testing problems. Lawrence Erlbaum.Google Scholar
Maydeu-Olivares, A. Goodness-of-fit assessment of item response theory models. Measurement: Interdisciplinary Research & Perspective, (2013). 11 (3 71–101Google Scholar
McDonald, R. P. (1999). Test theory: A unified approach. Erlbaum.Google Scholar
Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monograph No. 17.CrossRefGoogle Scholar
Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal, and structural equation models. CRC Press.CrossRefGoogle Scholar
Thomson, G. H. (1938). The factorial analysis of human ability. University of London Press.Google Scholar
Thurstone, L. L. The vectors of mind: Multiple-factor analysis for the isolation of primary traits. University of Chicago Press, (1935).Google Scholar