Hostname: page-component-745bb68f8f-v2bm5 Total loading time: 0 Render date: 2025-01-08T09:37:48.488Z Has data issue: false hasContentIssue false
  • English
  • Français

A Measure for the Reliability of a Rating Scale Based on Longitudinal Clinical Trial Data

Published online by Cambridge University Press:  01 January 2025

Annouschka Laenen ,
Ariel Alonso  and
Geert Molenberghs
Annouschka Laenen*
Affiliation:
Hasselt University
Ariel Alonso
Affiliation:
Hasselt University
Geert Molenberghs
Affiliation:
Hasselt University
*
Requests for reprints should be sent to Annouschka Laenen, Universiteit Hasselt, Centrum voor Statistiek, Agoralaan–Gelouw D, 3590 Diepenbeek, Belgium. E-mail: [email protected]
Get access Rights & Permissions [Opens in a new window]

Abstract

A new measure for reliability of a rating scale is introduced, based on the classical definition of reliability, as the ratio of the true score variance and the total variance. Clinical trial data can be employed to estimate the reliability of the scale in use, whenever repeated measurements are taken. The reliability is estimated from the covariance parameters obtained from a linear mixed model. The method provides a single number to express the reliability of the scale, but allows for the study of the reliability’s time evolution. The method is illustrated using a case study in schizophrenia.

Keywords

reliabilitylinear mixed modellongitudinal datapsychiatryrating scale
Type
Theory and Methods
Information
Psychometrika , Volume 72 , Issue 3 , September 2007 , pp. 443 - 448
Copyright
Copyright © 2007 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

The authors are grateful to J&J PRD for kind permission to use their data. We gratefully acknowledge support from the Belgian IUAP/PAI network “Statistical Techniques and Modeling for Complex Substantive Questions with Complex Data.”

References

Alonso, A., Geys, H., Molenberghs, G., Vangeneugden, T. (2002). Investigating the criterion validity of psychiatric symptom scales using surrogate marker validation methodology. Journal of Biopharmaceutical Statistics, 12, 161–179.CrossRefGoogle ScholarPubMed
Bock, R.D. (1966). Contributions of multivariate experimental designs to educational research. In Cattell, R.B. (Eds.), Handbook of multivariate experimental psychology, Chicago: Rand-McNally.Google Scholar
Bock, R.D. (1983). The discrete Bayesian. In Wainer, H. & Messick, S. (Eds.), Principals of modern psychological measurement. Hillsdale, NJ: Erlbaum.Google Scholar
Diggle, P.J., Liang, K.-Y., & Zeger, S.L. (1994). Analysis of longitudinal data. Oxford Science Publications. Oxford, UK: Clarendon Press.Google Scholar
Laird, N.M., Ware, J.H. (1982). Random effects models for longitudinal data. Biometrics, 38, 963–974.CrossRefGoogle ScholarPubMed
Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading, MA: Addison-Wesley.Google Scholar
Vangeneugden, T., Laenen, A., Geys, H., Renard, D., Molenberghs, G. (2004). Applying linear mixed models to estimate reliability in clinical trial data with repeated measurements. Controlled Clinical Trials, 25(1), 13–30.CrossRefGoogle ScholarPubMed
Verbeke, G., Molenberghs, G. (2000). Linear mixed models for longitudinal data, New York: Springer.Google Scholar