Hostname: page-component-745bb68f8f-f46jp Total loading time: 0 Render date: 2025-01-08T10:21:59.113Z Has data issue: false hasContentIssue false
  • English
  • Français

Measures of Reliability for Profiles and Test Batteries

Published online by Cambridge University Press:  01 January 2025

Anthony J. Conger  and
Raanan Lipshitz
Anthony J. Conger
Affiliation:
University of North Carolina
Raanan Lipshitz
Affiliation:
University of North Carolina
Get access Rights & Permissions [Opens in a new window]

Abstract

A general index of reliability, termed “canonical reliability,” is developed for use with profiles, or more generally, for use with vectors of random variables. Canonical reliability is defined as the ratio of the average squared distance among true scores to the average squared distance among observed scores. Based on Mahalonobis distances, canonical reliability is shown to be consistent with multivariate analogues of parallel form correlations, squared correlation between true and observed scores, and an analysis of variance formulation. The index of reliability based on Cronbach and Gleser's D2 is also derived from the general formulation. A comparison of the Mahalonobis and D2 approaches indicates that score vectors using D2 distances are more reliable; however, both methods of comparing profiles are useful depending on the nature of the information that is desired. Transforming the observed variables to independent canonical variates provides a basis for comparing profiles on maximally reliable profile dimensions. For illustrative purposes, profile reliability is calculated and interpreted for the WISC subscales for a 7 1/2 year age group.

Type
Original Paper
Information
Psychometrika , Volume 38 , Issue 3 , September 1973 , pp. 411 - 427
Copyright
Copyright © 1973 The Psychometric Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Footnotes

1

This research was supported in part by a PHS Research Grant No. HM-10006 from the National Institute of Mental Health, Public Health Service and in part by a Science Development Grant No. GU-2059 from the National Science Foundation.

2

The authors thank Dr. Elliot Cramer for many useful suggestions and Dr. Judith Conger for a careful reading of the manuscript. Requests for reprints should be sent to Anthony J. Conger, Psychology Department, Duke University, Durham, N. C., 27706.

References

Bock, R. D. Contributions of multivariate experimental designs to educational research. In Cattell, R. B. (Eds.), Handbook of Multivariate Experimental Psychology. Chicago: Rand McNally. 1966, 820–840Google Scholar
Cattell, R. B. r p and other coefficients of pattern similarity. Psychometrika, 1949, 14, 279–298CrossRefGoogle ScholarPubMed
Cohen, J. r e: a profile similarity coefficient invariant over variable reflection. Psychological Bulletin, 1969, 71, 281–284CrossRefGoogle Scholar
Conger, A. J. Estimating profile reliability and maximally reliable composities. Multivariate Behavioral Research, in press.Google Scholar
Cronbach, L. J. and Gleser, G. C. Assessing similarity between profiles. Psychological Bulletin, 1953, 50, 456–473CrossRefGoogle ScholarPubMed
Cronbach, L. J., Gleser, G. C., Nanda, H. and Rajaratnam, N. The Dependability of Behavioral Measurements, 1972, New York: WileyGoogle Scholar
Gaier, E. L. and Lee, M. C. Pattern analysis: the configural approach to predictive measurement. Psychological Bulletin, 1953, 50, 140–148CrossRefGoogle ScholarPubMed
Graybill, F. A. Introduction to Matrices with Applications in Statistics, 1969, Belmont, California: WadsworthGoogle Scholar
Guttman, L. A basis for analyzing test retest reliability. Psychometrika, 1945, 10, 255–282CrossRefGoogle ScholarPubMed
Harris, C. W. Characteristics of two measures of profile similarity. Psychometrika, 1955, 20, 289–297CrossRefGoogle Scholar
Hoyt, C. Test reliability estimated by analysis of variance. Psychometrika, 6, 153–160.CrossRefGoogle Scholar
Lord, F. M. and Novick, M. R. Statistical theories of mental test scores, 1968, Reading, Massachusetts: Addison-WesleyGoogle Scholar
Nunnally, J. The analysis of profile data. Psychological Bulletin, 1962, 59, 311–319CrossRefGoogle ScholarPubMed
Osgood, C. E. and Suci, G. A measure of relation determined by both mean difference and profile information. Psychological Bulltin, 1952, 59, 251–262CrossRefGoogle Scholar
Overall, J. Note on multivariate methods for profile analysis. Psychological Bulletin, 1964, 61, 195–198CrossRefGoogle ScholarPubMed
Tellegen, A. Direction of measurement: a source of misinterpretation. Psychological Bulletin, 1965, 63, 233–243CrossRefGoogle ScholarPubMed
Webster, H. A note on profile similarity. Psychological Bulletin, 1952, 49, 538–539CrossRefGoogle ScholarPubMed
Wechsler, D. Wechsler Intelligence Scale for Children Manual, 1949, New York: The Psychological CorporationGoogle Scholar
Winer, B. J. Statistical Principles in Experimental Design, 1962, New York: McGraw-HillCrossRefGoogle Scholar